Skip to content

Extend -SkipHeaderValidation to include -UserAgent to support non-standard User-Agent headers#4479

Merged
TravisEz13 merged 2 commits intoPowerShell:masterfrom
markekraus:UserAgentSkipValidation
Aug 4, 2017
Merged

Extend -SkipHeaderValidation to include -UserAgent to support non-standard User-Agent headers#4479
TravisEz13 merged 2 commits intoPowerShell:masterfrom
markekraus:UserAgentSkipValidation

Conversation

@markekraus
Copy link
Copy Markdown
Contributor

@markekraus markekraus commented Aug 3, 2017

Synopsis

Extends #4085 to include the -UserAgent parameter supplied User-Agent header.

Description

Some API's, such as Reddit require special User-Agent headers to identify application consumers of the API. Currently, Invoke-WebRequest and Invoke-RestMethod are unable to properly use non-compliant User-Agent headers. This extends the behavior of the -SkipHeaderValidation switch parameter to include the User-Agent header supplied by the -UserAgent parameter.

Test Code

$Uri = 'http://httpbin.org/headers'
Invoke-WebRequest -Uri $Uri -UserAgent 'Invalid:Agent' -SkipHeaderValidation
Invoke-RestMethod -Uri $Uri -UserAgent 'Invalid:Agent' -SkipHeaderValidation

Current Behavior:

Invoke-WebRequest : The format of value 'Invalid:Agent' is invalid.
At line:2 char:1
+ Invoke-WebRequest -Uri $Uri -UserAgent 'Invalid:Agent'
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Invoke-WebRequest], FormatException
    + FullyQualifiedErrorId : System.FormatException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand

Invoke-RestMethod : The format of value 'Invalid:Agent' is invalid.
At line:3 char:1
+ Invoke-RestMethod -Uri $Uri -UserAgent 'Invalid:Agent'
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Invoke-RestMethod], FormatException
    + FullyQualifiedErrorId : System.FormatException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand

New Behavior


StatusCode        : 200
StatusDescription : OK
Content           : {
                      "headers": {
                        "Connection": "close",
                        "Host": "httpbin.org",
                        "User-Agent": "Invalid:Agent"
                      }
                    }

RawContent        : HTTP/1.1 200 OK
                    Connection: keep-alive
                    Date: Thu, 03 Aug 2017 10:25:25 GMT
                    Via: 1.1 vegur
                    Server: meinheld/0.6.1
                    Access-Control-Allow-Origin: *
                    Access-Control-Allow-Credentials: true
                    X-Powered-...
Forms             :
Headers           : {[Connection, System.String[]], [Date, System.String[]], [Via, System.String[]], [Server,
                    System.String[]]...}
Images            : {}
InputFields       : {}
Links             : {}
ParsedHtml        :
RawContentLength  : 113
RelationLink      : {}


headers : @{Connection=close; Host=httpbin.org; User-Agent=Invalid:Agent}

@msftclas
Copy link
Copy Markdown

msftclas commented Aug 3, 2017

@markekraus,
Thanks for having already signed the Contribution License Agreement. Your agreement was validated by Microsoft. We will now review your pull request.
Thanks,
Microsoft Pull Request Bot

SteveL-MSFT
SteveL-MSFT requested changes Aug 4, 2017
View reviewed changes
Copy link
Copy Markdown
Member

@SteveL-MSFT SteveL-MSFT left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution, just one requested change

test/powershell/Modules/Microsoft.PowerShell.Utility/WebCmdlets.Tests.ps1 Outdated

try {
if ($Cmdlet -eq 'Invoke-WebRequest') {
$result.Output = Invoke-WebRequest -Uri $Uri -TimeoutSec 5 -UserAgent $UserAgent -SkipHeaderValidation:$SkipHeaderValidation.IsPresent
Copy link
Copy Markdown
Member

@SteveL-MSFT SteveL-MSFT Aug 4, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of duplicating all the parameters, you should use splatting:

$args = @{Uri=$uri;Timeout=5;UserAgent=$useragent;...}
Invoke-WebRequest @args
Invoke-RestMethod @args

Copy link
Copy Markdown
Contributor Author

@markekraus markekraus Aug 4, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Used $Params instead of $args to avoid conflicts with the automatic variable.

SteveL-MSFT
SteveL-MSFT approved these changes Aug 4, 2017
View reviewed changes
Copy link
Copy Markdown
Member

@SteveL-MSFT SteveL-MSFT left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@TravisEz13 TravisEz13 merged commit fc77c79 into PowerShell:master Aug 4, 2017
markekraus added a commit to markekraus/PowerShell that referenced this pull request Aug 6, 2017
@markekraus
Fix Tests from PowerShell#4479
f2234b4
@markekraus markekraus mentioned this pull request Aug 6, 2017
Merged
adityapatwardhan pushed a commit that referenced this pull request Aug 7, 2017
@markekraus @adityapatwardhan
Fix Tests from #4479 (#4512)
e4060ab 
* Fix Tests from #4479

* [Feature]
@markekraus markekraus deleted the UserAgentSkipValidation branch August 25, 2017 00:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@SteveL-MSFT SteveL-MSFT SteveL-MSFT approved these changes

Assignees

@TravisEz13 TravisEz13

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@markekraus @msftclas @SteveL-MSFT @TravisEz13