Test Availability and Inspect Web Pages using Powershell

Have you ever wanted to test the availability of a URL or web page?  or check for the presence of content or links on a page?  even clicking through a flow of web pages automatically?

Well, these things are not necessarily new concepts or challenges.  In fact, in the Application Testing space this is the type of thing we do day in and out.

I have recently been messing around with Selenium and Python web test scripts, and just for kicks wondered if similar things could be achieved using Powershell.  Funnily enough, the answer is yes, and quite easily too for anyone familiar with Powershell scripting.

So here are some basics just for fun, to get things going...   the magic sauce is in the Invoke-WebRequest cmdlet.  Let's check up on this blog's main page as follows by feeding the URL we want to check to the Invoke-WebRequest cmdlet, and assigning the resulting object to a variable called $webpage:

$webpage = Invoke-WebRequest https://blazingtm.com

Nice.  Exciting. Maybe not 😃

What can we do with this thing called $webpage you might ask.  Well, first let's see what type this object actually is.  You can do this calling

$webpage.GetType()

It's a thing called HtmlWebResponseObject.  Ok, so not just a simple "webpage", but an object.  Well, so it may have associated member properties we can inspect, maybe that could be interesting.  We can do this using the following cmdlet

$webpage| Get-Member

The result should look something like follows, listing all member properties and methods for the webpage:

Name	MemberType	Definition
----	----------	----------
Dispose	Method	void Dispose(), void IDisposable.Dispose()
Equals	Method	bool Equals(System.Object obj)
GetHashCode	Method	int GetHashCode()
GetType	Method	type GetType()
ToString	Method	string ToString()
AllElements	Property	Microsoft.PowerShell.Commands.WebCmdletElementCollection
BaseResponse	Property	System.Net.WebResponse BaseResponse {get;set;}
Content	Property	string Content {get;}
Forms	Property	Microsoft.PowerShell.Commands.FormObjectCollection Forms
Headers	Property	System.Collections.Generic.Dictionary[string,string] Hea
Images	Property	Microsoft.PowerShell.Commands.WebCmdletElementCollection
InputFields	Property	Microsoft.PowerShell.Commands.WebCmdletElementCollection
Links	Property	Microsoft.PowerShell.Commands.WebCmdletElementCollection
ParsedHtml	Property	mshtml.IHTMLDocument2 ParsedHtml {get;}
RawContent	Property	string RawContent {get;set;}
RawContentLength	Property	long RawContentLength {get;}
RawContentStream	Property	System.IO.MemoryStream RawContentStream {get;}
Scripts	Property	Microsoft.PowerShell.Commands.WebCmdletElementCollection
StatusCode	Property	int StatusCode {get;}
StatusDescription	Property	string StatusDescription {get;}

Let's have a little look at that StatusCode property.  To see what it is, use

$statuscode = $webpage.StatusCode

You would have guessed, if we look at the value of $statuscode we will see it's 200.  Provided the webpage was available earlier of course. 200 is good when it comes to web page availability.

We can test for this using something along the lines of

if ($statuscode -eq 200) {Write-Output("Brilliant, this page is available :)")}

else {Write-Output("Well, that did not work :(")}

Equally, we might want to make sure the page is not only available (ie a Status Code 200), but that it has a certain content on the page.  That is often useful to know to ensure a page has rendered the right results.

This is where the Content property comes in handy.  Let's say we wanted to check that the page contains the phrase "Late night musings".  Too easy, do this

$webpage.Content.Contains("Late night musings")

Now this will simply return True, or False, depending on whether the phrase is found.  Of course you can have further logic on the back of this, but you can see the basics of some interesting web tests evolving here.

That's all for tonight, but following on from this wouldn't it be great if we could script up a whole user flow navigating web pages to test availability of a web site, not just a page.  That's however a topic for another time.

Enjoy.

MB