Automating WebPageTest using the nodejs webpagetest-api package

WebPageTest-api NodeJS

Hopefully you’ve already had a chance to play around with the amazing WebPageTest during your website performance testing adventure so far.

In case not, I have a few articles you might like to browse, to help you get up to speed using this fantastic, free, open source, website performance testing tool.

It has a website interface and also an API, which I went through in the previous article.

In this article I’ll show you how to use the incredible webpagetest-api nodejs package to make the orchestration and automation of your WebPageTest setup even easier!

webpagetest-api

"But Robin, that’s the same heading as the previous article! What are you playing at, you silly man?" I hear you cry.

Worry not! It’s all good! webpagetest-api is a nodejs wrapper for the WebPageTest API that I demonstrated previously, making it much easier to codify and manage the whole process. You can use it from the command line or from within nodejs.

And I’m not a silly man.

First up install the webpagetest-api node package:

npm install webpagetest

Now we can do the equivalent HTTP requests to the WebPageTest API using either the command line interface (cli) or a little bit of JavaScript, such as these equivalent versions of the main API calls:

1) Executing a test run using runTest()

To queue up a test (previously we used runtest.php via the WebPageTest HTTP API):

// reference the webpagetest-api module
const WebPageTest = require("webpagetest");

// create a new instance of the wrapper with the address of
// the webpagetest server; this is the public instance,
// and you could use your private instance here too!
const wpt = new WebPageTest("www.webpagetest.org");

// dozens of options are available to be configured here:
// https://github.com/marcelduran/webpagetest-api
let options = {
    "key": "your api key",
    "firstViewOnly": true,
    "runs": 3
  };

// queue up the test
wpt.runTest(
    "https://www.microsoft.com",
     options,
     (err, result) => {

  // The test ID is returned in the response here:
    let testId = result.data.testId;

  /*
  {
    "statusCode": 200,
    "statusText": "Ok",
    "data": {
        "testId": "190618_SQ_51",
        ...
    }
  }
  */
});

2) Getting the status of a test with getTestStatus()

This method will retrieve the status of the test, returning the various possible status descriptions and codes within the response object, such as:

  • "Waiting behind 27 other tests…"
  • "Test Started 1 second ago"
  • "Completed 7 of 9 tests"
  • "Test Complete"

Previously we used testStatus.php to get this info; the equivalent within the webpagetest-api is getTestStatus:

wpt.getTestStatus("190616_Y7_c8", (err, res) => {
  console.log(err || res);
});

3) Getting the results of a test with getTestResults()

To retrieve the entire result of the test run, use getTestResults. Previously this was jsonResult.php or xmlResult:

wpt.getTestResults("190616_Y7_c8", (err, res) => {
  console.log(err || res);
});

"Ok, so whoop-de-doo, you’ve shown me another way to do the same thing – so what?" I hear you impatiently screech. Stop screeching and enhance your patience, I’m getting to the key part here, so chill!

Using the WebPageTest API directly meant that your options to run a test and subsequently retrieve the test results were pretty much limited to:

  1. execute a test and poll testStatus.php or jsonResults.php/xmlResults until we got the full payload, working through that data to pull out what we need, or
  2. pass in a pingback web hook URL when executing a test in order to get the test id of a completed test when it finishes, then have a separate process to pull and process the full results.

However, this nodejs API wrapper gives you multiple ways to queue up a test and get the full test results in one go!

A) Poll for results

The process will sit around and check the test results on a specified delay, and optionally timeout after a specified period.

wpt.runTest(
    "https://www.microsoft.com",
    {pollResults: 5, timeout: 60},
    (err, res) => {console.log(err || res);
});

Pro:

  • Very simple to set up; no need to have a separate webhook API or expose a port.

You can kick off a test and, if the test completes before the timeout has been reached, the full test results are returned for you to drill into:

{
  "data": {
      "id": "190628_D3_2MZ",
      "url": "https://www.apple.com/uk",
      "summary": "http://www.webpagetest.org/results.php?test=190628_D3_2MZ",
      "testUrl": "https://www.apple.com/uk",
      "location": "eu-west-1-linux:Chrome",
      "from": "Europe - Linux (Ireland) - <b>Chrome</b> - <b>Cable</b>",
      "connectivity": "Cable",
      "bwDown": 5000,
      "bwUp": 1000,
      "latency": 14,
      "plr": "0",
      "mobile": 0,
      "completed": 1561721766,
      "tester": "blah",
      "runs": { "1": "" },
      "fvonly": false,
      "successfulFVRuns": 1,
      "successfulRVRuns": 1,
      "average": { "firstView": "...", "repeatView": "..." },
      "standardDeviation": { "firstView": "...", "repeatView": "..." },
      "median": { "firstView": "...", "repeatView": "..." }
    },
  "statusCode": 200,
  "statusText": "Test Complete",
  "webPagetestVersion": "18.10"
}

Con:

  • Could lose results if timeout is reached.

When a timeout occurs this doesn’t mean that the test itself has failed, just that the test is taking longer to complete than your specified timeout value.

The test ID is in the error response so you could just put this test ID in a retry queue and check for results again later on:

{
    "error": {
        "code": "TIMEOUT",
        "testId": "190628_QZ_2MY",
        "message": "timeout"
    }
}

Con:

  • Could hang if no timeout is specified.

We can decide to have no timeout, in which case the process will just wait. And wait. And wait.

B) Pingback

Specify a pingback URL in the options that the completed testId will be sent to:

wpt.runTest("https://www.microsoft.com",
    {"pingback":"https://my.epic.webhook/handle-test"},
    (err, res) => {console.log(err || res);
});

Pros:

  • Complete separation of concerns; one process queues up tests, and then the results can be handled by another process.
  • No extra processes are spawned to wait for the results.

Cons:

  • Need to build and host a separate endpoint that WebPageTest can reach.
  • If something fails, extra work to retry that test would be needed.

C) Wait for results (simplifed pingback)

The process will just spin up a new http server locally, passing this local URL in as the pingback parameter to WebPageTest.

wpt.runTest("https://www.microsoft.com",
    {"waitResults":"localhost:8000"},
    (err, res) => {console.log(err || res);
});

Pro:

  • No need to build a separate API endpoint to process the pingback results.

Cons:

  • Need to make sure the machine running this process is accessible from your WebPageTest instance, which can be tricky.
  • A new httpserver is created for every test that is executed until that test is completed and the localhost pingback is called.

Take your pick; they will all work, so just think about your use case when making a choice.

Pingback WebHook Example

Below is a very simplistic example of a nodejs web hook; it will run on port 3000 and listen out for a request that contains an "id" querystring parameter, using that to get the test results from WebPageTest via webpagetest-api, and selecting a subset of the MASSIVE results dataset:

// webpagetest pingback webhook to receive the test Id
// and extract some specific data from the test results.

const http = require("http"),
    url = require("url"),
    qs = require('querystring'),
    WebPageTest = require('webpagetest'),
    wpt = new WebPageTest('www.webpagetest.org');

let port =  3000;

http.createServer((request, response) => {
    // get the test Id from the incoming URL
    let testId =
        qs.parse(
            url.parse(
                request.url
            )
        .query
    ).id;

    // use that test Id in the `getTestResults` method
    wpt.getTestResults(testId, (err, result) => {

        // Example: get the median run for firstView
        let median = result.data.median.firstView;

        // get a few specific values from the median
        let testResult = {};
        testResult.vc = median.visualComplete85;
        testResult.speedIndex = median.SpeedIndex;
        testResult.FirstPaintedHero = median.FirstPaintedHero;
        testResult.LastPaintedHero = median.LastPaintedHero;
        testResult.testId = testId;

        // do something with that data,
        // e.g. persist it somewhere (or just log it out!)
        const testResponse = JSON.stringify(testResult);
        console.log(testResponse);

        response.write("Ok");
        response.end();
    });
}).listen(port);

I always forget a simple way to set up a nodejs api, so regularly refer to this ancient article I wrote back in 2013!

Save this as wpt-pingback.js and run this little script using node wpt-pingback.js; it will start listening on port 3000.

Remember ngrok from the previous article? In case not, ngrok is an incredibly powerful tool that provides a secure tunnel to localhost. This means you can expose ports on your dev PC on various web ports temporarily, and have a packet inspector in a web interface. If you don’t already have it, definitely check it out; I find it invaluable for learning what comes back from a web hook.

Download ngrok and fire it up using a command like ngrok http 3000 (expose a local process running on port 3000 over HTTP) to get something like this:

ngrok by @inconshreveable                           (Ctrl+C to quit)

Session Status      online
Session Expires     7 hours, 59 minutes
Version             2.3.30
Region              United States (us)
Web Interface       http://127.0.0.1:4040
Forwarding          http://18e2a650.ngrok.io -> http://localhost:3000
Forwarding          https://18e2a650.ngrok.io -> http://localhost:3000

That means we now have a temporary end point on http(s)://18e2a650.ngrok.io; let’s use the resulting URL in the pingback parameter to get the results:

wpt.runTest('https://www.microsoft.com/en-gb/',
    {"pingback":"https://18e2a650.ngrok.io"},
    (err, res) => {console.log(err || res);
});

Once the test has completed, the subset of results as extracted by the WebHook logic should appear in the console:

{
    "vc": 1300,
    "speedIndex": 1313,
    "FirstPaintedHero": 1400,
    "LastPaintedHero": 1700,
    "testId": "190624_D2_19"
}

We can develop that script some more and push it out as an AWS Lambda, Azure Function, or similar, and we will have automated the process of receiving and inspecting WebPageTest results, extracting just the data we’re interested in.

Now we’re using WebPageTest as Performance Testing as a Service: PTaaS! (It’ll catch on, trust me! #PTaaS)

Summary

Thanks to the wonderful webagetest-api we can now:

  • run WebPageTest from within nodejs
  • easily both enqueue a test and process the results of a test in one go

Troubleshooting

webpagetest-api‘s waitResults actually spawns an httpServer running on a new port on the system that has called runtest, which can eventually become exhausted, especially on an AWS Lambda.

Increasing the available resources, or decreasing the number of concurrent tests that are waiting will help. On AWS Lambdas, just increase the memory which also increases the available CPU.

I hope that was useful; let me know how you get on and shout if you have any questions.

Leave a Reply

Your email address will not be published. Required fields are marked *