Tag Archives: aws

Automate Your WebPageTest Private Instance With Terraform: 2021 Edition

This article assumes you have an understanding of what Terraform is, what WebPageTest is and some AWS basics.

all the logos

Have you ever wanted to have your WebPageTest setup managed as infrastructure as code, so you can keep all those carefully tuned changes and custom settings in source control, ready to confidently and repeatedly destroy and rebuild at a whim?

Sure you have.

In this article I’ll show how to script the setup of your new WPT server, installing from a base OS, and configuring customisations – all within Terraform so you can easily rebuild it with a single command.

Continue reading →

Google’s Chrome User Experience Data in WebPageTest

This article assumes that you know the basics of AWS, WebPageTest, SSH, and at least one linux text editor.

Chrome User Experience Report logo + WebPageTest Logo + Core Web Vitals' LCP thresholds. Quite a busy hero image, I'll admit.

When talking to people about website performance stats, I’ll usually split it into Real User Metrics (RUM) and Automated (Synthetic/Test Lab):

  • RUM is performance data reported from the website you own, reported into the analytics tool you have integrated.
  • Automated are scripted tests that you run from your own performance testing tool against any website you like.

RUM is great: you get real performance details from real user devices and can investigate the difference in performance for many different options.

For example, iPhone vs Android, Mac vs Windows, Mobile vs Desktop, Chrome vs Firefox, UK vs US, even down to ISP and connection type, in order to see who is getting a good experience and who can be improved.

This data is invaluable in prioritising performance improvements, since you can tie it back to the approximate number of users it will affect, and therefore the impact on your business.

There are loads of vendors who can provide this for you (I’ve used many of them), or you can write your own – if you’re a glutton for punishment (and high AWS bills – ask me how I know…😁)

However, since this is measured on your own website and reported into your own tooling, you can only see such real-world performance detail for your own website; no real-world user experience data from your competitors.

Automated tests are great: you get detailed measurements of any website you can access – your own or competitors, or basically any website – in a repeatable fashion so that you can track changes over time.

You can have as many automated tests as you like, you can test from wherever you’re able to set up a test agent, and with whatever device you’re able to automate or emulate.

However, since these are all automated tests running because you said so, you can’t use them to understand how users are using your site, on which devices, what devices are underperforming others, and from where.

Again, there are a load of vendors who can provide this for you; writing your own is a bit more of a headache though – I wouldn’t recommend it, especially while wpt continues to be free for self hosting.

What if you could get some of the usual key performance metrics you’re used to with RUM, but for sites you don’t own such as your competitors?

In this article I’ll talk about the Google Chrome User Experience dataset and how to use it in your performance test setup to find the intersection of RUM and Automated performance test results, wiring it all up into your WebPageTest setup! Continue reading →

WebPageTest Private Instance: 2021 Edition

Catchpoint's WebPageTest

The fantastic WebPageTest, free to use and public, has been available to set up your own private instances for many years; I wrote this up a while back, and scripted a Terraform version to make this as easy and automated as possible.

For AWS it was just a case of creating an EC2 instance (other installation options are available) with a predefined WPT server AMI (amazon machine image), add in a few configuration options and boom – your very own, autoscaling, globally distributed, website performance testing solution! New test agents would spin up automatically in other AWS regions, all based on WebPageTest Agent AMIs.

In 2020 WebPageTest was bought by Catchpoint and we finally saw improvements being made, pull requests being closed, and the WebPageTest UI getting a huge update; things were looking great for WebPageTest enthusiasts! If you havent heard of Catchpoint before, they are a company who are all about global network and web application monitoring, so a good match for WebPageTest.

Unfortunately, however, this resulted in the handy WebPageTest server EC2 AMIs no longer existing. If you want your own private installation you now need to build your own WebPageTest server from a base OS. It can be a bit tricky, though it gives you greater understanding of how it works under the hood, so hopefully you’ll feel more confident extending your installation in future.

In this article, I’ll show you how to create a WebPageTest private instance on AWS from scratch (no AMI), create your own private WebPageTest agents using the latest and greatest version of WebPageTest, and wire it all up.

Continue reading →

Customized WebPageTest Lighthouse Results using a Custom Test Agent

In my recent article Eco Worriers: Saving the Planet, One Unoptimized Website at a Time for the fantastic annual Perf Planet advent calendar, I mentioned how I created a Private WebPageTest setup to use my own custom test agents, which were configured to use an extra Lighthouse plugin (The Green Web Foundation’s "greenhouse").

In this article I’ll show how to create custom WebPageTest agents, and how to configure your Private WebPageTest instance to use these instead of the default test agents.

Continue reading →

A Step by Step Guide to using Terraform to define an AutoScaling Private WebPageTest instance in code

Update: November 2021
This article is out of date; there are no longer WebPageTest Server AMIs, so you need to install WPT on a base OS. There is an updated article here Automate Your WebPageTest Private Instance With Terraform: 2021 Edition

WebPagetest Terraform

In a previous article I went through the steps needed to create your own private, autoscaling, WebPageTest setup in Amazon AWS. It wasn’t particularly complicated, but it was quite manual; I don’t like pointing and clicking in a GUI since I can’t easily put it in version control and run it again and again on demand.

Fortunately, whatever you create within AWS can be described using a language called CloudFormation which allows you to define your infrastructure as code.

Unfortunately it’s not easy to understand (in my opinion!) and I could never quite get my head around it, which annoyed me no end.

In this article I’ll show you how to use Terraform to define your private autoscaling WebPageTest setup in easily understandable infrastructure as code, enabling an effortless and reproducable web performance testing setup, which you can then fearlessly edit and improve!

Continue reading →

A Step by Step Guide to setting up an AutoScaling Private WebPageTest instance

Update: November 2021
This article is out of date; there are no longer WebPageTest Server AMIs, so you need to install WPT on a base OS. The updated article is here WebPageTest Private Instance: 2021 Edition

If you have any interest in website performance optimisation, then you have undoubtedly heard of WebPageTest. Being able to test your websites from all over the world, on every major browser, on different operating systems, and even on physical mobile devices, is the greatest ever addition to a web performance engineer’s toolbox.

One small shelf of Pat Meenan's epic WebPageTest device lab

The sheer scale of WebPageTest, with test agents literally global (even in China!), of course means that queues for the popular locations can get quite long – not great when you’re in the middle of a performance debug session and need answers FAST!

Also since these test agents query your website from the public internet they won’t be able to hit internal systems – for example pre-production or QA, or even just a corporate intranet that isn’t accessible outside of a certain network.

In this article I’ll show you how to set up your very own private instance of WebPageTest in Amazon AWS with autoscaling test agents to keep costs down

Continue reading →

Virtual Shop Assistant Chatbot with Amazing Image Recognition

There has been some significant progress in “deep learning”, AI, and image recognition over the past couple of years; Google, Microsoft, and Amazon each have their own service offering. But what is the service like? How useful is it?

Everyone’s having a go at making a chatbot this year (and if you’re not, perhaps you should contact me for consultancy or training!) – and although there are some great examples out there, I’ve not seen much in the e-commerce sector worth talking about.

In this article I’m going to show you a cool use case for an image recognition e-commerce chatbot via a couple of clever APIs wired together by botframework.

Continue reading →