How do You Stress Test Your Web Application?

Here’s a cunning way of writing a blog post, get other people to do it for you! Luckily for me, the people who (for some reason) follow me on Twitter are a really clever lot. Yesterday I asked the question, “What do people use for stress testing their web application?” the response was so good, that it has to be recorded as a blog post.

In addition to this, you should also check out this Stack Overflow post: How to you stress test a web application, People mention JMeter, but two tools not covered here, The Grinder and Pylot get recommendations.

Here are the tools Twitter people use in no particular order:

Users

This is the tool I’ve been using as well. It works great, but users get upset when the website is stressed beyond capacity. Also I really want to see how far I can push Suteki Shop before it breaks. Up to now, it’s coped very well with the small number of users thrown at it. I want to see what happens when it gets Slashdotted – and no please, that’s not an invitation :)

WCAT

http://www.iis.net/community/default.aspx?tabid=34&i=1466&g=6

This tool is a free download from Microsoft.

“Web Capacity Analysis Tool (WCAT) is a lightweight HTTP load generation tool primarily designed to measure the performance of a web server within a controlled environment.  WCAT can simulate thousands of concurrent users making requests to a single web site or multiple web sites.  The WCAT engine uses a simple script to define the set of HTTP requests to be played back to the web server. Extensibility is provided through plug-in DLLs and a standard, simple API.”

Several people said they’d successfully used this. The major irritation with WCAT is that you have to script every request, so to be realistic you’d have to log all the requests in a typical user session and then hand-craft a script to be played back. I wonder how hard it would be to write an IIS log-file to WCAT script tool?

Loadstorm

http://loadstorm.com/

This is an cloud based service that you pay a subscription for, although they do have a free option for up to 25 concurrent users. There seem to be a few of this kind of service out there.  I guess the advantage is that you get a much more realistic scenario, an external source hammering your application, than an internal tool. Beyond that, I’m not sure why I’d use such a service.

WatiN

http://watin.sourceforge.net/

This tool isn’t really intended as a stress test tool. It’s primary use is for UI testing. It provides a nice API to launch and operate Firefox and IE so that you can write C# UI tests. However by launching multiple instances one of my correspondents has been successfully using it for stress testing. I guess my main question would be, how much hardware I would need to run enough instances of Firefox or IE to simulate high loads.

Selenium Grid

http://selenium-grid.seleniumhq.org/

Selenium is another UI test tool, similar to WatiN. It’s what I use for UI based tests. Selenium Grid allows you run to many Selenium tests in parallel. Once again, it’s primary design goal is to aid UI integration build tests, but there’s no reason you couldn’t use it run stress tests.

Selenium 2 has merged in WebDriver which can drive the headless browser HtmlUnit (see below), this might mean that sufficiently high loads can be driven from minimal hardware.

Speed Tracer

http://code.google.com/webtoolkit/speedtracer/

This is a tool from Google. It’s a Chrome extension and looks like it’s primarily designed to profile AJAX applications, especially GWT based apps. For that it looks excellent. I wonder how easy to would be to use it for stress testing a web server though. At the very least it would face the same issues as WatiN and Selenium Grid; needing a large number of Chrome instances.

HtmlUnit

http://htmlunit.sourceforge.net/

“HtmlUnit is a "GUI-Less browser for Java programs". It models HTML documents and provides an API that allows you to invoke pages, fill out forms, click links, etc... just like you do in your "normal" browser.”

This looks interesting if you don’t mind playing with Java.

“HtmlUnit is used as the underlying "browser" by different Open Source tools like Canoo WebTest, JWebUnit, WebDriver, JSFUnit, Celerity, ...”

Apache JMeter

http://jakarta.apache.org/jmeter/

JMeter, another Java tool that looks just the ticket. It’s designed specifically for load testing:

“Apache JMeter may be used to test performance both on static and dynamic resources (files, Servlets, Perl scripts, Java Objects, Data Bases and Queries, FTP Servers and more). It can be used to simulate a heavy load on a server, network or object to test its strength or to analyze overall performance under different load types. You can use it to make a graphical analysis of performance or to test your server/script/object behavior under heavy concurrent load.”

Visual Studio Ultimate - Load Test Analyser

http://msdn.microsoft.com/en-us/library/ms404677.aspx

If you’ve already paid for Visual Studio Ultimate (or been lucky enough to have been given a copy, thanks Liam!) then it’s probably worth checking this out. At first glance it looks pretty comprehensive and you get all the nice VS integration.

So thanks everyone for some great tips. If you’ve had a good (or bad) experience with load testing tools, please leave a comment. I still haven’t decided what to go with for load testing Suteki Shop. I’m leaning towards Selenium 2 + HtmlUnit, but I might give the VS LTA and JMeter a spin too. Watch this space.