Writing Performance Tests in Grinder using a Framework

This project is now available on GitHub.

Before reading this tutorial (if you haven't already), please took at look at my last two tutorials (Performance Testing using The Grinder and Anatomy of a Grinder test-script). In this tutorial I'll talk about easily writing Grinder test-scripts using a framework I designed. As a disclaimer, I'd like to point out that I'm not a Python programmer and therefore certain things may not be very python-esque. If that's the case, I apologize. My personal opinion is that this framework is especially useful (of course, since I wrote it ;)) for web applications where you have a already have a lot of test data. In that case, you can simply record all your discrete tasks once and then construct different scenarios with them. But if you feel differently and have some constructive criticism, I do look forward to hearing from you! Also, if you'd like to try out the framework I've got a tarball and a zip file available for download on the very last page.

The motivation for a framework

Typically during performance-testing, you want to test different kinds of scenarios. These scenarios are made up of discrete tasks (also known as transactions), for example, consider a scenario where a user logs in, searches for a person, and then logs out. In this scenario, you have three separate tasks: logging in, searching for a person, and logging out. Using Grinder's TCPProxy tool, you can easily record this scenario and you can also parameterize it. What happens when you want to record another scenario? Say, one that involves logging in, searching for a person, adding a new person, and logging out? Sure, you can go ahead and record that, and even parameterize it. But consider the fact that the new scenario is a super-set of the old scenario. It has three tasks in common with the first scenario. What if there was a way to create new scenarios from scratch, not by recording, but by using previously recorded tasks? This way you would only have to record information once, and after that, you can reuse it. To do this, we need to first understand how to identify tasks within a recorded Grinder test-script. Using this as our foundation, we can figure out how to pull out the tasks into discrete units, which we can later reuse.

The Relationship between Requests and Tasks

In the previous tutorial, I went over the anatomy of a recorded Grinder test-script. There, we saw that the script defines a bunch of requests, and then has methods that correspond to each recorded page. In each method, the script executes a bunch of requests. Initially you might think that we would simply pull out these individual methods. However, that is not the case. A single task can involve more than one recorded page. For example, the act of logging into the app involves accessing the login page, then logging in, and then hitting whatever page the login process drops you into. This involves (at the very least) three pages. In fact, the relationship between requests, pages, and tasks looks something like this:

Here, you can see that a set of unique requests belong to a recorded page, and then a set of recorded pages belong to a unique task. Finally, a set of tasks belong to a scenario. What we want to do is pull out the individual tasks so that we can reuse them to create different scenarios. To do this, we will still be using Grinder's TCPProxy tool (at least to record the scenario), but our resulting script will not look like the typical Grinder test-script. It will, instead, conform to the testing framework.