Typically during performance-testing, you want to test different kinds of scenarios. These scenarios are made up of discrete tasks (also known as transactions), for example, consider a scenario where a user logs in, searches for a person, and then logs out. In this scenario, you have three separate tasks: logging in, searching for a person, and logging out. Using Grinder’s TCPProxy tool, you can easily record this scenario and you can also parameterize it. What happens when you want to record another scenario? Say, one that involves logging in, searching for a person, adding a new person, and logging out? Sure, you can go ahead and record that, and even parameterize it. But consider the fact that the new scenario is a super-set of the old scenario. It has three tasks in common with the first scenario. What if there was a way to create new scenarios from scratch, not by recording, but by using previously recorded tasks? This way you would only have to record information once, and after that, you can reuse it. To do this, we need to first understand how to identify tasks within a recorded Grinder test-script. Using this as our foundation, we can figure out how to pull out the tasks into discrete units, which we can later reuse.

The Relationship between Requests and Tasks

In the previous tutorial, I went over the anatomy of a recorded Grinder test-script. There, we saw that the script defines a bunch of requests, and then has methods that correspond to each recorded page. In each method, the script executes a bunch of requests. Initially you might think that we would simply pull out these individual methods. However, that is not the case. A single task can involve more than one recorded page. For example, the act of logging into the app involves accessing the login page, then logging in, and then hitting whatever page the login process drops you into. This involves (at the very least) three pages. In fact, the relationship between requests, pages, and tasks looks something like this:

The relation between requests and tasks

Here, you can see that a set of unique requests belong to a recorded page, and then a set of recorded pages belong to a unique task. Finally, a set of tasks belong to a scenario. What we want to do is pull out the individual tasks so that we can reuse them to create different scenarios. To do this, we will still be using Grinder’s TCPProxy tool (at least to record the scenario), but our resulting script will not look like the typical Grinder test-script. It will, instead, conform to the testing framework.

High-level structure of a Grinder script

The high-level structure of a recorded Grinder-test-script looks like this:

[
...
import statements
...
]

[
...
header definitions
...
]

[
...
url definitions
...
]

[
...
request and test definitions
...
]

class TestRunner:
     [
     ...
     method definitions - a method is defined for each recorded page
     ...
     ]

     def __call__(self):
     [
     ...
     calls to defined methods, which actually runs the requests
     ...
     ]

     [ utility function (I'll go over this later) ]

     [
     ...
     calls to utility function to wrap/instrument tests (I'll go over this later)
     ...
     ]

Let’s go over each of these sections one by one:

Import statements

If you know Java, then this section is pretty much self-explanatory. Here you import the stuff you need, to make the tests work. By default, the following libraries are imported:

from net.grinder.script import Test
from net.grinder.script.Grinder import grinder
from net.grinder.plugin.http import HTTPPluginControl, HTTPRequest
from HTTPClient import NVPair

The first two import statements import the Test and grinder objects, which give you access to Grinder’s testing framework, and the grinder environment. The next two import statements import some utility classes which perform HTTP Requests. The NVPair class lets you organize parameters and values for POST requests into name-value pairs.

Header definitions

When the recorder runs, it records every request that the browser makes. After that, it goes through all the requests and generates a set of unique headers. These headers are then used to create the request objects (which we’ll look at later). The generated code looks like this:

connectionDefaults = HTTPPluginControl.getConnectionDefaults()
httpUtilities = HTTPPluginControl.getHTTPUtilities()

# To use a proxy server, uncomment the next line and set the host and port.
# connectionDefaults.setProxyServer("localhost", 8001)

# These definitions at the top level of the file are evaluated once,
# when the worker process is started.

connectionDefaults.defaultHeaders = \
  ( NVPair('Accept-Language', 'en-us,en;q=0.5'),
    NVPair('Accept-Charset', 'ISO-8859-1,utf-8;q=0.7,*;q=0.7'),
    NVPair('Accept-Encoding', 'gzip,deflate'),
    NVPair('User-Agent', 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.11) Gecko/2009060308 Ubuntu/9.04 (jaunty) Firefox/3.0.11'), )

headers0= \
  ( NVPair('Accept', 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'),
    NVPair('Referer', 'https://local.infusiontest.com:8443/?msg=You%27ve+been+logged+out+-+thanks+for+playing%21&notification=You%27ve+been+logged+out+-+thanks+for+playing%21'), )

headers1= \
  ( NVPair('Accept', 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'),
    NVPair('Referer', 'https://local.infusiontest.com:8443/Admin/home.jsp'), )

The first statement creates an HTTPPluginConnection object, which lets you control the behavior of the connection. The next statement creates an instance of an HTTPUtilities object which provides access to various utility methods (you’ll see examples of this later). The next two commented lines let you know that you can proxy these requests through a proxy server if you wish. The two comments after that let you know that these definitions are evaluated once for each process. Essentially, all data defined at the top of a script, outside the class, are shared between threads belonging to a single worker process.

In the next few lines, the script actually defines the headers. First, it sets up the default headers that are used by each request (things like the User-Agent, Charset, and Encoding). After that, it goes through and defines the headers. As you can see, these headers are basically tuples of NVPair objects. One NVPair defines the Accept parameter, and the other defines the Referer. Those with an eye for detail will notice that there seems to be an extra comma at the end of the tuple definition. This isn’t a problem in Python because the parser will treat the last element as an empty element.

URL definitions

In the next section of the script, you’ll find URL definitions, which look like this:

url0 = 'https://local.infusiontest.com:8443'

In this example, there is only one URL. But let’s say that the app you’re testing hits a few different URL’s during the scenario that you’re recording. Then the script will have a definition for each unique URL that the recorder encountered. The definitions are of the form urlN = “protocol://recorded.url.here”. The combination of a URL and Header is what defines a request (more on that in the next section).

Request and test definitions

In the next section of the script, we define the actual requests that we’re going to use for our tests:

# Create an HTTPRequest for each request, then replace the
# reference to the HTTPRequest with an instrumented version.
# You can access the unadorned instance using request101.__target__.

request101 = HTTPRequest(url=url0, headers=headers0)
request101 = Test(101, 'POST processLogin.jsp').wrap(request101)

request102 = HTTPRequest(url=url0, headers=headers0)
request102 = Test(102, 'GET home.jsp').wrap(request102)

request201 = HTTPRequest(url=url0, headers=headers1)
request201 = Test(201, 'GET popUpTask.jsp').wrap(request201)

request301 = HTTPRequest(url=url0, headers=headers2)
request301 = Test(301, 'POST calendarBackend.jsp').wrap(request301)

The comments give a terse explanation of what’s happening, but allow me to elaborate. First, you create an instance of an HTTPRequest object using the previously defined URL’s and Headers. An important thing to note here is that there is a naming scheme for the requests, which is of the form request[Page Number][Request Number]. So, for example, request105 means the fifth request in the first recorded page, and request1101 means the first request in the eleventh recorded page (yes, there is an inherent ambiguity, because 1101 could also mean the one hundred and first request in the first recorded page. But if you have a hundred and one unique requests per page, then you have a problem).

The next statement is the most important one. The way Grinder records test statistics is by proxying whatever you want to test, through the Test object. What this means is that the Test object is a wrapper around the object you want to test. You can still treat the wrapped object like the original object, but under the hood, access to the methods of that object are proxied through the Test object. This way, Grinder can record statistics. The picture might help understand what’s going on:

On the top you can see how a regular request behaves when accessing the GET method. It’s a straight call. The bottom picture shows you what happens when you wrap your request object with a Test object. Now, the call to GET isn’t made on the actual request object. It’s proxied through the Test object, which has an attribute called __target__ (which is the actual request object). Once it gets to the actual request object, the actual GET method is called. The reason Grinder does this is so that it can collect statistics on actions performed by an object.

Method definitions

After the request objects have been defined, we come to the actual TestRunner class. This is where you define methods that correspond to each recorded page:

# A method for each recorded page.
  def page1(self):
    """GET / (requests 101-104)."""
    result = request101.GET('/')

    grinder.sleep(307)

    # Expecting 302 'Moved Temporarily'
    request102.GET('/slices/infusion-crm.gif')

    grinder.sleep(29)

    # Expecting 302 'Moved Temporarily'
    request103.GET('/login/defaultLogin.jsp')
    self.token_msg = \
      httpUtilities.valueFromLocationURI('msg') # 'Whoa,+easy+there+tiger.+You\'re+gonna+nee...'

    grinder.sleep(24)
    request104.GET('/index.jsp' +
      '?msg=' +
      self.token_msg)

    return result

  def page2(self):
    """POST processLogin.jsp (requests 201-202)."""

    # Expecting 302 'Moved Temporarily'
    result = request201.POST('/login/processLogin.jsp',
      ( NVPair('username', 'vivin'),
        NVPair('password', 'abAB12!@'),
        NVPair('Login', 'Login'), ),
      ( NVPair('Content-Type', 'application/x-www-form-urlencoded'), ))

    grinder.sleep(72)
    request202.GET('/Admin/home.jsp')

    return result

In the script, you have a method for each recorded page, which are called page1, page2, and so forth. Within each page, there are one or more GET and/or POST requests made via the request objects, which simulate the transactions made by a single page. Let’s take a look at the page1 method. In this method you can see that it makes three requests. The first two requests are simple requests without any parameters. The last request, however, is a GET request with a querystring. The script uses the httpUtilities variable (defined earlier, at the beginning of the script) to get the value of the msg parameter from the URI, and constructs the querystring for the request.

The page2 method provides an example of a POST request. The values to the POST are supplied in name-value pairs using the NVPair object. Also notice the calls to grinder.sleep(). These calls simulate “think time”.

In both methods, you can see that the return value from the first request is assigned to a variable called result. The method then returns this variable (which contains statistics from Grinder).

Utility method, and calls to the utility method

I’m going to go out of order here and talk about the instrumentMethod utility method before I talk about the __call__ method. The instrumentMethod and the calls to that method look like this:

def instrumentMethod(test, method_name, c=TestRunner):
  """Instrument a method with the given Test."""
  unadorned = getattr(c, method_name)
  import new
  method = new.instancemethod(test.wrap(unadorned), None, c)
  setattr(c, method_name, method)

# Replace each method with an instrumented version.
# You can call the unadorned method using self.page1.__target__().
instrumentMethod(Test(100, 'Page 1'), 'page1')
instrumentMethod(Test(200, 'Page 2'), 'page2')
instrumentMethod(Test(300, 'Page 3'), 'page3')
instrumentMethod(Test(400, 'Page 4'), 'page4')
instrumentMethod(Test(500, 'Page 5'), 'page5')
instrumentMethod(Test(600, 'Page 6'), 'page6')
instrumentMethod(Test(700, 'Page 7'), 'page7')

The instrumentMethod method is another way to proxy calls through the Test object, except, instead of proxying an object, it proxies a method. To accomplish this, instrumentMethod performs some metaprogramming. Metaprogramming lets you modify a class’s properties and methods during runtime. You can add, remove, and modify existing methods and properties. The instrumentMethod method has three arguments. The first argument is a test object, the second is the method name (as a string), and the third argument is a default argument that is set to TestRunner. First, instrumentMethod gets a reference to a TestRunner class method identified by method_name (assigned to the variable unadorned). Then it creates a new method (using new.instancemethod) which has the test object wrapped around the original method. instrumentMethod then assigns this new method back to the TestRunner class, effectively proxying all calls to the original method through the Test object.

__call__ method

In Python any method that implements the __call__ is _callable_, which basically means that this function is called when you call the class instance as a function. In the context of the this script, it just means that this is where all the testing starts off. In the __call__ method you can see that there are calls to all the recorded page methods (along with think time), and this essentially runs the recorded test.

Conclusion

Now you should have a general idea of what a recorded Grinder test-script looks like. In my next tutorial, I’ll go over writing performance tests in Grinder using a testing framework.

Introduction

In this article I’ll go over installing and setting up Grinder. I am not going to go into too many details since my aim is to provide information that will enable you to have Grinder up and running quickly. If you want more information, you can look at the rather thorough Grinder User Guide. At the end of this guide, you’ll know how to install and set up grinder, record a test, modify (and/or parameterize) it, start running it through the console, and record data from the test. Note: The instructions in this guide relate to a Linux environment. You can run Grinder in Windows; the set-up is not much different. The only differences will be in installation locations and shell scripts. If you want more information about setting up Grinder in a Windows environment, please take a look here.

Downloading everything you need

First you need to download Grinder. You can get it from here. You should only need to download the binary distribution and not the source distribution (unless you’re planning on extending Grinder). Unzip the installation file to somewhere convenient. I extracted the file to /usr/local/grinder. I recommend chown‘ing the installation directory to yourusername:yourusername. Make sure that your version of the JDK is greater than or equal to 1.4, otherwise Grinder won’t work! The second thing you need to install is Jython. Note that this is not absolutely necessary, but I found it useful to have around when I was trying to parameterize the test-scripts, and I needed to test something out first. I recommend not using the Jython that comes with Ubuntu. Instead, download and install Jython 2.2.1 from here (there are issues with Jython 2.5 and Grinder).

Setting up Grinder

For demonstration purposes, I’m going to show you how to set up Grinder with a local grinder.properties file, with a local console and a local agent running workers. To get an explanation as to what these terms mean, take a look at the picture below that I shamelessly stole from the Grinder documentation page:

As you can see, you essentially have a console that sits on top of everything. It acts like a controller, pushing out scenarios and test cases (scripts) to agents that are running on load-injection machines. Each agent is in charge of one or more workers that inject load into the system you are testing. The workers also report data back to the console. The console aggregates all the data it receives from the workers and presents it to you. This is the ideal scenario that you would use for actual performance testing, however like I said before, I’m not going to concentrate on that – I’m just going to show you how to quickly set up Grinder and run a performance test on your local machine. If you want to look into distributing scenarios and tests across different agents, you can take a look here.

The first thing you want to do is set up a grinder.properties file. I put this file into the root directory of the checked out app (so basically, wherever you checked out your app). It’s not necessary to put it there; you can put it wherever you want. So open up your favorite text editor, and create the following file:

# Tell Grinder how many processes, threads and runs (iterations) we have in this scenario
grinder.processes = 1
grinder.threads = 5
grinder.runs = 10

# grinder.jvm.arguments sends arguments to the JVM. The reason I have the following value
# is to prevent an error (Jython complains about caches) from popping up when you start
# a test. Jython caches imports and so you need to let it know where it can store the imports.
# Though it's not strictly necessary, you do need it if you are performing wildcard imports
# (like java.util.*)
grinder.jvm.arguments = -Dpython.cachedir=/tmp/mycache
grinder.script = grinder.py

# Here you set up the arguments that tells the agent where the console lives.
grinder.hostID = dauntless
grinder.consoleHost = dauntless
grinder.consolePort = 6372
grinder.useConsole = true
grinder.reportToConsole.interval = 500

# sleepTime is a synonym for "think time". You don't want the load-generator hammering away at the
# app one request after another with a uniform interval. sleepTime introduces time variations between
# different requests
grinder.initialSleepTime = 0
grinder.sleepTimeFactor = 1
grinder.sleepTimeVariation = 0.2

# These properties are used to configure the behavior of Grinder with respect to logging
grinder.logProcessStreams = true
grinder.reportTimesToConsole = true
grinder.debug.singleProcess = false
grinder.useNanoTime = false

If you want more information regarding the grinder.properties file, take a look at this page.

Alright, now that you have the grinder.properties created, you also need a few shell-scripts to easily start and stop the console, agent, and TCP Proxy. You will have to change the values of GRINDERPATH, GRINDERPROPERTIES, and JAVA_HOME to values that make sense on your system.

#!/bin/bash
GRINDERPATH=/usr/local/grinder/grinder-3.2
GRINDERPROPERTIES=./grinder.properties
CLASSPATH=$GRINDERPATH/lib/grinder.jar:$CLASSPATH
JAVA_HOME=/usr/local/java/6u13
PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH PATH GRINDERPROPERTIES

#!/bin/bash
. ./setGrinderEnv.sh
java -cp $CLASSPATH net.grinder.TCPProxy -console -http > grinder.py

#!/bin/bash
. ./setGrinderEnv.sh
java -cp $CLASSPATH net.grinder.Console

#!/bin/bash
. ./setGrinderEnv.sh
java -cp $CLASSPATH net.grinder.Grinder $GRINDERPROPERTIES

Creating a Test

Before you start performance testing, you need to have a test. Grinder test scripts are written in the Jython language. You can write your tests from scratch if you wish (it can be useful if you’re trying to test things like XML-RPC), but more often than not you will be using Grinder’s proxy tool to record a usage scenario and generate a test script. The tool that Grinder comes with is called TCP Proxy. It sits between the browser and the server and records requests and responses that move between the browser and the server. By itself, it simply echoes whatever goes through it (in that sense, it makes a pretty good debugging tool), but we want it to generate Jython test scripts. Note: TCPProxy actually uses XSLT to translate the header information into a Jython script. This is pretty neat, actually because you can use XSLT to translate the code to Groovy, or even JRuby.

First, you need to start up TCP Proxy. Simply run startProxy.sh and you should be good to go. A little window will pop-up letting you know that the proxy is running. There’s a textbox in the window that lets you insert comments too. Once you the proxy is started up, you need to configure your browser to use the proxy at localhost:8001. Now access the app you want to test, in your browser. Log in and do whatever it is you need to do (search for a contact, add a contact, send and email, etc.). In the scenario I created, I logged in, searched for contacts, added a new contact, searched for more contacts, and then logged out. Note: If you get an alert box that complains about security certificates, you can add an exception in Firefox by going to Edit->Preferences->Advanced->Encryption Tab->View Certificates->Add Exception. Put in the URL to your server and Firefox should grab the certificate and add an exception.

Once you’re done recording the test, press the Stop button in TCPProxy.

WARNING: When you’re running TCPProxy, make sure that you have no other tabs open! Also make sure that you have no plugins trying to phone home (through the browser). TCPProxy intercepts every communication that your browser makes so you don’t want to pollute your results with irrelevant requests (I think it’s possible to filter the data using the domain name, but I haven’t looked into how to do it yet)

Parameterizing the test

There are a few tests where you would like to supply data (for example, when adding a contact). In the context of performance-testing, this is called parameterizing. Since Grinder uses Jython, it’s actually pretty easy to parameterize a test. First, let’s take a look at a TCPProxy-generated Jython-script. I’m only going to reproduce parts of the script since the entire script itself is rather large (however, if you want to look at the entire script, you can download it):

# The Grinder 3.2
# HTTP script recorded by TCPProxy at Jun 17, 2009 4:57:02 PM

from net.grinder.script import Test
from net.grinder.script.Grinder import grinder
from net.grinder.plugin.http import HTTPPluginControl, HTTPRequest
from HTTPClient import NVPair
connectionDefaults = HTTPPluginControl.getConnectionDefaults()
httpUtilities = HTTPPluginControl.getHTTPUtilities()

# To use a proxy server, uncomment the next line and set the host and port.
# connectionDefaults.setProxyServer("localhost", 8001)

# These definitions at the top level of the file are evaluated once,
# when the worker process is started.

connectionDefaults.defaultHeaders = \
  ( NVPair('Accept-Language', 'en-us,en;q=0.5'),
    NVPair('Accept-Charset', 'ISO-8859-1,utf-8;q=0.7,*;q=0.7'),
    NVPair('Accept-Encoding', 'gzip,deflate'),
    NVPair('User-Agent', 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.11) Gecko/2009060308 Ubuntu/9.04 (jaunty) Firefox/3.0.11'), )

headers0= \
  ( NVPair('Accept', 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'),
    NVPair('Referer', 'https://local.infusiontest.com:8443/?msg=You%27ve+been+logged+out+-+thanks+for+playing%21&notification=You%27ve+been+logged+out+-+thanks+for+playing%21'), )

headers1= \
  ( NVPair('Accept', 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'),
    NVPair('Referer', 'https://local.infusiontest.com:8443/Admin/home.jsp'), )

...
...

headers9= \
  ( NVPair('Accept', 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'),
    NVPair('Referer', 'https://local.infusiontest.com:8443/Admin/home.jsp?revertCalUser=true'),
    NVPair('Cache-Control', 'no-cache'), )

headers10= \
  ( NVPair('Accept', 'image/png,image/*;q=0.8,*/*;q=0.5'),
    NVPair('Referer', 'https://local.infusiontest.com:8443/?msg=You%27ve+been+logged+out+-+thanks+for+playing%21&notification=You%27ve+been+logged+out+-+thanks+for+playing%21'), )

url0 = 'https://local.infusiontest.com:8443'

# Create an HTTPRequest for each request, then replace the
# reference to the HTTPRequest with an instrumented version.
# You can access the unadorned instance using request101.__target__.
request101 = HTTPRequest(url=url0, headers=headers0)
request101 = Test(101, 'POST processLogin.jsp').wrap(request101)

request102 = HTTPRequest(url=url0, headers=headers0)
request102 = Test(102, 'GET home.jsp').wrap(request102)

request201 = HTTPRequest(url=url0, headers=headers1)
request201 = Test(201, 'GET popUpTask.jsp').wrap(request201)

...
...

request3004 = HTTPRequest(url=url0, headers=headers0)
request3004 = Test(3004, 'GET defaultLogin.jsp').wrap(request3004)

request3005 = HTTPRequest(url=url0, headers=headers0)
request3005 = Test(3005, 'GET index.jsp').wrap(request3005)

class TestRunner:
  """A TestRunner instance is created for each worker thread."""

  # A method for each recorded page.
  def page1(self):
    """POST processLogin.jsp (requests 101-102)."""

    # Expecting 302 'Moved Temporarily'
    result = request101.POST('/login/processLogin.jsp',
      ( NVPair('username', 'myusername'),
        NVPair('password', 'myp@55W0rd'),
        NVPair('Login', 'Login'), ),
      ( NVPair('Content-Type', 'application/x-www-form-urlencoded'), ))

    grinder.sleep(49)
    request102.GET('/Admin/home.jsp')

    return result

  def page2(self):
    """GET popUpTask.jsp (request 201)."""
    result = request201.GET('/files/popUpTask.jsp' +
      '?0.6697580414936783')

    return result

  def page3(self):
    """POST calendarBackend.jsp (request 301)."""
    result = request301.POST('/Calendar/calendarBackend.jsp',
      ( NVPair('calType', 'Day'),
        NVPair('userId', '1'),
        NVPair('calDate', '17'),
        NVPair('calMonth', '5'),
        NVPair('calYear', '2009'),
        NVPair('weekStartDate', '-1'),
        NVPair('weekStartMonth', '-1'),
        NVPair('weekStartYear', '-1'),
        NVPair('weekEndDate', '-1'),
        NVPair('weekEndMonth', '-1'),
        NVPair('weekEndYear', '-1'), ),
      ( NVPair('Content-Type', 'application/x-www-form-urlencoded; charset=UTF-8'), ))

    return result

  ...
  ...

  def page18(self):
    """POST processContact.jsp (requests 1801-1802)."""

    # Expecting 302 'Moved Temporarily'
    result = request1801.POST('/Contact/processContact.jsp',
      ( NVPair('addingFormId', '0'),
        NVPair('tabs_sel', 'main'),
        NVPair('Contact0Id', '0'),
        NVPair('Contact0FirstName', "Mikey"),
        NVPair('Contact0LastName', "Mike"),
        ...
        ...
        NVPair('Contact0Website', ''),
        NVPair('Contact0Email', "mikey.mike@mike.com"),
        NVPair('email_Contact0Email', '1'),
        ...
        ...
        NVPair('view', 'add'),
        NVPair('Save', 'Save'), ),
      ( NVPair('Content-Type', 'application/x-www-form-urlencoded'), ))
    self.token_view = \
      httpUtilities.valueFromLocationURI('view') # 'edit'
    self.token_tabs_sel = \
      httpUtilities.valueFromLocationURI('tabs_sel') # 'main'
    self.token_msg = \
      httpUtilities.valueFromLocationURI('msg') # 'Person Added'

    grinder.sleep(46)
    request1802.GET('/Contact/manageContact.jsp' +
      '?view=' +
      self.token_view +
      '&tabs_sel=' +
      self.token_tabs_sel +
      '&msg=' +
      self.token_msg)

    return result

  ...
  ...

  def __call__(self):
    """This method is called for every run performed by the worker thread."""
    self.page1()      # POST processLogin.jsp (requests 101-102)

    grinder.sleep(350)
    self.page2()      # GET popUpTask.jsp (request 201)

    grinder.sleep(385)
    self.page3()      # POST calendarBackend.jsp (request 301)

  ...
  ...

    grinder.sleep(407)
    self.page29()     # POST calendarBackend.jsp (request 2901)

    grinder.sleep(3661)
    self.page30()     # GET logout.jsp (requests 3001-3005)

def instrumentMethod(test, method_name, c=TestRunner):
  """Instrument a method with the given Test."""
  unadorned = getattr(c, method_name)
  import new
  method = new.instancemethod(test.wrap(unadorned), None, c)
  setattr(c, method_name, method)

# Replace each method with an instrumented version.
# You can call the unadorned method using self.page1.__target__().
instrumentMethod(Test(100, 'Page 1'), 'page1')
instrumentMethod(Test(200, 'Page 2'), 'page2')
instrumentMethod(Test(300, 'Page 3'), 'page3')
...
...
instrumentMethod(Test(2900, 'Page 29'), 'page29')
instrumentMethod(Test(3000, 'Page 30'), 'page30')

From the above listing, you can get a general idea of the anatomy of a test script. Of course, there’s a little bit more that goes into this, so if you want a detailed description, take a look at this page (in my next article, I’ll go into more detail about the anatomy of a recorded script). By itself, this script is 95% useful. However, to make it exactly what we need, we need to parameterize certain areas. The first section we need to parameterize is where we add a new contact (look at the page18 method). Right now, it inserts a contact called “Mikey Mike” with the email “mikey.mike@mike.com”. That’s not particularly useful if we’re running this test a bunch of times. We want to parameterize this value. What we’re going to do is create a random first name, last name, and email address, and use those values in the request. Here’s how we do it (once again, I’m only showing the pertinent sections of the code i.e., the sections that have changed):

# The Grinder 3.2
# HTTP script recorded by TCPProxy at Jun 17, 2009 4:57:02 PM

# We need to import java.util.Random for random number generation
from java.util import Random

...
...

  def page18(self):
    """POST processContact.jsp (requests 1801-1802)."""

    chars = [ "b", "c", "d", "f", "g", "h", "j", "l", "m", "n", "p", "q", "r", "s", "t", "v", "w", "x", "y", "z", "th", "dh", "gh", "ph", "ch", "st", "sw", "sp", "kh" ];
    vowels = [ "a", "e", "i", "o", "u" ];
    r = Random();

    fname = "";
    lname = "";

    for i in range(0, r.nextInt(3) + 2):
       fname += chars[r.nextInt(len(chars))] + vowels[r.nextInt(len(vowels))];

    for i in range(0, r.nextInt(3) + 2):
       lname += chars[r.nextInt(len(chars))] + vowels[r.nextInt(len(vowels))];

    # Expecting 302 'Moved Temporarily'
    result = request1801.POST('/Contact/processContact.jsp',
      ( NVPair('addingFormId', '0'),
        NVPair('tabs_sel', 'main'),
        NVPair('Contact0Id', '0'),
        NVPair('Contact0FirstName', fname.capitalize()),
        NVPair('Contact0LastName', lname.capitalize()),
        ...
        ...
        NVPair('Contact0Website', ''),
        NVPair('Contact0Email', fname + "." + lname + "@" + lname + ".com"),
        NVPair('email_Contact0Email', '1'),
        ...
        ...
        NVPair('Save', 'Save'), ),
      ( NVPair('Content-Type', 'application/x-www-form-urlencoded'), ))
    self.token_view = \
      httpUtilities.valueFromLocationURI('view') # 'edit'
    self.token_tabs_sel = \
      httpUtilities.valueFromLocationURI('tabs_sel') # 'main'
    self.token_msg = \
      httpUtilities.valueFromLocationURI('msg') # 'Person Added'

    grinder.sleep(46)
    request1802.GET('/Contact/manageContact.jsp' +
      '?view=' +
      self.token_view +
      '&tabs_sel=' +
      self.token_tabs_sel +
      '&msg=' +
      self.token_msg)

    return result

As you can see, we’ve now parameterized the first name and last name values that go into processContact.jsp. There is still one more thing we need to parameterize. If you go back and take a look at grinder.properties, you can see that I set grinder.threads to 5. This means that there are going to be 5 threads running. We don’t want the same user logging in more than once (our app doesn’t allow it). So what we also need to do is parameterize user logins:

# The Grinder 3.2
# HTTP script recorded by TCPProxy at Jun 17, 2009 4:57:02 PM

from java.util import Random
from net.grinder.script import Test
from net.grinder.script.Grinder import grinder
from net.grinder.plugin.http import HTTPPluginControl, HTTPRequest
from HTTPClient import NVPair
connectionDefaults = HTTPPluginControl.getConnectionDefaults()
httpUtilities = HTTPPluginControl.getHTTPUtilities()

# To use a proxy server, uncomment the next line and set the host and port.
# connectionDefaults.setProxyServer("localhost", 8001)

# These definitions at the top level of the file are evaluated once,
# when the worker process is started.

# Setting up an array of dictionaries (i.e., hash/map/associative-array) of username and password name-value pairs
loginCredentials = [{"username":"vivin", "password":"abAB12!@"},
                    {"username":"jimbo", "password":"abAB12!@"},
                    {"username":"hippy", "password":"abAB12!@"},
                    {"username":"flippy", "password":"abAB12!@"},
                    {"username":"batman", "password":"abAB12!@"}];

...
...

class TestRunner:
  """A TestRunner instance is created for each worker thread."""

  # A method for each recorded page.
  def page1(self):
    """POST processLogin.jsp (requests 101-102)."""

    loginCredential = loginCredentials[grinder.threadNumber];
    username = loginCredential["username"];
    password = loginCredential["password"];
    # Expecting 302 'Moved Temporarily'
    result = request101.POST('/login/processLogin.jsp',
      ( NVPair('username', username),
        NVPair('password', password),
        NVPair('Login', 'Login'), ),
      ( NVPair('Content-Type', 'application/x-www-form-urlencoded'), ))

    grinder.sleep(49)
    request102.GET('/Admin/home.jsp')

    return result

To parameterize logins, I created an array of dictionaries that store username and password combinations. Then, based on the thread number (which is zero-based and which I can access via grinder.threadNumber) I select the appropriate username and password combination, to log in.

Running the tests

Now we can actually start running the tests and recording data! The first thing you need to do is start up your app. Then, you need to start the Grinder console. Use the startConsole.sh shell-script and start up the Grinder console. After that, use startAgent.sh to start up the agent. You should see something like this in your shell:

vivin@dauntless ~/Projects/crm-1.16.0.x
$ startConsole.sh &
[1] 16180

vivin@dauntless ~/Projects/crm-1.16.0.x
$ startAgent.sh
6/18/09 4:43:19 PM (agent): The Grinder 3.2
6/18/09 4:43:19 PM (agent): connected to console at dauntless/127.0.1.1:6372
6/18/09 4:43:19 PM (agent): waiting for console signal

If you see this, it means that your agent started up, and was able to talk to the Grinder console. In the Grinder console, there are four icons in the top-left corner of the window. If you mouse-over the leftmost icon, you will see a tooltip that says “Start the worker processes”. Click that button. A dialog box will pop up saying that you haven’t selected a properties file and that the worker processes will run the script set in the agent’s properties file. This is fine, since you are running Grinder locally and not distributing a file. Click Ok. You should now see something like this in your shell:

vivin@dauntless ~/Projects/crm-1.16.0.x
$ startConsole.sh &
[1] 16180

vivin@dauntless ~/Projects/crm-1.16.0.x
$ startAgent.sh
6/18/09 4:50:12 PM (agent): The Grinder 3.2
6/18/09 4:50:12 PM (agent): connected to console at dauntless/127.0.1.1:6372
6/18/09 4:50:12 PM (agent): waiting for console signal
6/18/09 4:51:48 PM (agent): received a start message
6/18/09 4:51:48 PM (agent): Worker process command line: java '-Dpython.cachedir=/tmp/mycache' -classpath '/usr/local/grinder/grinder-3.2/lib/grinder.jar:' net.grinder.engine.process.WorkerProcessEntryPoint
6/18/09 4:51:48 PM (agent): worker dauntless-0 started
*sys-package-mgr*: processing new jar, '/usr/local/grinder/grinder-3.2/lib/grinder.jar'
*sys-package-mgr*: processing new jar, '/usr/local/java/6u13/jre/lib/resources.jar'
*sys-package-mgr*: processing new jar, '/usr/local/java/6u13/jre/lib/rt.jar'
*sys-package-mgr*: processing new jar, '/usr/local/java/6u13/jre/lib/jsse.jar'
*sys-package-mgr*: processing new jar, '/usr/local/java/6u13/jre/lib/jce.jar'
*sys-package-mgr*: processing new jar, '/usr/local/java/6u13/jre/lib/charsets.jar'
*sys-package-mgr*: processing new jar, '/usr/local/java/6u13/jre/lib/ext/sunpkcs11.jar'
*sys-package-mgr*: processing new jar, '/usr/local/java/6u13/jre/lib/ext/sunjce_provider.jar'
*sys-package-mgr*: processing new jar, '/usr/local/java/6u13/jre/lib/ext/localedata.jar'
*sys-package-mgr*: processing new jar, '/usr/local/java/6u13/jre/lib/ext/dnsns.jar'
*sys-package-mgr*: processing new jar, '/usr/local/grinder/grinder-3.2/lib/jython.jar'
6/18/09 4:51:53 PM (process dauntless-0): starting threads

If you check back into the Grinder console, you can see that it is grabbing data from the agent. Now just sit back and let the test run. Once the test is complete, hit the stop button and exit out of console. If you go to the directory from where you started up the agent, you will see three log files: data_<hostname>-N.log, error_<hostname>-N.log, and out_<hostname>-N.log. data_<hostname>-N.log is the file that contains all the data gathered (like mean response time, transactions per second, etc.). It is a comma-separated-values file and you can open it up in Excel or Openoffice Calc. error_<hostname>-N.log contains any errors encountered while running the test, and out_<hostname>-N.log contains logging data from the script file. You are now in a position to analyze data from the log files. WARNING: I recommend NOT using the version of Java that comes with Ubuntu (GCJ/IcedTea or whatever). Instead, use Sun’s Java6. I ran into a whole bunch of problems and wasted an entire day (you can view the thread describing the problem here).

Analyzing Grinder-Generated Data

Grinder spits out raw data. So it’s not that user-friendly when compared to, say, WebLoad. This makes sense when you consider the fact that Grinder is geared towards developers. Grinder says “Here is the data I collected, do whatever you want with it”. If you want a quick overview of your performance data, there is a tool called GrinderAnalyzer that you can download from here. Usage is pretty straightfoward (check out the documentation). Extract it to your user home directory. Then run the shell script (run.sh) in the bin directory while providing it your data_<hostname>-N.log and out_<hostname>-N.log files. GrinderAnalyzer will generate some pretty graphs and some pretty tables for you to view. Your other option is to examine the data on your own and generate graphs. This, while a bit more involved, gives you the freedom to display the data in a manner you see fit.

Conclusion

Grinder is a pretty flexible and powerful performance-testing tool. I’m hoping that after this tutorial you have enough knowledge to go out and start creating and running tests of your own. Like I mentioned before, this guide is by no meand comprehensive. It’s simply something to get you started. If you want more information, I highly recommend looking at the excellent Grinder Documentation.

Stay tuned for the next article, where I’ll be talking about the anatomy of a recorded Grinder test-script…