Writing Performance Tests in Grinder using a Framework
Before reading this tutorial (if you haven’t already), please took at look at my last two tutorials (Performance Testing using The Grinder and Anatomy of a Grinder test-script). In this tutorial I’ll talk about easily writing Grinder test-scripts using a framework I designed. As a disclaimer, I’d like to point out that I’m not a Python programmer and therefore certain things may not be very python-esque. If that’s the case, I apologize. My personal opinion is that this framework is especially useful (of course, since I wrote it 
) for web applications where you have a already have a lot of test data. In that case, you can simply record all your discrete tasks once and then construct different scenarios with them. But if you feel differently and have some constructive criticism, I do look forward to hearing from you! Also, if you’d like to try out the framework I’ve got a tarball and a zip file available for download on the very last page.
The motivation for a framework
Typically during performance-testing, you want to test different kinds of scenarios. These scenarios are made up of discrete tasks (also known as transactions), for example, consider a scenario where a user logs in, searches for a person, and then logs out. In this scenario, you have three separate tasks: logging in, searching for a person, and logging out. Using Grinder’s TCPProxy tool, you can easily record this scenario and you can also parameterize it. What happens when you want to record another scenario? Say, one that involves logging in, searching for a person, adding a new person, and logging out? Sure, you can go ahead and record that, and even parameterize it. But consider the fact that the new scenario is a super-set of the old scenario. It has three tasks in common with the first scenario. What if there was a way to create new scenarios from scratch, not by recording, but by using previously recorded tasks? This way you would only have to record information once, and after that, you can reuse it. To do this, we need to first understand how to identify tasks within a recorded Grinder test-script. Using this as our foundation, we can figure out how to pull out the tasks into discrete units, which we can later reuse.
The Relationship between Requests and Tasks
In the previous tutorial, I went over the anatomy of a recorded Grinder test-script. There, we saw that the script defines a bunch of requests, and then has methods that correspond to each recorded page. In each method, the script executes a bunch of requests. Initially you might think that we would simply pull out these individual methods. However, that is not the case. A single task can involve more than one recorded page. For example, the act of logging into the app involves accessing the login page, then logging in, and then hitting whatever page the login process drops you into. This involves (at the very least) three pages. In fact, the relationship between requests, pages, and tasks looks something like this:
The relation between requests and tasks
Here, you can see that a set of unique requests belong to a recorded page, and then a set of recorded pages belong to a unique task. Finally, a set of tasks belong to a scenario. What we want to do is pull out the individual tasks so that we can reuse them to create different scenarios. To do this, we will still be using Grinder’s TCPProxy tool (at least to record the scenario), but our resulting script will not look like the typical Grinder test-script. It will, instead, conform to the testing framework.
Dear vivin
Happy onam to you. we are a group of students from cochin who are currently building a web portal on kerala. in which we wish to include a kerala blog roll with links to blogs maintained by malayali’s or blogs on kerala.
you could find our site here: http://enchantingkerala.org
the site is currently being constructed and will be finished by 1st of sep 2009.
we wish to include your blog located here
http://vivin.net/
we’ll also have a feed fetcher which updates the recently updated blogs from among the listed blogs thus generating traffic to your recently posted entries.
If you are interested in listing your site in our blog roll; kindly include a link to our site in your blog in the prescribed format and send us a reply to enchantingkerala.org@gmail.com and we’ll add your blog immediatly.
pls use the following format to link to us
Kerala
hoping to hear from you soon.
warm regards
Biby Cletus
Hi,
I was evaluating Grinder tool..
If the Tester Runner method is big . i mean if Testrunner class exceeds certain length then i am unable to run the test and the tool is throwing following exception…
As a workaround we can divide the script into more than once script but the problem here is how can i call one script in other…?? could you help me..
Thanks In Advance
Kamalakar
@kamalakar
That’s strange – I’ve never run into that issue before. What is the exception you’re getting? You might consider putting each one into a class of its own and since they’re callable, you might be able to call them split them into separate classes and then write a wrapper that calls them? I think you’ll probably have better luck on the grinder-use mailing list – have you asked your question on there?
I have read some of your posts and find your style of writing very interersting. Salute you on how neatly the articles are presented.
@Avinash Mangipudi
Thanks Avinash!
I get following error for all import statements of my framework scripts (ImportError: no module named sci).
This goes away if I compile all my dependent scripts and add jar to classpath of agent. But then I cant parameterize methods in those dependent scripts as they are already classes.
How do I solve this?
I tried settting PYTHONPATH, passing python.home in grinder.properties. nothing seems to work for me.
I found out the issue. I had to put my script sources folder in Jython registry file.@Mukes Nair
@Mukesh Nair
Sorry I didn’t get back to you sooner. I didn’t check my email until now! I’m glad that you were able to fix the issue. I wasn’t clear on your initial problem though. Did you have problems importing the actual scripts (that you created using the framework) themselves?
Hi ,
I am trying to use torque framework , facing problem with the scenario class.
As scenario class showing error at following code.
” urlDict = property(getUrlDict, setUrlDict)” because my eclipse is unable to find property import.
i have jython 2.1 with me. it seems the property import is not available on the jython 2.1
Could you please help me wat might be the problem..?
Thanks
Kamalakar
I need a a way to share some data between tasks..
How can i do this?
Do we need to add other prop in the scenario class
@kamalakar
Hi
i am able to fix this error.. the problem is with my environment
Actually i am using GrinderStone its eclipse plugin for Grinder Tool.
Thanks
Kamalakar
@kamalakar
Sorry I wasn’t able to get to you sooner! Glad you were able to fix the issue – it did sound like an environment problem.
What data are you trying to share between tasks?Tasks are discrete entities that should have nothing in common with each other. Hence, they shouldn’t be sharing state-data. However, they can share input data (from a common resource – like a file).
suppose consider following usecase:
1.login is one task: after user got logged in we will get a userid .
2.second task :
— Here the request in this task need the userid from the previous task.
So, we need a global parameters that can be accessible throughout the scenario.
Could you suggest me how we can achieve . I have one approach in my mind.
— Add another param like ‘urlDict’ in the Scenario Class . what do you say.?
Thanks
Kamalakar
@kamalakar
In this case I’d parameterize the usernames and userId’s and link them to the thread number. You’d have to do this even if you were testing a pure Grinder-script that didn’t use the framework.
I have another issue to resolve. Any help will be appreciated. Lets take a scenario of a billing counter like of BestBuy. The associate will log in and load the billing screen. Then he will enter items and print receipt for a customer. Then for the next customer and so on. Here I have classified tasks in the scenario as follows
1. Login Task
2. Load Billing Screen Task
3. Enter Items Task
4.Print receipt Task
5. Logout task
Now in the run I want to run tasks 1, 2 and 5 only once. I want to repeat 3 and 4.
How do I do this?
Thanks
Mukesh
@Mukesh Nair
I’m assuming that you know how many times you need to repeat tasks 3 and 4? You can simply create n “Enter Items” tasks and n “Print receipt” tasks (through a loop perhaps, and maybe have an array for each, for example enterItemsTaskArray and printReceiptTaskArray).
When you build your scenario, you add Task 1 and 5, then iterate through both arrays at the same time and them to the scenario (so you’d add one “Enter items” task and then one “Print receipt” task). After you’re done you add Task 5.
Actually I don’t know how many times I will be repeating 3 and 4. I found another way to tackle the issue. Like this.
class TestRunner:
def __init__(self):
log(“initializing scenario %s” % self.__class__.__name__)
myScenario.initialize()
def __call__(self):
log(“running scenario %s” % self.__class__.__name__)
myScenario.run()
def __del__(self):
log(“finalizing scenario %s” % self.__class__.__name__)
myScenario.finalize()
@Mukesh Nair
so my scenario will look like:
myScenario.addInitTask(loginTask)
myScenario.addInitTask(loadPackScreenTask)
myScenario.addTask(scanShipmentTask)
myScenario.addTask(newCaseTask)
myScenario.addTask(scanItemTask)
myScenario.addTask(closeContainerTask)
myScenario.addFinalTask(logoutTask)
I think this should solve my current issue.
@Mukesh Nair
That would work as well. I’m assuming that you’re using meta-programming and extending the Scenario class at runtime?
Even if you didn’t know how many times you’d be repeating it, as in let’s say that it will be repeated a random number of times, you can add a random number of items to the arrays. But I do like your solution. It’s a little more elegant.
I have a problem of data getting mixed up between threads. Look at the log below.
initially thread-1 had value 100000019. Once thread -0 got the value 100000024, i printed out values again. Now both threads have the same value.
I decompiled one of the class files and I found that self is declared as:
static final py self;
Did you face similar issue?
11/20/09 10:56:09 AM (thread 1 run 0 test 101): self.parameters["SelectStationAndScanShipment2"]["201"]["BarcodeData"] 100000019
11/20/09 10:56:09 AM (thread 0 run 0 test 101): suggestedShipments ['100000019', '100000024']
11/20/09 10:56:09 AM (thread 0 run 0 test 101): self.parameters["SelectStationAndScanShipment2"]["201"]["BarcodeData"] 100000024
11/20/09 10:56:09 AM (thread 1 run 0 test 4201): http://10.10.20.80:7099/smcfs/console/exuipack.exui -> 200 OK, 365 bytes
11/20/09 10:56:09 AM (thread 1 run 0 test 4201): sleeping for 29 ms
11/20/09 10:56:09 AM (thread 0 run 0 test 4201): http://10.10.20.80:7099/smcfs/console/exuipack.exui -> 200 OK, 919 bytes
11/20/09 10:56:09 AM (thread 0 run 0 test 4201): sleeping for 31 ms
11/20/09 10:56:09 AM (thread 1 run 0 test 4301): http://10.10.20.80:7099/smcfs/console/exuipack.exui -> 200 OK, 244 bytes
11/20/09 10:56:09 AM (thread 1 run 0 test 4301): sleeping for 34 ms
11/20/09 10:56:09 AM (thread 0 run 0 test 4301): http://10.10.20.80:7099/smcfs/console/exuipack.exui -> 200 OK, 244 bytes
11/20/09 10:56:09 AM (thread 0 run 0 test 4301): sleeping for 30 ms
11/20/09 10:56:09 AM (thread 1 run 0 test 4401): http://10.10.20.80:7099/smcfs/console/exuipack.exui -> 200 OK, 919 bytes
11/20/09 10:56:09 AM (thread 0 run 0 test 4401): http://10.10.20.80:7099/smcfs/console/exuipack.exui -> 200 OK, 919 bytes
11/20/09 10:56:09 AM (thread 1 run 0 test 4401): sleeping for 63 ms
11/20/09 10:56:09 AM (thread 0 run 0 test 4401): sleeping for 60 ms
11/20/09 10:56:09 AM (thread 1 run 0 test 4501): http://10.10.20.80:7099/smcfs/console/exuipack.exui -> 200 OK, 341 bytes
11/20/09 10:56:09 AM (thread 1 run 0 test 4600): self.parameters["SelectStationAndScanShipment2"]["201"]["BarcodeData"] 100000024
11/20/09 10:56:09 AM (thread 0 run 0 test 4501): http://10.10.20.80:7099/smcfs/console/exuipack.exui -> 200 OK, 341 bytes
11/20/09 10:56:09 AM (thread 0 run 0 test 4600): self.parameters["SelectStationAndScanShipment2"]["201"]["BarcodeData"] 100000024
@Mukesh Nair
How are you parameterizing the data? Are you doing it based on the thread number?
@vivin
am setting data in the task itself using one method call to a utility script, like below
def SelectStationAndScanShipment2(self):
“”"Selecting station and scanning shipment POST exuipack.exui (request 201).”"”
self.parameters["SelectStationAndScanShipment2"]["201"]["BarcodeData"] = self.StringUtil.getAPickedShipmentNumber(self.parameters["SelectStationAndScanShipment2"]["201"]["OrganizationCode"])
self.log(“self.parameters[\"SelectStationAndScanShipment2\"][\"201\"][\"BarcodeData\"] %s” % self.parameters["SelectStationAndScanShipment2"]["201"]["BarcodeData"])
@Mukesh Nair
I recommend not modifying the actual task. What you want to do is create a parameterizing method that sets the values in the parameters dictionary for you. Check out my examples in this post.
@vivin
i wanna show you a grinder xml which is not producing correct url in the task created by the perl script. I directly created grinder script using proxy. I can see a lot of arguements witht he url. But when it is converted using using the perl script into a task, all the arguements are missing. let me know where to send the related files so that you can take a look. I haven’t looked into perl yet. So I thought you could resolve it faster.
@Mukesh Nair
Is it a POST or a GET? I can take a look at it. I DM’ed you my email on twitter. So just send me the XML and I’ll take a look at it.
Hi Vivin,
Here i need your help. I am working in web application and i wanna to test the load of the application. For this i am using the grinder tool and under stand all the things except properties file of grinder framework. Is it possible to run more than one script or divide the scripts in threads on same time. Here i can explain you through example. Suppose we have 4 scripts and want 4 scripts are equally divided in 100 user’s. is it possible?
@vicky
I looked around for ways to run multiple scripts and I ran into this post here that says you can do it by making each script as a separate property and not space delimited. So what you need to do is divide the load (and select a user) based on the thread number. Script 1 will use user1-user25 (if the thread number is between 1 and 25), script 2 will use user26-user50 (if the thread number is between 26 and 50). This basically means that you need to have 100 threads.
The Perl file under the link on page 9 is not the same as the one you packaged in the ZIP.
@Marcel
Thanks for pointing that out Marcel; I’ll fix it!
@Mukesh Nair
Could you show the details about it. I have the same problem.