Introduction to JMeter : Load Testing using JMeter

JMeter is load testing tool used for testing load on site by multiple requests of users. JMeter version 2.4 does not support https requests. If you are using JMeter 2.4 version then you can use BadBoy recording tool for processing https requests. But Latest version of JMeter available in market is 2.8 which supports https requests.

Operating System:
 
JMeter is a 100% Java application and should run correctly on any system that has a compliant Java implementation.
JMeter has been tested and works under:
• Unix (Solaris, Linux, Fedora etc)
• Windows (98, NT, XP, Vista, 7 etc)
• OpenVMS Alpha 7.3+

Install JMeter:

1. JRE or SDK first needs to be installed with a JAVA_HOME environment variable.
2. Download Latest version of JMeter from http://jmeter.apache.org/download_jmeter.cgi
3. Extract this .zip or .tgz file in any directory.
4. Go to that path and open jmeter folder.
5. Type ./bin/jmeter on command prompt(for Unix) or Run bin/jmeter.bat (for Windows)
6. JMeter screen will open with 2 main components:
    - Test Plan
    - Workbench 

Recording in JMeter:

1. JMeter:
- Right click on Workbench component and go to Add >> Non-Test Elements >> HTTP Proxy Server.
- Right click on TestPlan component and go to Add >> Threads (Users) >> Thread Group.
- Right click on Thread Group component and go to Add >> Logic Controller >> Recording Controller.

2. Firefox settings:
- In JMeter, recording supports in Firefox browser. So, open Firefox and go to Tools >> Options >> Network >> Settings >> Manual proxy cofiguration.
- Add “localhost” or “127.0.0.1” in HTTP Proxy and "8080" in Port field. If added port is in use, we can add some other port number also.

3. JMeter:
- In HTTP Proxy server screen, add same port number in Port field which we have entered in Firefox browser.
- From Target Controller dropdown, select option “Thread Group > Recording Controller”
- In some cases, when we don't want to consider page load time for images, javascripts, etc. then we can exclude them. In “URL Patterns to Exclude” section, Click on ADD button and Add exclude formats in the way like this .*\.jpg
- Now from HTTP Proxy settings screen, click on Start button to start recording.

4. Firefox: Open URL which needs to be recorded.

5. JMeter: Check the recorded requests are coming under “Thread Group >> Recording Controller” component. Click on Stop button, when you want to stop the recording.

Add Listeners in JMeter to check results of the recorded requests:

1. Right click Thread Group and go to Add >> Listener >> Aggregate Report. We can also add Summary Report to check the details.
2. To verify whether requests are running correctly or not, we can verify it by adding a listener as “View Results Tree”. We can check error by clicking on each request in View Results Tree section. 

Use of Thread Group Properties to apply Load to the recorded requests:

    

1. Click on Thread group.
2. In Thread group screen, check the “Thread Properties” option.
3. We can add the number of users for load test in “Number of Thread (Users)” option.
4. To increase the load gradually, add total time to add full load in “Ramp-Up Period (in sec)” field. The Ramp-Up Period tells JMeter how long to take to "ramp-up" to the full number of threads chosen. If Number of Thread (Users) is 10 and the Ramp-Up Period is 100 seconds, then JMeter will take 100 seconds to get all 10 threads up and running. Each thread will start 10 (100/10) seconds after the previous thread was begun. If there are 30 threads and a ramp-up period of 120 seconds, then each successive thread will be delayed by 4 seconds.
5. To run the script multiple times, we can add value in Loop count field.

Example 1: 
Number of Threads (Users) = 100
Ramp up period(in seconds) = 600
Loop Count = 1
Every 6 second (600/100) one request hits the server. 

Example 2: 

Number of threads(users) = 100
Ramp up period(in seconds) = 600
Loop Count = 3
Every 6 second (600/100) 3 request hits the server.

Run the script:

From the Menu, go to Run >> Start and script will start executing.

Verify result in Reports:

As mentioned above, we can check results of the script in Aggregate Report, Summary Report, View Results Tree etc and also we can export the results to a file by using “Save Response to a File”. This file will get saved in /bin folder in 1.html format.

Thanks and Regards,

Prashant Vadher | QC Engineer

Read more »

Distributed Load Testing in JMeter

ApacheJMeter is open source desktop application tool designed to simulate a heavy load on a server, network or object to test its strength or to analyze overall performance under different load types. The reason of using more than one system for load testing is the limitation of single system to generate large number of requests, threads and users.

In JMeter this is achieved by creating a Master - Slave configuration. Distributed load testing is the process using which multiple systems are used for simulating load of large number of users. In a distributed setup there is one controller called master and number of slaves controlled by master.

Distributed load testing can be tricky and may produce incorrect results if not configured correctly. Test results depends on the following factors.

1. Network bandwidth. 
2. Master and Slave Systems configuration including memory, processor speed.
3. Configuration of the load testing tool like number of users running in parallel. 

For distributed load testing we need to create Master-slave configuration using following steps where Master will control all the slaves and collect the test results.

Step 1: Points to remember

1. Make sure that all the systems are running exactly the same version of Jmeter. Mixing versions may not work correctly.
2. As far as possible, also use the same version of Java on all systems. Using different versions of Java may work-but is best avoided.
3. To make the system work firewall needs to be turned off. 
4. All the systems need to be in same Subnet. 
5. If the test uses any data files, note that these are not sent across by the client so make sure that these are available in the appropriate directory on each server. If necessary you can define different values for properties by editing the user.properties or system.properties files on each server. These properties will be picked up when the server is started and may be used in the test plan to affect its behavior (Exa. Connecting to a different remote server). Alternatively use different content in any datafiles used by the test (e.g. if each server must use unique ids, divide these between the data files).

Step 2: Configure Slave Systems

1. Go to Jmeter/bin folder and Open jmeter-server.bat in a Text Editor.
2. Find “:setCP” and Edit “START rmiregistry” to the full path. Example: “START C:\j2sdk1.4.2\jre\bin\rmiregistry”

3. Now Execute jmeter-server.bat (jmeterserver on unix).
4. On windows, you should see a dos window appear with “jre\[version]\bin\rmiregistry.exe”. If this doesn't happen, it means either the environment settings are not right, or there are multiple JRE installed on the system. [version] would be the jre version installed on the system.

Step 3: Configure Master Systems

1. Master system will act as the console.
2. Open windows explorer and go to jmeter/bin directory
3. Open jmeter.properties in notepad or wordpad.
4. Find the property named, "remote_hosts" and Edit the line “remote_hosts=127.0.0.1”. Add the Slave System's IP address where jmeter server is running.
   For example, If Jmeter server running on 192.168.0.1, 192.168.0.2, 192.168.0.3, 192.168.0.4 and 192.168.0.5 Slave Systems then the entry would like like this:
   remote_hosts=192.168.0.1,192.168.0.2,192.168.0.3,192.168.0.4,192.168.0.5

5. Start jmeter file by executing jmeter.bat file.
6. Open the test plan you want to use.
7. You will notice that the Run menu contains two new sub-menus: "Remote Start" and "Remote Stop". These menus contain the client that you set in the properties file.
8. To Execute script in all Slave Systems select “Remote Start All” option from the Run menu.

Thanks & Regards,

Prashant Vadher | QC Engineer

Read more »

Crowdsource Testing

What is Crowdsource Testing?

Crowdsource testing is an emerging trend in software testing which covers benefits, effectiveness, and efficiency of crowdsourcing and the cloud platform. It differs from traditional testing methods in that the testing is carried out by a number of different testers from different places, and not by hired consultants and professionals. The software is put to test under diverse realistic platforms which makes it more reliable, cost-effective, fast, and bug-free.

This method of testing is considered when the software is more user-centric: i.e., software whose success is determined by its user feedback and which has a diverse user space. It is frequently implemented with gaming, mobile applications, when experts who may be difficult to find in one place are required for specific testing, or when the company lacks the resources or time to carry out the testing internally. Crowdsourcing your software testing consists of delegating onto a number of internet users the task of testing your web or software project while in development to ensure that it contains no defects.

Crowdsource testing companies provide the platform for the testing cycles. They then crowdsourse the Product to a community of testers, who register for testing the software voluntarily.

Testers are paid per bug, depending on type of bug and its market price. The crowdsource testing team is usually in addition to the organization's testing team, and not usually a replacement.

Crowdsource testing vs Outsource testing 

The main difference is that, in crowdsource testing, testers may belong to different workplaces. In outsource testing, the testers are from the same company or workplace that is responsible for outsourcing. In crowdsource testing, people voluntarily test a software with the possibility of not being paid (if no bugs are discovered). Outsource testers always get paid for their work.

Advantages of Crowdsource testing:

• Real insights from the real world, not just made up test case results 
• Rapid feedback right away. 
• With crowdsourcing, testers automatically test your interactive project against a variety of platforms 
• It is cost effective, as the product company pays only for the valid bugs reported. 
• The pool of testers is diverse with variations in languages as well as locales. This helps in testing applications which are based on localization. 
• Testing done by hundreds of people at the same time. As there are large number of testers testing a software simultaneously, testing can be done quickly, giving more time to market. Time to test the software is comparably less. 
• Leads to better productivity. 

Disadvantages of Crowdsource testing: 

• Security and Confidentiality : When offering a project to a crowd for testing, it is exposed to a large number of internet users worldwide. 
• If the project is not released, a large number of users are able to access it fully and discover its secrets. 
• Inconsistent quality and increased workload : The users that compose your crowd of testers are from different backgrounds, speak different languages and possess different levels of experience. 
• They may be a number of poorly written bugs, duplicate bugs and erroneous bugs.

Some Crowdsource testing Tools:

• InnoCentive
• TopCoder
• Amazon Mechanical Turk
• uTest
• Passbrains
• Bugfinders
• TXTeagle

Thanks and Regards,

Prashant Vadher | QC Engineer

Read more »

How to test Flex application using Selenium RC?

Testing Flex applications was difficult because the logic or behavior is encapsulated from the browser. Selenium RC uses JavaScript to communicate with the browser. Flex External Interface provides a mechanism for which you can use JavaScript to call an ActionScript function in a SWF file embedded in an HTML page. Therefore, we use FlexSelenium, a Selenium RC client extension that uses JavaScript as the medium between Selenium RC and the Flex application.

You can also test flex application by having a flex monkium plugin in selenium IDE. You need to compile your client application with sfapi.swc and automation_monkey.swc and the flex libs. This becomes your application to test. You can record your test and convert that into any format which you are comfortable in.

Recently I just got the chance to test one Flex application using selenium. To provide flex support to selenium, you just have to add few JAR files. But for this you need to rebuild your application with provided library file (SeleniumFlexAPI.swc) by selenium flex. 

Below are the steps to test flex application using Selenium RC.

1. Rebuild your flex application with SeleniumFlexAPI.swc
   Download the “Selenium-Flex API” zip file and extract the zip file. In FlexBuilder, add this SeleniumFlexAPI.swc in the /src folder, then build your application with -include-libraries SeleniumFlexAPI.swc as the additional compiler argument.
   
   
   Click Here to Download SeleniumFlexAPI
   
   
2. Add JAR files in the project
   Download “Flash Selenium Java client extension” and “Flex UI Selenium” jar files. Now Right click on Project name in Eclipse and Select “Build Path >> Configure Build Path >> Library Tab”. Add these jar files by selecting “Add External Jar files” button.
   
   
   Click Here to Download Flash Selenium Java client extension
   Click Here to Download Flash UI Selenium

3. Write Selenium Script
   Before we write the script in Selenium RC we need to identify the elements of the flex application. So for this use FlashFirebug (extension of the firebug add-on) Firefox add-on to identify the elements.
    
   Click Here to Download FlashFirebug
   
   
   Example:
   
   

   selenium_flex.swf:

   
   
   
   
   Selenium Script:
   

   /*
    @author Software Testing Diary
   */
   
   package practice;
   import org.junit.After;
   import org.junit.Before;
   import org.junit.Test;
   import static org.junit.Assert.*;
   import com.thoughtworks.selenium.DefaultSelenium;
   import com.thoughtworks.selenium.FlexUISelenium;
   import com.thoughtworks.selenium.Selenium;
   
   public class flexSelenium {
       private final static String OPEN_URL = "http://www.softwaretestingdiary.com/";
       private final static String OPEN_PAGE = "2012/07/how-to-test-flex-application-using.html";
       private Selenium selenium;
       private FlexUISelenium flexUITester;
      
       @Before
       public void setUp() throws Exception
       {
           selenium = new DefaultSelenium("localhost", 4444, "*chrome",OPEN_URL);
           selenium.start();
           selenium.open(OPEN_PAGE);
           flexUITester = new FlexUISelenium(selenium, "selenium_flex.swf");
       }
        @Test
        public void test()
       {
       flexUITester.type("Software Testing Diary").at("myInput");
       flexUITester.click("myButton");
       assertEquals("Software Testing Diary", flexUITester.readFrom("myText"));
       }
       @After
       public void tearDown() throws Exception
       {
       selenium.stop();
       }
   }
   
   
   
   
   
   
   
   Thanks and Regards,
   Prashant Vadher | QC Engineer 
    
   
   
   
   
   
   
    

Read more »

What is Testopia and How to set access control in Testopia at Test Plan Level?

What is Testopia?

Testopia is a test case management extension for Bugzilla. This allows a single user experience and point of product management for both defect tracking and test case management. It is designed to be a generic tool for tracking test cases, allowing for testing organizations to integrate bug reporting with their test case run results.

Bugzilla is one of the most popular open source issue tracking systems available. Testopia integrates with Bugzilla products, components, versions, and milestones to allow a single management interface for high level objects. Testopia allows users to attach bugs to test case run results for centralized management of the software engineering process.

Testopia allows users to login to one tool and uses Bugzilla group permissions to limit access to modifying test objects.

Test Plans:

At the top of the Testopia hierarchy are test plans. So you need to create a Test Plan before you can do anything else in Testopia. Test plans are associated with a single product in Bugzilla. You can create multiple test plans for each product. This test plan will use as the storage point for all related test cases and test runs and it will act as the dashboard for your testing. It will also determine who will have access to update test cases.

Test Cases:

Test cases are semi-independent in Testopia. Each test case can be associated with multiple test plans. Test cases are associated with one or more test plans and with zero or more test runs. Test cases can be divided into categories. You can define as many categories for your product as you like from the test plan page. Each product in Bugzilla is divided into components and you can apply multiple components to each test case, however each test case can only belong to one category at a time.

Test Runs:

Once you have defined a set of test cases, you are ready to run through those tests in a test run. Each run is associated with a single test plan and Environment. It contains any number of test cases from that plan. It contains a list of test cases to be examined and stores the results in the case-runs table.

Test Case-run:

The union of a test case and a test run. Each time a test case is included in a new test run, an entry is made for it in the test case-runs table. This captures whether the test case passed or failed in the given run. Each case-run should be associated with only one build for a given status.

When you create a test run, records for each test case in that run are created. By default these take on the build and environment of the test run, however, it is possible to change these attributes on a particular case-run, essentially creating a new case-run for each combination.

How to set access control in Testopia at Test Plan Level?

When you first install Testopia, it will create a Bugzilla group called “Testers”. Members of this group have access to view and update all test plans and their associated objects such as cases and runs. Membership in this group is required in order to create new test plans, clone test plans, and administer environments. If the “testopia-allow-group-member-deletes parameter” is on, members of this group will also have rights to delete any object in Testopia. Membership in this group is checked first and supersedes the access control lists for individual plans.

Testopia checks which test plans your user has access to / remember access control in testopia is at the test plan level (Permissions tab in a test plan). Testopia checks if all the test plans to which the user can access are associated with products that the user has access to. If not then it complains: "You are not authorized to edit product X contact administrator etc."

In addition to the Testers group, each test plan maintains it's own access control list which can be used to allow or deny access to test plans based on email domain or explicit inclusion. Each test plan has its own access list. For a user that is not in the Testers group to access a test plan or any associated cases, runs, or case runs, he or she must be included on the list either by matching a regular expression, or explicit inclusion. To edit the access control list for a plan, navigate to the test plan and click the Permissions tab.

User Regular Expression:

Users with login names (email addresses) matching a supplied regular expression can be given rights to a particular test plan. The regular expression should be crafted with care to prevent unintentional access to the test plan by outsiders.

For example :

To grant access to your test plan by all users at SoftwareTestingDiary.com you would supply the following regular expression:

^.*@SoftwareTestingDiary\.com$

To provide access to all users at gmail.com and SoftwareTestingDiary.com you would use:

^.*@(gmail\.com|SoftwareTestingDiary\.com)$

To provide public access (all users) you would use:

.*

An empty regular expression does not match anything meaning leaving this field blank will mean the test plan will rely solely on explicit membership. Once you have supplied the regular expression, you must select the access level.

Explicit Inclusion:

If you do not wish to grant access to a whole group at once, you can add individual users by entering their Bugzilla login id in the field provided and clicking the Add User button. This allows the most fine grained control as to who can do what within your test plan. However, if you add a user that matches the regular expression they will have the greater of the two rights.

Access Rights

Users on the test plan access control lists can be granted rights to read, write, delete, and admin test plans and their associated objects.

Read: Allows viewing rights to the plan and all test cases, test runs, and test case-runs associated with it. Test cases linked to more than one plan will be visible to users in both plans.

Write: Implies Read. Allows rights to modify the plan and associated cases, runs, and case-runs. Test cases linked to more than one plan will not be writable unless the user has write rights in all plans.

Delete: Implies Read and Write. Allows rights to delete the plan and associated cases, runs, and case-runs. Test cases linked to more than one plan will not be deletable unless the user has

delete rights in all plans.

Admin: Implies Read, Write, and Delete. Allows rights to modify the plan's access controls.

Thanks and Regards,

Prashant Vadher | QC Engineer

Read more »

How to set Product level permissions in Bugzilla?

I have two products (ProductA and ProductB), groups (GroupA and GroupB) and users (UserA and UserB) in bugzilla.

If I want to set permission such that UserA can create/edit the fields of ProductA but UserB can not edit the bug of ProductB.

1. Login with a user with Admin rights and Open "Administration" page.
   
   
   
   
   
   
2. Click on “Groups” option and Create two Groups : GroupA and GroupB
   
   
   
   
   
3. Select “Administration >> Users” options and Click on UserA and Select “User is a member of these groups ” checkbox for GroupA only and Click on Save Changes button. 
   
   
   
   
   
   
4. Again Select “Administration >> Users” options and Click on UserB and Select “User is a member of these groups ” checkbox for GroupB only and Click on Save Changes button.
5. Now Select “Administration >> Products >> ProductA >> Edit Group Access Controls ” options. “On Edit Group Controls for ProductA” Page Both groups will be displayed. Now Select following options and Click on Submit option.

GroupA > Entry:Checked > MemberControl:Shown > OtherControl :N/A > canconfirm:Checked > editbugs:Checked

6. Again Select “Administration >> Products >> ProductB >> Edit Group Access Controls ” options. “On Edit Group Controls for ProductB” Page Both groups will be displayed. Now Select following options and Click on Submit option.

GroupB > Entry:Checked > MemberControl:Shown > OtherControl :N/A > canconfirm:Checked > editbugs:Checked

After step -5, UserA will be able to place and edit bugs for ProductA only and UserB will be able to place and edit bugs for ProductB.

You can also make all projects readable anonymously by default by setting "requirelogin" to off in “Adminitration >> Parameters >> User Authentication”. Then you would need to check if those products that you don't want to be publicly visible have correct group security.

Thanks and Regards,

Prashant Vadher | QC Engineer

Read more »