LarryDpk
发布于 2023-05-16 / 282 阅读 / 0 评论 / 0 点赞

Performance Test With JMeter Distributed Testing (Step-By-Step Guide)

Performance Test With JMeter Distributed Testing (Step-By-Step Guide)

Concepts of Performance Test

Performance testing is a type of software testing that evaluates responsiveness, stability, scalability, reliability, speed, and resource usage of your software and infrastructure. The goal of performance testing is to identify and eliminate performance bottlenecks in the system, ensuring that it meets the required performance criteria.

Performance testing can be performed on a variety of systems, including web applications, databases, and operating systems.

Performance testing is an important part of the software development process. By identifying and eliminating performance bottlenecks early in the development process, developers can avoid costly and time-consuming problems later on.

Here are some of the benefits of performance testing:

  • Improves the user experience by ensuring that the system is responsive and stable under load.
  • Helps to identify potential failure points in the system.
  • Can help to prevent costly and time-consuming problems later on.
  • Can help to improve the system’s scalability and reliability.
  • Can help to meet performance requirements.

Types Of Performance Test

There are a few different types of performance test, you can check the table as below:

Type of Performance Testing Description Volume Time
Load Testing Measures how a system performs under a specific load. This can be done by simulating a certain number of users or transactions. Load testing can help to identify performance bottlenecks and ensure that the system can handle the expected load. High Varies
Baseline Testing Used to establish a baseline for performance testing. This is done by running the application under a known load and measuring the results. The baseline can then be used to compare the performance of the application over time. Low Varies
Smoke Testing A quick test that is used to verify that the application is working as expected. Smoke tests are typically used to check for basic functionality, such as the ability to log in and access the main features of the application. Low Short
Stress Testing Used to test the limits of an application. Stress testing is typically done by simulating a large number of users or transactions. Stress testing can help to identify performance bottlenecks and ensure that the system can handle unexpected loads. High Varies
Soak Testing Used to test the performance of an application over a long period of time. Soak testing is typically done by simulating a constant load on the application. Soak testing can help to identify performance degradation over time. High Long
Endurance Testing A type of soak testing that is used to test the performance of an application under extreme conditions. Endurance testing is typically done by simulating a very high load on the application for a long period of time. Endurance testing can help to identify performance problems that may not be evident under normal conditions. Very high Very long
Capacity Testing Used to determine the maximum load that an application can handle. Capacity testing is typically done by gradually increasing the load on the application until it reaches a point where it can no longer function properly. Capacity testing can help to determine the amount of resources that are needed to support the application. High Varies
Longevity Testing Used to test the performance of an application over a long period of time. Longevity testing is typically done by running the application for a set period of time and monitoring its performance. Longevity testing can help to identify performance problems that may develop over time. High Long
Peak Testing Used to test the performance of an application under peak load conditions. Peak load conditions are typically the highest load that an application is expected to experience. Peak testing can help to identify performance problems that may occur during peak load conditions. Very high Varies
Spike Testing A type of peak testing that is used to test the performance of an application under sudden spikes in load. Spike testing is typically done by simulating a sudden increase in the load on the application. Spike testing can help to identify performance problems that may occur during sudden spikes in load. Very high Short
Volume Testing Used to test the performance of an application under a large volume of data. Volume testing is typically done by simulating a large number of transactions or by loading a large amount of data into the application. Volume testing can help to identify performance problems that may occur when the application is processing a large amount of data. High Varies
Capacity Tests Used to determine the maximum amount of data that an application can process. Capacity tests are typically done by gradually increasing the amount of data that is processed by the application until it reaches a point where it can no longer function properly. Capacity tests can help to determine the amount of resources that are needed to support the application. High Varies
Reliability Testing Used to test the reliability of an application. Reliability testing is typically done by simulating failures in the application and then measuring the time it takes to recover from the failure. Reliability testing can help to identify performance problems that may occur when the application fails. High Varies
Scalability Testing Used to test the scalability of an application. Scalability testing is typically done by gradually increasing the load on the application and measuring how it performs. Scalability testing can help to determine how well the application can handle increasing loads. High Varies

Some docs Soak Test is the same with endurance testing, capacity testing, or longevity testing.

Tools for Performance Test

Here is a list of tools you can use to do performance test:

Tool Features Pros Cons
Apache JMeter Open source, easy to use, and can be used to test a variety of applications. It is also highly customizable and can be used to test a wide range of performance metrics. Free, easy to learn, and highly customizable. Can be complex to set up and use, and does not have as many features as some commercial tools.
HP LoadRunner Commercial tool that is powerful and has a wide range of features. It is also easy to use and can be used to test a wide range of performance metrics. Powerful, easy to use, and has a wide range of features. Expensive, and can be difficult to learn.
Gatling Open source tool that is powerful and has a wide range of features. It is also easy to use and can be used to test a wide range of performance metrics. Free, easy to learn, and highly scalable. Can be complex to set up and use, and does not have as many open source options as some other tools.
Blazemeter Cloud-based tool that is easy to use and has a wide range of features. It is also affordable and can be used to test a wide range of performance metrics. Affordable, easy to use, and has a wide range of features. Does not have as many open source options as some other tools.
k6 Open source tool that is easy to use and can be used to test a variety of applications. It is also highly customizable and can be used to test a wide range of performance metrics. Free, easy to learn, and highly scalable. Can be complex to set up and use, and does not have as many open source options as some other tools.
NeoLoad Commercial tool that is powerful and has a wide range of features. It is also easy to use and can be used to test a wide range of performance metrics. Powerful, easy to use, and has a wide range of features. Expensive, and can be difficult to learn.
TestComplete Commercial tool that is powerful and has a wide range of features. It is also easy to use and can be used to test a wide range of performance metrics. Powerful, easy to use, and has a wide range of features. Expensive, and can be difficult to learn.
Silk Performer Commercial tool that is powerful and has a wide range of features. It is also easy to use and can be used to test a wide range of performance metrics. Powerful, easy to use, and has a wide range of features. Expensive, and can be difficult to learn.
Rational Performance Tester Commercial tool that is powerful and has a wide range of features. It is also easy to use and can be used to test a wide range of performance metrics. Powerful, easy to use, and has a wide range of features. Expensive, and can be difficult to learn.
Apica LoadView Commercial tool that is powerful and has a wide range of features. It is also easy to use and can be used to test a wide range of performance metrics. Powerful, easy to use, and has a wide range of features. Expensive, and can be difficult to learn.
AppDynamics Commercial tool that is powerful and has a wide range of features. It is also easy to use and can be used to test a wide range of performance metrics. Powerful, easy to use, and has a wide range of features. Expensive, and can be difficult to learn.
Dynatrace Commercial tool that is powerful and has a wide range of features. It is also easy to use and can be used to test a wide range of performance metrics. Powerful, easy to use, and has a wide range of features. Expensive, and can be difficult to learn.
New Relic Commercial tool that is powerful and has a wide range of features. It is also easy to use and can be used to test a wide range of performance metrics. Powerful, easy to use, and has a wide range of features. Expensive, and can be difficult to learn.

The best tool for you will depend on your specific needs and budget. If you are looking for an open source tool that is easy to use, Apache JMeter is a good option. If you are looking for a commercial tool that is powerful and has a wide range of features, LoadRunner or NeoLoad are good options.

It is important to note that no single tool is perfect for all performance testing needs. It is often helpful to use a combination of tools to get the best results.

JMeter Distributed Testing

Now let me introduce how to do a Soak Test with JMeter distributed testing.

We need one controller node with multiple worker nodes to do the testing, if you need more workload, just increase the number of worker nodes. All the workers will connect to the target system to perform the pre-defined tests.

The workers can run in different servers, that’s why it can create plenty of requests at the same time.

Test Step by Step

I will show how to do the test step by step. This is the overview of the servers.

I run the controller on my Macbook, and created two workers on Ubuntu servers. The target system is Bing, stackoverflow, QQ.

Step 1: Install JDK for controller and workers

The Apache JMeter is a Java based application, it needs a Java runtime to start up. Install JDK on Ubuntu:

sudo apt install openjdk-11-jdk

Stpe 2: Download Apache JMeter

Go the the download page and download the apache-jmeter-5.5.zip, and unzip the package:

unzip apache-jmeter-5.5.zip

We need to download for both controller and workers.

For controller, I place JMeter here:

/Users/larry/Software/apache-jmeter-5.5

For workers, I place JMeter here:

home/larry/software/apache-jmeter-5.5

Step 3: Prepare the JMX file

JMeter stores the defined test cases in .jmx file. Open the JMeter UI and write the cases:

I run the test with 8 threads, 60s ramp-up and infinite loop for 120s.

I saved the file here:

$ ls /Users/larry/Software/apache-jmeter-5.5/projects/pkslow
performance-test.jmx

Just controller need the jmx file, worker will receive the info from controller.

Step 4: Start up the workers

Start up the two workers with this command:

  • Worker 1:
$ /home/larry/software/apache-jmeter-5.5/bin/jmeter-server -Dserver.rmi.ssl.disable=true -Djava.rmi.server.hostname=192.168.50.171
Created remote object: UnicastServerRef2 [liveRef: [endpoint:[192.168.50.171:41709](local),objID:[-2af2edf6:1882245cfde:-7fff, 1980465993472976097]]]
  • Worker 2:
$ /home/larry/software/apache-jmeter-5.5/bin/jmeter-server -Dserver.rmi.ssl.disable=true -Djava.rmi.server.hostname=192.168.50.60
Created remote object: UnicastServerRef2 [liveRef: [endpoint:[192.168.50.60:45337](local),objID:[18489452:1882246a88b:-7fff, 4749047852254882466]]]

Note: You may hit java.rmi.RemoteException without specify the java.rmi.server.hostname.

$ ./software/apache-jmeter-5.5/bin/jmeter-server -Dserver.rmi.ssl.disable=true
May 15, 2023 5:43:16 PM java.util.prefs.FileSystemPreferences$1 run
INFO: Created user preferences directory.
Created remote object: UnicastServerRef2 [liveRef: [endpoint:[127.0.1.1:33021](local),objID:[-7155c199:1881ecaf233:-7fff, -6972385484103483700]]]
Server failed to start: java.rmi.RemoteException: Cannot start. ubuntu is a loopback address.
An error occurred: Cannot start. ubuntu is a loopback address.

Step 5: Start the controller to run the test

$ cd /Users/larry/Software/apache-jmeter-5.5/

$ ./bin/jmeter -n -t projects/pkslow/performance-test.jmx -l projects/pkslow/performance-test.csv -e -o projects/pkslow/Reports -Dserver.rmi.ssl.disable=true -R 192.168.50.171,192.168.50.60
Creating summariser <summary>
Created the tree successfully using projects/pkslow/performance-test.jmx
Configuring remote engine: 192.168.50.171
Configuring remote engine: 192.168.50.60
Starting distributed test with remote engines: [192.168.50.171, 192.168.50.60] @ May 16, 2023 10:37:31 PM CST (1684247851079)
Remote engines have been started:[192.168.50.171, 192.168.50.60]
Waiting for possible Shutdown/StopTestNow/HeapDump/ThreadDump message on port 4445
summary +      1 in 00:00:02 =    0.7/s Avg:  1260 Min:  1260 Max:  1260 Err:     0 (0.00%) Active: 2 Started: 2 Finished: 0
summary +    111 in 00:02:31 =    0.7/s Avg:  4433 Min:    41 Max: 50474 Err:     4 (3.60%) Active: 15 Started: 16 Finished: 1
summary =    112 in 00:02:33 =    0.7/s Avg:  4405 Min:    41 Max: 50474 Err:     4 (3.57%)
summary +     64 in 00:00:19 =    3.3/s Avg: 25706 Min:    62 Max: 66745 Err:    31 (48.44%) Active: 0 Started: 16 Finished: 16
summary =    176 in 00:02:52 =    1.0/s Avg: 12151 Min:    41 Max: 66745 Err:    35 (19.89%)
Tidying up remote @ May 16, 2023 10:40:24 PM CST (1684248024629)
... end of run

You can check more log in jmeter.log:

$ tail -f jmeter.log 
2023-05-16 22:40:29,956 INFO o.a.j.r.d.JsonExporter: Found data for consumer statisticsSummary in context
2023-05-16 22:40:29,957 INFO o.a.j.r.d.JsonExporter: Creating statistics for overall
2023-05-16 22:40:29,957 INFO o.a.j.r.d.JsonExporter: Creating statistics for other transactions
2023-05-16 22:40:29,957 INFO o.a.j.r.d.JsonExporter: Checking output folder
2023-05-16 22:40:29,958 INFO o.a.j.r.d.JsonExporter: Writing statistics JSON to /Users/larry/Software/apache-jmeter-5.5/projects/pkslow/Reports/statistics.json
2023-05-16 22:40:30,006 INFO o.a.j.r.d.ReportGenerator: Exporting data using exporter:'html' of className:'org.apache.jmeter.report.dashboard.HtmlTemplateExporter'
2023-05-16 22:40:30,007 INFO o.a.j.r.d.HtmlTemplateExporter: Will generate dashboard in folder: /Users/larry/Software/apache-jmeter-5.5/projects/pkslow/Reports
2023-05-16 22:40:30,142 INFO o.a.j.r.d.HtmlTemplateExporter: Report will be generated in: /Users/larry/Software/apache-jmeter-5.5/projects/pkslow/Reports, creating folder structure
2023-05-16 22:40:30,145 INFO o.a.j.r.d.TemplateVisitor: Copying folder from '/Users/larry/Software/apache-jmeter-5.5/bin/report-template' to '/Users/larry/Software/apache-jmeter-5.5/projects/pkslow/Reports', got message: /Users/larry/Software/apache-jmeter-5.5/projects/pkslow/Reports, found non empty folder with following content [/Users/larry/Software/apache-jmeter-5.5/projects/pkslow/Reports/statistics.json], will be ignored
2023-05-16 22:40:30,472 INFO o.a.j.JMeter: Dashboard generated

The workers will print some simple log:

$ /home/larry/software/apache-jmeter-5.5/bin/jmeter-server -Dserver.rmi.ssl.disable=true -Djava.rmi.server.hostname=192.168.50.171
Created remote object: UnicastServerRef2 [liveRef: [endpoint:[192.168.50.171:44621](local),objID:[4d13369d:18824fd059a:-7fff, 2287886479123818682]]]
Starting the test on host 192.168.50.171 @ 2023 May 16 22:37:31 HKT (1684247851980)
Warning: Nashorn engine is planned to be removed from a future JDK release
Finished the test on host 192.168.50.171 @ 2023 May 16 22:40:24 HKT (1684248024679)

Step 6: result and report

After the whole test completed, you can find the result and report:

$ ls -l /Users/larry/Software/apache-jmeter-5.5/projects/pkslow/
total 72
drwxr-xr-x  6 larry  staff    192 May 16 22:40 Reports
-rw-r--r--  1 larry  staff  23472 May 16 22:40 performance-test.csv
-rw-r--r--  1 larry  staff  10593 May 16 09:50 performance-test.jmx

We can open the index.html in the Reports folder:

Other tips

Change the worker server port

For workers, the default server port is 1099, you can change the port in properties file or command line parameter:

$ /home/larry/software/apache-jmeter-5.5/bin/jmeter-server -Dserver.rmi.ssl.disable=true -Djava.rmi.server.hostname=192.168.50.60 -Dserver_port=1984

So the controller need to change the port:

$ ./bin/jmeter -n -t projects/pkslow/performance-test.jmx -l projects/pkslow/performance-test.csv -e -o projects/pkslow/Reports -Dserver.rmi.ssl.disable=true -R 192.168.50.171:1984,192.168.50.60:1984

What’s more, with different port, you can start up more workers in one server.

run command with nohup

As the test may take a long time to complete, it’s better run all the command with nohup:

$ nohup ./bin/jmeter -n -t projects/pkslow/performance-test.jmx -l projects/pkslow/performance-test.csv -e -o projects/pkslow/Reports -Dserver.rmi.ssl.disable=true -R 192.168.50.171:1984,192.168.50.60:1984 &
[1] 81248

Save the result and resport with datetime

$ nohup ./bin/jmeter -n -t projects/pkslow/performance-test.jmx -l projects/pkslow/performance-test."$(date +%F.%T)".csv -e -o projects/pkslow/Reports-"$(date +%F.%T)" -Dserver.rmi.ssl.disable=true -R 192.168.50.171:1984,192.168.50.60:1984 &

Summary

Performance test is quite important and Apache JMeter can help us. Hope this article help you to understand and set up the remote test.

Please find the jmx code in: LarryDpk/pkslow-samples


References:
awesome-jmeter
JMeter distributed testing
How to install ssh on Ubuntu Linux using apt-get
API Performance Testing for Success