Continuous Stress Testing for your JAX-RS (and JavaEE) applications with Gatling + Gradle + Jenkins Pipeline

In this post I am going to explain how to use Gatling project to write stress tests for your JAX-RS Java EE endpoints, and how to integrate them with Gradle and Jenkins Pipeline, so instead of having a simple stress tests, what you have is a continuous stress testing, where each commit might fire these kind of tests automatically, providing automatic assertions and more important graphical feedback of each execution so you can monitorize how the performance is evolving in your application.

First thing to develop is the JAX-RS JavaEE service:

There is nothing special, this is an asynchronous JAX-RS endpoint that connects to swapi.co site, retrieves all the information of Star Wars planets, calculates the average of orbital period and finally it returns it in form of text. For sake of simplicity, I am not going to show you all the other classes but they are quite simple and at the end of the post I will provide you the github repository. 

The application is packaged inside a war file and deployed into an application server. In this case into an Apache TomEE 7 deployed inside the official Apache TomEE Docker image.

Next step is configuring Gradle build script with Gatling dependencies. Since Gatling is written in Scala you need to use Scala plugin.

After that, it is time to write our first stress test. It is important to notice that writing stress tests for Gatling is writing a Scala class using the provided DSL. Even for people who has never seen Scala is pretty intuitive how to use it.

So create a directory called src/test/scala and create a new class called AverageOrbitalPeriodSimulation.scala with next content:

Every simulation must extends Simulation object. This simulation takes base URL of the service from starwars_planets_url environment or system property, it creates the scenario pointing to the endpoint defined in JAX-RS,  and finally during 3 seconds it will gradually add users until 10 users are running at the same time. The test will pass only if all the requests succeed in less than 3 seconds.

Now we need to run this test. You will notice that this is not a JUnit test, so you cannot do a Run As JUnit test. What you need to do is use a runnable class provided by Gatling which requires you pass as argument the simulation class. This is really easy to do with Gradle.

We are defining a Gradle task of type JavaExec, since what we want is to run a runnable class. Then we make the life a bit easier for developer by automatically detect that if starwars_planets_url is not set, we are running this test into a machine that has Docker installed so probably this is the host to be used.
Finally we override the environment variable if it is required, we set the runnable class with required properties and we configure Gradle to execute this task every time the test task is executed (./gradlew test).

If you run it, you might see some output messages from Gatling, and after all a message like: please open the following file: /Users/..../stress-test/build/reports/gatling-results/averageorbitalperiodsimulation-1459413095563/index.html and this is where you can get the report. Notice that a random number is appended at the end of the directory and this is important as we are going to see later. The report might looks like:

At this time we have Gatling integrated with Gradle, but there is a missing piece here, and it is adding the continuous part on the equation. For adding continuous stress testing we are going to use Jenkins and Jenkins Pipeline as CI server so for each commit stress tests are executed among other tasks such as compile, run unit, integration tests, or code quality gate.

Historically Jenkins jobs were configured using web UI, requiring users to manually create jobs, fill the details of the job and create the pipeline through web browser. Also this makes keeping configuration of the job separated from the actual code being built.

With the introduction of Jenkins Pipeline plugin. This plugin is a Groovy DSL that let's implement you the entire build process in a file and store that alongside its code. Jenkins 2.0 comes by default with this plugin, but if you are using Jenkins 1.X you can install it as any other plugin (https://wiki.jenkins-ci.org/display/JENKINS/Pipeline+Plugin)

So now we can start coding our release plugin but for the purpose of this post only stress part is going to be covered. You need to create a file called Jenkinsfile (the name is not mandatory but it is the de-facto name) on the root of your project, and in this case with next content:

In this case we are defining a new stage which is called Stress Test. Stage step is only used as informative and it will be used for logging purposes. Next a node is defined. A node is a Jenkins executor where to execute the code. Inside this node, the source code is checked out from the same location where Jenkinsfile is placed, sets a new environment variable pointing out to the location where the application is deployed, and finally a shell step which executes the Gradle test task.

Last step in Jenkins is to create a new job of type Pipeline and set the location of the Jenkinsfile. So go to Jenkins > New Item > Pipeline and give a name to the job.

Then you only need to go to Pipeline section and configure the SCM repository where the project is stored.

And then if you have correctly configured the hooks from Jenkins and your SCM server, this job is going to be executed for every commit, so your stress tests are going to run continuously.

Of course probably you have noticed that stress tests are executed but no reports are published in Jenkins, so you have no way to see or compare results from different executions. For this reason you can use publishHtml plugin to store the generated reports in Jenkins. If you don't have the plugin installed yet, you need to install it as any other Jenkins plugin.

PublishHtml plugin allows us to publish some html files generated by our build tool to Jenkins so they are available to users and also categorised by build number. You need to configure the location of the directory of files to publish, and here we find the first problem, do you remember that Gatling generates a directory with a random number? So we need to fix this first. You can follow different strategies, but the easiest one is simply rename the directory to a known static name after the tests.

Open Gradle build file and add next content.

We are creating a new task executed at the end of test task that renames the last created directory to averageorbitalperiodsimulation.

Final step is add after shell call in Jenkinsfile next call:

publishHTML(target: [reportDir:'stress-test/build/reports/gatling-results/averageorbitalperiodsimulation', reportFiles: 'index.html', reportName: 'Gatling report', keepAll: true])

After that you might see a link in the job page that points to the report.

And that's all, thanks of Gradle and Jenkins you can implement a continuous stress testing strategy in an easy way and just using code the language all developers speak.

We keep learning,
Alex.

I can live whatever way I please, I move around among the seven seas, No one will miss me when the sun goes down, and in the morning I'be out of town (Movin' Cruisin' - The Fantastic Oceans)

Music: https://www.youtube.com/watch?v=Byg5Xq_pb74
Source Code: https://github.com/lordofthejars/starwars