Load-testing Jira with Gatling

Comments: 0
Computers  

As I mentioned earlier, I’ve been doing some simple load-testing of Jira instances using Gatling. Detailed sample code after the jump, because I couldn’t find anyone else’s and I’ve got decent pagerank.

There are three placeholder strings in the code below:

  • JIRA_BASEURL = https://jira.example.com/
  • JIRA_USERNAME = mytestuser
  • JIRA_PASSWORD = reallygoodpassword

Boilerplate

Pretty much every Gatling test starts out like this:

import java.util.concurrent.ThreadLocalRandom
import scala.concurrent.duration._
import io.gatling.core.Predef._
import io.gatling.http.Predef._
import io.gatling.jdbc.Predef._

class mytest extends Simulation {
  val httpProtocol = http
    // don't hit gatling.io at start of every run
    .warmUp("JIRA_BASEURL")
    .baseUrl("JIRA_BASEURL")
    .inferHtmlResources()
    .acceptHeader("application/json, text/javascript, */*; q=0.01")
    .acceptEncodingHeader("gzip, deflate")
    .acceptLanguageHeader("en-US,en;q=0.5")
    .userAgentHeader("Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:80.0) Gecko/20100101 Firefox/80.0")

  val headers = Map(
    "Accept" -> "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",
    "Upgrade-Insecure-Requests" -> "1",
    "Origin" -> "JIRA_BASEURL")

(note that we’re still in class mytest)

Login to Jira

Strictly speaking, there’s no need to visit the Dashboard URL and then wait 1-2 seconds before POSTing the login; I’m doing it for realism.

  object Login {
    val login = exec(http("Homepage")
        .get("/secure/Dashboard.jspa")
        .headers(headers))
      .pause(ThreadLocalRandom.current.nextInt(1,3))
      .exec(http("Login")
        .post("/login.jsp")
        .formParam("os_username", "JIRA_USERNAME")
        .formParam("os_password", "JIRA_PASSWORD")
        .formParam("os_destination", "")
        .formParam("user_role", "")
        .formParam("atl_token", "")
        .formParam("login", "Log In"))
  }

Browse N issues

resources/issues.tsv is a single-field file containing a list of ~3,000 issue IDs selected at random. Basically I just tinkered with filters until I had a list that included stories, epics, bugs, and tasks from a number of different projects, in a number of different states.

IssueId
BOGO-50
HHGTG-42
PRIME-17
DJT-45
...

Since we’re using .random for the iterator, it doesn’t matter how many rows are in the file. Adding more just keeps us from frequently hitting issues that are already cached.

  val getIssue = tsv("issues.tsv").random
  object Browser {
    val browse = repeat(50, "n") {
      feed(getIssue)
      .pause(ThreadLocalRandom.current.nextInt(2,6))
      .exec(http("Browse ${IssueId}")
        .get("/browse/${IssueId}")
        .headers(headers))
    }
  }

Including the IssueID in the label argument to http() makes the resulting charts more useful. In my first run, one issue was very slow to load, and it turned out to be a bug that complained about ridiculously verbose log messages that included an example. That one was sluggish on Production as well, even with no artificial load, because Jira was struggling to do syntax highlighting and emoji detection on it.

Login once, browse many 

  val browser1 = scenario("IssueBrowser1")
    .exec(Login.login, Browser.browse)

Run N JQL searches

resources/jql.tsv contains a large set of JQL queries. Since I’m loading it with .shuffle to guarantee that all of my queries get run exactly once, I need to set my repetition and scaling parameters carefully, or the simulation will terminate when it runs out of data.

Label	JQL
Simple1	text ~ "fribble" order by priority DESC,updated DESC
...

The reason I’m using TSV instead of the CSV parser is because JQL queries can contain commas and double-quotes that I’d have to escape. Note that Gatling automatically handles the URL encoding for %20, %22, %2C, etc, but only in actual requests; attempting to use it as the label argument to http() breaks the log parser, producing empty graphs.

  val getJQL = tsv("jql.tsv").shuffle
  object Searcher {
    val search = repeat(10, "n") {
      feed(getJQL)
      .pause(ThreadLocalRandom.current.nextInt(2,6))
      .exec(http("Search ${Label}")
        .get("/issues/?jql=${JQL}")
        .headers(headers))
    }
  }

Login once, search many

Reusing my Login object in a different scenario. One simple enhancement to this script would be to make the Browser object repeat only a few times, and then alternate between browsing and searching several times; this is trivial to do by adding additional comma-separated calls to the .exec().

  val searcher1 = scenario("IssueSearcher1")
    .exec(Login.login, Searcher.search)

Just Add Users

The most basic test is to run a single scenario once with a single user:

  setUp(
    browser1.inject(atOnceUsers(1))
  ).protocols(httpProtocol)

Once you know it works, you can replace that with something that ramps up from a single browsing session to N, and starts running searches somewhere in the middle:

  setUp(
    browser1.inject(rampUsers(20) during (30 seconds)),
    searcher1.inject(nothingFor(10 seconds), atOnceUsers(5))
  ).protocols(httpProtocol)

Note the use of nothingFor() so that the searches don’t start until a fair number of browsers are already running.

And when you really want to load it up:

  setUp(
    browser1.inject(
      incrementUsersPerSec(5)
        .times(5)
        .eachLevelLasting(10 seconds)
        .startingFrom(5)
    )
  ).protocols(httpProtocol)

(this scales up fast, and can easily overload your server if you don’t do the math; the total number of planned sessions will be printed at the start of the run, so you can quickly Control-C out of it)

Note that each method can be referenced only once in .inject(), so if you wanted to run additional copies of the exact same test with different scaling parameters, you’d need to create a “browser2” object, etc.

Hey, remember that class definition? 

}

Wrap-up

There are a lot of things that could be added. You could parse the retrieved page to run checks and populate variables to be used in subsequent requests. You could add more complex actions using the REST interface and files containing JSON data.

But at least now you have a complete working example to start from.

Notes

  • When you run Gatling, it will compile every test it finds in any directory under user-files/simulations/, so nuke anything you’re not actively using, or store your tests and resource files somewhere else and point to them with the -sf and -rsf options, respectively.

  • Restart your browser before trying to view the results of a large test run; all the graphs are generated with Javascript, so you might end up load-testing Chrome.

  • You can test the REST interface for Jira as well, which is easier for constructing tests that inject data, but I wanted something read-only that could freely be run against any instance, even Production, without making any changes.

  • If you want to merge the output of multiple runs (for instance, manually testing each node in a cluster, testing from multiple hosts simultaneously, etc), you can run the simulations with -nr, then collect all the simulation.log files in one directory and generate the reports with -ro $logdir.

  • There’s a Git plugin for Gatling, if you want to thrash your repos or test your CI environment.

Comments via Isso

Markdown formatting and simple HTML accepted.

Sometimes you have to double-click to enter text in the form (interaction between Isso and Bootstrap?). Tab is more reliable.