I learnt about Artillery.IO a few months ago but never used it. For a while, my team utilized this for quick performance appraisals. Because the current sprint was scheduled to feature some performance enhancements, I had to assist by obtaining some baseline measurements. I started working with Artillery.io, and here are my early thoughts on the tool.
GitHub sample project can be found here
What is Artillery.io?
It is an open-source, user-friendly performance testing toolbox. Use it to build scalable apps that remain responsive and robust even under heavy load. It adheres to the “batteries-included” principle and puts developer productivity and satisfaction as a top priority.This tool can be used for both performance tests and production level functional checks like Synthetic tests, Semantic tests, etc.
Services supported by Artillery are
- API
- Backend
- Databases
- Chat system
Initial Set-up
Artillery is supported on Node.js, which makes the installation journey a little easy. If your team is already working on Node this step would be a cakewalk as you may have to just install an npm package.
npm install -g artillery@latestThis command installs the latest stable version of Artillery onto your system. You can also install this on Docker, just ensure you are not installing it at the root user.
Test File
Writing your first test?
Get to know some basic details
Test files are in “.yml” format, and test steps should be written using YAML commands. The whole test file is divided into 2 main sections,
File structure
Configuration – a place where you define all prerequisites and settings required to execute that test file. Example: target URL, test environments, payloads, plugins, etc. This section is initiated using keyword “config:”
Scenarios – a place where you define the test steps in detail. You can also access the response body using “capture” in order to verify response details. This section is initiated through keyword “scenarios:”
Phases
Artillery creates a virtual user, “phases” is the section where you can draw up the backdrop of your test. Questions like
- How many users?
- What duration?
- At what interval? etc can be filled up
Payload
Artillery has a provision to pass dynamic data for the test scripts through a .csv file. This file should be defined under the “config” section. There are 2 ways in which the payload can be utilised by virtual user
- All Virtual user use every row data defined in csv file
- Each virtual user gets data from just one row
- Some payload options provided are
- order : Control how rows are selected for each virtual user
- skipHeader: To skip the first row in the file
- delimiter: define any delimiter other than the default one
- skipEmptyLines: skips the empty rows.
Learn more about test scripts here
Execution
Executing artillery scripts is pretty straightforward, you can run CLI command to trigger the test.
npm run artillery --run <testfile_destination>
eg: npm run artillery -- run tests/name-api/test.ymlReports
Artillery provides plugins like “metrics-by-endpoint” to capture the metrics for each endpoint. The metrics are defined in percentile which helps find clear outliers. In the config section, define the plugin’s key with value metrics-by-endpoint. However, now this command is provided by default and may not require any explicit mentioning.
Execute: artillery -- run --output report.json <testfile>
Generate HTML: artillery report report.json 
Authorisation
Authorization can be settled by creating a new config.csv file in your project structure and provide the basic authentication details .
example: your_test_URL/token,username,password
Pricing
Artillery is an open source tool however the “Dev” version of the product is free, but there are other versions with loaded features which comes at a cost. More details can be found here.
Alternatives for Artillery.io
Since this is a first impressions post, I refrain from diving deeper into topics right away. I am planning to continue to write on deeper technical details as I continue to explore this tool, so stay tuned.
| Parameters | My score (Scale 10) |
|---|---|
| Ease of Installation | 9 |
| Ease of Understanding | 7 |
| Clarity in Documentation | 6 |
| Ease of execution | 8 |
| Report Generation | 7 |
| Integration with CI | 8 |
| Serverless Load runs | 9 |
someooow 
Leave a Reply