Use Allure to generate language- and framework-independent test reports!
Overview
Introducing Allure based on the translation of the official document.
- There is no doubt that it does not depend on the language or framework, but in this article, we will use
RSpec(Ruby) for convenience.
About Allure
Speaking of test frameworks, there are many great tools available in various languages, such as JUnit`` RSpec jest`` pytest.
However, many tools do not have the ability to generate visually stunning test reports.
Allure supplements it with OSS to generate test execution reports that are easy for everyone on the team to understand, independent of language and framework. is.
demo
Seeing is believing. Let's open the Demo Report (https://demo.qameta.io/allure/).
You can see that the test execution report can be viewed graphically and interactively from many perspectives.
Again, the language and framework are irrelevant to these generations. You can use the test code currently in operation as it is from JUnit, RSpec, jest, or pytest.
How to make a report
Allure is based on the output of the general xUnit (a generic term for unit testing frameworks) plus supplementary data.
The test report is generated in the following two steps.
** 1. Test execution **
When various test frameworks execute tests, a lightweight library called adapter generates information about the executed tests in XML or JSON format.
The adapter is officially provided for Java, PHP, Ruby, Python, Scala, C # and supports each major testing framework.
** 2. Report generation **
After the test is completed, an HTML-formatted report will be generated based on the generated data.
Report generation can be done from the CLI tool.
Usage examples by various frameworks
Execution example with RSpec (Ruby)
For convenience, this article uses RSpec as an example, but many languages and frameworks can follow similar steps.
CLI installation
Install ʻallure` itself, a CLI tool for generating reports. Since we are using a Mac this time, install it with brew.
$ brew install allure
It is OK if you can confirm the version.
$ allure --version
2.13.0
$ allure --version
No Java runtime present, requesting install.
If you get something like that, you need to include the Java runtime.
Preparation of test code
This time, we will prepare a unit test that verifies the behavior of the Calculator class as follows.
The Calculator class is a class that can add and subtract based on the initial value. Only one is intentionally set so that the test fails.
describe Calculator do
subject { described_class.new(initial_value) }
describe '#add addition' do
context 'When the initial value is 0' do
let(:initial_value) { 0 }
it 'Add 10 to get 10 back' do
expect(subject.add(10)).to eq 10
end
end
context 'When the initial value is 5' do
let(:initial_value) { 5 }
it 'Add 10 to get 15 back' do
expect(subject.add(10)).to eq 15
end
end
end
describe '#sub subtraction' do
context 'When the initial value is 0' do
let(:initial_value) { 0 }
#Fall test
it 'Subtract 10 to return 0' do
expect(subject.sub(10)).to eq 0
end
end
context 'When the initial value is 30' do
let(:initial_value) { 30 }
it 'Subtract 10 to return 20' do
expect(subject.sub(10)).to eq 20
end
end
end
end
When rpsec is executed normally, it is as follows
$ bundle exec rspec spec/sample_spec.rb -f d
Calculator
#add addition
When the initial value is 0
Add 10 to get 10 back
When the initial value is 5
Add 10 to get 15 back
#sub subtraction
When the initial value is 0
Subtract 10 to return 0(FAILED - 1)
When the initial value is 30
Subtract 10 to return 20
Failures:
1) Calculator#sub subtraction When the initial value is 0, subtracting 10 returns 0.
Failure/Error: expect(subject.sub(10)).to eq 0
expected: 0
got: -10
(compared using ==)
# ./sample_spec.rb:42:in `block (4 levels) in <top (required)>'
Finished in 0.11199 seconds (files took 2.06 seconds to load)
4 examples, 1 failure
Failed examples:
rspec ./sample_spec.rb:41 # Calculator#sub subtraction When the initial value is 0, subtracting 10 returns 0.
Generate report data using an adapter
When running Rspec, use the Allure adapter for Ruby to generate data for report creation.
The adapter for RSpec uses allure-rspec.
This time, add the following to the Gemfile and install it.
gem "allure-rspec"
Load the adapter with spec_helper
require "allure-rspec"
Specify the adapter as the formatter at run time. This time nothing is output to standard output.
$ bundle exec rspec spec/sample_spec.rb -f AllureRspecFormatter
By default, the reports directory is created and report data is generated with the following structure.
$ tree reports/
reports/
└── allure-results
├── 4ee53de0-790a-0138-ab5e-3d13be3ed9ba-container.json
├── 4ee54750-790a-0138-ab5e-3d13be3ed9ba-container.json
├── 4ee54a80-790a-0138-ab5e-3d13be3ed9ba-container.json
├── 4ee64530-790a-0138-ab5e-3d13be3ed9ba-result.json
├── 4ee7fad0-790a-0138-ab5e-3d13be3ed9ba-container.json
├── 4ee80710-790a-0138-ab5e-3d13be3ed9ba-result.json
├── 4ee83ae0-790a-0138-ab5e-3d13be3ed9ba-container.json
├── 4ee83e90-790a-0138-ab5e-3d13be3ed9ba-container.json
├── 4ee84860-790a-0138-ab5e-3d13be3ed9ba-result.json
├── 4eefe9b0-790a-0138-ab5e-3d13be3ed9ba-container.json
└── 4eeff800-790a-0138-ab5e-3d13be3ed9ba-result.json
1 directory, 11 files
Generate a report
Use the report data output in the previous section to generate a report with the ʻallure` command.
The serve subcommand allows you to generate a report locally and access it in your browser.
$ allure serve -h 0.0.0.0 ./reports/allure-results/
Generating report to temp directory...
Report successfully generated to /var/folders/3_/r5h56zzn1130yr7vkpjftqpw0000gp/T/15980098365829452874/allure-report
Starting web server...
2020-05-16 04:02:58.509:INFO::main: Logging initialized @3541ms to org.eclipse.jetty.util.log.StdErrLog
Server started at <http://0.0.0.0:57717/>. Press <Ctrl+C> to exit
In the above example, when you access 0.0.0.0:57717, you can see that the report is generated successfully.
Impressions
As the title of this article suggests, it is very versatile and versatile because it does not depend on the language or framework, so I had the impression that it would be Too much for individual use.
It may be a good idea to design a test assuming Allure from the beginning, but it seems difficult to put existing test code unless it is well structured.
Recommended Posts