Automated testing is an essential component in software development, ensuring the quality and reliability of the software. The Allure Report is a tool that has significantly enhanced the control and management of automated testing. This blog post will provide an in-depth exploration of the Allure Report, including its installation process, benefits, and compatibility with various testing frameworks.

What is an Allure Report?

Allure Report is an open-source framework designed to create test execution reports that are clear, concise, and attractive. It provides a visual and user-friendly interface that allows developers and testers to obtain detailed insights into what is happening in their tests. With its unique features, Allure Report has become a popular choice among developers for reporting in automated testing.

Benefits

1. Detailed Test Analysis: Allure Report provides a comprehensive analysis of test execution, including test steps, start time, duration, and status. This level of detail allows you to understand exactly what happened during each test, making it easier to identify any issues or areas for improvement.
           
2. Visual Appeal: Allure Report presents test data in a visually appealing and intuitive manner. This includes pie charts, graphs, and timelines that make it easier to understand the test results at a glance. This visual representation can be particularly useful when presenting test results to stakeholders who may not be familiar with the technical aspects of testing.
           
3. Historical Trend Analysis: Allure Report stores historical data, enabling you to track progress and trends over time. This can be particularly useful for identifying patterns and making predictions about future test performance. It also allows you to demonstrate improvements or regressions in test performance over time.
           
4. Facilitates Debugging: Allure Report's detailed error reporting makes it easier to identify and debug issues. It provides stack traces, screenshots, and even video recordings of failed tests, which can be invaluable for troubleshooting. This can significantly reduce the time spent on debugging and help ensure issues are resolved more quickly.
           
5. Seamless Integration: Allure Report can be integrated with a wide range of testing frameworks and Continuous Integration (CI) tools like Jenkins, Bamboo, and TeamCity. This makes it a versatile tool in a DevOps environment, facilitating continuous testing and delivery.
           
6. Customizable: Allure Report allows for customization of the report's appearance and content. This means you can tailor the reports to meet your specific needs and preferences. For example, you can add custom labels and categories, or include additional information such as logs or attachments.
           
7. Collaboration: Allure reports can be shared and viewed by anyone, making it easier for teams to collaborate on testing. This can be particularly useful in remote or distributed teams, where team members may not be able to discuss test results in person.
           
8. Test Case Management: Allure allows you to link your test cases with corresponding issues in your issue tracker, or with requirements in your test management system. This makes it easier to manage your test cases and ensure that all requirements are being adequately tested.

Allure Report is a powerful tool that can enhance your testing process in numerous ways. From detailed test analysis and visual reporting to seamless integration and customization, Allure Report offers a range of benefits that can help you improve the efficiency and effectiveness of your testing.

How to Install Allure Report in Your Project

Installing Allure Report in a project is a simple process. Here are the basic steps needed to install and generate reports. Keep in mind that you need to adjust the configurations for different projects

1. First, you need to download and install Allure Commandline. This can be done using npm (Node Package Manager) with the following command: npm install -g allure-commandline --save-dev.
           
2. Once installed, you can generate Allure Reports using the command: allure generate [allure_output_dir] && allure open. Replace [allure_output_dir] with the directory where your Allure results are stored.

Sample general steps for Configuring Allure Reports in Google Cloud Platform (GCP)

While there isn't a direct guide available on the Google Cloud documentation, I can provide a general outline of the steps you would need to follow:

1. Create a Google Cloud Project: You'll first need to create a new project in the Google Cloud Console. This will provide you with a project ID, which you'll use in subsequent steps.
           
2. Set up a Compute Engine Instance: You'll need a server to host your Allure Reports. You can do this by setting up a Compute        Engine instance in your Google Cloud project. You'll need to select an operating system for your instance - Allure Reports can run on any system that supports Java, so a Linux-based system like Ubuntu would be a good choice.
           
3. Install Java: Allure Reports requires Java, so you'll need to install this on your Compute Engine instance. You can do this by connecting to your instance via SSH and running the appropriate commands to install Java.
           
4. Install Allure Commandline: You'll need to install the Allure Commandline tool on your Compute Engine instance. To do this, download the latest release from the Allure GitHub page and follow the installation instructions.
           
5. Generate Allure Reports: Once Allure is installed, you can generate reports by running your tests with Allure enabled, and then using the Allure Commandline tool to generate a report. The report will be generated as a set of static HTML files.
           
6. Serve Allure Reports: To view your Allure Reports, you'll need to serve the static HTML files over HTTP. You can do this by installing a web server like Apache or Nginx on your Compute Engine instance and configuring it to serve your Allure Report files.
           
7. Access Allure Reports: Once your web server is set up, you can access your Allure Reports by navigating to the external IP address of your Compute Engine instance in a web browser.

Please note that these steps are quite general and the exact commands you'll need to run will depend on the specifics of your setup, such as the operating system you're using and the configuration of your tests. It's also worth noting that while this setup will allow you to view your Allure Reports, it won't automatically update them when new tests are run - you'll need to set up a continuous integration (CI) pipeline to do this. Remember, Allure Report can be integrated into any testing framework, and the installation process may vary slightly depending on the specific framework you are using.

Possibilities for allure on the example of the cloudia project

In the Cloudia project we’ve placed an allure server on Google Kubernetes Engine. By doing that, we reuse already existing infrastructure which enables us to expose the server with test reports easily, secure it with Identity-Aware Proxy(IAP) and make it available via web browser.
The allure pod is also connected to GCS Bucket where we store our test reports. By grouping each test report by typer, date and id, it’s easy to navigate to get the exact test report we are looking for.

Entering the directory of a particular day there are directories with the test of a particular day. It also depends on our configuration and how many tests will be on a particular day.

Main allure interface

Clicking on the selected folder takes you to the allure interface with a report on the individual status of the written tests for the day. In our interface configuration, the main page shows a side menu with many tabs, only some of which will be discussed. In the main area of the interface, you can see the date of the report along with the time it was created, the number of tests and the percentage of tests passed. In addition, you can see our test packages sorted into folders and subfolders, depending on the module being tested. Thanks to the coloring and counting, we have a preview of the status of the tests in each module. There is also a test trend graph, where we can see a preview with the trend of tests by day. In addition, a helpful widget in the interface is categories, where tests that have a defect are visible.

In this blog post, we will focus on two tabs: Categories, where we can analyze all the defects in our automated tests, and a tab suites, where you can find all the e2e tests we wrote in the playwright framework

Categories tab

This tab contains all the defects in our tests. Opening any defect displays information about the selected defect in the interface on the right. In the Overview tab, in addition to the defect itself, we can view the path to the file where the problem occurs. In addition, Allure allows us to trace all the steps in the test body. We can watch a video of our test executing the steps we wrote. We can also download the trace of our tests. In addition, it is possible to preview a screenshot of the place where the test failed to execute further.

The history tab allows us to preview the status of a particular test with specific dates and times of launch. In addition, the success rate of a given test is calculated.

The "Retries" tab in Allure reports is used to track and display information about attempts to restart tests that failed the first time.

Suites tab

The "Suites" tab in Allure reports is used to present organized information about the test suites that have been performed. "Suite" is a term used to describe a group of tests that are related and are typically run together.

Under the "Suites" tab, we can see a list of all the test sets that have been run, along with a summary of their results. For each test set, Allure displays the number of tests that passed, that failed, that were skipped, as well as the duration of the entire test set.

When you click on a specific test suite, you can see detailed information about each test in the suite, including a full description of the test, its status (e.g. success, failure, skip), duration, and any error or exception messages that may have occurred during test execution.

The "Suites" tab is very useful for quickly reviewing test results and identifying areas that may require additional attention or analysis.

With tests that have a status of failed, allure proves to be very helpful.

Allure allows us to preview what is wrong with the test, in addition, it creates a video file that shows the playback of the steps of our written test. This allows us to more quickly locate what could be a potential problem.

We can also download the entire trace of our test

Or just see a screenshot of where our test stopped working

The Perks of Using Allure Report

Allure Report offers several benefits that make it a valuable tool for automated testing:

1. Comprehensive Test Analysis: Allure Report provides a detailed analysis of test execution, including test steps, start time,      duration, and status. This allows for a more in-depth understanding of test performance and helps identify areas that may need improvement.
2. Visual Appeal: Allure Report presents test data in a visually appealing and intuitive manner. This includes pie charts, graphs, and timelines that make it easier to understand the test results at a glance.
3. Historical Trend Analysis: Allure Report stores historical data, enabling you to track progress and trends over time. This can be particularly useful for identifying patterns and making predictions about future test performance.
4. Facilitates Debugging: Allure Report's detailed error reporting makes it easier to identify and debug issues. It provides stack traces, screenshots, and even video recordings of failed tests, which can be invaluable for troubleshooting.
5. Seamless Integration: Allure Report can be integrated with Continuous Integration (CI) tools like Jenkins, Bamboo, and TeamCity. This makes it a versatile tool in a DevOps environment, facilitating continuous testing and delivery.
6. Customizable: Allure Report allows for customization of the report's appearance and content. This means you can tailor the reports to meet your specific needs and preferences.

Testing Frameworks Compatible with Allure Report

Allure Report is compatible with a wide range of testing frameworks, making it a versatile tool for any project. Here are some of the popular testing frameworks it can integrate with:

1. Playwright: Playwright is a Node.js library to automate the Chromium, WebKit, and Firefox browsers with a single API. Allure integrates with Playwright via the Jest testing framework. By using the test-allure reporter, you can generate Allure reports from your Playwright tests. This allows you to leverage the power of Playwright's browser automation capabilities with the detailed reporting of Allure.
           
2. PyTest: A mature full-featured Python testing tool. Allure has a PyTest plugin that allows for the generation of Allure reports from PyTest tests.
           
3. TestNG: A testing framework inspired by JUnit and NUnit but introducing some new functionalities. Allure provides a listener for TestNG that can generate Allure reports.
           
4. RSpec: A Behavior Driven Development (BDD) framework for Ruby programming. Allure has an RSpec formatter that can generate Allure reports from RSpec tests.
           
5. Mocha: A feature-rich JavaScript test framework running on Node.js. Allure has a Mocha reporter that can generate Allure reports from Mocha tests.
           
6. NUnit: A unit-testing framework for all .Net languages. Allure provides an NUnit adapter that can generate Allure reports from NUnit tests.
7.  JUnit: A simple framework to write repeatable tests in Java. Allure provides a custom runner that can be used to generate Allure reports from JUnit tests.

Conclusion

Allure Report is a powerful tool that enhances control over automated testing. With its detailed reporting, visual representation, and compatibility with various testing frameworks, it is an invaluable asset in any software development project. By integrating Allure Report into your project, you can ensure a smoother, more efficient testing process, leading to higher quality software.