Visualize test methods running in TestNG with listeners

TestNG A certain "Where is this?"

When I run a TestNG test normally on the console, it doesn't give me any information about which test method is being executed. If all the tests run smoothly, that's fine, but

% mvn clean test
[INFO] Scanning for projects...
(Omission)
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running TestSuite

↑ Occasionally, I get stuck with this display, and I am at a loss because I do not know which test is hung.

So, I'll show you how to output on the console which test is being executed by TestNG so that you don't have to worry about it. The image of the log output after the action is like this:

[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running TestSuite
[CustomTestListener] [INFO] Running: com.example.calc.test.CalcTest#testAdd(0, 0, 0)
[CustomTestListener] [INFO] Running: com.example.calc.test.CalcTest#testAdd(0, 1, 1)

In this article, I will introduce the following three methods.

Sample code

The sample code used in this article can be found on GitHub.

Method 1: Put the log directly in the test method

It is a simple and easy method to put the log directly in the test method itself. It's nice if you have a limited number of methods you want to log, but if you have a lot of them, it's a big deal ...

public class CalcTest {
    @Test(dataProvider = "testAdd")
    public void testAdd(int x, int y, int expected) {
        //Output test method information to log
        LOGGER.info(String.format(
        "Running: com.example.calc.test.CalcTest#testAdd(%d, %d, %d)", x, y, expected));

        Calc calc = new Calc();
        assertEquals(calc.add(x, y), expected);
    }
    // ...
}
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running TestSuite
[CalcTest] [INFO] Running: com.example.calc.test.CalcTest#testAdd(0, 0, 0)
[CalcTest] [INFO] Running: com.example.calc.test.CalcTest#testAdd(0, 1, 1)

Method 2: Put the log in the @BeforeMethod method of the test class

Define a method with @BeforeMethod annotation in the test class and put the log in it. It's a reasonable method that is easy to get started and easy to understand because you only have to play with the test class a little.

import org.testng.annotations.BeforeMethod;

public class CalcTest {
    @BeforeMethod
    public void beforeMethod(Method method, Object[] data) {
        //Output test method information to log
        LOGGER.info(String.format(
            "Running: %s#%s(%s)",
            method.getDeclaringClass().getCanonicalName(),
            method.getName(),
            Arrays.stream(data).map(x -> x.toString()).collect(Collectors.joining(", "))));
    }
    // ...
}
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running TestSuite
[CalcTest] [INFO] Running: com.example.calc.test.CalcTest#testAdd(0, 0, 0)
[CalcTest] [INFO] Running: com.example.calc.test.CalcTest#testAdd(0, 1, 1)

Method 3: Add a listener and populate the ʻonTestStart` method with logs

The method is to add a TestNG listener and use the listener's ʻonTestStart` method to prepare the log. Although it takes time to set the listener, it is effective when there are many target test methods because the setting works globally. It's also useful if you can't change the test set (such as running TCK) because you don't have to mess with the test class.

TestNG listeners are prepared as the following classes that implement the ʻITestListener` interface.

import org.testng.ITestContext;
import org.testng.ITestListener;
import org.testng.ITestResult;

public class CustomTestListener implements ITestListener {
    private static final Logger LOGGER = Logger.getLogger(CustomTestListener.class);

    public void onTestStart(ITestResult result) {
        //Output test method information to log
        LOGGER.info(String.format(
            "Running: %s#%s(%s)",
            result.getMethod().getTestClass().getName(),
            result.getMethod().getMethodName(),
            Arrays.stream(result.getParameters()).map(x -> x.toString()).collect(Collectors.joining(", "))));
    }

    public void onTestSuccess(ITestResult result) { }
    public void onTestFailure(ITestResult result) { }
    public void onTestSkipped(ITestResult result) { }
    public void onTestFailedButWithinSuccessPercentage(ITestResult result) { }
    public void onStart(ITestContext context) { }
    public void onFinish(ITestContext context) { }
}

Also, edit testng.xml or specify the @Listeners annotation in the test class to register the listener with TestNG. To register a listener with testng.xml, write as follows.

<suite>
    <listeners>
        <listener class-name="com.example.calc.test.util.CustomTestListener"/>
    </listeners>
   <!-- ... -->
</suite>

To specify the @Listeners annotation in the test class, write as follows.

import org.testng.annotations.Listeners;

@Listeners({ CustomTestListener.class })
public class CalcTest {
    // ...
}

The expected log output looks like this.

[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running TestSuite
[CustomTestListener] [INFO] Running: com.example.calc.test.CalcTest#testAdd(0, 0, 0)
[CustomTestListener] [INFO] Running: com.example.calc.test.CalcTest#testAdd(0, 1, 1)

Summary

I introduced how to visualize the test being executed by TestNG. There are many ways to do this, as introduced in this article, so give it a try on a case-by-case basis!

References

Recommended Posts

Visualize test methods running in TestNG with listeners
Test private methods in JUnit
Test private methods in JUnit
[Java] Test private methods with JUnit
Control test order in Junit4 with enumeration
Test controller with Mock MVC in Spring Boot
Mixin test cases with JUnit 5 and default methods
Setup with initial test data inserted in Db2 / DB container
Compile with Java 6 and test with Java 11 while running Maven on Java 8
In Ruby, methods with? Do not always return true/false.