What are the rules in JUnit?

What is a rule?

A JUnit extension framework added in JUnit 4.7. JUnit has various mechanisms for extracting common processes for fixture setup, Rules are a mechanism that makes it easier and more flexible to extract common processes.

The features are as follows.

--Common processing can be defined as an independent class --Access to metadata when running tests

Due to the above characteristics, the rules are easy to reuse and can be written declaratively. It can be said that it is a mechanism that can be easily extended at the time of test execution.

Rule implementation

Promise of rules (confusing)

1. Must be an implementation class of the org.junit.rules.TestRule interface
2. Add org.junit.Rule annotation or org.junit.ClassRule Define it as a public field
3. Create an instance at the same time as declaring the field Or create an instance with the constructor

How the rules work

The timing when the rule is applied differs between the Rule annotation and the ClassRule annotation. The Rule annotation is applied every time the test method is executed (same as the Before annotation). The ClassRule annotation is applied to each test class (same as the BeforeClass annotation).

Declaration of multiple rules

Multiple rules can be defined in one test class. However, if RuleChain (described later) is not used, the execution order cannot be controlled and will be random.

Rule implementation example

RuleExampleTest.java

public class RuleExampleTest {
    @Rule
    public Timeout timeout = new TimeOut(100);
    @ClassRule
    public TestName testName = new TestName();

    @Test
public void Tests that can take a long time to run() throws Exception {
        doLongTask();
    }
}

Rules provided by JUnit

TemporaryFolder: Create and release temporary folders

Note: A temporary folder is a folder that temporarily stores necessary files.

Usually, when running tests that deal with file systems, every test Maintain test independence by creating folders in pre-processing and deleting them in post-processing. If you define the org.junit.rules.TemporaryFolder class as a rule, every time the rule is executed Since folders are created and deleted, there is no need to implement these processes.

public class TemporaryFolderExampleTest {
    @Rule
    public TemporaryFolder tmpFolder = new TemporaryFolder();

    @Test
public void mkFiles creates two files() throws Exception {
        File folder = tmpFolder.getRoot();
        TemporaryFolderExample.mkDefaultFiles(folder);
        String[] actualFiles = folder.list();
        Arrays.sort(actualFiles);
        assertThat(actualFiles.length, is(2));
        assertThat(actualFiles[0], is("UnitTest"));
        assertThat(actualFiles[1], is("readme.txt")); 
    }
}

TemporaryFolderExample.java

public class TemporaryFolderExample {

	public static void mkDefaultFiles(File folder) {
		String rootPath = folder.getPath();
		File file = new File(rootPath + "/UnitTest");
		File file2 = new File(rootPath + "/readme.txt");
		try {
			file.createNewFile();
			file2.createNewFile();
		} catch (IOException e) {
			e.printStackTrace();
		}
		
	}

}

Note: Extension of Temporary Folder

The TemporaryFolder class before extension was set to Java system property "java.io.tmpdir" It only prepares and releases resources directly under the folder, but you can extend the process by creating a subclass.

TemporaryFolder is a subclass of org.junit.rules.ExternalResource, so Overrides before and after methods. Since they are called before and after the test is executed, the processing to be performed at that time can be expanded.

The following creates a subclass directly under the temporary folder before execution, An example of extending the folder name created after execution to output to the console.

SubTemporaryFolder.java

public class SubTemporaryFolder extends TemporaryFolder {

	//Subfolder
	public File srcFolder;
	public File testFolder;
	
	@Override
	protected void before() throws Throwable {
		//Temporary Folder before method(Mandatory)
		super.before();
		srcFolder = newFolder("src");
		testFolder = newFolder("test");
	}
	
	@Override
	protected void after() {
		//TemporaryFolder after method(Mandatory)
		super.after();
		System.out.println(srcFolder.getName());
		System.out.println(testFolder.getName());
	}
}

SubTemporaryFolderExampleTest.java

public class SubTemporaryFolderExampleTest {
	   @Rule
	    public SubTemporaryFolder tmpFolder = new SubTemporaryFolder();

	    @Test
public void mkFiles creates two files() throws Exception {
	        File folder = tmpFolder.getRoot();
	        File srcFolder = tmpFolder.srcFolder;
	        File testFolder = tmpFolder.testFolder;
	        assertThat(folder.list().length, is(2));
	        assertThat(srcFolder.list().length, is(0));
	        assertThat(testFolder.list().length, is(0));
	    }
}

ExternalResource: Base class when dealing with external resources

The org.junit.rules.ExternalResource class A rule that "prepares resources before running a test" and "releases resources after running a test". Since the ExternalResource class is an abstract class, implement a subclass when using it.

In the example we defined the subclass as a nested class of test classes, If this is defined as an independent class, it can be used by multiple test classes.

python

public class ExternalResourceExampleTest() {
	
	@Rule
	public ServerResource resource = new ServerResource();

	static class ServerResource extends ExternalResource {
		ServerSocket server;
		
		@Override
		protected void before() throws Throwable {
			server = new ServerSocket(8080);
			server.accept();
		}
		
		@Override
		protected void after() {
			try {
				server.close();
			} catch (IOException e) {
				e.printStackTrace();
			}
		}
	}
	@Test
	public void test() {
		fail("Not yet implemented");
	}
}

Verifier: Verify post-test conditions

The org.junit.rules.Verifier class is a rule that validates post-test conditions. Like the ExternalResource class, it's an abstract class, so create a subclass and Override and use the verify method.

After annotation and AfterClass annotation were added to verify the post-condition of the test. Since it is also possible to do it with a post-processing method, the verification content is more complicated and I want to do it across multiple classes In that case, it is convenient to use the Verifier class properly.

• When used across multiple classes, implement as an independent class instead of a nested class.

The execution order is as follows. Before annotation → Test annotation → After annotation → verify method

public class VerifierExampleTest {
    //Implemented as a disposable anonymous class
    @Rule
    public Verifier verifier = new Verifier() {
        protected void verify() throws Throwable {
            assertThat("value should be 0.", sut.value, is(0));
        }
    };
    VerifierExample sut;

    @Before
    public void setUp() throws Exception {
        sut = new VerifierExample();
    }

    @After
    public void tearDown() throws Exception {
        // do nothing.
    }

    @Test
Initialize the calculation result to 0 with the public void clear method() throws Exception {
        sut.set(0);
        sut.add(140);
        sut.minus(5);
        assertThat(sut.getValue(), is(135));
        sut.clear();
    }
}

VerifierExample.java

public class VerifierExample {

	protected Integer value;

	public void set(int i) {
		value = i;
	}

	public void add(int i) {
		value += i;
	}

	public void minus(int i) {
		value -= i;
	}

	public Integer getValue() {
		return value;
	}

	public void clear() {
		value = 0;
	}
}

ErrorCollector: Customize handling of exceptions during testing

Normally, the processing of the test method in JUnit ends when an error occurs. However, if you use the org.junit.rules.ErrorCollector class after running the test to the end, Outputs information about errors collectively.

public class ItemInfoTest {
	
	@Rule
	public ErrorCollector errCollector = new ErrorCollector();

	@Test
public void Verification when applying rules() throws Exception {
		ItemInfo itemInfo = new ItemInfo();
		errCollector.checkThat(itemInfo, is(nullValue())); /*Where the error occurred*/
		errCollector.checkThat(itemInfo.getId(), is(""));
		errCollector.checkThat(itemInfo.getName(), is(not(""))); /*Where the error occurred*/
		errCollector.checkThat(itemInfo.getStockNum(), is(0));
	}
	
	@Test
public void Normal verification() throws Exception {
		ItemInfo itemInfo = new ItemInfo();
		assertThat(itemInfo, is(nullValue())); /*Where the error occurred*/
		assertThat(itemInfo.getId(), is(""));
		assertThat(itemInfo.getName(), is(not(""))); /*Where the error occurred*/
		assertThat(itemInfo.getStockNum(), is(0));
	}

}

ItemInfo.java

public class ItemInfo {
	
	private String id;
	
	private String name;
	
	private int stockNum;
	
	public ItemInfo() {
		this.id = "";
		this.name = "";
		this.stockNum = 0;
	}

/*Below, each member's Getter and Setter method*/

}

Error report

~ Verification when applying rules ~

Expected: is null
     but: was <com.example.junit.rules.errcollect.ItemInfo@506c589e>

java.lang.AssertionError: 
Expected: is not ""
     but: was ""

~ Normal verification ~

Expected: is null
     but: was <com.example.junit.rules.errcollect.ItemInfo@1698c449>

ExpectedException: Examine the exception in detail

The org.junit.rules.ExpectedException class validates the thrown exception It is a rule that provides a mechanism to do it simply. In order to compare the implementation method, the case where the rule is not used and the case where the rule is used are compared below.

public class ExpectExceptionExampleTest {
	
	@Rule
	public ExpectedException exException = ExpectedException.none();
	
	@Test
public void Validate exception messages using rules() throws Exception {
		exException.expect(ArithmeticException.class);
		exException.expectMessage(containsString("by zero"));
		int result = 1 / 0;
		System.out.println(result);
	}

	@Test
public void Validate exception messages in a standard way() throws Exception {
		try {
			int result = 1 / 0;
			System.out.println(result);
			//Detect when no exception occurs
			fail("No exception");
		} catch (ArithmeticException e) {
			assertThat(e.getMessage(), is(containsString("by zero")));
		}
	}
}

Timeout: Control the timeout

A rule that allows you to set the time until the timeout as you read it. You can also set the timeout by passing a value to the timeout attribute of the Test annotation, When setting common to all test methods in the test class The description can be combined in one place by using the Timeout class.

public class TimeoutExampleTest {
	@Rule
	public Timeout timeout = new Timeout(1000);

	@Test
	public void sleep1() throws InterruptedException {
		while(true) {
		}
	}
	
	@Test
	public void sleep2() throws InterruptedException {
		while(true) {
		}
	}
}

TestWatcher: Record test execution time

The org.junit.rules.TestWatcher class can be used in conjunction with logs It is a rule that can monitor (follow) the execution status of the test. Since TestWatcher is an abstract class, use it by overriding the method in the subclass.

The execution order of TestWatcher when the test succeeds and fails is as follows.

[At the time of success] starting → succeeded → finished

[At the time of failure] starting → failed → finished

public class TestWatcherExampleTest {
	
	@Rule
	public TestWatcher testWatcher = new TestWatcher() {
		
		@Override
		protected void starting(Description desc) {
                    Logger.getAnonymousLogger()
                        .info("start: " + desc.getMethodName());
		}
		
		@Override
		protected void succeeded(Description desc) {
                    Logger.getAnonymousLogger()
                        .info("succeeded: " + desc.getMethodName());
		}
		
		@Override
		protected void failed(Throwable e, Description desc) {
                    Logger.getAnonymousLogger()
                        .log(Level.WARNING, "failed: " + desc.getMethodName(), e);
		}
		
		@Override
		protected void finished(Description desc) {
			Logger.getAnonymousLogger()
                    .info("finished: " + desc.getMethodName());
		}
	};

	@Test
public void Successful test() throws Exception {
	}
	
	@Test
public void test that fails() throws Exception {
		fail("NG");
	}
}

Log output

4 23, 2020 7:11:08 pm com.example.junit.rules.testwatcher.TestWatcherExampleTest$1 starting
information: start:Tests that fail
4 23, 2020 7:11:08 pm com.example.junit.rules.testwatcher.TestWatcherExampleTest$1 failed
warning: failed:Tests that fail
java.lang.AssertionError: NG
	at org.junit.Assert.fail(Assert.java:88)
	at com.example.junit.rules.testwatcher.TestWatcherExampleTest.Tests that fail(TestWatcherExampleTest.java:45)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)

4 23, 2020 7:11:08 pm com.example.junit.rules.testwatcher.TestWatcherExampleTest$1 finished
information: finished:Tests that fail
4 23, 2020 7:11:08 pm com.example.junit.rules.testwatcher.TestWatcherExampleTest$1 starting
information: start:Successful test
4 23, 2020 7:11:08 pm com.example.junit.rules.testwatcher.TestWatcherExampleTest$1 succeeded
information: succeeded:Successful test
4 23, 2020 7:11:08 pm com.example.junit.rules.testwatcher.TestWatcherExampleTest$1 finished
information: finished:Successful test

TestName: Refers to the name of the running test method

The org.junit.rules.TestName class is A rule to get the name of the method running in the test method. By being able to get the method name in the test method, the method name and the external resource There are various ways to use it, such as getting the file name linked.

public class TestNameExampleTest {
    @Rule
    public TestName testName = new TestName();

    @Test
public void test method name() throws Exception {
        fail(testName.getMethodName() + " is unimplements yet.");
    }
}

Create custom rules

To create a custom rule

1. Create a subclass of an abstract class that is already implemented (for example, the ExternalResource class)
2. Create a class that implements the TestRule interface

There are two methods.

TestRule interface

The following methods are defined in the org.junit.rules.TestRule interface.

Statement apply(Statement base, Description description);

Statement object

The org.junit.runners.model.Statement class is the class that controls the execution of tests. The test is executed when the Statement.evaluate method is called. The Statement object passed as an argument to the apply method is in the evaluate method. The tests are set up to run in the following order:

1. Create an instance of the test class
2. Execution of method with Before annotation (pre-processing)
3. Execution of test method
4. Execution of method with After annotation (post-processing)

Description object

The org.junit.runner.Description class is a class that holds test case meta information. You can get information such as test class, test method name and added annotation.

Implementation template

A common implementation of a rule is in the Statement object of the argument. Create a proxy object (proxy object) and return it as a return value.

public abstract class custom rule name implements TestRule{

	@Override
	public Statement apply(Statement base, Description description) {
		return new Statement() {
			@Override
			public void evaluate() throws Throwable {
				//You can call your own pre-processing as pre- and post-processing,
				customedBySubClazz();
				//Get test meta information and conditional branch,
				if (description.getMethodName().equals("Applicable test method name")) {
					//Depending on the judgment result, the original evaluate method may or may not be executed.
					base.evaluate();
				} else {
					fail("The test method is different");
				}

			}
		};
	}
	//Abstract method for implementing your own processing when defining a subclass
	protected abstract void customedBySubClazz() throws Throwable;
}

Custom rules to check preconditions

Create a custom rule to check the test preconditions. The difference from the precondition check by the method with Before annotation is Extensions can be provided more independently of the test class. Easy access to meta information. There is such a thing. For implementation, refer to the implementation of Verifier class.

Custom rule class

public abstract class PreProcess implements TestRule {

	@Override
	public Statement apply(Statement base, Description description) {
		return new Statement() {
			@Override
			public void evaluate() throws Throwable {
				//Proprietary pre-processing
				verify();
				//Original evaluate method
				base.evaluate();
			}
		};
	}

	//Implement your own preprocessing when defining a subclass
	protected abstract void verify() throws Throwable;
}

Test class with PreProcess applied to the rule

public class PreProcessExampleTest {
	
	@Rule
	public PreProcess preProcess = new PreProcess() {
		@Override
		protected void verify() throws Throwable {
			LocalDate now = LocalDate.now();
			LocalDate limitDate = LocalDate.of(2015, 4, 1);
			if (now.isAfter(limitDate)) {
				throw new AssertionError();
			}
		}
	};

	@Test
public void Tests that do not run() {
		assertThat(1, is(1));
	}
}

Custom rules for OS-dependent testing

Create a custom rule for OS-dependent testing. In evaluate the proxy object, determine the OS of the execution environment and Only when it matches the assumed OS (OS specified by the RunOn annotation of the original implementation) Call the original evaluate method.

Create your own annotation to set the OS to run the test

For details on the annotation related to the meta information used in creating the annotation, see See Articles posted on TECHSCORE.

Roughly speaking

@Retention(RetentionPolicy.RUNTIME)→ You can also refer to it when executing the JUnit test

#### **`@Target({ElementType.METHOD})→ It is an annotation given to the method`**
```METHOD})→ It is an annotation given to the method

 Feeling like that.


#### **`Proprietary implementation annotation`**
```java

@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.METHOD})
public @interface RunOn {
	public enum OS {
		WINDOWS, MAC, LINUX
	}
	public OS value();
}

Implementation of custom rules

In the custom rule, with the OS obtained from the RunOn annotation Judge whether the OS of the execution environment obtained from the Java system properties matches, and If they match, call the original evaluate method and Implement the process of doing nothing if they do not match.

public class OSDepend implements TestRule {
	
	@Override
	public Statement apply(Statement base, Description desc) {
		return new Statement() {
			@Override
			public void evaluate() throws Throwable {
				//Get the annotation given to the method from Description
				RunOn env = desc.getAnnotation(RunOn.class);
				//Determine if the OS specified in the annotation matches the OS of the execution environment
				if (env == null || canEvaluate(env.value())) {
					base.evaluate();
				} else {
					// don't evaluate
				}
			}
			
			private boolean canEvaluate(OS os) {
				/*Get the OS of the execution environment from the system properties
Returns true if it matches the OS specified in the RunOn annotation*/
				String osName = System.getProperty("os.name");
				if (osName == null) {
					return false;
				}
				if (os == OS.WINDOWS && osName.startsWith("Windows")) {
					return true;
				}
				if (os == OS.MAC && osName.startsWith("Mac OS X")) {
					return true;
				}
				if (os == OS.LINUX && osName.startsWith("Linux")) {
					return true;
				}
				return false;
			}
		};
	}
}

Test class for OS-dependent testing

If you run the following test class, two tests will be performed, Only Mac has console output (because my environment is Mac) You can see that the test cases in the Windows environment have not been executed.

public class OSDependExampleTest {
	@Rule
	public OSDepend osDepend = new OSDepend();

	@Test
	//Annotation that defines Enum to judge OS
	@RunOn(OS.WINDOWS)
public void Tests that run only in a Windows environment() {
		System.out.println("test: onlyWindows");
		assertThat(File.separator, is("¥¥"));
	}

	@Test
	@RunOn(OS.MAC)
public void Tests run only in Mac environment() {
		System.out.println("test: onlyMac");
		assertThat(File.separator, is("/"));
	}
}

↓ JUnit execution result

Console output

test: onlyMac

Controlling the order of rule execution by RuleChain

When controlling external resources using rules, there are cases where it is necessary to specify the execution order. For example, if there is a test case that links the AP server and DB server, The execution order must be DB server startup → AP server startup.

In this case, use the org.junit.rules.RuleChain class to You can control the execution order in the form of ** chaining rule executions **. (Chain of Responsibility pattern: a type of design pattern)

Implementation example of RuleChain

The following methods are implemented in the RuleChain class.

static rulechain outerrule(testrule outerrule) Specify a rule that initializes first and finishes afterwards

rulechain around(testrule enclosedrule) Specify a rule that initializes later and finishes first

If it is a letter, it did not come in quickly, so it is illustrated.

public class RuleChainExampleTest {
    @Rule
    public RuleChain ruleChain = RuleChain
                                    .outerRule(new PreRule())
                                    .around(new PostRule());

    @Test
public void test() throws Exception {
    }
}

class PreRule extends ExternalResource {
    @Override
    protected void before() throws Throwable {
        System.out.println("1 start: PreRule");
    }

    @Override
    protected void after()  {
        System.out.println("4 finish: PreRule");
    }
}

class PostRule extends ExternalResource {
    @Override
    protected void before() throws Throwable {
        System.out.println("2 start: PostRule");
    }

    @Override
    protected void after()  {
        System.out.println("3 finish: PostRule");
    }
}

Console output

1 start: PreRule
2 start: PostRule
3 finish: PostRule
4 finish: PreRule

Application of RuleChain by ClassRule

Since starting and stopping the server is a heavy process, slow test problems occur as the number of test cases increases. Therefore, if you do not need special control at startup and termination, use the ClassRule annotation and use it. It is possible to save resources by starting and stopping the server for each test class.

References

This article was written with reference to the following information.

-[Introduction to JUnit Practice ── Systematic Unit Testing Techniques](https://www.amazon.co.jp/JUnit%E5%AE%9F%E8%B7%B5%E5%85%A5%E9 % 96% 80-% E4% BD% 93% E7% B3% BB% E7% 9A% 84% E3% 81% AB% E5% AD% A6% E3% 81% B6% E3% 83% A6% E3% 83% 8B% E3% 83% 83% E3% 83% 88% E3% 83% 86% E3% 82% B9% E3% 83% 88% E3% 81% AE% E6% 8A% 80% E6% B3% 95-WEB-PRESS-plus-ebook / dp / B07JL78S95)