1. Overview
By default, JUnit runs tests using a deterministic but unpredictable order (MethodSorters.DEFAULT).
In most cases, that behavior is perfectly fine and acceptable. But there are cases when we need to enforce a specific ordering.
2. Test Order in JUnit 5
In JUnit 5, we can use @TestMethodOrder to control the execution order of tests.
We can use our own MethodOrderer, as we’ll see later.
Or we can select one of three built-in orderers:
- Alphanumeric Order
- @Order Annotation
- Random Order
2.1. Using Alphanumeric Order
JUnit 5 comes with a set of built-in MethodOrderer implementations to run tests in alphanumeric order.
For example, it provides MethodOrderer.MethodName to sort test methods based on their names and their formal parameter lists:
@TestMethodOrder(MethodOrderer.MethodName.class)
public class AlphanumericOrderUnitTest {
private static StringBuilder output = new StringBuilder("");
@Test
void myATest() {
output.append("A");
}
@Test
void myBTest() {
output.append("B");
}
@Test
void myaTest() {
output.append("a");
}
@AfterAll
public static void assertOutput() {
assertEquals("ABa", output.toString());
}
}
Similarly, we can use MethodOrderer.DisplayName to sort methods alphanumerically based on their display names.
Please keep in mind that MethodOrderer.Alphanumeric is another alternative. However, this implementation is deprecated and will be removed in 6.0.
2.2. Using the @Order Annotation
We can use the @Order annotation to enforce tests to run in a specific order.
In the following example, the methods will run firstTest(), then secondTest() and finally thirdTest():
@TestMethodOrder(OrderAnnotation.class)
public class OrderAnnotationUnitTest {
private static StringBuilder output = new StringBuilder("");
@Test
@Order(1)
void firstTest() {
output.append("a");
}
@Test
@Order(2)
void secondTest() {
output.append("b");
}
@Test
@Order(3)
void thirdTest() {
output.append("c");
}
@AfterAll
public static void assertOutput() {
assertEquals("abc", output.toString());
}
}
2.3. Using Random Order
We can also order test methods pseudo-randomly using the MethodOrderer.Random implementation:
@TestMethodOrder(MethodOrderer.Random.class)
public class RandomOrderUnitTest {
private static StringBuilder output = new StringBuilder("");
@Test
void myATest() {
output.append("A");
}
@Test
void myBTest() {
output.append("B");
}
@Test
void myCTest() {
output.append("C");
}
@AfterAll
public static void assertOutput() {
assertEquals("ACB", output.toString());
}
}
As a matter of fact, JUnit 5 uses System.nanoTime() as the default seed to sort the test methods. This means that the execution order of the methods may not be the same in repeatable tests.
However, we can configure a custom seed using the junit.jupiter.execution.order.random.seed property to create repeatable builds.
We can specify the value of our custom seed in the junit-platform.properties file:
junit.jupiter.execution.order.random.seed=100
2.4. Using a Custom Order
Finally, we can use our own custom order by implementing the MethodOrderer interface.
In our CustomOrder, we’ll order the tests based on their names in a case-insensitive alphanumeric order:
public class CustomOrder implements MethodOrderer {
@Override
public void orderMethods(MethodOrdererContext context) {
context.getMethodDescriptors().sort(
(MethodDescriptor m1, MethodDescriptor m2)->
m1.getMethod().getName().compareToIgnoreCase(m2.getMethod().getName()));
}
}
Then we’ll use CustomOrder to run the same tests from our previous example in the order myATest(), myaTest() and finally myBTest():
@TestMethodOrder(CustomOrder.class)
public class CustomOrderUnitTest {
// ...
@AfterAll
public static void assertOutput() {
assertEquals("AaB", output.toString());
}
}
2.5. Set Default Order
JUnit 5 provides a convenient way to set a default method orderer through the junit.jupiter.testmethod.order.default parameter.
Similarly, we can configure our parameter in the junit-platform.properties file:
junit.jupiter.testmethod.order.default = org.junit.jupiter.api.MethodOrderer$DisplayName
The default orderer will be applied to all tests that aren’t qualified with @TestMethodOrder.
Another important thing to mention is that the specified class must implement the MethodOrderer interface.
3. Test Order in JUnit 4
For those still using JUnit 4, the APIs for ordering tests are slightly different.
Let’s go through the options to achieve this in previous versions as well.
3.1. Using MethodSorters.DEFAULT
This default strategy compares test methods using their hash codes.
In case of a hash collision, the lexicographical order is used:
@FixMethodOrder(MethodSorters.DEFAULT)
public class DefaultOrderOfExecutionTest {
private static StringBuilder output = new StringBuilder("");
@Test
public void secondTest() {
output.append("b");
}
@Test
public void thirdTest() {
output.append("c");
}
@Test
public void firstTest() {
output.append("a");
}
@AfterClass
public static void assertOutput() {
assertEquals(output.toString(), "cab");
}
}
When we run the tests in the class above, we will see that they all pass, including assertOutput().
3.2. Using MethodSorters.JVM
Another ordering strategy is MethodSorters.JVM.
This strategy utilizes the natural JVM ordering, which can be different for each run:
@FixMethodOrder(MethodSorters.JVM)
public class JVMOrderOfExecutionTest {
// same as above
}
Each time we run the tests in this class, we get a different result.
3.3. Using MethodSorters.NAME_ASCENDING
Finally, this strategy can be used for running tests in their lexicographic order:
@FixMethodOrder(MethodSorters.NAME_ASCENDING)
public class NameAscendingOrderOfExecutionTest {
// same as above
@AfterClass
public static void assertOutput() {
assertEquals(output.toString(), "abc");
}
}
When we run the tests in this class, we see that they all pass, including assertOutput(). This confirms the execution order that we set with the annotation.
4. Conclusion
In this quick article, we went through the ways of setting the execution order available in JUnit.
As always, the examples used in this article can be found over on GitHub.