@TestFactory and Dynamic Tests

Sometimes you do not know the test cases at compile time. Maybe you need to generate tests from a database query, a file, or some other dynamic source. @TestFactory lets you create tests at runtime — they are called dynamic tests.

@TestFactory Basics

import org.junit.jupiter.api.TestFactory;
import org.junit.jupiter.api.DynamicTest;
import java.util.stream.Stream;
import java.util.Collection;

class DynamicTestDemo {

    @TestFactory
    Collection<DynamicTest> dynamicTests() {
        return List.of(
            DynamicTest.dynamicTest("addition", () -> assertEquals(4, 2 + 2)),
            DynamicTest.dynamicTest("subtraction", () -> assertEquals(2, 4 - 2)),
            DynamicTest.dynamicTest("multiplication", () -> assertEquals(6, 2 * 3))
        );
    }
}
    

Dynamic Tests from Streams

@TestFactory
Stream<DynamicTest> dynamicTestsFromStream() {
    return IntStream.rangeClosed(1, 5)
        .mapToObj(i ->
            DynamicTest.dynamicTest(
                "test " + i,
                () -> assertTrue(i > 0)
            )
        );
}

@TestFactory
Stream<DynamicTest> calculatorTests() {
    return Stream.of(
        Arguments.of(2, 3, 5),
        Arguments.of(0, 0, 0),
        Arguments.of(-1, 1, 0),
        Arguments.of(100, 200, 300)
    ).map(args ->
        DynamicTest.dynamicTest(
            String.format("%d + %d = %d",
                args.get(0), args.get(1), args.get(2)),
            () -> assertEquals(
                args.get(2),
                calc.add(args.get(0), args.get(1))
            )
        )
    );
}
    

Dynamic Test Naming

The first argument to DynamicTest.dynamicTest() is the display name. Make it descriptive:

@TestFactory
Stream<DynamicTest> palindromeTests() {
    return Stream.of(
        new TestInput("racecar", true),
        new TestInput("hello", false),
        new TestInput("level", true),
        new TestInput("world", false)
    ).map(input ->
        DynamicTest.dynamicTest(
            String.format("isPalindrome(\"%s\") should return %s",
                input.word, input.expected),
            () -> assertEquals(input.expected,
                helper.isPalindrome(input.word))
        )
    );
}

record TestInput(String word, boolean expected) {}
    

DynamicTest vs @ParameterizedTest

+-------------------+---------------------------+---------------------------+
|                   |  DynamicTest              |  @ParameterizedTest       |
+-------------------+---------------------------+---------------------------+
|  Source           |  Runtime computation      |  Compile-time sources     |
|  Display          |  In @TestFactory method   |  In @Test class           |
|  Individual       |  Yes                      |  Yes                      |
|  tests visible    |                           |                           |
|  Lifecycle        |  No @BeforeEach etc.      |  Full lifecycle support   |
|  Disable/Ignore   |  Not directly             |  @Disabled works          |
|  Use case         |  Generate from data       |  Fixed set of inputs      |
+-------------------+---------------------------+---------------------------+
    

Practical Example: Testing from CSV

@TestFactory
Stream<DynamicTest> testsFromCsvFile() throws IOException {
    Path csvPath = Path.of("src/test/resources/test-data.csv");

    return Files.lines(csvPath)
        .skip(1) // Skip header
        .map(line -> {
            String[] parts = line.split(",");
            String input = parts[0];
            int expected = Integer.parseInt(parts[1]);

            return DynamicTest.dynamicTest(
                String.format("length(\"%s\") = %d", input, expected),
                () -> assertEquals(expected, input.length())
            );
        });
}
    

Practical Example: Testing All Enum Values

@TestFactory
Stream<DynamicTest> allHttpStatusCodes() {
    return Arrays.stream(HttpStatus.values())
        .map(status ->
            DynamicTest.dynamicTest(
                String.format("%d %s", status.value(), status.name()),
                () -> {
                    assertTrue(status.value() >= 100);
                    assertTrue(status.value() < 600);
                }
            )
        );
}
    

Executable Interface

DynamicTest implements the Executable interface, which is just a functional interface with execute():

@TestFactory
Collection<DynamicTest> dynamicTests() {
    List<DynamicTest> tests = new ArrayList<>();

    for (int i = 1; i <= 10; i++) {
        final int n = i;
        tests.add(DynamicTest.dynamicTest(
            "factorial of " + n,
            () -> assertEquals(factorial(n), helper.factorial(n))
        ));
    }

    return tests;
}
    

When to Use Dynamic Tests

• Generating tests from external data (CSV, JSON, database)
• Testing combinatorial scenarios
• Testing all values of an enum
• Property-based testing patterns
• When test cases are determined at runtime