Spring 集成测试优化建议

1. Introduction

In this article, we’ll have a holistic discussion about integration tests using Spring and how to optimize them.

First, we’ll briefly discuss the importance of integration tests and their place in modern Software focusing on the Spring ecosystem.

Later, we’ll cover multiple scenarios, focusing on web-apps.

Next, we’ll discuss some strategies to improve testing speed, by learning about different approaches that could influence both the way we shape our tests and the way we shape the app itself.

Before getting started, it is important to keep in mind this is an opinion article based on experience. Some of this things might suit you, some might not.

Finally, this article uses Kotlin for the code samples to keep them as concise as possible, but the concepts aren’t specific to this language and code snippets should feel meaningful to Java and Kotlin developers alike.

2. Integration Tests

Integration tests are a fundamental part of automated test suites. Although they shouldn’t be as numerous as unit tests if we follow a healthy test pyramid. Relying on frameworks such as Spring leave us needing a fair amount of integration testing in order to de-risk certain behaviors of our system.

The more we simplify our code by using Spring modules (data, security, social…), the bigger a need for integration tests. This becomes particularly true when we move bits and bobs of our infrastructure into @Configuration classes.

We shouldn’t “test the framework”, but we should certainly verify the framework is configured to fulfill our needs.

Integration tests help us build confidence but they come at a price:

That is a slower execution speed, which means slower builds
Also, integration tests imply a broader testing scope which is not ideal in most cases

With this in mind, we’ll try to find some solutions to mitigate the above-mentioned problems.

3. Testing Web Apps

Spring brings a few options in order to test web applications, and most Spring developers are familiar with them, these are:

MockMvc: Mocks the servlet API, useful for non-reactive web apps
TestRestTemplate: Can be used pointing to our app, useful for non-reactive web apps where mocked servlets are not desirable
WebTestClient: Is a testing tool for reactive web apps, both with mocked requests/responses or hitting a real server

As we already have articles covering these topics we won’t spend time talking about them.

Feel free to have a look if you’d like to dig deeper.

4. Optimizing Execution Time

Integration tests are great. They give us a good degree of confidence. Also if implemented appropriately, they can describe the intent of our app in a very clear way, with less mocking and setup noise.

However, as our app matures and the development piles up, build time inevitably goes up. As build time increases it might become impractical to keep running all tests every time.

Thereafter, impacting our feedback loop and getting on the way of best development practices.

Furthermore, integration tests are inherently expensive. Starting up persistence of some sort, sending requests through (even if they never leave localhost), or doing some IO simply takes time.

It’s paramount to keep an eye on our build time, including test execution. And there are some tricks we can apply in Spring to keep it low.

In the next sections, we’ll cover a few points to help us out optimize our build time as well as some pitfalls that might impact its speed:

Using profiles wisely – how profiles impact performance
Reconsidering @MockBean – how mocking hits performance
Refactoring @MockBean – alternatives to improve performance
Thinking carefully about @DirtiesContext – a useful but dangerous annotation and how not to use it
Using test slices – a cool tool that can help or get on our way
Using class inheritance – a way to organize tests in a safe manner
State management – good practices to avoid flakey tests
Refactoring into unit tests – the best way to get a solid and snappy build

Let’s get started!

4.1. Using Profiles Wisely

Profiles are a pretty neat tool. Namely, simple tags that can enable or disable certain areas of our App. We could even implement feature flags with them!

As our profiles get richer, it’s tempting to swap every now and then in our integration tests. There are convenient tools to do so, like @ActiveProfiles. However, every time we pull a test with a new profile, a new ApplicationContext gets created.

Creating application contexts might be snappy with a vanilla spring boot app with nothing in it. Add an ORM and a few modules and it will quickly skyrocket to 7+ seconds.

Add a bunch of profiles, and scatter them through a few tests and we’ll quickly get a 60+ seconds build (assuming we run tests as part of our build – and we should).

Once we face a complex enough application, fixing this is daunting. However, if we plan carefully in advance, it becomes trivial to keep a sensible build time.

There are a few tricks we could keep in mind when it comes to profiles in integration tests:

Create an aggregate profile, i.e. test, include all needed profiles within – stick to our test profile everywhere
Design our profiles with testability in mind. If we end up having to switch profiles perhaps there is a better way
State our test profile in a centralized place – we’ll talk about this later
Avoid testing all profiles combinations. Alternatively, we could have an e2e test-suite per environment testing the app with that specific profile-set

4.2. The Problems with @MockBean

@MockBean is a pretty powerful tool.

When we need some Spring magic but want to mock a particular component, @MockBean comes in really handy. But it does so at a price.

Every time @MockBean appears in a class, the ApplicationContext cache gets marked as dirty, hence the runner will clean the cache after the test-class is done. Which again adds an extra bunch of seconds to our build.

This is a controversial one, but trying to exercise the actual app instead of mocking for this particular scenario could help. Of course, there’s no silver bullet here. Boundaries get blurry when we don’t allow ourselves to mock dependencies.

We might think: Why would we persist when all we want to test is our REST layer? This is a fair point, and there’s always a compromise.

However, with a few principles in mind, this might actually can be turned into an advantage that leads to better design of both tests and our app and reduces testing time.

4.3. Refactoring @MockBean

In this section, we’ll try to refactor a ‘slow’ test using @MockBean to make it reuse the cached ApplicationContext.

Let’s assume we want to test a POST that creates a user. If we were mocking – using @MockBean, we could simply verify that our service has been called with a nicely serialized user.

If we tested our service properly this approach should suffice:

class UsersControllerIntegrationTest : AbstractSpringIntegrationTest() {

    @Autowired
    lateinit var mvc: MockMvc
    
    @MockBean
    lateinit var userService: UserService

    @Test
    fun links() {
        mvc.perform(post("/users")
          .contentType(MediaType.APPLICATION_JSON)
          .content("""{ "name":"jose" }"""))
          .andExpect(status().isCreated)
        
        verify(userService).save("jose")
    }
}

interface UserService {
    fun save(name: String)
}

We want to avoid @MockBean though. So we’ll end up persisting the entity (assuming that’s what the service does).

The most naive approach here would be to test the side effect: After POSTing, my user is in my DB, in our example, this would use JDBC.

This, however, violates testing boundaries:

@Test
fun links() {
    mvc.perform(post("/users")
      .contentType(MediaType.APPLICATION_JSON)
      .content("""{ "name":"jose" }"""))
      .andExpect(status().isCreated)

    assertThat(
      JdbcTestUtils.countRowsInTable(jdbcTemplate, "users"))
      .isOne()
}

In this particular example we violate testing boundaries because we treat our app as an HTTP black box to send the user, but later we assert using implementation details, that is, our user has been persisted in some DB.

If we exercise our app through HTTP, can we assert the result through HTTP too?

@Test
fun links() {
    mvc.perform(post("/users")
      .contentType(MediaType.APPLICATION_JSON)
      .content("""{ "name":"jose" }"""))
      .andExpect(status().isCreated)

    mvc.perform(get("/users/jose"))
      .andExpect(status().isOk)
}

There are a few advantages if we follow the last approach:

Our test will start quicker (arguably, it might take a tiny bit longer to execute though, but it should pay back)
Also, our test isn’t aware of side effects not related to HTTP boundaries i.e. DBs
Finally, our test expresses with clarity the intent of the system: If you POST, you’ll be able to GET Users

Of course, this might not always be possible for various reasons:

We might not have the ‘side-effect’ endpoint: An option here is to consider creating ‘testing endpoints’
Complexity is too high to hit the entire app: An option here is to consider slices (we’ll talk about them later)

4.4. Thinking Carefully About @DirtiesContext

Sometimes, we might need to modify the ApplicationContext in our tests. For this scenario, @DirtiesContext delivers exactly that functionality.

For the same reasons exposed above, @DirtiesContext is an extremely expensive resource when it comes to execution time, and as such, we should be careful.

Some misuses of @DirtiesContext include application cache reset or in memory DB resets. There are better ways to handle these scenarios in integration tests, and we’ll cover some in further sections.

4.5. Using Test Slices

Test Slices are a Spring Boot feature introduced in the 1.4. The idea is fairly simple, Spring will create a reduced application context for a specific slice of your app.

Also, the framework will take care of configuring the very minimum.

There are a sensible number of slices available out of the box in Spring Boot and we can create our own too:

@JsonTest: Registers JSON relevant components
@DataJpaTest: Registers JPA beans, including the ORM available
@JdbcTest: Useful for raw JDBC tests, takes care of the data source and in memory DBs without ORM frills
@DataMongoTest: Tries to provide an in-memory mongo testing setup
@WebMvcTest: A mock MVC testing slice without the rest of the app
… (we can check the source to find them all)

This particular feature if used wisely can help us build narrow tests without such a big penalty in terms of performance particularly for small/medium sized apps.

However, if our application keeps growing it also piles up as it creates one (small) application context per slice.

4.6. Using Class Inheritance

Using a single AbstractSpringIntegrationTest class as the parent of all our integration tests is a simple, powerful and pragmatic way of keeping the build fast.

If we provide a solid setup, our team will simply extend it, knowing that everything ‘just works’. This way we can worry less about managing state or configuring the framework and focus on the problem at hand.

We could set all the test requirements there:

The Spring runner – or preferably rules, in case we need other runners later
profiles – ideally our aggregate test profile
initial config – setting the state of our application

Let’s have a look at a simple base class that takes care of the previous points:

@SpringBootTest
@ActiveProfiles("test")
abstract class AbstractSpringIntegrationTest {

    @Rule
    @JvmField
    val springMethodRule = SpringMethodRule()

    companion object {
        @ClassRule
        @JvmField
        val SPRING_CLASS_RULE = SpringClassRule()
    }
}

4.7. State Management

It’s important to remember where ‘unit’ in Unit Test comes from. Simply put, it means we can run a single test (or a subset) at any point getting consistent results.

Hence, the state should be clean and known before every test starts.

In other words, the result of a test should be consistent regardless of whether it is executed in isolation or together with other tests.

This idea applies just the same to integration tests. We need to ensure our app has a known (and repeatable) state before starting a new test. The more components we reuse to speed things up (app context, DBs, queues, files…), the more chances to get state pollution.

Assuming we went all in with class inheritance, now, we have a central place to manage state.

Let’s enhance our abstract class to make sure our app is in a known state before running tests.

In our example, we’ll assume there are several repositories (from various data sources), and a Wiremock server:

@SpringBootTest
@ActiveProfiles("test")
@AutoConfigureWireMock(port = 8666)
@AutoConfigureMockMvc
abstract class AbstractSpringIntegrationTest {

    //... spring rules are configured here, skipped for clarity

    @Autowired
    protected lateinit var wireMockServer: WireMockServer

    @Autowired
    lateinit var jdbcTemplate: JdbcTemplate

    @Autowired
    lateinit var repos: Set<MongoRepository<*, *>>

    @Autowired
    lateinit var cacheManager: CacheManager

    @Before
    fun resetState() {
        cleanAllDatabases()
        cleanAllCaches()
        resetWiremockStatus()
    }

    fun cleanAllDatabases() {
        JdbcTestUtils.deleteFromTables(jdbcTemplate, "table1", "table2")
        jdbcTemplate.update("ALTER TABLE table1 ALTER COLUMN id RESTART WITH 1")
        repos.forEach { it.deleteAll() }
    }

    fun cleanAllCaches() {
        cacheManager.cacheNames
          .map { cacheManager.getCache(it) }
          .filterNotNull()
          .forEach { it.clear() }
    }

    fun resetWiremockStatus() {
        wireMockServer.resetAll()
        // set default requests if any
    }
}

4.8. Refactoring into Unit Tests

This is probably one of the most important points. We’ll find ourselves over and over with some integration tests that are actually exercising some high-level policy of our app.

Whenever we find some integration tests testing a bunch of cases of core business logic, it’s time to rethink our approach and break them down into unit tests.

A possible pattern here to accomplish this successfully could be:

Identify integration tests that are testing multiple scenarios of core business logic
Duplicate the suite, and refactor the copy into unit Tests – at this stage, we might need to break down the production code too to make it testable
Get all tests green
Leave a happy path sample that is remarkable enough in the integration suite – we might need to refactor or join and reshape a few
Remove the remaining integration Tests

Michael Feathers covers many techniques to achieve this and more in Working Effectively with Legacy Code.

5. Summary

In this article, we had an introduction to Integration tests with a focus on Spring.

First, we talked about the importance of integration tests and why they are particularly relevant in Spring applications.

After that, we summarized some tools that might come in handy for certain types of Integration tests in Web Apps.

Finally, we went through a list of potential issues that slow down our test execution time, as well as tricks to improve it.

Persistence

REST

Security