1. Overview
In this tutorial, we’ll understand the popular software-testing model called the test pyramid.
We’ll see how it’s relevant in the world of microservices. In the process, we’ll develop a sample application and relevant tests to conform to this model. In addition, we’ll try to understand the benefits and boundaries of using a model.
2. Let’s Take a Step Back
Before we start to understand any particular model like the test pyramid, it’s imperative to understand why we even need one.
The need to test software is inherent and perhaps as old as the history of software development itself. Software testing has come a long way from manual to automation and further. The objective, however, remains the same — to deliver software conforming to specifications.
2.1. Types of Tests
There are several different types of tests in practice, which focus on specific objectives. Sadly, there is quite a variation in vocabulary and even understanding of these tests.
Let’s review some of the popular and possibly unambiguous ones:
- Unit Tests: Unit tests are the tests that target small units of code, preferably in isolation. The objective here is to validate the behavior of the smallest testable piece of code without worrying about the rest of the codebase. This automatically implies that any dependency needs to be replaced with either a mock or a stub or such similar construct.
- Integration Tests: While unit tests focus on the internals of a piece of code, the fact remains that a lot of complexity lies outside of it. Units of code need to work together and often with external services like databases, message brokers, or web services. Integration tests are the tests that target the behavior of an application while integrating with external dependencies.
- UI Tests: A software we develop is often consumed through an interface, which consumers can interact with. Quite often, an application has a web interface. However, API interfaces are becoming increasingly popular. UI tests target the behavior of these interfaces, which often are highly interactive in nature. Now, these tests can be conducted in an end-to-end manner, or user interfaces can also be tested in isolation.
2.2. Manual vs. Automated Tests
Software testing has been done manually since the beginning of testing, and it’s widely in practice even today. However, it’s not difficult to understand that manual testing has restrictions. For the tests to be useful, they have to be comprehensive and run often.
This is even more important in agile development methodologies and cloud-native microservice architecture. However, the need for test automation was realized much earlier.
If we recall the different types of tests we discussed earlier, their complexity and scope increase as we move from unit tests to integration and UI tests. For the same reason, automation of unit tests is easier and bears most of the benefits as well. As we go further, it becomes increasingly difficult to automate the tests with arguably lesser benefits.
Barring certain aspects, it’s possible to automate testing of most software behavior as of today. However, this must be weighed rationally with the benefits compared to the effort needed to automate.
3. What Is a Test Pyramid?
Now that we’ve gathered enough context around test types and tools, it’s time to understand what exactly a test pyramid is. We’ve seen that there are different types of tests that we should write.
However, how should we decide how many tests should we write for each type? What are the benefits or pitfalls to look out for? These are some of the problems addressed by a test automation model like the test pyramid.
Mike Cohn came up with a construct called Test Pyramid in his book “Succeeding with Agile”. This presents a visual representation of the number of tests that we should write at different levels of granularity.
The idea is that it should be highest at the most granular level and should start decreasing as we broaden our scope of the test. This gives the typical shape of a pyramid, hence the name:
While the concept is pretty simple and elegant, it’s often a challenge to adopt this effectively. It’s important to understand that we must not get fixated with the shape of the model and types of tests it mentions. The key takeaway should be that:
- We must write tests with different levels of granularity
- We must write fewer tests as we get coarser with their scope
4. Test Automation Tools
There are several tools available in all mainstream programming languages for writing different types of tests. We’ll cover some of the popular choices in the Java world.
4.1. Unit Tests
- Test Framework: The most popular choice here in Java is JUnit, which has a next-generation release known as JUnit5. Other popular choices in this area include TestNG, which offers some differentiated features compared to JUnit5. However, for most applications, both of these are suitable choices.
- Mocking: As we saw earlier, we definitely want to deduct most of the dependencies, if not all, while executing a unit test. For this, we need a mechanism to replace dependencies with a test double like a mock or stub. Mockito is an excellent framework to provision mocks for real objects in Java.
4.2. Integration Tests
- Test Framework: The scope of an integration test is wider than a unit test, but the entry point is often the same code at a higher abstraction. For this reason, the same test frameworks that work for unit testing are suitable for integration testing as well.
- Mocking: The objective of an integration test is to test an application behavior with real integrations. However, we may not want to hit an actual database or message broker for tests. Many databases and similar services offer an embeddable version to write integration tests with.
4.3. UI Tests
- Test Framework: The complexity of UI tests varies depending on the client handling the UI elements of the software. For instance, the behavior of a web page may differ depending upon device, browser, and even operating system. Selenium is a popular choice to emulate browser behavior with a web application. For REST APIs, however, frameworks like REST-assured are the better choices.
- Mocking: User interfaces are becoming more interactive and client-side rendered with JavaScript frameworks like Angular and React. It’s more reasonable to test such UI elements in isolation using a test framework like Jasmine and Mocha. Obviously, we should do this in combination with end-to-end tests.
5. Adopting Principles in Practice
Let’s develop a small application to demonstrate the principles we’ve discussed so far. We’ll develop a small microservice and understand how to write tests conforming to a test pyramid.
Microservice architecture helps structure an application as a collection of loosely coupled services drawn around domain boundaries. Spring Boot offers an excellent platform to bootstrap a microservice with a user interface and dependencies like databases in almost no time.
We’ll leverage these to demonstrate the practical application of the test pyramid.
5.1. Application Architecture
We’ll develop an elementary application that allows us to store and query movies that we’ve watched:
As we can see, it has a simple REST Controller exposing three endpoints:
@RestController
public class MovieController {
@Autowired
private MovieService movieService;
@GetMapping("/movies")
public List<Movie> retrieveAllMovies() {
return movieService.retrieveAllMovies();
}
@GetMapping("/movies/{id}")
public Movie retrieveMovies(@PathVariable Long id) {
return movieService.retrieveMovies(id);
}
@PostMapping("/movies")
public Long createMovie(@RequestBody Movie movie) {
return movieService.createMovie(movie);
}
}
The controller merely routes to appropriate services, apart from handling data marshaling and unmarshaling:
@Service
public class MovieService {
@Autowired
private MovieRepository movieRepository;
public List<Movie> retrieveAllMovies() {
return movieRepository.findAll();
}
public Movie retrieveMovies(@PathVariable Long id) {
Movie movie = movieRepository.findById(id)
.get();
Movie response = new Movie();
response.setTitle(movie.getTitle()
.toLowerCase());
return response;
}
public Long createMovie(@RequestBody Movie movie) {
return movieRepository.save(movie)
.getId();
}
}
Furthermore, we have a JPA Repository that maps to our persistence layer:
@Repository
public interface MovieRepository extends JpaRepository<Movie, Long> {
}
Finally, our simple domain entity to hold and pass movie data:
@Entity
public class Movie {
@Id
private Long id;
private String title;
private String year;
private String rating;
// Standard setters and getters
}
With this simple application, we’re now ready to explore tests with different granularity and quantity.
5.2. Unit Testing
First, we’ll understand how to write a simple unit test for our application. As evident from this application, most of the logic tends to accumulate in the service layer. This mandates that we test this extensively and more often — quite a good fit for unit tests:
public class MovieServiceUnitTests {
@InjectMocks
private MovieService movieService;
@Mock
private MovieRepository movieRepository;
@Before
public void setUp() throws Exception {
MockitoAnnotations.initMocks(this);
}
@Test
public void givenMovieServiceWhenQueriedWithAnIdThenGetExpectedMovie() {
Movie movie = new Movie(100L, "Hello World!");
Mockito.when(movieRepository.findById(100L))
.thenReturn(Optional.ofNullable(movie));
Movie result = movieService.retrieveMovies(100L);
Assert.assertEquals(movie.getTitle().toLowerCase(), result.getTitle());
}
}
Here, we’re using JUnit as our test framework and Mockito to mock dependencies. Our service, for some weird requirement, was expected to return movie titles in lower case, and that is what we intend to test here. There can be several such behaviors that we should cover extensively with such unit tests.
5.3. Integration Testing
In our unit tests, we mocked the repository, which was our dependency on the persistence layer. While we’ve thoroughly tested the behavior of the service layer, we still may have issues when it connects to the database. This is where integration tests come into the picture:
@RunWith(SpringRunner.class)
@SpringBootTest
public class MovieControllerIntegrationTests {
@Autowired
private MovieController movieController;
@Test
public void givenMovieControllerWhenQueriedWithAnIdThenGetExpectedMovie() {
Movie movie = new Movie(100L, "Hello World!");
movieController.createMovie(movie);
Movie result = movieController.retrieveMovies(100L);
Assert.assertEquals(movie.getTitle().toLowerCase(), result.getTitle());
}
}
Note a few interesting differences here. Now, we’re not mocking any dependencies. However, we may still need to mock a few dependencies depending upon the situation. Moreover, we’re running these tests with SpringRunner.
That essentially means that we’ll have a Spring application context and live database to run this test with. No wonder, this will run slower! Hence, we much choose fewer scenarios to tests here.
5.4. UI Testing
Finally, our application has REST endpoints to consume, which may have their own nuances to test. Since this is the user interface for our application, we’ll focus to cover it in our UI testing. Let’s now use REST-assured to test the application:
@RunWith(SpringRunner.class)
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
public class MovieApplicationE2eTests {
@Autowired
private MovieController movieController;
@LocalServerPort
private int port;
@Test
public void givenMovieApplicationWhenQueriedWithAnIdThenGetExpectedMovie() {
Movie movie = new Movie(100L, "Hello World!");
movieController.createMovie(movie);
when().get(String.format("http://localhost:%s/movies/100", port))
.then()
.statusCode(is(200))
.body(containsString("Hello World!".toLowerCase()));
}
}
As we can see, these tests are run with a running application and access it through the available endpoints. We focus on testing typical scenarios associated with HTTP, like the response code. These will be the slowest tests to run for obvious reasons.
Hence, we must be very particular to choose scenarios to test here. We should only focus on complexities that we’ve not been able to cover in previous, more granular tests.
6. Test Pyramid for Microservices
Now we’ve seen how to write tests with different granularity and structure them appropriately. However, the key objective is to capture most of the application complexity with more granular and faster tests.
While addressing this in a monolithic application gives us the desired pyramid structure, this may not be necessary for other architectures.
As we know, microservice architecture takes an application and gives us a set of loosely coupled applications. In doing so, it externalizes some of the complexities that were inherent to the application.
Now, these complexities manifest in the communication between services. It’s not always possible to capture them through unit tests, and we have to write more integration tests.
While this may mean that we deviate from the classical pyramid model, it does not mean we deviate from principle as well. Remember, we’re still capturing most of the complexities with as granular tests as possible. As long as we’re clear on that, a model that may not match a perfect pyramid will still be valuable.
The important thing to understand here is that a model is only useful if it delivers value. Often, the value is subject to context, which in this case is the architecture we choose for our application. Therefore, while it’s helpful to use a model as a guideline, we should focus on the underlying principles and finally choose what makes sense in our architecture context.
7. Integration with CI
The power and benefit of automated tests are largely realized when we integrate them into the continuous integration pipeline. Jenkins is a popular choice to define build and deployment pipelines declaratively.
We can integrate any tests which we’ve automated in the Jenkins pipeline. However, we must understand that this increases the time for the pipeline to execute. One of the primary objectives of continuous integration is fast feedback. This may conflict if we start adding tests that make it slower.
The key takeaway should be to add tests that are fast, like unit tests, to the pipeline that is expected to run more frequently. For instance, we may not benefit from adding UI tests into the pipeline that triggers on every commit. But, this is just a guideline and, finally, it depends on the type and complexity of the application we’re dealing with.
8. Conclusion
In this article, we went through the basics of software testing. We understood different test types and the importance of automating them using one of the available tools.
Furthermore, we understood what a test pyramid means. We implemented this using a microservice built using Spring Boot.
Finally, we went through the relevance of the test pyramid, especially in the context of architecture like microservices.