1. Overview
In this tutorial, we’ll understand the twelve-factor app methodology.
We’ll also understand how to develop a microservice with the help of Spring Boot. In the process, we’ll see how to apply the twelve-factor methodology for developing such a microservice.
2. What Is the Twelve-Factor Methodology?
The twelve-factor methodology is a set of twelve best practices to develop applications developed to run as a service. This was originally drafted by Heroku for applications deployed as services on their cloud platform, back in 2011. Over time, this has proved to be generic enough for any software-as-a-service (SaaS) development.
So, what do we mean by software-as-a-service? Traditionally we design, develop, deploy, and maintain software solutions to derive business value from it. But, we don’t have to engage in this process to achieve the same result necessarily. For instance, calculating applicable tax is a generic function in many domains.
Now, we may decide to build and manage this service our selves or subscribe to a commercial service offering. Such service offerings are what we know as software-as-a-service.
While software-as-a-service doesn’t impose any restriction on the architecture it’s developed on; it’s quite useful to adopt some best practices.
If we design our software to be modular, portable, and scalable on modern cloud platforms, it’s quite amenable to our service offerings. This is where the twelve-factor methodology helps in. We’ll see them in action later in the tutorial.
3. Microservice with Spring Boot
Microservice is an architectural style to develop software as loosely coupled services. The key requirement here is that the services should be organized around business domain boundaries. This is often the most difficult part to identify.
Moreover, a service here has the sole authority over its data and exposes operations to other services. Communication between services is typically over lightweight protocols like HTTP. This results in independently deployable and scalable services.
Now, microservice architecture and software-as-a-service are not dependent on each other. But, it’s not difficult to understand that, when developing software-as-a-service, leveraging the microservice architecture is quite beneficial. It helps to achieve a lot of goals we discussed earlier, like modularity and scalability.
Spring Boot is an application framework based on Spring which takes away a lot of boilerplate associated with developing an enterprise application. It gives us a highly-opinionated but flexible platform to develop microservices. For this tutorial, we’ll leverage Spring Boot to deliver a microservice using the twelve-factor methodology.
4. Applying Twelve-Factor Methodology
Let’s now define a simple application that we’ll try to develop with the tools and practices we just discussed. We all love watching movies, but it’s challenging to keep track of the movies we’ve already watched.
Now, who would like to start a movie and then abandon it later? What we need is a simple service to record and query movies that we’ve watched:
This is quite a simple and standard microservice with a data store and REST endpoints. We need to define a model which will map to persistence as well:
@Entity
public class Movie {
@Id
private Long id;
private String title;
private String year;
private String rating;
// getters and setters
}
We’ve defined a JPA entity with an id and a few other attributes. Let’s now see what the REST controller looks like:
@RestController
public class MovieController {
@Autowired
private MovieRepository movieRepository;
@GetMapping("/movies")
public List<Movie> retrieveAllStudents() {
return movieRepository.findAll();
}
@GetMapping("/movies/{id}")
public Movie retrieveStudent(@PathVariable Long id) {
return movieRepository.findById(id).get();
}
@PostMapping("/movies")
public Long createStudent(@RequestBody Movie movie) {
return movieRepository.save(movie).getId();
}
}
This covers the base of our simple service. We’ll go through the rest of the application as we discuss how we implement the twelve-factor methodology in the following subsections.
4.1. Codebase
The first best practice of twelve-factor apps is to track it in a version control system. Git is the most popular version control system in use today and is almost ubiquitous. The principle states that an app should be tracked in a single code repository and must not share that repository with any other apps.
Spring Boot offers many convenient ways to bootstrap an application, including a command-line tool and a web interface. Once we generate the bootstrap application, we can convert this into a git repository:
git init
This command should be run from the root of the application. The application at this stage already contains a .gitignore file which effectively restricts generated files from being version-controlled. So, we can straight away create an initial commit:
git add .
git commit -m "Adding the bootstrap of the application."
Finally, we can add a remote and push our commits to the remote if we wish to (this is not a strict requirement):
git remote add origin https://github.com/<username>/12-factor-app.git
git push -u origin master
4.2. Dependencies
Next, the twelve-factor app should always explicitly declare all its dependencies. We should do this using a dependency declaration manifest. Java has multiple dependency management tools like Maven and Gradle. We can use one of them to achieve this goal.
So, our simple application depends on a few external libraries, like a library to facilitate REST APIs and to connect to a database. Let’s see how can we declaratively define them using Maven.
Maven requires us to describe a project’s dependencies in an XML file, typically known as Project Object Model (POM):
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<scope>runtime</scope>
</dependency>
</dependencies>
Although this looks plain and simple, these dependencies usually have other transitive dependencies. This complicates it to an extent but helps us achieve our goal. Now, our application doesn’t have a direct dependency which is not explicitly described.
4.3. Configurations
An application typically has lots of configuration, some of which may vary between deployments while others remain the same.
In our example, we’ve got a persistent database. We’ll need the address and credentials of the database to connect to. This is most likely to change between deployments.
A twelve-factor app should externalize all such configurations that vary between deployments. The recommendation here is to use environment variables for such configurations. This leads to a clean separation of config and code.
Spring provides a configuration file where we can declare such configurations and attach it to environment variables:
spring.datasource.url=jdbc:mysql://${MYSQL_HOST}:${MYSQL_PORT}/movies
spring.datasource.username=${MYSQL_USER}
spring.datasource.password=${MYSQL_PASSWORD}
Here, we’ve defined the database URL and credentials as configurations and have mapped the actual values to be picked from the environment variable.
On Windows, we can set the environment variable before starting the application:
set MYSQL_HOST=localhost
set MYSQL_PORT=3306
set MYSQL_USER=movies
set MYSQL_PASSWORD=password
We can use a configuration management tool like Ansible or Chef to automate this process.
4.4. Backing Services
Backing services are services that the application depends on for operation. For instance a database or a message broker. A twelve-factor app should treat all such backing services as attached resources. What this effectively means is that it shouldn’t require any code change to swap a compatible backing service. The only change should be in configurations.
In our application, we’ve used MySQL as the backing service to provide persistence.
Spring JPA makes the code quite agnostic to the actual database provider. We only need to define a repository which provides all standard operations:
@Repository
public interface MovieRepository extends JpaRepository<Movie, Long> {
}
As we can see, this is not dependent on MySQL directly. Spring detects the MySQL driver on the classpath and provides a MySQL-specific implementation of this interface dynamically. Moreover, it pulls other details from configurations directly.
So, if we’ve to change from MySQL to Oracle, all we’ve to do is replace the driver in our dependencies and replace the configurations.
4.5. Build, Release and Run
The twelve-factor methodology strictly separates the process of converting codebase into a running application as three distinct stages:
- Build Stage: This is where we take the codebase, perform static and dynamic checks, and then generate an executable bundle like a JAR. Using a tool like Maven, this is quite trivial:
mvn clean compile test package
- Release Stage: This is the stage where we take the executable bundle and combine this with the right configurations. Here, we can use Packer with a provisioner like Ansible to create Docker images:
packer build application.json
- Run Stage: Finally, this is the stage where we run the application in a target execution environment. If we use Docker as the container to release our application, running the application can be simple enough:
docker run --name <container_id> -it <image_id>
Finally, we don’t necessarily have to perform these stages manually. This is where Jenkins comes in as pretty handy with their declarative pipeline.
4.6. Processes
A twelve-factor app is expected to run in an execution environment as stateless processes. In other words, they can not store persistent state locally between requests. They may generate persistent data which is required to be stored in one or more stateful backing services.
In the case of our example, we’ve got multiple endpoints exposed. A request on any of these endpoints is entirely independent of any request made before it. For instance, if we keep track of user requests in-memory and use that information to serve future requests, it violates a twelve-factor app.
Hence, a twelve-factor app imposes no such restriction like sticky sessions. This makes such an app highly portable and scalable. In a cloud execution environment offering automated scaling, it’s quite a desirable behavior from applications.
4.7. Port Binding
A traditional web application in Java is developed as a WAR or web archive. This is typically a collection of Servlets with dependencies, and it expects a conformant container runtime like Tomcat. A twelve-factor app, on the contrary, expects no such runtime dependency. It’s completely self-contained and only requires an execution runtime like Java.
In our case, we’ve developed an application using Spring Boot. Spring Boot, apart from many other benefits, provides us with a default embedded application server. Hence, the JAR we generated earlier using Maven is fully capable of executing in any environment just by having a compatible Java runtime:
java -jar application.jar
Here, our simple application exposes its endpoints over an HTTP binding to a specific port like 8080. Upon starting the application as we did above, it should be possible to access the exported services like HTTP.
An application may export multiple services like FTP or WebSocket by binding to multiple ports.
4.8. Concurrency
Java offers Thread as a classical model to handle concurrency in an application. Threads are like lightweight processes and represent multiple paths of execution in a program. Threads are powerful but have limitations in terms of how much it can help an application scale.
The twelve-factor methodology suggests apps to rely on processes for scaling. What this effectively means is that applications should be designed to distribute workload across multiple processes. Individual processes are, however, free to leverage a concurrency model like Thread internally.
A Java application, when launched gets a single process which is bound to the underlying JVM. What we effectively need is a way to launch multiple instances of the application with intelligent load distribution between them. Since we’ve already packaged our application as a Docker container, Kubernetes is a natural choice for such orchestration.
4.9. Disposability
Application processes can be shut down on purpose or through an unexpected event. In either case, a twelve-factor app is supposed to handle it gracefully. In other words, an application process should be completely disposable without any unwanted side-effects. Moreover, processes should start quickly
For instance, in our application, one of the endpoints is to create a new database record for a movie. Now, an application handling such a request may crash unexpectedly. This should, however, not impact the state of the application. When a client sends the same request again, it shouldn’t result in duplicate records.
In summary, the application should expose idempotent services. This is another very desirable attribute of a service destined for cloud deployments. This gives the flexibility to stop, move, or spin new services at any time without any other considerations.
4.10. Dev/Prod Parity
It’s typical for applications to be developed on local machines, tested on some other environments and finally deployed to production. It’s often the case where these environments are different. For instance, the development team works on Windows machines whereas production deployment happens on Linux machines.
The twelve-factor methodology suggests keeping the gap between development and production environment as minimal as possible. These gaps can result from long development cycles, different teams involved, or different technology stack in use.
Now, technology like Spring Boot and Docker automatically bridge this gap to a great extent. A containerized application is expected to behave the same, no matter where we run it. We must use the same backing services – like the database – as well.
Moreover, we should have the right processes like continuous integration and delivery to facilitate bridging this gap further.
4.11. Logs
Logs are essential data that an application generates during its lifetime. They provide invaluable insights into the working of the application. Typically an application can generate logs at multiple levels with varying details and output ii in multiple different formats.
A twelve-factor app, however, separates itself from log generation and its processing. For such an app, logs are nothing but a time-ordered stream of events. It merely writes these events to the standard output of the execution environment. The capture, storage, curation, and archival of such stream should be handled by the execution environment.
There are quite several tools available to us for this purpose. To begin with, we can use SLF4J to handle logging abstractly within our application. Moreover, we can use a tool like Fluentd to collect the stream of logs from applications and backing services.
This we can feed into Elasticsearch for storage and indexing. Finally, we can generate meaningful dashboards for visualization in Kibana.
4.12. Admin Processes
Often we need to perform some one-off tasks or routine procedure with our application state. For instance, fixing bad records. Now, there are various ways in which we can achieve this. Since we may not often require it, we can write a small script to run it separately from another environment.
Now, the twelve-factor methodology strongly suggests keeping such admin scripts together with the application codebase. In doing so, it should follow the same principles as we apply to the main application codebase. It’s also advisable to use a built-in REPL tool of the execution environment to run such scripts on production servers.
In our example, how do we seed our application with the already watched movies so far? While we can use our sweet little endpoint, but that may seem to be impractical. What we need is a script to perform a one-time load. We can write a small Java function to read a list of movies from a file and save them in batch into the database.
Moreover, we can use Groovy integrated with Java runtime to start such processes.
5. Practical Applications
So, now we’ve seen all the factors suggested by the twelve-factor methodology. Developing an application to be a twelve-factor app certainly has its benefits, especially when we wish to deploy them as services on the cloud. But, like all other guidelines, framework, patterns, we must ask, is this a silver bullet?
Honestly, no single methodology in software design and development claim to be a silver bullet. The twelve-factor methodology is no exception. While some of these factors are quite intuitive, and most likely we’re already doing them, others may not apply to us. It’s essential to evaluate these factors in the backdrop of our objectives and then choose wisely.
It’s important to note that all these factors are there to help us develop an application which is modular, independent, portable, scalable, and observable. Depending upon the application, we may be able to achieve them through other means better. It’s also not necessary to adopt all the factors together, adopting even some of these may make us better than we were.
Finally, these factors are quite simple and elegant. They hold greater importance in an age where we demand our applications to have higher throughput and lower latency with virtually no downtime and failure. Adopting these factors gives us the right start from the beginning. Combined with microservice architecture and containerization of applications, they just seem to hit the right spot.
6. Conclusion
In this tutorial, we went through the concepts of twelve-factor methodology. We discussed how to leverage a microservice architecture with Spring Boot to deliver them effectively. Further, we explored each factor in detail and how to apply them to our application. We also explored several tools to apply these individual factors in an effective manner successfully.