1. Introduction
In modern-day application development, it’s a common practice to split large applications into multiple independent, smaller components. Together, these application components address the business problem. However, the key challenge in this approach is to ensure effective communication between the components.
In this tutorial, we’ll explore the various aspects of message brokers and Enterprise Service Buses (ESB). They are two different messaging middlewares that facilitate communication between application components.
2. Message Broker
In this section, we’ll talk about message brokers, the communication patterns they support, and their advantages and disadvantages.
2.1. What’s a Message Broker?
A message broker is a software component that facilitates message flow between application components.
A message is formally defined and not tied to the technologies used by the application components. For instance, a Java application can communicate to a JavaScript application through a message broker with a message defined to be in the JSON format.
Besides that, a message broker also provides message routing, translation, persistence, and data marshaling. The following diagram shows the basic usage of a message broker:
In the above figure, we’ve Application A which may be a RESTful application, and Application B may be a SOAP web service. The message broker facilitates communication between these two applications even though the applications are discrete and do not have a common communication interface.
2.2. How Does a Message Broker Work?
Let’s talk about the building blocks involved in using a message broker.
The first item is a message producer or publisher. As the name suggests, they produce data and send messages to a message broker.
The next item is a message consumer or subscriber. They consume, i.e., read messages from a message broker.
Finally, there are a queue and a topic. These are storage components inside the message broker. Based on the defined communication pattern, a message broker stores the messages inside a queue or topic.
A message broker facilitates two major forms of communication: point-to-point and publish-subscribe messaging.
2.3. Point-to-Point Messaging
In this form of communication, there’s a one-to-one communication between the message producer and consumer. We achieve it by creating a queue in the message broker. The producer application connects to the queue to put a message. The consumer application polls the queue to retrieve the message:
In this type of communication, the producer sends a message only once, and the consumer consumes it only once. After consumption, the message disappears from the queue. The broker guarantees that unless the message is consumed by a consumer, it is safely stored inside the queue.
2.4. Publish-Subscribe Messaging
The other type is one-to-many communication between the message publisher and the message subscriber. We call it the publish-subscribe pattern.
Here, we use the terms publisher and subscriber rather than producer and consumer. The reason is that the publisher publishes a message on a topic and one or more subscribers read it. In contrast, reading a message in the producer-consumer pattern deletes it. Visually:
2.5. Advantages
Let’s now discuss a few advantages of using a message broker.
First, it allows the decoupling of application components. With a message broker, applications are decoupled and not dependent on each other’s availability for communication. For example, let’s assume application A needs to send a message to application B, but B isn’t available. Application A can put the message in the message broker and continue. Once B is up, it can access the message from the broker.
Second, a broker provides processing disparity. The message broker can act as an intermediate buffer in case there’s a processing disparity between components. For example, let’s say that producer A can process 100 messages/second and consumer B can process only 80 messages/second. The message broker bridges this processing disparity by acting as a buffer and stores the additional 20 messages B can’t consume at a time.
Lastly, there’s an option for message redelivery: in case of a consumer failure, the message broker can redeliver the messages once the consumer is available. Message brokers also provide a dead-letter queue that holds the messages that can’t be delivered to the consumers.
2.6. Disadvantages
First, the introduction of a message broker increases the complexity of an application. Splitting applications into small components and using a message broker for communication can lead to complex application architecture**.**
Second, the use of a message broker leads to eventual consistency. Although message brokers enable near real-time communication, several factors can lead to eventual consistency in the system. A delay in retrieving messages or network latency can delay the propagation of messages to consumer applications. Thus, the system can get into an inconsistent state. However, the inconsistency is only temporary since it will resolve as consumers process the messages. This temporary inconsistency of the system and eventually becoming consistent s known as eventual consistency.
Lastly, the message broker can become a single point of failure. A broker plays a critical role as an intermediary to facilitate communication. If it fails, it leads to a communication breakdown. We can minimize the risk by using a reliable broker.
3. Enterprise Service Bus
In this section, we’ll talk about an Enterprise Service Bus (ESB).
3.1. What Is an ESB?
As its name suggests, an ESB provides a bus-like architecture for connecting applications to communicate and coordinate. The ESB sets the rules of communication through the bus.
The following diagram shows an example of the ESB architecture:
There are several ESB products available and each has its own set of rules and principles that determines the communication strategies. But, why do we need an ESB in the first place? Why a broker isn’t sufficient?
3.2. Why Do We Need ESBs?
We’ve seen that message brokers facilitate point-to-point communication between two applications. If an enterprise has a lot of applications and they need to communicate with each other, enabling point-to-point communication between each pair will lead to a complex architecture that is difficult to manage and troubleshoot for issues. Additionally, even one-to-many communication also leads to a mesh-like communication pattern that is hard to manage.
An ESB solves this issue by introducing a central bus-like component to which all applications connect. That way, an ESB replaces the need for a one-to-one connection between the applications. It provides a pluggable communication architecture that makes connecting new applications with the existing infrastructure relatively easy.
3.3. Advantages
First, an ESB can translate messages between formats if needed. For example, it can convert a CSV file into a JSON message if one application sends CSVs but the other accepts only JSON.
Second, ESBs facilitate the notion of Service Orchestration. It’s common to combine and coordinate simpler services to define more complex ones. This promotes the service reusability and manageability of the existing granular services. For instance, let’s assume there’s a banking application that uses an ESB for service orchestration. Let’s also assume we need to find out the customer’s address, which requires multiple service invocations:
- Given there is a customer identifier, we first need to obtain an authentication token
- Second, we use the authentication token to retrieve the customer details that provide the primary address identifier
- Last, we use the primary address identifier to retrieve the complete address details
We can see the above constraints in the following image:
Lastly, ESBs offer Service Mediation. An ESB offers multiple interfaces for backward compatibility and allows multiple channels to interact with the same service component through service mediation.
In a heterogeneous application, there are diverse applications. Applications may run with different technologies, protocols, interfaces etc. An ESB can facilitate communication across the applications with adapters, protocol translations, and service mappings.
4. Brokers vs. ESBs
When should we use a message broker and when an ESB?
If we have a complex application design with multiple application components, a broker may not be sufficient. An ESB can be more suitable in that case. In contrast, a message broker can establish simple asynchronous communication between applications. So, if communication patterns aren’t complex, a broker can do the job.
Further, ESBs are generally expensive in comparison to message brokers. So, we should consider our budget limitations when making a choice.
5. Conclusion
In this article, we talked about message brokers and enterprise service buses. These components play a crucial role in modern application architecture by enabling the decoupling of application components with asynchronous communication.