1. Introduction
In this tutorial, we’ll talk about two of the top services that AWS provides to users: SNS and SQS. First of all, we’ll have a short description of one and the other, peeking at some simple use cases. Then we’ll point out the main differences between them, looking from different angles. Finally, we’ll see how those services express great power and capabilities when coupled together.
2. SNS Definition and Use Cases
Users utilize Amazon Simple Notification Service as a managed service for sending real-time notifications. To understand SNS easily, we can focus specifically on three objects: the topic, the publisher, and the subscriber. One topic can receive messages from multiple publishers and can deliver the same message to multiple subscribers. Every message a publisher sends to the topic will reach all subscribers registered:
Let’s talk about the journey of a message from a publisher to a subscriber. First of all, both publishers and subscribers need to have permission to read and write from the SNS topic. We can define permissions with IAM Access Policies. Then, when the message reaches the topic, it’s stored in an encrypted-at-rest storage that is used to retry the delivery of the message if something fails.
Looking at the image, we can observe that there are two types of topics: Standard and FIFO. The key differentiation lies in the fact that only the latter ensures that the sequence of message delivery matches the order of their publication.
Prior to dispatching messages, they undergo a procedure that involves the activation of Data Protection Policies. These policies are crafted to furnish an elevated level of safeguarding for Personally Identifiable Information (PII) and other forms of sensitive data. Techniques such as data masking are implemented as integral components of this protective process.
Finally, subscribers can receive the message using various protocols. It’s also possible to define specific filtering policies for every subscriber so that some messages are discarded and not sent, and a Dead Letter Queue (SQS queue) to handle delivery failures and manual retries for a specific subscriber.
Users can deliver time-critical notifications using Amazon SNS. For example, with a monitoring tool such as Datadog, we can use SNS to send system alerts triggered by predefined thresholds in case something in our application is misbehaving. Another example could be to use SNS as a messaging system to send updates to every user subscribed to our application, using email, SMS, and mobile push notifications. SNS with SQS is the fundamental building block of any AWS Cloud Based application for implementing the “Fanout” scenario.
3. SQS Definition and Use Cases
Amazon launched AWS SQS Simple Queue Service back in 2004 and it was one of the first managed services available to users. It has become one of the fundamental building blocks of many cloud-based applications. It finds its primary use in enabling asynchronous communication between different software components. We know from experience that managing a queue presents a series of challenges, such as the bounded-buffer problems, and that managing a distributed queue is even more difficult because of the communication that we need to enable between components to manage concurrent writes and reads. SQS helps solve all those problems in an easy way.
Let’s now talk about a message’s journey from a publisher to a subscriber. The first part is similar to SNS: publishers and subscribers need permission to read and write from the SQS queue. As in SQS, we can define those permissions with IAM Access Policies. Then, when the message reaches the queue, the system stores it in encrypted-at-rest storage. A delivery retry mechanism will read from this storage if something fails.
The queue, in this case, can be Standard, FIFO, or Delay. We can pick a FIFO queue to maintain the ordering or a delay queue to delay the delivery of the message by a predefined amount of time. The consumer can read from the queue using two different mechanisms: short polling and long polling.
Any use case in which we need a decoupling of communication between software components could make good use of the SQS service. For example, in the SNS infrastructure, SQS serves as the underlying implementation of the Dead Letter Queue, handling delivery failures and manual retries for a specific subscriber.
4. SNS vs. SQS: Differences
Let’s now summarize the key differences between SQS and SNS:
Comparison
SNS
SQS
Entity Type
Topic (standard and FIFO)
Queue (standard, FIFO, and Delayed)
Message consumption
Push Mechanism, SNS push messages to subscribers
Pull Mechanism (Long and Short Polling)
Delivery Guarantee
At least once Delivery
Exactly once Delivery
Number of Subscribers
Best suited for multiple subscriber use cases
Best suited for single-subscriber use cases
Communication Type
Real-Time, A2A and A2P
Delayed communication, only A2A
5. Why Coupling SNS and SQS Together?
The “fanout” scenario is the typical use case in which we need both SNS and SQS working together. In this case, messages are sent from SNS topics and then replicated to different SQS queues. The queues are subscribed to the service. This, of course, allows for parallel asynchronous processing.
For example, let’s suppose we need to build a video streaming platform. When a user uploads a new video, we publish an SNS message to a topic with the link to the item (stored in an S3 bucket). The topic establishes connections with several SQS queues. The queues will process (concurrently) different encoding and video quality for the same video. Then, a series of independent applications read from those queues and process the workload asynchronously:
The Dead-Letter Queue used to retry SNS message delivery is another example of using SNS and SQS together. In this way, clients and applications achieve real-time communication with better resiliency and fault tolerance.
6. Conclusion
In this article, we have described two of the most used software solutions of the AWS Cloud Provider: the Simple Queue Service and the Simple Notification Service. We discussed the key characteristics, we peeked at the basic functionalities, and we compared the most important features. Finally, we shared a common example of usage in which SNS and SQS combined solve some typical software design problems.