SQS — Simple Queue Service

The other day I was reading through Amazon documentation for SQS. I found it to be too elaborate with too many details and also didn’t find any good video tutorials explaining the basic of SQS.

So, here I am on medium writing this post to spread my knowledge of what SQS is, sharing my notes so that, it will be easy for others to quickly understand the important terminology and concepts behind SQS.

SQS is the oldest offering from AWS. It is scalable and fully managed message queuing service which provides a way to decouple cloud components.

Features of SQS:

• Provides high security, durability and availability.

• The default retention of messages is 4 days, and a maximum of 14 days.

• There is no limit on number of messages in queue. It has a very low latency of less than 10ms. It can have duplicate messages, as well as the messages can be out of order.

• The limitation on the size of message is 256 Kb.

Lifecycle of SQS :

• Component 1 sends message x to a queue to distributed across SQS servers.

• Component 2 retrieves the message x from the queue and start processing it. While component 2 is processing this message, other component are not able to see this message in queue for certain period of time i.e. Visibility timeout.

• Component 2 deletes message from the queue after successful processing.

• If processing fails, Component 2 doesn’t delete this message from queue so that any other component can pick it and process it.

There are two types of queue available in AWS.

• Standard (async)

• FIFO queues (Not available in all regions)

Based on your requirements you can choose any one type of the queue. But the major difference is that, in FIFO the order of message is maintained.

Delay Queue :

If you have a scenario where you don’t want the queue to receive message immediately after you send it, in that case you can go for Delay Queue. Such kind of queue can delay message upto 15 minutes. The default delay is 0 secs in standard queue. This Default value can be overridden using DelaySeconds parameter.

Visibility Timeout :

When a consumer polls for a message, the message is invisible to other consumer for a certain period. This time period is referred as visibility timeout. This is there to ensure that a particular message is not processed multiple times. Visibility timeout can be set between 0 to 12 hours ( Default is 30 sec).

Dead Letter Queue:

If message processing fails too many times then, there is an option to define a threshold after which the message can be moved to dead letter queue, where it could be debugged accordingly.

Long polling :

If a consumer is polling the queue for messages, but at that moment of time, no messages are available, then consumer consumer can optionally wait for certain period of time, for the message to become available. This is done to reduce the number of API calls made to queue. Long polling can be enabled at the queue level or at the API level using waitTimeSeconds. This time period can be set between 1 sec to 20 sec.

Extended Client :

If you want to send a message which is larger than 256kb, in that case what you can do is send the actually message to S3 (Simple Storage Service provided by AWS), and send small metadata message to SQS which contains the location of file in S3. So, when the consumer process this message, it can retrieve the appropriate file from S3 using this metadata.

Hope you liked this post. Please support by giving a like to this post. Look for this space for more such articles on AWS services.