This lab will walk you through the steps to create a SQS Queue, set visibility and delivery delay time. Send a message from the queue and receive the message in lambda function.
Duration: 30 minutes
AWS Region: US East (N. Virginia)
Fact: This was the first ever AWS service offering made publicly available.
Amazon SQS is a reliable, easy to manage, scalable queuing service. SQS is a simple and cost effective cloud application.
AWS SQS can be used to transmit any amount of data, at any level of throughput, without losing messages.It doesn’t interrupt other services when run continuously.
SQS helps to reduce admirative task by scaling high-available messaging clusters. While we pay only for what we use. AWS SQS helps us to save important data which could be lost if an entire application goes down or if any component becomes unavailable.
SQS queue acts as a buffer between the application components that receive the data and the other parts that process the data in the system.
SQS is used for message-oriented architecture. If the processing server cannot process the work fast enough (for whatever reason) the work is queued so that the processing servers can work on it when they have available resources to process the request. This means that work is not lost due to insufficient resources.
The default Amazon SQS ensures that each message is delivered at least once.
There are two types of Queues:
Visibility timeout is a time duration which AWS SQS avoids the other components from receiving and processing the messages.
Lets try to understand the definition by an example. So if you keep your visibility timeout to 1 minute for a big data job, what's going to happen is the message is going to come back into the queue because it's not going to complete in 1 minute.
Let's say it takes five minutes to actually process the large amount of data.The message is going to become visible in the queue and then another EC2 instance will pick it up. Because of this you could be delivering your messages multiple times because your visibility timeout is too low.
Default Visibility timeout is 30 seconds.
Increase it if your task takes > 30 seconds.
Maximum it can go up to is 12 hours.
Delay queue allows us to delay/postpone the delivery of a new message for a given period of time. We can easily turn any queue into a delay queue by configuring SetQueueAttributes to set the queue’s DelaySeconds attribute.
If we create a delay queue, any message that has been sent to this queue will remain invisible to the consumer for the configured delay time.
To create a delay queue set DelaySeconds attribute to a value between 0 to 900 seconds.
Let's try to understand the definition by an example. Consider a case where an application is trying to insert millions of data into a database. The application will send a message about the availability of this new data that has been recently inserted to other subsystems, which in turn process this message and subsequently make updates to the same row. Now there is a dependency that until the first batch completes its job and commits after the updates the next batch should not get triggered. If the message gets triggered before the updates complete, it would fail the next batch. In this case, delay delivery helps.
Create an IAM Role.
Create a SQS queue.
Create a Lambda function.
Send a message with Visibility Timeout.
Purge the Queue
Send a message with Delivery Delay
Purge the Queue