Support Documents

Creating CloudWatch Logs for SQS by Triggering a Lambda function

Lab Details

  1. This lab walks you through the steps to creating CloudWatch Logs for SQS using a Lambda Function.

  2. You will Practice using SQS with Lambda and CloudWatch.

  3. Duration: 45 minutes    

  4. AWS Region: US East (N. Virginia) us-east-1


Amazon Simple Queue Service (SQS)

Fun fact: SQS was the first-ever AWS service that was publicly available.

  1. Amazon SQS is a reliable, easy to manage, scalable queuing service. SQS is a simple and cost-effective cloud service.

  2. AWS SQS can be used to transmit any amount of data, at any level of throughput, without losing messages. It doesn’t interrupt the other services to run continuously.

  3. SQS helps to reduce admirative tasks by scaling highly-available messaging clusters. We pay only for what we use. AWS SQS helps us to save important data that might be lost if an entire application goes down or if any component becomes unavailable. 

  4. SQS acts as a buffer between the application components that receive data and the other parts that process data.

  5. SQS is used for message-oriented applications. Sometimes processing servers cannot process requests fast enough (due to any possible reason) so the requests are queued so that the processing servers can work on them when they have available resources to process the request. This means that requests are not lost due to insufficient resources.

  6. Amazon SQS ensures that each message is delivered at least once.

There are two types of queues:

Architecture Diagram

A Simple Use Case

  1. Consider an example of a major online flash sale where people are buying a wide-range of products sold by a company. The requirement is all the purchase requests hould be processed in the order that they are received in the queue. To fulfill this requirement, we’ll be using a FIFO (first in, first out) queue.

  1. The company only has a limited amount of goods in stock, so it's important to track orders by their time of arrival. Once the request is received, it is sent to a FIFO queue before being processed.

  2. Let’s understand how messages get in and out of the queue. Assume the consuming server asks for a batch of up to 300 messages. AWS SQS starts filling the batch with the oldest message (REQ A1). SQS keeps filling the queue until the batch is full. In our example above, assume a batch contains only three requests and now the queue is empty. Once the message batch has left the queue, SQS considers the batch to be In-flight until the batch gets processed completely.

  1. When you have a single consumer, this is easy to process. The consumer gets a batch of messages, does its processing and deletes the messages. Now the consumer is ready to process and take up the next batch of the messages. You can also add the auto-scaling group to scale your processing power depending upon the requirement.

Note: SQS won’t release the next batch of messages until the first batch has been deleted.

Task Details

  1. Log into the AWS Management Console. 

  2. Create an SQS.

  3. Create a Lambda Function.

  4. Check the CloudWatch Logs.

  5. Validation of the lab