Configuring DynamoDB Streams Using Lambda

Lab Details

  1. This lab walks you through the steps to launch an Amazon DynamoDB table, configuring DynamoDB Streams and trigger a Lambda function to dump the items in the table as a text file and then move the text file to an S3 bucket.

  2. You will practice it using Amazon DynamoDB, S3, and lambda.

  3. Duration: 1 hour

  4. AWS Region: US East (N. Virginia) us-east-1

Introduction

Amazon DynamoDB

  • Amazon DynamoDB is a fully managed NoSQL database service where maintenance, administrative burden, operative and scaling are taken care of by AWS.

  • We don't need to provide the specifications of how much we are going to save.

  • It provides single digit latency even for terabytes of data and is used for applications where very fast reads are required.

  • It is used in applications like gaming where data needs to be captured and changes take place very quickly.

Amazon DynamoDB Streams     

  • Amazon DynamoDB stream is a feature that emits events when record modifications or changes occur in a DynamoDB table.

  • DynamoDB streams captures the time level modification in the DynamoDB table in a time ordered sequence.

  • When a DynamoDB stream is enabled, it captures the changes happening in the DynamoDB tables in an orderly manner.

  • DynamoDB streams can be used to replicate the data from one DynamoDB Table of region 1 to another DynamoDB Table of region 2.

  • Events can be of the following types:

    • INSERT

    • UPDATE

    • REMOVE

  • Each event carries the contents of the rows that are being modified.

  • It records all the modifications through encrypted logs and stores them for 24 hours.

  • Whenever there is a change, DynamoDB creates a Stream records along with the primary key attribute of the items.

  • A DynamoDB Stream consists of stream record and an individual stream record, which represents a unique data modification in the DynamoDB table.

  • Each stream record is assigned through sequence numbering, which gives us the exact order in which the modification occurred.

  • Stream records are coupled into groups called shards. Each shard contains multiple stream records.

  • The shard also contains information required to access the stream records.

  • The stream records inside the shard will be time-limited with a lifetime of 24 hours and after that time the Stream records will be deleted automatically.

  • The shard can be split into multiple shards if needed for processing the records parallely with one parent shard and multiple child shards.

  • The order of processing will preference the parent first, followed by child shards.

  • Events are recorded in near real time.   

  • In real time applications, we can access the stream records that contain events where changes take place.

  • The DynamoDB Streams can be configured to catch additional information like capturing the image of the modified items before and after the modification takes place. 

  • The DynamoDB stream records can be read and processed with the help of the DynamoDB Streams endpoint.

  • Streams can be enabled while creating a DynamoDB table and can be disabled at any time.

  • Performance of the table won't be affected by both enabling and disabling the Streams because DynamoDB streams operate asynchronously.

  • Once the DynamoDB Stream is disabled, the data in the Stream will be available for 24 Hours. There is no methodology available for manually deleting the existing streams.

  • DynamoDB streams can be used as an event source for Lamda functions so you can create applications which take actions based on the events in the DynamoDB table.

  • The events that the DynamoDB captures can be analysed by moving the data into other AWS services like S3 or Cloudwatch.

Architecture Diagram

  Lab Tasks

In this lab we are going to:

  1. Launch an Amazon DynamoDB table. 

  2. Insert items into the DynamoDB table.

  3. Create a Lambda function.

  4. Enable Triggers to the DynamoDB table.

  5. Make changes to the contents of the DynamoDB Table.  

  6. While the changes take place in the DynamoDB table , the DynamoDB Streams will trigger the Lambda function which will push the data to S3 bucket as a text file.

  7. Download and Verify the contents of the S3 bucket.

 



Join Whizlabs_Hands-On to Read the Rest of this Lab..and More!

Step 1 : Login to My-Account
Step 2 : Click on "Access Now" to view the course you have purchased
Step 3 : You will be taken to our Learn Management Solution (LMS) to access your Labs,Quiz and Video courses