Configuring DynamoDB Streams Using Lambda

Lab Details

  1. This lab walks you through the steps to launch an Amazon DynamoDB table, configuring DynamoDB Streams and trigger a Lambda function to dump the items in the table as a text file and then move the text file to an S3 bucket.

  2. You will practice it using Amazon DynamoDB, S3, and lambda.

  3. Duration: 1 hour

  4. AWS Region: US East (N. Virginia) us-east-1

Introduction

Amazon DynamoDB

  • Amazon DynamoDB is a fully managed NoSQL database service where maintenance, administrative burden, operative and scaling are taken care of by AWS.

  • We don't need to provide the specifications of how much we are going to save.

  • It provides single digit latency even for terabytes of data and is used for applications where very fast reads are required.

  • It is used in applications like gaming where data needs to be captured and changes take place very quickly.

Amazon DynamoDB Streams     

  • Amazon DynamoDB stream is a feature that emits events when record modifications or changes occur in a DynamoDB table.

  • DynamoDB streams captures the time level modification in the DynamoDB table in a time ordered sequence.

  • When a DynamoDB stream is enabled, it captures the changes happening in the DynamoDB tables in an orderly manner.

  • DynamoDB streams can be used to replicate the data from one DynamoDB Table of region 1 to another DynamoDB Table of region 2.

  • Events can be of the following types:

    • INSERT

    • UPDATE

    • REMOVE

  • Each event carries the contents of the rows that are being modified.

  • It records all the modifications through encrypted logs and stores them for 24 hours.

  • Whenever there is a change, DynamoDB creates a Stream records along with the primary key attribute of the items.

  • A DynamoDB Stream consists of stream record and an individual stream record, which represents a unique data modification in the DynamoDB table.

  • Each stream record is assigned through sequence numbering, which gives us the exact order in which the modification occurred.

  • Stream records are coupled into groups called shards. Each shard contains multiple stream records.

  • The shard also contains information required to access the stream records.

  • The stream records inside the shard will be time-limited with a lifetime of 24 hours and after that time the Stream records will be deleted automatically.

  • The shard can be split into multiple shards if needed for processing the records parallely with one parent shard and multiple child shards.

  • The order of processing will preference the parent first, followed by child shards.

  • Events are recorded in near real time.   

  • In real time applications, we can access the stream records that contain events where changes take place.

  • The DynamoDB Streams can be configured to catch additional information like capturing the image of the modified items before and after the modification takes place. 

  • The DynamoDB stream records can be read and processed with the help of the DynamoDB Streams endpoint.

  • Streams can be enabled while creating a DynamoDB table and can be disabled at any time.

  • Performance of the table won't be affected by both enabling and disabling the Streams because DynamoDB streams operate asynchronously.

  • Once the DynamoDB Stream is disabled, the data in the Stream will be available for 24 Hours. There is no methodology available for manually deleting the existing streams.

  • DynamoDB streams can be used as an event source for Lamda functions so you can create applications which take actions based on the events in the DynamoDB table.

  • The events that the DynamoDB captures can be analysed by moving the data into other AWS services like S3 or Cloudwatch.

Tasks

In this lab we are going to:

  1. Launch an Amazon DynamoDB table. 

  2. Insert items into the DynamoDB table.

  3. Create a Lambda function.

  4. Enable Triggers to the DynamoDB table.

  5. Make changes to the contents of the DynamoDB Table.  

  6. While the changes take place in the DynamoDB table , the DynamoDB Streams will trigger the Lambda function which will push the data to S3 bucket as a text file.

  7. Download and Verify the contents of the S3 bucket.

Architecture Diagram

 



Join Whizlabs_Hands-On to Read the Rest of this Lab..and More!

Step 1 : Login to My-Account
Step 2 : Click on "Access Now" to view the course you have purchased
Step 3 : You will be taken to our Learn Management Solution (LMS) to access your Labs,Quiz and Video courses