How to implement end to end VPC Endpoint service

Lab Details

  1. This lab walks you through the steps to set up an end to end connection between two VPC’s (Services Provider and customer) using Endpoint service

  2. You will practice the lab using VPC, ELB and EC2.

  3. Duration: 120 minutes

  4. AWS Region: US East (N. Virginia) us-east-1


What is the AWS VPC Endpoint ?

  • If you have an EC2 in the public subnet and want to communicate with public services in cloud-like S3, DynamoDB. This can be achieved very easily using an Internet gateway and the traffic flows through the internet.

  • Suppose if you have an EC2 in a private subnet/network where you need to read/write data to S3 (take this as an example). It is not possible to use an internet gateway.

  • Now, this can be achieved using a NAT Gateway or NAT Instance but many companies are strict about data security and won’t allow the use of NAT to connect to S3 or DynamoDB.

  • Use of NAT will cause High cost (1$ per hour), the bottleneck for IGW (huge data transfer)

  • This is why VPC Endpoint is introduced, it uses AWS Private network to communicate with your private EC2 to S3/DynamoDB and doesn’t require an IGW and NAT.

What is the AWS VPC Endpoint Services ?

  • Let’s understand this service with the help of a use case.

  • Many companies utilize service providers like DataDog, New Relic, etc which helps to monitor your cloud servers, databases and other services. For this, you need to provide your server metric (CPU utilization, Application metric, etc) of EC2 in your VPC to the service provider’s EC2 that sits in a VPCover internet.

  • Using these metrics the service provider will create a dashboard view where you can monitor your resources.

  • Since both the EC2 are in a VPC and both customer and service provider are in AWS, why send the data through the internet?

  • Instead, it uses the AWS Private network to send the data. This not only makes it secure but also saves costs.

  • Yes, You can use VPC Peering to achieve this, but what if the service provider has a huge number of customers? then we can’t use VPC Peering (VPC CIDR block should be unique).

  • In the VPC Endpoint service, the customer/consumer EC2 uses an endpoint interface network and then connects to the service provider LoadBalancer and then reaches the services provider EC2.

Network Load Balancer

  1. The network load balancer (NLB) distributes the traffic based on network variables, such as IP address and destination ports.

  2. NLB is capable of processing traffic and scaling at a much higher rate than the application load balancer.

  3. We can't use some features of the Application Load Balancer such as SSL-offloading, host-based routing, cross-zone load balancing, and a few others.

  4. The complete comparison among load balancers can be found in the link Load balancer differences.

  5. It is not designed to take into consideration anything at the application layer, such as content type, cookie data, custom headers, user location, or the application behavior.

  6. For TCP traffic, NLB selects a target using a flow hash algorithm based on the type of protocol, source IP address, source port, destination IP address and destination port.

  7. TCP connections from a client have different source ports and sequence numbers compared with NLB and can be routed to different targets. 

  8. Each individual TCP connection is routed to a single target for a connection.

  9. A UDP flow has the same source and destination, so it is consistently routed to a single target throughout its lifetime. 

  10. Different UDP flows have different source IP addresses and ports, so they can be routed to different targets. 

  11. The advantage of NLB is that it can manage the traffic to a different port to the same instance.

  12. We can split the request based on the port to different services using the Network Load Balancer, thus NLB allows you to route the traffic among multiple applications running on the same server.

Architecture Diagram

Case study

We will be creating a service provider VPC (VPC 1) with a public subnet which contains a Network load balancer with one EC2 Instance having a sample webpage and then create a VPC Endpoint service for the Network Load balancer.

For the customer side, we will be creating a Customer VPC (VPC 2) with a public subnet and create a VPC Endpoint and test the connectivity using an EC2 Instance.

Case study Diagram

Task Details

  1. Launching Lab Environment.

  2. Create a service provider VPC.

  3. Create and attach an Internet Gateway.

  4. Create a Public subnet.

  5. Create a Public Route Table and associate it with the subnet.

  6. Add the public Route in the Route table.

  7. Create an EC2 Instance.

  8. Create a Network LoadBalancer.

  9. Create an Endpoint service.

  10. Create a customer VPC.

  11. Create and attach an Internet Gateway.

  12. Create a Public subnet.

  13. Create a Public Route Table and associate it with the subnet.

  14. Add the public Route in the Route table.

  15. Create an EC2 Instance.

  16. Create a VPC Endpoint.

  17. Test the connectivity.

  18. Validation of the lab.

  19. Deleting AWS resources