Author: PH

PH http://awsnetworks.wordpress.com
PH's avatar

4. AWS Direct Connect (DX)

AWS Direct Connect becomes appealing when you need more than a best-effort assurance of reachability to one or more AWS regions.

Many people think of Direct Connect as more of a privacy construct or something you use when you move to a colocation facility. But really, it’s not – both from an exam and practical perspective. Direct Connect gives you a few different capabilities:

  • Deterministic bandwidth availability and latency
  • Increased privacy IF you encrypt your traffic over direct connect (VPN over direct connect)
  • Attach non-AWS services to your VPC (security infrastructure, legacy or license-restricted compute, VDI, high performance database or compute).

Fundamentals:

  • Physical: 1 (1000base-LX) or 10 Gb (10Gbase-LR) Ethernet connection over Single Mode Fiber (1310nm) to AWS or an AWS partner. The only time you would want to use Direct Connect via a partner are when either you want to subscribe to less than 1Gb OR when Direct Connect is not available through your facility.
    • Redundancy in a single location can be an inexact science. You can order multiple direct connect handoffs to a given region, but it is still possible for AWS Direct Connect maintenance or that rare outage to affect all of your handoffs.
    • The most common AWS-provided recommendation has been to order two Direct Connect connections at the same time, and AWS will do their best to engineer them on separate physical paths as much as possible.
  • Error Detection:
    • (BFD) Bidrectional Forwarding Detection is optionally supported on your Direct Connect circuit. This is used to trigger faster failover when you have redundant connections.
    • BGP peering will also go down with or without BFD. If you have no secondary connection you might be better off relying on a 30- to 90-second BGP failure as your official warning that AWS has gone unreachable, but if you have a redundant path the BGP outage will probably come too late for a failover to be hitless.
  • Logical: 802.1Q VLANs are used to segment the direct connect to several AWS termination constructs available. A common construct from 2013-2018 was to assign one vlan to each VPC and connect a trunk with all the VPC vlans to your enterprise. Newer constructs like AWS Direct Connect Gateway are starting to change this.
  • Routing: BGP is the only dynamic routing protocol supported. From my observation, it appears AWS uses a feature in Junos called Maximum Prefix which makes very important to you never advertise more than 100 prefixes to your AWS peer construct (see feature below).
  • AWS Account: An important infrastructure management and IT controls decision is which AWS account to request the Direct Connect under. Typically this is performed from either a designated master account or shared infrastructure account rather than one used by application or business teams.
Junos prefix-limit options. Observed behavior is AWS will tear down the BGP sessions indefinitely until the prefix length is 100 or less.

AWS Construct Fundamentals (Points of Attachment)

After understanding the networking fundamentals, you need to understand the AWS constructs that are between your organization and your VPC(s).

Slide from a 2018 AWS presentation (can’t find actual source.)
  • DX PoP: The true DX PoP is an AWS-operated cage inside of a colocation provider – typically near an AWS Region. Equinix and CoreSite are examples of these colo providers. [Todo: add colo provider section. See question: Do I need to be in a colo to use direct connect?]
  • DXCON: DX Port is the 1G (1000Base-LX) or 10Gb (10Gbase-LR) physical network interface and then DXCON is logical AWS construct that is your connection to AWS. Only single more fiber is permitted for this connection. If you require more than 10Gb of Direct Connect service you will likely have a bundle of 2 or 4 DX Ports (max allowed) associated with a single DXCON. You can bundle 1Gb ports as well, but in my experience if you require more than 1Gb, jumping to 10Gb is something you won’t regret.
  • Virtual Interface

Questions

 

VPC Routing – Internet

You have web services and an EC2 database server running in your VPC.  You want IPv6  anccrm for both layers.    Your VPC has one private subnet 10.0.1.0/24 and hosts the DB instance.  This subnet will require only outbound IPv6 and currently has outbound IPv4 using a NAT GW.  What should you add to this subnet for IPv6?

  • Add an Egress Only IGW to the subnet and add a static route for all v6 traffic to use this.
  • Or add an IGW and static route

What is the best way to make two EC2 instances in private subnets that are the back-end of an internet-facing load balancer reachable?

[missing]

Your VPC route table has two routes to 10.1.1.0/24.  One has a propagated VGW target, and one has a manually added IGW target.  Which will be preferred?

  • Static routes take priority over propagated routes

What are three types of subnets in an AWS VPC?

  • Public subnet: has an Internet Gateway and a default route to that IGW.  You can only have one IGW for a VPC.
  • Private subnet: does not have internet gateway.
  • VPN only subnet: only routes traffic to a Virtual Private Gateway
  • Worth knowing that any HA design requires minimum 2 public and 2 private subnets.

What limits and special conditions apply to Elastic IP addresses?

  • 5 Elastic IP addresses per region.
  • They are free if in used, but a charge applies for unused EIPs.

VPC Peering and Endpoints

A customer has an S3 bucket in US-East-1 and a VPC with EC2 instances on private subnet 10.75.85.0/24.  EC2 instances need to be able to access S3 bucket without traversing internet.  The customer would like to use the most cost effective solution possible.  Options?

  • Create a VPC endpoint for the S3 bucket and update the routing table for the EC2 subnet
  • Add an Elastic IP to one fo the EC2 instances and use it to access S3 bucket

 

You use a NAT GW for your EC2 fleet to send content to and from your S3 bucket.  Data transfer costs are getting expensive.   How can you reduce costs?

  • Create a VPC endpoint for your S3 bucket

 

Transitive routing:  If A is peered to B, and A is peered to C, there is nothing you can add to C’s route table to reach subnets in B.

You have a Postgres cluster in AWS using private IP addresses using a security group to control access to all nodes in the cluster.   How can you ensure disaster recovery for these nodes in another region SECURELY?

  • Create a VPN IPSec tunnel and ensure the nodes in the other region allow the VPC CIDR block in their security group.

Direct Connect and VPN

Company has a Direct Connect endpoint in US-West to on-premise infrastructure using a direct connect partner.  Currently this endpoint has 9 VIFs  You would like to add more.  What are your next steps?

  • Log into the AWS console and start provisioning

 

Company has a Direct Connect in US-East-1 (10.1.0.0/16) to their data center  but also wants to use it to connect to US-East-2 (10.2.0.0/16).   After updating BGP, traffic from East-1 is not reaching East-2.  Both VPCs can still access the corporate network.  What might be the cause?

  • Forgot to enable route propagation?
  • VGWs in East-1 and East-2 are using the same AS

 

What is the limit on the number of propagated BGP routes in a VPC route table?

  • 100

 

How many Direct Connect (DX) endpoints do you need to access all US regions over direct connect?

  • 1

 

You have 3 VPCs, 10 S3 endpoints and 10 EC2 instances in each VPC.  What is the minimum number of Virtual Interfaces for all of your resources to have access to the Direct Connect(ion)?

  • 4 – one public VIF for the S3 endoints, plus one for each VPC

 

What are the minimum requirements for your network device to support Direct Connect?

  • VLAN support (802.1Q)
  • BGP and BGP MD5 support
  • Disable autonegotiation
  • Single-mode 1000Base-LX or 10GBASE-LR physical interface

 

You are using two Direct Connect circuits in a Primary/Backup  pattern, but you are finding it takes too long for your Backup to take over when the Primary connection becomes sick but not dead.  The same solution should apply if you are using VPN as backup.  What will make failover faster?

  • BFD

 

You need to aggregate and scale Direct Connects to achieve more data throughput to AWS.  You decide to use a LAG group.  What rules apply?

  • Direct Connect links must al be the same bandwidth (1Gb or 10Gb)
  • Direct Connect links must terminate on the same endpoint o the AWS side
  • 4 connections max per LAG
  • Each connection counts against your region connection limit

 

You have a large amount of on-premise traffic going to S3 that needs to be low, consistent latency and encrypted.  What solution will help?

  • Use an AWS Managed VPN (for encryption) over Direct Connect.

 

You are setting up a VPN connection between your VPC and on-premise infrastructure.   You need to ensure the firewall rules allow this on premise.   What ports do you need to allow?

  • UDP port 500
  • IP protocol 50 (ESP)

 

You are setting up a VPC Managed VPN.  IKE Phase 1 is established but IPSec Phase 2 is failing.  You have the correct IKE version, customer gateway and encryption method.  What else might the problem be?

  • UDP 500 might be blocked

 

From an AWS perspective, what VPC architecture is recommended for a highly available, low-cost VPN connection to your VPC?

  • VPN connection is already highly available.   AWS provisions two separate VPN endpoints for you to peer from your CGW.  Anything more would increase cost.

 

IPSec VPNs are also supported on EC2 instances.  Why might the AWS Managed IPSec VPN be more resilient?

  • EC2 instances place more responsibility on the customer as AWS only manages the underlying host and has no responsibility for resiliency if the host fails health checks.

 

Your AWS Managed IPSec VPN to on-premise customer gateway is configured, but when you check the console tunnel status it still shows down.  What will bring it up?

  • Ping an AWS destination (behind the tunnel) from your CGW.

 

You require a GRE VPN between your AWS VPC and a remote site.  How?

  • AWS Managed VPNs support only IPSec.  So you will have to create an EC2 instance that runs the GRE VPN endpoint and connect that to an Internet Gateway.

 

You have an AWS Direct Connect in US-East-1 and are using a public VIF to access an S3 bucket in the region.  You want to use the same Direct Connect to access a S3 bucket in US-West-2 for safekeeping.  How can this be achieved at the lowest cost?

  • Create another public VIF of of your direct connect.   This allows all network traffic to remain on the AWS network backbone.

 

You have a Direct Connect in US-East-1 and want to use it to access a VPC in another region?  What are two ways to achieve this?

  • Create a Direct Connect Gateway and connect it to a private VIF in the remote VPC
  • Create a Public VIF off your Direct Connect  and then a VPN connection to remote VPC

 

Load Balancing

You have just provisioned an Application Load Balancer to distribute your web traffic to a cluster of EC2 instances in the same subnet of a VPC.   When you test the web site you receive HTTP 504: Gateway Timeout.   What are likely causes of this error?

  • EC2 subnet is missing a NACL to allow high ports out (1024-65535)

Company wants to send mobile web traffic (url is /mobile) to memory optimized instances and api calls (url is /wapi) to their largest compute instances.  What can enable this?

  • Use an ALB with path-based routing (“You can use path conditions to define rules that forward requests to different target groups based on the URL in the request (also known as path-based routing”).

You are using a Classic Load Balancer for both http:// and https:// connections, but you want your back-end logs to show the client IP, not the load balancer IP.

  • Set the X-Forwarded-For header on the load balancer

You have an online commerce application running on EC2 instances (autoscaling and multi-AZ) behind Application Load Balancer service.  The application tier must read and write data to customer managed (not RDS) database cluster.  You want to ensure no access to these databases from the internet, but you do need to retrieve database patches over internet.  What meets this need?

  • Public subnets for the application tier.  NAT-GW and private subnets for db cluster.

DNS – Route 53

You want to access your RDS DB using your private domain name mydb.acme.org instead of the AWS-provided name.  What will enable this?

  • Create a CNAME

For both performance and reduced cost, what is a good pattern to use to allow all of your VPCs to access your on-premise DNS servers?

  • Create a hub VPC and replicate your on-premise DNS there.  Then allow all your other VPCs to access this either through peering, VPC endpoints or proxy.

You have a private hosted zone in AWS but want to sure on-premise resources can lookup resources in the AWS zone.  Which approach is least effort:

  • Simple AD.  Point on-premise systems to Simple AD; allows resolution of external DNS queries.  Don’t forget to open your NACL to allow traffic from outside the VPC.
  • AWS-Simple-AD-Blog

 

SECURITY POLICIES

Your EC2 web instance has both a Security Group (allow inbound  on port 80 from 0.0.0.0/0, deny outgoing traffic) and NACL (same rules as SG).  Users cannot access the web site.  What needs to change?

  • NACL needs to allow outgoing traffic for ephemeral ports (1024-65535 is the broadest range for these ports)

 

You are unable to ping an instance in another subnet in your VPC.  Flow logs show your inbound ping as ACCEPT OK followed by REJECT OK.   What is the likely fix?

  • NACL needs to allow outbound ICMP

 

 

 

 

 

PERFORMANCE

[Jumbo frames] A peer manager suggested using jumbo frames to improve performance on a VPN connection between your data center and VPC.  In what situations can jumbo frames help?

  • Jumbo frames allow packet payload to exceed the 1500 byte limit – fewer packets are needed to send the same amount of data.
  • Within AWS Jumbo Frames function: within a VPC or VPC peering connection within the same AWS region.
  • Best used in Placement Groups for cluster computing.
  • If accidentally set for traffic leaving the VPC,  jumbo frames can hurt performance

 

Your EC2 instance will be processing a large inbound traffic flow, and you want to ensure maximum performance for network packet processing.  Two suggestions?

  • Choose an instance type that supports Enhanced Networking (SR-IOV)
  • Enable Jumbo frames (MTU 9001) on the instance

 

You need optimal network performance between EC2 instances in different regions with peered VPCs.  What tools do you have?

  • Ensure the OS and instance type support Enhanced Networking
  • Jumbo frames are NOT supported in VPC peering.

 

 

 

 

MONITORING

Application teams who do not manage Direct Connect (or VPN) want to be alerted when the connection to the headquarters office is down.  How can they do this?

  • Use Cloudwatch metrics to check for the state of the direct connect (or tunnel).
  • Metric is called ConnectionState (0=Down, 1=Up), and you create an alarm for it

 

You want to be alerted any time resources are created.  Which 3 services are most helfpul for this?

  • AWS Config – continuously monitors and records your AWS resource configurations.  Was created specifically to trigger on resource changes.
  • AWS Lambda can respond to that trigger to send a message to SNS
  • SNS

 

You have a Lambda function that looks for EC2 events and sends requests to SQS.  How is this built?

  • Create a NAT instance in your VPC to allow Lambda to talk to services that require internet.
  • Ensure the VPC configuration is added to the Lambda function.

 

 

CLOUDFRONT

Your EC2 instance will be a custom origin for a Cloudfront web distribution, but you want to ensure traffic is encrypted in transit.  What is required?

  • Configure the Cloudfront Viewer protocol policy to redirect HTTP to HTTPS and change the Origin Protocol Policy to Match Viewer.

 

BILLING $$$

A department has set up their own AWS account that is not part of consolidated billing used by the rest of the company.  They have a Direct Connect to their VPC using a private VIF.   When downloading data from an EC2 instance in the VPC, how could the charges comes across?

  • They would be charged for the data transfer out.

 

 

3. Hybrid AWS – VPN Attachment

 

You need a VGW regardless of how you connect your VPC to private infrastructure –

  • Virtual Private Gateway is an AWS-managed gateway for your VPC, and it is used for both VPN and Direct Gateway.   It is created and “attached” to the VPC
  • Create the VGW under VPC and give it a BGP Autonomous System Number (specify AWS default or give your own preferred) and then attach to a VPC.
  • ASN cannot be changed after creation.
  • VPC can have only 1 VGW, and it can be detached if you need to change VGWs.
  • You receive an error if you try to attach a second VGW to your VPC
    Screen Shot 2018-08-23 at 9.46.26 PM
  • With VPNs, the VGW provides an additional level of resiliency because it establishes egress points in two Availability Zones

Screen Shot 2018-08-23 at 9.54.55 PM

 

Site-to-Site VPN

  • Site-to-Site IPSec VPN connects your private network (wherever it is – data center, small office, home, Digital Ocean, Azure, even another VPC) to your VPC over public internet.  VPC only supports this type of VPN.
  • By default you receive TWO VPN connections to AWS. This is most useful for avoiding outages when AWS performs maintenance.  In my time with AWS this has happened no more than once per year, but never happens at a convenient time.
  • VGW is the preferred VPN termination point.   Creating an internet-accessible EC2 instance inside of your VPC is a secondary option for custom requirements.
    • AES 256, SHA2, Phase 1 DH groups 2, 14-18,22,23,24 Phase 2 add 1 and 5
    • VGW does not have backward compatibility with older protocols
    • Static and BGP routing only (standard TCP port 179)
    • Max 100 propagated routes before AWS terminates the peering session.
    • No debug information since VGW is a managed entity.  AWS TAC can provide.
  • VGW support only IPSec VPN in ESP mode.  Less secure options are a reason for some VPCs to use the EC2 VPN option.

 

Internet Protocol Security (IPSec) Basics

  • Suite of protocols to provide Confidentiality (encryption), Integrity (prevent data corruption) and Authentication (keys and certificates).
  • Core components are:
    • IPSec Authentication Header (AH – IP protocol 50) to allow parties to verify authenticity
    • Encapsulating Security Payload (ESP – IP Protocol 51)
    • Supporting components include Encryption Hashing Algorithms, Security Policy Associations, Internet Key Exchange (IKE) Key Management
  • Protocol Options for IPsec Functionality:
    • Confidentiality: DES, 3DES, AES
    • Integrity: MD5, SHA
    • Authentication: Preshared Keys, RSA
    • Key Exchange Algorithms: DH1, DH2, DH5, DH7
  • IPSec connection establishment happens in two phases:
    • IKE Phase 1 – endpoints authenticate and agree on a keying material and create session keys
      • AWS requirement for this uses preshared keys.
    • IKE Phase 2 –  endpoints use the new secure tunnel to negotiate Security Associations that are used to encrypt the traffic.   A separate tunnel is created for the data transport.
      • AWS allows AES 128- or 256-bit encrtyption, SHA-1 or 2 (256) Hashing
      • AWS uses DH Perfect Forward Secrecy or other supported DH groups
  • IPSec Transport Mode encrypts only the payload, leaving IP header unprotected.  Not supported in AWS.
  • IPSec Tunnel Mode builds a new IP header, so even the original IP header is encapsulated in the tunnel.  AWS VPN Concentrator (Juniper) only supports tunnel mode.  Packet fragmentation is performed prior to encapsulation.
  • Firewall requirements to allow IPSec communication to flow:
    • IP Protocol 50 (ESP) in/out
    • IP Protocol 51 (AH) in/out
    • UDP port 500 in/out
    • Optional UDP 4500 (NAT-Traversal)
  • IPSec VPN routing can be static or dynamic (BGP)
    • BGP peering must have tunnels bound to logical interfaces (route-based VPN)
    • BGP peering must support dead peer detection
  • Charged at 5 cents per connection hour, regardless of status.
  • VPN Limits on your account:
    • 50 customer gateways and 50 VPN connections per region
    • 5 Virtual Private Gateways per region
    • 10 VPN connection per VPC (per VPG)

 

Setting up the VPN

  1. Create the Virtual Private Gateway and Attach to your VPC:

Screen Shot 2018-09-01 at 9.14.05 PM

2. Create a Customer Gateway (basically provide AWS with the device type and public IP address of your on-premise VPN termination).

3. Create a VPN connection which binds your VGW and CGW.

Screen Shot 2018-09-01 at 9.26.53 PM

4. Download and Apply the VPN configuration file for your on-premise device.

5. Generate traffic from your on-premise network to AWS to initiate the VPN connection.  The Customer Gateway must initiate the VPN connection, and with many appliances it requires either keepalive traffic or traffic destined to the VPC (“interesting traffic”)

 

2. VPC Networking Fundamentals

The fundamental topics in this section underpin most of your communication options in AWS.  They are essential building blocks for how your servers communicate, how you route traffic between environments, and how you uncover why your traffic reaching its destination.

VPC Creation

  • VPC is the fundamental network container for a set of AWS resources (EC2 instances, RDS databases, connections to outside networks) that you define.
  • VPC spans an entire region and then breaks down into subnets in different AZs.
  • On creation, you assign a private CIDR block (RFC1918 or public IP addresses if you wish but not recommended)
  • For IPv4: Max size /16.  Smallest size /28.
  • For IPv6: fixed size of /56.  The address range is assigned by AWS as it comes from their Global Unicast Address space – AWS ip-ranges.
  • Dual stack IPv4/v6 is possible, but understand the routing and security groups are handled independently.
  • VPC CIDR block cannot be changed after VPC creation.  A common exam question here is what to do when you want to change instance IPs to a different CIDR block.  The answer is create a new VPC and migrate the instances.
  • New accounts always start with a default VPC if 172.31/16 (and no IPv6)
  • “DoNotDelete” appears in the name of the default VPC because if you delete it, according to AWS you will experience issues using other AWS services that expect a default VPC to exit – even if that is not where your EC2 instances reside.
    Screen Shot 2018-08-18 at 9.54.49 PM

Subnets

Screen Shot 2018-08-18 at 9.56.46 PM

  • The default VPC is then broken up into /20 subnets for each AZ.
  • A subnet is a segment of the VPC that must reside within a single AZ.
  • Each AZ can have Zero-to-multiple subnets.
  • IPv4 max subnet size must be within the VPC CIDR range, and minimum is /28.
  • IPv6 subnets are fixed at /64 and can be disassociated and readded later.
  • Subnets are classified as
    • public: route table must target an Internet Gateway
    • private: route table does not contain IGW, but perhaps a NAT Gateway
    • VPN-only: route table must target the VPC VGW or EC2 instance vpn
  • Default VPC contains one public subnet in every AZ.  Default NACL allows everything in and out.
  • Default additioan
  • .1, .2, .3 are reserved in every VPC subnet.  .1 is the gateway, .2 is DNS, .3 future.
  • AWS default DNS server runs on the .2 address of the base VPC CIDR range (e.g. 172.16.0.2) and is reachable at 169.254.169.253
  • Tags are helpful for identifying subnet use
  • The main difference between Public and Private subnets is simply that Public subnets have an IGW and NACL that allows internet flows.

Route Tables

  • Default VPC contains a main route table that is used by all subnets.

Screen Shot 2018-09-01 at 4.59.24 PM

  • Every VPC subnet contains an implicit router – the next-hop gateway for the subnet.  This can be customized for each subnet, and one of the later recommendations is to have a dedicated route table per subnet for maximum flexibility.
  • Route table is the set of route entries used by the implicit router

Screen Shot 2018-09-01 at 5.07.37 PM

  • Targets of route tables (the way out) have special names and can include
    • Internet Gateway (igw-qwert123)
    • NAT gateway
    • egress-only IGW (eigw-qwer123)
    • Virtual Private Gateway
    • VPC gateway endpoints (vpce-qwert123)
    • VPC peers (pcx-qwert123)
    • Elastic Network Interfaces
  • Targets that are “local” cannot be removed.  These are entities on the subnet.
  • In general, the Local Routes are preferred first, followed by Longest Prefix routes.
  • When there are multiple options for a route, Route Priority is used to prefer:
    • 1. Local route (even if more specific route exists)
    • 2. Most specific route (longest prefix match)
    • 3. Static routes (preferred over dynamic for equal prefix)
    • 4. Dynamic routes from Direct Connect
    • 5. Static routes configured on a VGW VPN
    • 6. Dynamic routes propagated over VPN
  • Longest prefix preference is not applied when propagated routes and static routes conflict.  Instead the static route is preferred in this order of targets:
    • IGW
    • VGW
    • Network Interface
    • Instance ID
    • VPC Peering (PX)
    • Nat GW
    • VPC endpoint
  • If you fail to associate a subnet with a specific route table, Main RT is used.
  • The Main Route Table cannot be deleted, but you can move subnets to a different route table, and once moved the Old Main can be removed.   This can be a nice way to safely stage/test/migrate your Direct Connect environment
  • Limits:
    • Hard: 200 Routes/VPC
    • Default: 50 Non-propaged routes/RT (Max 100 at potential network performance cost)
    • Separate (same) limits for IPv4 and IPv6

IPv4 Addressing

  • Private (you assign) or Public (assigned from AWS owned IP address space)
  • Manually assigned (if within your VPC CIDR) or automatic.
  • EC2 primary interface IP addresses are retained until the instance is terminated.
  • EC2 instances can also have Elastic Network Interfaces and the private IP is listed as a secondary address on those interfaces (also retained until ENI is deleted).
  • EC2 Public IP addresses can be automatically assigned at launch or added later, but you cannot keep an EC2 public IP if the instance is stopped or terminated.

IPv6 Addressing

  • Link-local addresses (fe80://10) are required on every interface for DHCPv6, and neighbor discovery.  The VPC implicit router expects a modified EUI-64 format where the mac address of an ENI is converted into the interface IP with FF:FE as follows: [first 24 bits of mac]:FF:FE:[second 24 bits of mac]
  • Global Unicast Address (public) is allocated for all IPv6 assignments.  Because AWS owns these allocations from the relevant Regional Internet Registry, the public IPv6 block is always assigned by AWS starting with a /56 for the entire VPC and /64 for each subnet in each region.
  • You can assign the host-level bits for an Elastic Network Interface within your allocated subnet.  You can also create an attribute that takes precedence over the automatic assignment and influences the assignment.
  • Like IPv4, the IPv6 can be added after launch.

Elastic IP Address

  • Static, public IP that you purchase from AWS and allocate in your account
  • Ideal for maintaining a fixed IP regardless of what you are doing with the underlying set of EC2 instances or other created services.
  • EIP is first assigned to a VPC within a region and then can be assigned to an instance.
  • Assignment is essentially a mapping of the EIP to a Private IP.  Mapping allows you to move the EIP to another instance.
  • EIP remains associated with your AWS account until you release it.
  • You are not charged for the first Elastic IP address assigned to an instance as long as it is running.  Additional EIPs per instance or on a stopped instance charge hourly.

Screen Shot 2018-08-19 at 11.36.03 AM

Security Groups (sg-xxxxxxxx)

  • Allow only stateful firewall.  Default allows all outbound/denies all inbound.
  • Stateful means you don’t have to explicity allow reverse traffic.  If you allow ssh or https outbound the virtual firewall will allow the response to come back.
  • With VPC peering, you can reference a security group ID from a peer VPC to automatically adjust the sg- for autoscaling events.
  • Max 500 SG per VPC (50 inbound rules/50 outbound rules per SG)
  • Up to 5 SG can be associated with any network interface.  You can change the assignment of an SG to an instance or interface, and change applies immediately.
  • New Security Group always has a default allow all outbound rule.  Can be removed.
  • Instances associated with the same SG cannot communicate with each other unless you add a rule allowing the SG to communicate with itself :
Inbound Rules
Protocol Type Protocol Port Range Source Comments
All traffic All All sg-1a2b3c4d Enables instances in same security group to communicate w/each other.

Network Access Control Lists (NACLs)

  • Stateless firewall on a VPC subnet
  • Remember to explicitly allow return traffic.  This is a very common mistake.
  • Ordered list of rules that AWS evaluates lowest # first until either an allow condition is met or mandatory final deny all rule is reached.
  • Elements: Rule #, type, protocol, port range, source, allow/deny (see below)
  • Default NACL has two inbound, two outbound rules to allow all before deny all.

Screen Shot 2018-08-19 at 11.23.23 AM

Internet Gateway

  • AWS managed Target in VPC route table for internet-bound traffic.  Horizontally scaled, highly available.
  • Default VPC always has a default IGW in the main route table:

Screen Shot 2018-08-19 at 11.38.18 AM

  • If you build one manually the steps are to create + attach the IGW to your VPC, create a default route in the VPC route table with IGW as target, ensure NACLs and SGs allow traffic to and from the IGW instance.
  • If you do not use an IGW, you still still reach the internet by:
    • Assign public IPv4 or Elastic IP
    • Assign IPv6 (always public)
  • IPv6 also has Egress-Only Internet Gateway, since all instance IPv6 instances have a public address (IPv6 does not require or support NAT).

NAT Instances and NAT Gateways

  • Specifically exist for private subnets to have a method for accessing public services, including repo updates, development libraries, applications, and AWS public endpoints.
  • Port Address Translation (many servers to 1 public IP) is used. Max 65,000 active flows.
  • NAT GW is generally recommended over NAT instance: higher bandwidth, easier, HA.  NAT GW uses ports in the range 1024-65535.

Screen Shot 2018-08-19 at 11.56.10 AM

  • NAT instance maintains a translation table of transit traffic.  Can use either automatically assigned public IP or EIP.  Forgetting to disable the instance source/destination check on the instance is a common mistake.
  • amzn-ami-vpc-nat is the prebuilt AMI for NAT instance usage published by AWS.
  • NAT GW requires an EIP.
  • For AZ resiliency, you should have separate NAT GW in each AZ.  This is party why the recommended use of route tables is a separate route table for every AZ subnet.

Attaching External Networks

  • Virtual Private Gateway is VPC-level construct that performs routing to an external connection.  The VGW manages all edge routing separate from your VPC route tables. It is separately created and attached to a VPC.
  • VPN and Direct Connect are the two types of external connection you can have.
  • Customer Gateway is the physical or virtual router on the customer side of the VPN or Direct Connect link.  All tunnel negotiations are initiated by the CGW.
  • Virtual Private Network connections are configured after you have created a VGW and specific the IP address and device type of your CGW.   Firewall rules supporting the VPN must UDP 500 and IP protocol 50 (ESP).Screen Shot 2018-08-19 at 12.15.19 PM
  • Two IPsec tunnels are built by default for each VPN connection to keep the connection up during AWS maintenance.
  • VPN connections allow static or BGP routing, but Route Propagation must be enabled for the VPN routes to be advertised into your VPC route tables.
  • Direct Connect basic requirements are simply single mode 1Gb or 10Gb connection and a customer gateway capable of VLANs and BGP with MD5 Auth.  BFD is automatically enabled on the AWS side, but optional on the customer side.
  • Direct Connect has the unique benefit of being able to attach public VIFs to services like S3 in ANY REGION, enabling multi-region from a single direct connect.

VPC Endpoints

  • An offshoot of AWS PrivateLink, VPC Endpoints allow you to create private pathways to public AWS services like S3, Kinesis, KMS, SNS and AWS control APIs.
  • Endpoint is identified as pl-qwert123 (pl for privatelink)
  • Your EC2 instances can use their private IP to reach endpoint services
  • Two types of endpoints: interface and gateway.
    • Interface endpoints are effectively an ENI with a private IP for traffic to the desired service.  Interface endpoints support Kinesis, APIs, AWS Systems Manager, Service Catalog and endpoint services hosted by other marketplace partners.
    • Gateway endpoints are a route table target for the service you are reaching.  Gateway endpoints support S3, DynamoDB
  • Gateway endpoint you specify the VPC, service, policy and route table:

Screen Shot 2018-08-19 at 7.21.17 PM

Screen Shot 2018-08-19 at 7.23.30 PM

  • Remember to tailor the policies of the service to expect traffic from the endpoint (e.g. S3 bucket police should access from the VPC or VPC endpoint id (pl-xxxxxxx)

VPC Peering (pcx-qwert123)

  • Connect two VPCs together as if they were on a shared network.  Always 1:1.
  • Can be in same region (additional advantages) or different region, but must be in the same AWS partition (aws, aws-cn, aws-us-gov).
  • Request accept proposal made to accountID/VPC ID.  7-day response window.
  • Once established, both VPCs need to add the subnet/PCX targets to their route tables
  • Overlapping CIDR blocks won’t work.  Partial prefixes can be used if you don’t want to restrict yourself from an entire large block in the future.
  • Within region peering allows you to:
    • Reference shared security groups
    • Enable DNS hostname resolution across VPCs
    • Use Jumbo frames and IPv6
  • Across region peering:   AWS encrypts traffic between regions.

The features below primarily assist EC2 connectivity and performance:

Placement Groups

  • Logical grouping of instances within single AZ for lower latency, high pkt-per-second performance and high network throughput – High Performance Computing
  • Best used with instance types that support enhanced networking (v4 or v6)
  • Launch all the instances you need together at the same time or you risk not being able to add later.
  • Max throughput is gated by the slowest instance.
  • Max combined traffic leaving the placement group is capped at 25Gbps for EC2 and S3 and 5 Gbps for all other services.

Elastic Network Interfaces

  • Detachable network interface only available in a VPC and associated with a subnet.
  • Must have primary IPv4 address, MAC address, at least one security group.
  • Can add more private v4/v6 IP addresses, Elastic IP, one public IP.
  • Attaching a second ENI to an EC2 instance gives it presence on two networks.  This is common with network and security appliances.  Max # of interfaces varies.
  • EC2 instance and ENIs must all reside in same AZ.
  • Not usable for NIC teaming (two interfaces on same subnet).
  • The detachable nature makes it useful for high availability designs (e.g. detach an IP from a failing instance and reattach to a running instance).
  • You cannot detach a hosts primary network interface.
  • Autoscaling only supports single-interface launch configurations.

DHCP Option Sets

  • VPC MUST HAVE ONE DHCP option set assigned
    • Cannot be changed, but you can create and associate a new option set to VPC
    • Once associated, existing servers (no restart) and all new ones use the new set.
    • If you delete a VPC, the option set associated with it is also deleted.
  • AWS creates and associates a DHCP option set on VPC creation:
  • domain-name-server is set to AmazonProvidedDNS (must be enabled) – up to 4
  • domain-name is set to the .ec2.internal
  • Additional options include ntp (up to four), netbios,

Continue

[Back to AWS Networks Home]

1. AWS Global Infrastructure

AWS regions, availability zones, and Edge locations:

  • Latency: Most Availability Zones are less than 2ms apart
  • Latency is designed to be close, so that you can use Placement Groups to have instances in different data centers close enough for high-bandwidth traffic.

Virtual Private Cloud:

VPC uses a mapping service which abstracts all of your VPC resource information from the underlying infrastructure. Because of this it is likely there is additional header overhead (VPC header, likely q in q) for all traffic in and out of your VPC.

This abstraction is why broadcast and multicast traffic are not supported.  There are tutorials on methods to use when your application requires broadcast, but the general exam answer is likely to be rearchitect your application to not require these protocols.

Pay attention to where services reside.

  • Outside you VPC: Edge locations, Route 53, Cloudfront, S3, Lambda, etc.
  • Inside your VPC: EC2, RDS, Workspaces
  • On VPC Endpoints in your VPC: DynamoDB, S3
  • On public endpoints outside your VPC: S3, SQS.  A good rule of thumb might be that region-based services where you don’t specify an AZ live outside the VPC.

What makes up “AWS Networking?”

Many of the exam questions in this section should be very similar to those from the Solution Architect exam – mostly understanding what services do.

Knowing your management plane risk is important.

  • AWS Console sits on underlying regional services:  Cloudfront, DynamoDB, S3, SQS
  • Preferring use of AWS SDK or AWS CLI could give you an additional chance of control during outages
  • Generally a control plane outage won’t affect standalone EC2 services, but will likely affect your ability to communicate with other AWS services.

[Back to AWS Networks Home]