AWS Direct Connect with Transit Gateway

Complete Configuration Guide with Network Diagrams and CLI Examples

Table of Contents

Overview and Architecture

AWS Direct Connect with Transit Gateway creates a scalable, centralized hub for connecting your on-premises network to multiple VPCs across different AWS regions. This architecture eliminates the need for multiple Direct Connect connections and provides a single point of management.

Key Benefits:
  • Centralized connectivity hub
  • Cross-region VPC connectivity
  • Simplified routing and management
  • Cost optimization through shared connectivity
  • Enhanced security with segmentation

Traffic Flow Overview

graph TB subgraph "On-Premises" OP[On-Premises Network
192.168.0.0/16] CE[Customer Edge Router] end subgraph "AWS Direct Connect" DXL[Direct Connect Location] DXC[Direct Connect Connection
1Gbps/10Gbps] end subgraph "AWS Cloud" DXGW[Direct Connect Gateway] TGW[Transit Gateway] subgraph "VPC-A us-east-1" VPCA[VPC-A
10.1.0.0/16] TGWA[TGW Attachment] end subgraph "VPC-B us-west-2" VPCB[VPC-B
10.2.0.0/16] TGWB[TGW Attachment] end end OP --> CE CE --> DXL DXL --> DXC DXC --> DXGW DXGW --> TGW TGW --> TGWA TGW --> TGWB TGWA --> VPCA TGWB --> VPCB

Complete Direct Connect VIF Options Matrix

All Possible Direct Connect Virtual Interface Configurations
This matrix shows every possible way to configure Direct Connect VIFs for different use cases.

Primary VIF Types and Connection Options

VIF Type Connection Path Components Required Max Bandwidth Use Case IP Type BGP Required
Private VIF Direct to VGW Private VIF → VGW → Single VPC 10 Gbps Single VPC, Legacy Private (RFC 1918) Yes
Via DX Gateway to VGW Private VIF → DX Gateway → VGW → Single VPC 10 Gbps Single VPC, Cross-account Private (RFC 1918) Yes
Via DX Gateway to TGW Private VIF → DX Gateway → TGW → Multiple VPCs 10 Gbps Multiple VPCs, Traditional Private (RFC 1918) Yes
Public VIF AWS Public Services Public VIF → AWS Public Zone 10 Gbps S3, DynamoDB, Public APIs Public Yes
Transit VIF Direct to TGW Transit VIF → DX Gateway → TGW → Multiple VPCs 100 Gbps Modern Multi-VPC, High Performance Private (RFC 1918) Yes

Detailed Configuration Matrix

Configuration VGW Required DX Gateway Required TGW Required Cross-Region Support Multiple VPCs Complexity Recommended For
Private VIF → VGW ✅ Yes ❌ No ❌ No ❌ No ❌ Single Only Low Simple single VPC
Private VIF → DX GW → VGW ✅ Yes ✅ Yes ❌ No ✅ Yes ❌ Single Only Medium Cross-account single VPC
Private VIF → DX GW → TGW ❌ No ✅ Yes ✅ Yes ✅ Yes ✅ Multiple High Traditional multi-VPC
Transit VIF → DX GW → TGW ❌ No ✅ Yes ✅ Yes ✅ Yes ✅ Multiple Medium Modern multi-VPC, high bandwidth
Public VIF → AWS Public ❌ No ❌ No ❌ No ✅ Yes N/A Low AWS public services access

VIF Parameter Comparison Matrix

Parameter Private VIF Public VIF Transit VIF Notes
VLAN Range 1-4094 1-4094 1-4094 Must be unique per connection
Customer ASN Private: 64512-65534
Public: Any registered		 Public registered ASN required Private: 64512-65534
Public: Any registered		 Private ASN sufficient for most cases
Amazon ASN 64512 (default)
4200000000-4294967294		 7224 (fixed) 64512 (default)
4200000000-4294967294		 Can customize for Private/Transit VIFs
IP Addressing /30 or /31 private /30 or /31 public /30 or /31 private /31 saves IP addresses
BGP Authentication Optional MD5 key Optional MD5 key Optional MD5 key Recommended for security
MTU Size 1500 (standard)
9000 (jumbo)		 1500 (standard)
9000 (jumbo)		 1500-8500 (standard)
9000 (jumbo)		 Higher MTU = better performance
Route Filters Supported AWS advertises all public prefixes Supported Control route advertisements

Use Case Decision Matrix

Your Requirement Recommended Configuration Alternative Options Why This Choice
Single VPC, Simple Setup Private VIF → VGW → VPC Private VIF → DX GW → TGW → VPC Lowest complexity, lowest cost
Single VPC, Future Growth Transit VIF → DX GW → TGW → VPC Private VIF → DX GW → TGW → VPC Easy to add more VPCs later
Multiple VPCs, Same Region Transit VIF → DX GW → TGW → VPCs Private VIF → DX GW → TGW → VPCs Higher bandwidth, better performance
Multiple VPCs, Cross-Region Transit VIF → DX GW → TGW → VPCs
+ TGW Peering		 Multiple Private VIFs per region Single connection, TGW peering
High Bandwidth (>10G) Transit VIF → DX GW → TGW Multiple Private VIFs (ECMP) 100G per VIF vs 10G per VIF
AWS Public Services Only Public VIF → AWS Public Zone Private VIF + NAT Gateway Direct path, no VPC needed
Hybrid (VPCs + Public Services) Transit VIF + Public VIF Private VIF + Public VIF Dedicated bandwidth per use case
Environment Separation Multiple Transit VIFs
(1 per environment)		 Single Transit VIF + TGW Route Tables Complete isolation, separate BGP
Disaster Recovery Transit VIF in primary + secondary region VPN backup to Private VIF Regional redundancy
Service Provider/Multi-Tenant Multiple Private/Transit VIFs
(1 per customer)		 Shared VIF + route filtering Customer isolation, separate billing

Cost Comparison Matrix

Component Hourly Cost Data Transfer Cost When You Pay Cost Optimization Tips
Direct Connect Port $0.30/hour (1G)
$2.25/hour (10G)
$22.50/hour (100G)		 $0.02/GB outbound Always (when provisioned) Right-size port capacity
Private VIF $0.05/hour Included When VIF exists Delete unused VIFs
Public VIF $0.05/hour $0.02/GB outbound When VIF exists Use for high-volume public service access
Transit VIF $0.05/hour Included When VIF exists Higher bandwidth efficiency
Virtual Private Gateway $0.05/hour N/A When attached to VPC Use TGW for multiple VPCs
Direct Connect Gateway Free N/A Never No additional cost
Transit Gateway $0.05/hour $0.02/GB processed When TGW exists Consolidate multiple VGWs

Performance Comparison Matrix

Configuration Max Bandwidth Latency Packet Processing Scalability Performance Notes
Private VIF → VGW 10 Gbps Lowest Hardware optimized Single VPC only Best for single VPC performance
Private VIF → DX GW → TGW 10 Gbps Medium Additional hop Multiple VPCs Extra hop adds ~1ms latency
Transit VIF → DX GW → TGW 100 Gbps Medium Optimized for TGW Multiple VPCs 10x bandwidth of Private VIF
Multiple VIFs (ECMP) Aggregate bandwidth Medium Load balanced Highest Linear scaling with VIF count
Public VIF 10 Gbps Variable AWS public network Global AWS services Performance varies by service

Virtual Interface Types & VGW vs Transit Gateway

Key Point: You DON'T need a VGW if you use Transit Gateway, even with just 1 VIF!
Modern AWS architecture recommends Transit Gateway over VGW for scalability and simplicity.

Does Transit Gateway Require a Direct Connect Gateway?

NO - Transit Gateway does NOT always require a Direct Connect Gateway!
It depends on your VIF type and architecture choice.
graph TB subgraph "Option A: Private VIF (Requires DX Gateway)" A1[On-Premises] --> A2[Private VIF] --> A3[Direct Connect Gateway] --> A4[Transit Gateway] --> A5[VPCs] A3 -.->|Required| A4 end subgraph "Option B: Transit VIF (NO DX Gateway Needed!)" B1[On-Premises] --> B2[Transit VIF] --> B3[Transit Gateway] --> B4[VPCs] B2 -.->|Direct Connection| B3 end style A3 fill:#ffeb3b style B3 fill:#4caf50

When You DO Need Direct Connect Gateway

Private VIF → DX Gateway → Transit Gateway
  • Using Private VIF type
  • Private VIFs cannot connect directly to Transit Gateway
  • Must go through Direct Connect Gateway as an intermediary
  • More hops = slightly more latency
  • Traditional/legacy approach

When You DON'T Need Direct Connect Gateway

Transit VIF → Transit Gateway (Direct)
  • Using Transit VIF type
  • Transit VIFs connect directly to Transit Gateway
  • No Direct Connect Gateway required
  • Fewer hops = better performance
  • Modern/recommended approach
  • Supports higher bandwidth (up to 100G)

Side-by-Side Comparison

Aspect Private VIF + DX Gateway Transit VIF (Direct)
DX Gateway Required? ✅ YES - Required ❌ NO - Not needed
Network Hops VIF → DX GW → TGW → VPC VIF → TGW → VPC
Maximum Bandwidth 10 Gbps per VIF 100 Gbps per VIF
Latency Higher (more hops) Lower (fewer hops)
Complexity More complex (more components) Simpler (fewer components)
When to Use Legacy deployments, existing Private VIFs New deployments, high performance needs

Configuration Examples: With vs Without DX Gateway

Option A: Private VIF (Requires DX Gateway)

# This approach REQUIRES Direct Connect Gateway # 1. Create Transit Gateway aws ec2 create-transit-gateway --description "TGW with DX Gateway" # 2. Create Direct Connect Gateway (REQUIRED for Private VIF) aws directconnect create-direct-connect-gateway \ --name "Required-DX-Gateway" \ --amazon-side-asn 64512 # 3. Associate DX Gateway with Transit Gateway aws directconnect create-direct-connect-gateway-association-proposal \ --direct-connect-gateway-id dxgw-12345678 \ --gateway-id tgw-0abcd1234efgh5678 # 4. Create Private VIF pointing to DX Gateway aws directconnect create-private-virtual-interface \ --connection-id dxcon-xxxxxxxxx \ --new-private-virtual-interface '{ "virtualInterfaceName": "Private-VIF-via-DXGW", "vlan": 100, "asn": 65001, "amazonAddress": "192.168.254.2/30", "customerAddress": "192.168.254.1/30", "directConnectGatewayId": "dxgw-12345678" }'

Option B: Transit VIF (NO DX Gateway Needed)

# This approach does NOT need Direct Connect Gateway! # 1. Create Transit Gateway aws ec2 create-transit-gateway --description "TGW with Transit VIF" # 2. Create Transit VIF directly to Transit Gateway # NO Direct Connect Gateway creation needed! aws directconnect create-transit-virtual-interface \ --connection-id dxcon-xxxxxxxxx \ --new-transit-virtual-interface '{ "virtualInterfaceName": "Direct-Transit-VIF", "vlan": 200, "asn": 65001, "amazonAddress": "192.168.254.6/30", "customerAddress": "192.168.254.5/30", "transitGatewayId": "tgw-0abcd1234efgh5678" }' # That's it! No DX Gateway needed.
Important Note: In the Transit VIF command above, notice we use transitGatewayId directly instead of directConnectGatewayId. This creates a direct connection from the VIF to the Transit Gateway.

Architecture Options for Single VIF

graph TB subgraph "Option 1: Direct VIF to VGW (Legacy)" O1A[On-Premises] --> O1B[Private VIF] --> O1C[Virtual Gateway VGW] --> O1D[Single VPC] end subgraph "Option 2: VIF via DX Gateway to TGW (Recommended)" O2A[On-Premises] --> O2B[Private VIF] --> O2C[DX Gateway] --> O2D[Transit Gateway] --> O2E[Single VPC] end subgraph "Option 3: Transit VIF to TGW (Most Efficient)" O3A[On-Premises] --> O3B[Transit VIF] --> O3C[Transit Gateway] --> O3D[Single VPC] end style O1D fill:#ffcccc style O2E fill:#ccffcc style O3D fill:#ccffff

Architecture Comparison Table

Architecture Components Needed When to Use Limitations
Direct to VGW Private VIF → VGW → VPC Legacy deployments, single VPC only No multi-VPC, no cross-region
Via DX Gateway Private VIF → DX Gateway → TGW → VPC(s) Multiple VPCs, cross-region connectivity Additional hop, more complex
Transit VIF Transit VIF → TGW → VPC(s) Modern deployments, highest performance Requires newer DX infrastructure

When You DON'T Need a VGW

Modern Recommendation: Skip VGW entirely and use Transit Gateway
Even for a single VPC, Transit Gateway provides better scalability and future-proofing.
# Single VPC Scenarios - VGW vs TGW Decision CHOOSE VGW ONLY IF: ❌ Legacy deployment that can't be changed ❌ Very simple setup with no growth plans ❌ Cost is extremely sensitive (VGW is cheaper for single VPC) CHOOSE TRANSIT GATEWAY IF: ✅ Any possibility of adding more VPCs in future ✅ Need cross-region connectivity ✅ Want simplified routing management ✅ Plan to connect to other AWS services (VPN, other DX) ✅ Modern deployment with best practices

Configuration Examples: Single VIF, No VGW

# Example: Single VIF to Single VPC via Transit Gateway # NO VGW required! # 1. Create Transit Gateway aws ec2 create-transit-gateway \ --description "Single VPC Transit Gateway" \ --options DefaultRouteTableAssociation=enable,DefaultRouteTablePropagation=enable # 2. Create Direct Connect Gateway aws directconnect create-direct-connect-gateway \ --name "Single-VPC-DX-Gateway" \ --amazon-side-asn 64512 # 3. Associate DX Gateway with Transit Gateway aws directconnect create-direct-connect-gateway-association-proposal \ --direct-connect-gateway-id dxgw-12345678 \ --gateway-id tgw-0abcd1234efgh5678 \ --add-allowed-prefixes-to-direct-connect-gateway Cidr=10.1.0.0/16 # 4. Create Private VIF pointing to DX Gateway (NOT VGW) aws directconnect create-private-virtual-interface \ --connection-id dxcon-xxxxxxxxx \ --new-private-virtual-interface '{ "virtualInterfaceName": "Single-VPC-VIF", "vlan": 100, "asn": 65001, "amazonAddress": "192.168.254.2/30", "customerAddress": "192.168.254.1/30", "directConnectGatewayId": "dxgw-12345678" }' # 5. Attach your single VPC to Transit Gateway aws ec2 create-transit-gateway-vpc-attachment \ --transit-gateway-id tgw-0abcd1234efgh5678 \ --vpc-id vpc-0123456789abcdef0 \ --subnet-ids subnet-0123456789abcdef0

Alternative: Even Simpler with Transit VIF

# Most efficient: Transit VIF directly to Transit Gateway # Eliminates DX Gateway entirely! # 1. Create Transit Gateway (same as above) aws ec2 create-transit-gateway \ --description "Single VPC Transit Gateway" # 2. Create Transit VIF (NO DX Gateway needed!) aws directconnect create-transit-virtual-interface \ --connection-id dxcon-xxxxxxxxx \ --new-transit-virtual-interface '{ "virtualInterfaceName": "Direct-Transit-VIF", "vlan": 200, "asn": 65001, "amazonAddress": "192.168.254.6/30", "customerAddress": "192.168.254.5/30", "directConnectGatewayId": "dxgw-12345678" }' # 3. Attach VPC to Transit Gateway (same as above) aws ec2 create-transit-gateway-vpc-attachment \ --transit-gateway-id tgw-0abcd1234efgh5678 \ --vpc-id vpc-0123456789abcdef0 \ --subnet-ids subnet-0123456789abcdef0

3 Types of Virtual Interfaces (VIFs) - Complete Overview

YES - There are exactly 3 types of VIFs in AWS Direct Connect:
  1. Private VIF - Connects to VPCs via VGW or DX Gateway
  2. Public VIF - Connects to AWS public services
  3. Transit VIF - Connects directly to Transit Gateway
graph TB subgraph "3 Types of Virtual Interfaces" subgraph "1. Private VIF" P1[On-Premises] --> P2[Private VIF
RFC 1918 IPs] P2 --> P3A[Option A: VGW → Single VPC] P2 --> P3B[Option B: DX Gateway → TGW → Multiple VPCs] end subgraph "2. Public VIF" PU1[On-Premises] --> PU2[Public VIF
Public IPs] --> PU3[AWS Public Services
S3, DynamoDB, etc.] end subgraph "3. Transit VIF" T1[On-Premises] --> T2[Transit VIF
RFC 1918 IPs] --> T3[Transit Gateway] --> T4[Multiple VPCs
Cross-Region] end end style P2 fill:#e3f2fd style PU2 fill:#fff3e0 style T2 fill:#e8f5e8

Complete VIF Type Comparison

VIF Type Purpose IP Addressing Connects To Max Bandwidth Use Case
Private VIF VPC access Private (RFC 1918) VGW or DX Gateway 10 Gbps Traditional VPC connectivity
Public VIF AWS public services Public IPs AWS Public Zone 10 Gbps S3, DynamoDB, CloudFront, etc.
Transit VIF Multi-VPC via TGW Private (RFC 1918) Transit Gateway directly 100 Gbps Modern multi-VPC architecture

Important Clarification About Transit VIF

Key Point: Transit VIF is a newer VIF type introduced specifically for Transit Gateway. In my earlier example, I made an error - you still need to create a Direct Connect Gateway even for Transit VIF, but the Transit VIF connects directly to the Transit Gateway through that DX Gateway.

Corrected Transit VIF Configuration

# CORRECTED: Transit VIF still needs a DX Gateway # But it's optimized for Transit Gateway connectivity # 1. Create Transit Gateway aws ec2 create-transit-gateway --description "TGW for Transit VIF" # 2. Create Direct Connect Gateway (still needed) aws directconnect create-direct-connect-gateway \ --name "Transit-VIF-DX-Gateway" \ --amazon-side-asn 64512 # 3. Associate DX Gateway with Transit Gateway aws directconnect create-direct-connect-gateway-association-proposal \ --direct-connect-gateway-id dxgw-12345678 \ --gateway-id tgw-0abcd1234efgh5678 # 4. Create Transit VIF (optimized for TGW) aws directconnect create-transit-virtual-interface \ --connection-id dxcon-xxxxxxxxx \ --new-transit-virtual-interface '{ "virtualInterfaceName": "Transit-VIF-for-TGW", "vlan": 300, "asn": 65001, "amazonAddress": "192.168.254.10/30", "customerAddress": "192.168.254.9/30", "directConnectGatewayId": "dxgw-12345678" }'

Why Use Each VIF Type?

1. Private VIF - When to Use

  • Single VPC: Connect directly via VGW
  • Multiple VPCs: Use DX Gateway → Transit Gateway
  • Legacy environments: Already established Private VIF infrastructure
  • Simple setups: Basic VPC connectivity needs

2. Public VIF - When to Use

  • AWS Public Services: S3, DynamoDB, SQS, SNS access
  • Bypass Internet: Private connection to public AWS services
  • Compliance: Avoid internet routing for public service access
  • Performance: Dedicated bandwidth for public service access

3. Transit VIF - When to Use

  • High Performance: Need more than 10G bandwidth
  • Modern Architecture: New deployments with Transit Gateway
  • Multiple VPCs: Optimized for TGW routing
  • Cross-Region: TGW peering across regions
  • Future-Proof: Latest AWS networking technology

Can You Have Multiple Transit VIFs?

YES! You can have multiple Transit VIFs on the same Direct Connect connection:
  • Multiple Transit VIFs per connection: Each using different VLANs
  • Different Transit Gateways: Each Transit VIF can connect to different TGWs
  • Different regions: Transit VIFs can connect to TGWs in different AWS regions
  • Bandwidth aggregation: Combine multiple Transit VIFs for higher total bandwidth
  • Redundancy: Multiple paths for high availability

Multiple Transit VIF Scenarios

graph TB subgraph "Single Direct Connect Connection" DC[Direct Connect
dxcon-123456789] subgraph "Multiple Transit VIFs" TVIF1[Transit VIF 1
VLAN 100
Production] TVIF2[Transit VIF 2
VLAN 200
Development] TVIF3[Transit VIF 3
VLAN 300
Cross-Region] end end subgraph "Multiple Transit Gateways" TGW1[TGW Production
us-east-1] TGW2[TGW Development
us-east-1] TGW3[TGW Staging
us-west-2] end DC --> TVIF1 DC --> TVIF2 DC --> TVIF3 TVIF1 --> TGW1 TVIF2 --> TGW2 TVIF3 --> TGW3

Use Cases for Multiple Transit VIFs

Scenario Why Multiple Transit VIFs Configuration Benefits
Environment Separation Isolate Prod/Dev/Test 1 Transit VIF per environment Security isolation, separate routing
Cross-Region Connect to multiple AWS regions 1 Transit VIF per region Regional redundancy, compliance
High Bandwidth Aggregate bandwidth >100G Multiple Transit VIFs to same TGW Higher total throughput
Redundancy Active/passive or load balancing 2+ Transit VIFs, different paths High availability, failover
Different Customers Service provider scenario 1 Transit VIF per customer account Customer isolation, billing

Configuration Example: Multiple Transit VIFs

# Example: 3 Transit VIFs for different environments # Get Direct Connect connection ID DX_CONNECTION_ID="dxcon-123456789abcdef0" # Create 3 different Direct Connect Gateways aws directconnect create-direct-connect-gateway \ --name "Production-DX-Gateway" \ --amazon-side-asn 64512 aws directconnect create-direct-connect-gateway \ --name "Development-DX-Gateway" \ --amazon-side-asn 64513 aws directconnect create-direct-connect-gateway \ --name "Cross-Region-DX-Gateway" \ --amazon-side-asn 64514 # Create Transit VIF #1 - Production (VLAN 100) aws directconnect create-transit-virtual-interface \ --connection-id $DX_CONNECTION_ID \ --new-transit-virtual-interface '{ "virtualInterfaceName": "Production-Transit-VIF", "vlan": 100, "asn": 65001, "amazonAddress": "192.168.254.2/30", "customerAddress": "192.168.254.1/30", "directConnectGatewayId": "dxgw-prod123456" }' # Create Transit VIF #2 - Development (VLAN 200) aws directconnect create-transit-virtual-interface \ --connection-id $DX_CONNECTION_ID \ --new-transit-virtual-interface '{ "virtualInterfaceName": "Development-Transit-VIF", "vlan": 200, "asn": 65001, "amazonAddress": "192.168.254.6/30", "customerAddress": "192.168.254.5/30", "directConnectGatewayId": "dxgw-dev123456" }' # Create Transit VIF #3 - Cross-Region (VLAN 300) aws directconnect create-transit-virtual-interface \ --connection-id $DX_CONNECTION_ID \ --new-transit-virtual-interface '{ "virtualInterfaceName": "CrossRegion-Transit-VIF", "vlan": 300, "asn": 65001, "amazonAddress": "192.168.254.10/30", "customerAddress": "192.168.254.9/30", "directConnectGatewayId": "dxgw-xregion123456" }'

Customer Router Configuration for Multiple Transit VIFs

! Configure multiple sub-interfaces for different Transit VIFs ! Transit VIF 1 - Production (VLAN 100) interface GigabitEthernet0/0/1.100 description Production Transit VIF to AWS encapsulation dot1Q 100 ip address 192.168.254.1 255.255.255.252 no ip redirects ! Transit VIF 2 - Development (VLAN 200) interface GigabitEthernet0/0/1.200 description Development Transit VIF to AWS encapsulation dot1Q 200 ip address 192.168.254.5 255.255.255.252 no ip redirects ! Transit VIF 3 - Cross-Region (VLAN 300) interface GigabitEthernet0/0/1.300 description Cross-Region Transit VIF to AWS encapsulation dot1Q 300 ip address 192.168.254.9 255.255.255.252 no ip redirects ! BGP Configuration for multiple neighbors router bgp 65001 bgp log-neighbor-changes ! Production Transit VIF neighbor neighbor 192.168.254.2 remote-as 64512 neighbor 192.168.254.2 description AWS-Production-TGW neighbor 192.168.254.2 password prodSecretKey123 ! Development Transit VIF neighbor neighbor 192.168.254.6 remote-as 64513 neighbor 192.168.254.6 description AWS-Development-TGW neighbor 192.168.254.6 password devSecretKey456 ! Cross-Region Transit VIF neighbor neighbor 192.168.254.10 remote-as 64514 neighbor 192.168.254.10 description AWS-CrossRegion-TGW neighbor 192.168.254.10 password xregionSecretKey789 ! Address family configuration address-family ipv4 ! Advertise different networks to different environments network 192.168.0.0 mask 255.255.0.0 ! Activate all neighbors neighbor 192.168.254.2 activate neighbor 192.168.254.6 activate neighbor 192.168.254.10 activate ! Apply different route policies per environment neighbor 192.168.254.2 prefix-list PROD-OUT out neighbor 192.168.254.6 prefix-list DEV-OUT out neighbor 192.168.254.10 prefix-list XREGION-OUT out exit-address-family ! Different prefix lists for different environments ip prefix-list PROD-OUT seq 10 permit 192.168.0.0/16 ip prefix-list DEV-OUT seq 10 permit 192.168.0.0/16 ip prefix-list XREGION-OUT seq 10 permit 192.168.0.0/16

Bandwidth Aggregation with Multiple Transit VIFs

Important: Multiple Transit VIFs to the same Transit Gateway can provide:
  • Bandwidth Aggregation: 2x 100G Transit VIFs = 200G total bandwidth
  • Load Distribution: Traffic automatically load-balances across VIFs
  • Redundancy: If one VIF fails, traffic continues on remaining VIFs
  • ECMP (Equal Cost Multi-Path): BGP supports multiple equal-cost paths

Limitations and Considerations

Keep in Mind:
  • VLAN Limits: Each VIF needs a unique VLAN ID (1-4094)
  • BGP Sessions: Each Transit VIF creates a separate BGP session
  • Routing Complexity: More VIFs = more complex routing policies
  • Cost: Each VIF has hourly charges, plan accordingly
  • Port Capacity: Total bandwidth limited by DX connection speed
YES! You can have multiple VIFs of different types on the same Direct Connect connection:
  • VLAN 100: Private VIF for legacy VPC
  • VLAN 200: Public VIF for S3 access
  • VLAN 300: Transit VIF for new multi-VPC architecture
Each VIF uses a different VLAN ID for traffic separation.

Private VIFs connect to VPCs through various paths depending on your architecture choice.

Private VIF Characteristics:
  • Uses private IP addressing (RFC 1918)
  • Connects to VPCs via VGW, DX Gateway, or directly to TGW (Transit VIF)
  • Supports BGP routing
  • Can connect to Transit Gateway via Direct Connect Gateway OR directly

2. Public VIF

Public VIFs provide access to AWS public services using public IP addresses.

Public VIF Characteristics:
  • Uses public IP addressing
  • Accesses AWS public services (S3, DynamoDB, etc.)
  • Does NOT connect to VPC resources
  • Requires public ASN and public IP prefixes

3. Transit VIF

Transit VIFs are specifically designed to connect directly to Transit Gateway, providing the most efficient path for multi-VPC connectivity.

Transit VIF Characteristics:
  • Connects directly to Transit Gateway (no Direct Connect Gateway needed)
  • Supports up to 100 Gbps bandwidth
  • Provides the most efficient routing path
  • Supports advanced routing features
graph LR subgraph "VIF Types Comparison" subgraph "Private VIF via DX Gateway" P1[On-Premises] --> P2[Private VIF] --> P3[DX Gateway] --> P4[Transit Gateway] end subgraph "Transit VIF (Direct)" T1[On-Premises] --> T2[Transit VIF] --> T3[Transit Gateway] end subgraph "Public VIF" PU1[On-Premises] --> PU2[Public VIF] --> PU3[AWS Public Services] end end

Detailed Architecture Diagrams

Complete Network Architecture

graph TB subgraph "On-Premises Data Center" DC[Data Center
192.168.0.0/16] CE[Customer Edge Router
BGP ASN: 65001] SW[Core Switch] SRV[Servers/Workloads] end subgraph "AWS Direct Connect Location" DXL[Direct Connect Location
Equinix/CoreSite/etc.] DXP[Direct Connect Port
1G/10G/100G] AWSR[AWS Router] end subgraph "AWS us-east-1" DXGW[Direct Connect Gateway
dxgw-12345678] TGW1[Transit Gateway
tgw-0abcd1234efgh5678] subgraph "Production VPC" PROD[VPC-Production
10.1.0.0/16] PRODSUB1[Private Subnet
10.1.1.0/24] PRODSUB2[Public Subnet
10.1.2.0/24] PRODEC2[EC2 Instances] end subgraph "Development VPC" DEV[VPC-Development
10.2.0.0/16] DEVSUB1[Private Subnet
10.2.1.0/24] DEVEC2[EC2 Instances] end end subgraph "AWS us-west-2" TGW2[Transit Gateway
tgw-0wxyz9876abcd1234] subgraph "Staging VPC" STAGE[VPC-Staging
10.3.0.0/16] STAGESUB1[Private Subnet
10.3.1.0/24] STAGEEC2[EC2 Instances] end end %% Connections DC --> SW SW --> SRV SW --> CE CE -.->|Cross Connect| DXL DXL --> DXP DXP --> AWSR AWSR --> DXGW DXGW --> TGW1 TGW1 --> PROD TGW1 --> DEV PROD --> PRODSUB1 PROD --> PRODSUB2 PRODSUB1 --> PRODEC2 DEV --> DEVSUB1 DEVSUB1 --> DEVEC2 %% Cross-region peering TGW1 -.->|TGW Peering| TGW2 TGW2 --> STAGE STAGE --> STAGESUB1 STAGESUB1 --> STAGEEC2

BGP Routing Flow

sequenceDiagram participant CE as Customer Edge participant AWS as AWS Router participant DXGW as DX Gateway participant TGW as Transit Gateway participant VPC as VPC Route Tables Note over CE,VPC: BGP Session Establishment CE->>AWS: BGP OPEN (ASN 65001) AWS->>CE: BGP OPEN (ASN 64512) Note over CE,VPC: Route Advertisement from On-Premises CE->>AWS: BGP UPDATE: 192.168.0.0/16 AWS->>DXGW: Route: 192.168.0.0/16 DXGW->>TGW: Route: 192.168.0.0/16 TGW->>VPC: Route: 192.168.0.0/16 Note over CE,VPC: Route Advertisement from AWS VPC->>TGW: Route: 10.1.0.0/16, 10.2.0.0/16 TGW->>DXGW: Route: 10.1.0.0/16, 10.2.0.0/16 DXGW->>AWS: Route: 10.1.0.0/16, 10.2.0.0/16 AWS->>CE: BGP UPDATE: 10.1.0.0/16, 10.2.0.0/16

Prerequisites and Planning

Network Planning

Component Requirement Example
On-Premises CIDR Non-overlapping with AWS VPCs 192.168.0.0/16
BGP ASN (Customer) Private or Public ASN 65001 (Private)
VLAN ID Unique per VIF (100-4094) 100
BGP Authentication Key MD5 key for BGP security mySecretKey123
Connection Speed 1G, 10G, or 100G 10G

IP Addressing Scheme

# On-Premises Network On-Premises CIDR: 192.168.0.0/16 - Management Network: 192.168.1.0/24 - Server Network: 192.168.10.0/24 - Point-to-Point Link: 192.168.254.0/30 # AWS VPC Networks Production VPC: 10.1.0.0/16 - Private Subnets: 10.1.1.0/24, 10.1.2.0/24 - Public Subnets: 10.1.10.0/24, 10.1.11.0/24 Development VPC: 10.2.0.0/16 - Private Subnets: 10.2.1.0/24, 10.2.2.0/24 # BGP Peering Network (for VIF) Customer Side: 192.168.254.1/30 AWS Side: 192.168.254.2/30

Transit Gateway Setup

Create Transit Gateway

# Create Transit Gateway aws ec2 create-transit-gateway \ --description "Main Transit Gateway for Direct Connect" \ --options DefaultRouteTableAssociation=enable,DefaultRouteTablePropagation=enable,AutoAcceptSharedAttachments=enable \ --tag-specifications 'ResourceType=transit-gateway,Tags=[{Key=Name,Value=Main-TGW},{Key=Environment,Value=Production}]' \ --region us-east-1
Expected Output:
{ "TransitGateway": { "TransitGatewayId": "tgw-0abcd1234efgh5678", "TransitGatewayArn": "arn:aws:ec2:us-east-1:123456789012:transit-gateway/tgw-0abcd1234efgh5678", "State": "pending", "OwnerId": "123456789012", "Description": "Main Transit Gateway for Direct Connect", "CreationTime": "2025-07-01T10:30:00.000Z", "Options": { "AmazonSideAsn": 64512, "AutoAcceptSharedAttachments": "enable", "DefaultRouteTableAssociation": "enable", "DefaultRouteTablePropagation": "enable" }, "Tags": [ { "Key": "Name", "Value": "Main-TGW" } ] } }
Save this ID: tgw-0abcd1234efgh5678 - you'll need it for VPC attachments and DX Gateway association.

Transit Gateway Parameters Explained

Parameter Description Default
DefaultRouteTableAssociation Automatically associate attachments with default route table enable
DefaultRouteTablePropagation Automatically propagate routes to default route table enable
AutoAcceptSharedAttachments Automatically accept shared resource attachments disable
DnsSupport Enable DNS resolution for VPC attachments enable

Create VPC Attachments

# Get VPC IDs first PROD_VPC_ID=$(aws ec2 describe-vpcs --filters "Name=tag:Name,Values=Production-VPC" --query 'Vpcs[0].VpcId' --output text) # Expected output: vpc-0123456789abcdef0 DEV_VPC_ID=$(aws ec2 describe-vpcs --filters "Name=tag:Name,Values=Development-VPC" --query 'Vpcs[0].VpcId' --output text) # Expected output: vpc-0fedcba9876543210 # Get subnet IDs for attachments PROD_SUBNET_ID=$(aws ec2 describe-subnets --filters "Name=vpc-id,Values=$PROD_VPC_ID" "Name=tag:Name,Values=*Private*" --query 'Subnets[0].SubnetId' --output text) # Expected output: subnet-0abcdef1234567890 DEV_SUBNET_ID=$(aws ec2 describe-subnets --filters "Name=vpc-id,Values=$DEV_VPC_ID" "Name=tag:Name,Values=*Private*" --query 'Subnets[0].SubnetId' --output text) # Expected output: subnet-09876543210fedcba # Create VPC attachments aws ec2 create-transit-gateway-vpc-attachment \ --transit-gateway-id tgw-0abcd1234efgh5678 \ --vpc-id $PROD_VPC_ID \ --subnet-ids $PROD_SUBNET_ID \ --tag-specifications 'ResourceType=transit-gateway-attachment,Tags=[{Key=Name,Value=Production-VPC-Attachment}]'
Expected Output for VPC Attachment:
{ "TransitGatewayVpcAttachment": { "TransitGatewayAttachmentId": "tgw-attach-0123456789abcdef0", "TransitGatewayId": "tgw-0abcd1234efgh5678", "VpcId": "vpc-0123456789abcdef0", "VpcOwnerId": "123456789012", "State": "pending", "SubnetIds": [ "subnet-0abcdef1234567890" ], "CreationTime": "2025-07-01T10:35:00.000Z", "Tags": [ { "Key": "Name", "Value": "Production-VPC-Attachment" } ] } }
Save this ID: tgw-attach-0123456789abcdef0 - you'll need it for route table associations.
aws ec2 create-transit-gateway-vpc-attachment \ --transit-gateway-id tgw-0abcd1234efgh5678 \ --vpc-id $DEV_VPC_ID \ --subnet-ids $DEV_SUBNET_ID \ --tag-specifications 'ResourceType=transit-gateway-attachment,Tags=[{Key=Name,Value=Development-VPC-Attachment}]'
Expected Output for Dev VPC Attachment:
{ "TransitGatewayVpcAttachment": { "TransitGatewayAttachmentId": "tgw-attach-0fedcba9876543210", "TransitGatewayId": "tgw-0abcd1234efgh5678", "VpcId": "vpc-0fedcba9876543210", "State": "pending", "SubnetIds": [ "subnet-09876543210fedcba" ] } }
Save this ID: tgw-attach-0fedcba9876543210
Important: VPC attachments require at least one subnet per Availability Zone. For high availability, attach subnets from multiple AZs.

Direct Connect Gateway Configuration

Create Direct Connect Gateway

# Create Direct Connect Gateway aws directconnect create-direct-connect-gateway \ --name "Main-DX-Gateway" \ --amazon-side-asn 64512
Expected Output:
{ "DirectConnectGateway": { "DirectConnectGatewayId": "dxgw-12345678", "DirectConnectGatewayName": "Main-DX-Gateway", "AmazonSideAsn": 64512, "OwnerAccount": "123456789012", "DirectConnectGatewayState": "available", "StateChangeError": "" } }
Save this ID: dxgw-12345678 - you'll need this for VIF creation and TGW association.

Associate Direct Connect Gateway with Transit Gateway

# Create association proposal aws directconnect create-direct-connect-gateway-association-proposal \ --direct-connect-gateway-id dxgw-12345678 \ --direct-connect-gateway-owner-account 123456789012 \ --gateway-id tgw-0abcd1234efgh5678 \ --add-allowed-prefixes-to-direct-connect-gateway Cidr=10.1.0.0/16 \ --add-allowed-prefixes-to-direct-connect-gateway Cidr=10.2.0.0/16
Expected Output for Association Proposal:
{ "DirectConnectGatewayAssociationProposal": { "ProposalId": "dxgwassocprop-0123456789abcdef0", "DirectConnectGatewayId": "dxgw-12345678", "DirectConnectGatewayOwnerAccount": "123456789012", "ProposalState": "requested", "AssociatedGateway": { "Id": "tgw-0abcd1234efgh5678", "Type": "transitGateway", "Name": "Main-TGW", "OwnerAccount": "123456789012", "Region": "us-east-1" }, "ExistingAllowedPrefixesToDirectConnectGateway": [], "RequestedAllowedPrefixesToDirectConnectGateway": [ { "Cidr": "10.1.0.0/16" }, { "Cidr": "10.2.0.0/16" } ] } }
Save this ID: dxgwassocprop-0123456789abcdef0 - you'll need it to accept the proposal.
# Accept the association (if in same account) PROPOSAL_ID=$(aws directconnect describe-direct-connect-gateway-association-proposals \ --direct-connect-gateway-id dxgw-12345678 \ --query 'directConnectGatewayAssociationProposals[0].proposalId' --output text) # Expected output: dxgwassocprop-0123456789abcdef0 aws directconnect accept-direct-connect-gateway-association-proposal \ --direct-connect-gateway-association-proposal-id $PROPOSAL_ID \ --associated-gateway-owner-account 123456789012 \ --override-allowed-prefixes-to-direct-connect-gateway Cidr=10.1.0.0/16 \ --override-allowed-prefixes-to-direct-connect-gateway Cidr=10.2.0.0/16
Expected Output for Accept Proposal:
{ "DirectConnectGatewayAssociation": { "DirectConnectGatewayId": "dxgw-12345678", "DirectConnectGatewayOwnerAccount": "123456789012", "AssociationState": "associating", "StateChangeError": "", "AssociatedGateway": { "Id": "tgw-0abcd1234efgh5678", "Type": "transitGateway", "Name": "Main-TGW", "OwnerAccount": "123456789012", "Region": "us-east-1" }, "AssociationId": "dxgwassoc-0123456789abcdef0", "AllowedPrefixesToDirectConnectGateway": [ { "Cidr": "10.1.0.0/16" }, { "Cidr": "10.2.0.0/16" } ] } }
Save this ID: dxgwassoc-0123456789abcdef0 - this represents the active association.

Direct Connect Gateway Parameters

Parameter Description Value/Range
amazon-side-asn AWS side BGP ASN for the gateway 64512 (default) or 4200000000-4294967294
allowed-prefixes CIDR blocks allowed to be advertised from AWS VPC CIDR blocks (e.g., 10.1.0.0/16)
gateway-id Transit Gateway ID to associate tgw-xxxxxxxxxxxxxxxxx

Virtual Interface Configuration

Create Private Virtual Interface

# Get your Direct Connect connection ID first DX_CONNECTION_ID=$(aws directconnect describe-connections --query 'connections[0].connectionId' --output text) # Create Private VIF aws directconnect create-private-virtual-interface \ --connection-id $DX_CONNECTION_ID \ --new-private-virtual-interface '{ "virtualInterfaceName": "Production-Private-VIF", "vlan": 100, "asn": 65001, "authKey": "mySecretKey123", "amazonAddress": "192.168.254.2/30", "customerAddress": "192.168.254.1/30", "directConnectGatewayId": "dxgw-12345678", "tags": [ { "key": "Name", "value": "Production-Private-VIF" }, { "key": "Environment", "value": "Production" } ] }'
# Get your Direct Connect connection ID first DX_CONNECTION_ID=$(aws directconnect describe-connections --query 'connections[0].connectionId' --output text) # Expected output: dxcon-0123456789abcdef0 # Create Private VIF aws directconnect create-private-virtual-interface \ --connection-id $DX_CONNECTION_ID \ --new-private-virtual-interface '{ "virtualInterfaceName": "Production-Private-VIF", "vlan": 100, "asn": 65001, "authKey": "mySecretKey123", "amazonAddress": "192.168.254.2/30", "customerAddress": "192.168.254.1/30", "directConnectGatewayId": "dxgw-12345678", "tags": [ { "key": "Name", "value": "Production-Private-VIF" }, { "key": "Environment", "value": "Production" } ] }'
Expected Output for Private VIF Creation:
{ "virtualInterface": { "ownerAccount": "123456789012", "virtualInterfaceId": "dxvif-abcd1234efgh5678", "location": "EqDC2", "connectionId": "dxcon-0123456789abcdef0", "virtualInterfaceType": "private", "virtualInterfaceName": "Production-Private-VIF", "vlan": 100, "asn": 65001, "amazonSideAsn": 64512, "authKey": "mySecretKey123", "amazonAddress": "192.168.254.2/30", "customerAddress": "192.168.254.1/30", "addressFamily": "ipv4", "virtualInterfaceState": "pending", "customerRouterConfig": "...", "mtu": 1500, "jumboFrameCapable": true, "virtualGatewayId": "", "directConnectGatewayId": "dxgw-12345678", "routeFilterPrefixes": [], "bgpPeers": [ { "bgpPeerId": "dxpeer-fg123456", "asn": 65001, "authKey": "mySecretKey123", "addressFamily": "ipv4", "amazonAddress": "192.168.254.2/30", "customerAddress": "192.168.254.1/30", "bgpPeerState": "pending", "bgpStatus": "down", "awsDeviceV2": "EqDC2-19y7z3w8uk" } ], "region": "us-east-1", "awsDeviceV2": "EqDC2-19y7z3w8uk", "awsLogicalDeviceId": "EqDC2-19y7z3w8uk-VIF", "tags": [ { "key": "Name", "value": "Production-Private-VIF" }, { "key": "Environment", "value": "Production" } ] } }
Save these IDs:
  • dxvif-abcd1234efgh5678 - Virtual Interface ID for monitoring
  • dxpeer-fg123456 - BGP Peer ID for troubleshooting
  • EqDC2-19y7z3w8uk - AWS Device ID for support cases

Virtual Interface Parameters Detailed

Parameter Description Example/Notes
virtualInterfaceName Friendly name for the VIF Production-Private-VIF
vlan VLAN ID for traffic separation 100 (range: 1-4094, must be unique per connection)
asn Customer BGP ASN 65001 (private: 64512-65534, public: registered ASN)
authKey BGP MD5 authentication key Optional but recommended for security
amazonAddress AWS side IP address 192.168.254.2/30 (must be /30 or /31)
customerAddress Customer side IP address 192.168.254.1/30 (must be in same subnet as amazonAddress)

Alternative: Create Transit Virtual Interface

# Create Transit VIF (direct to Transit Gateway - more efficient) aws directconnect create-transit-virtual-interface \ --connection-id $DX_CONNECTION_ID \ --new-transit-virtual-interface '{ "virtualInterfaceName": "Production-Transit-VIF", "vlan": 200, "asn": 65001, "authKey": "mySecretKey123", "amazonAddress": "192.168.254.6/30", "customerAddress": "192.168.254.5/30", "directConnectGatewayId": "dxgw-12345678", "tags": [ { "key": "Name", "value": "Production-Transit-VIF" }, { "key": "Type", "value": "Transit" } ] }'
Expected Output for Transit VIF Creation:
{ "virtualInterface": { "ownerAccount": "123456789012", "virtualInterfaceId": "dxvif-wxyz9876abcd1234", "location": "EqDC2", "connectionId": "dxcon-0123456789abcdef0", "virtualInterfaceType": "transit", "virtualInterfaceName": "Production-Transit-VIF", "vlan": 200, "asn": 65001, "amazonSideAsn": 64512, "authKey": "mySecretKey123", "amazonAddress": "192.168.254.6/30", "customerAddress": "192.168.254.5/30", "addressFamily": "ipv4", "virtualInterfaceState": "pending", "mtu": 8500, "jumboFrameCapable": true, "directConnectGatewayId": "dxgw-12345678", "bgpPeers": [ { "bgpPeerId": "dxpeer-hij789012", "asn": 65001, "authKey": "mySecretKey123", "addressFamily": "ipv4", "amazonAddress": "192.168.254.6/30", "customerAddress": "192.168.254.5/30", "bgpPeerState": "pending", "bgpStatus": "down", "awsDeviceV2": "EqDC2-19y7z3w8uk" } ], "region": "us-east-1", "awsDeviceV2": "EqDC2-19y7z3w8uk", "tags": [ { "key": "Name", "value": "Production-Transit-VIF" }, { "key": "Type", "value": "Transit" } ] } }
Key Differences in Transit VIF Output:
  • virtualInterfaceType: "transit" - Confirms it's a Transit VIF
  • mtu: 8500 - Higher MTU supported (vs 1500 for Private VIF)
  • dxvif-wxyz9876abcd1234 - Save this Transit VIF ID
Transit VIF vs Private VIF:
  • Transit VIF: Connects directly to Transit Gateway, supports higher bandwidth (up to 100G), more efficient routing
  • Private VIF: Connects via Direct Connect Gateway, traditional approach, maximum 10G per VIF

Customer Router Configuration (Cisco Example)

! Configure the sub-interface for VLAN 100 interface GigabitEthernet0/0/1.100 description Direct Connect to AWS Production VIF encapsulation dot1Q 100 ip address 192.168.254.1 255.255.255.252 no ip redirects no ip proxy-arp ! Configure BGP router bgp 65001 bgp log-neighbor-changes bgp graceful-restart ! Neighbor configuration for AWS neighbor 192.168.254.2 remote-as 64512 neighbor 192.168.254.2 description AWS-DirectConnect-VIF neighbor 192.168.254.2 password mySecretKey123 neighbor 192.168.254.2 timers 10 30 neighbor 192.168.254.2 soft-reconfiguration inbound ! Address family configuration address-family ipv4 network 192.168.0.0 mask 255.255.0.0 neighbor 192.168.254.2 activate neighbor 192.168.254.2 prefix-list ADVERTISE-TO-AWS out neighbor 192.168.254.2 prefix-list ACCEPT-FROM-AWS in maximum-paths 4 exit-address-family ! Prefix lists for route filtering ip prefix-list ADVERTISE-TO-AWS seq 10 permit 192.168.0.0/16 ip prefix-list ADVERTISE-TO-AWS seq 20 deny 0.0.0.0/0 le 32 ip prefix-list ACCEPT-FROM-AWS seq 10 permit 10.1.0.0/16 ip prefix-list ACCEPT-FROM-AWS seq 20 permit 10.2.0.0/16 ip prefix-list ACCEPT-FROM-AWS seq 30 deny 0.0.0.0/0 le 32 ! Static route for redundancy (higher metric) ip route 10.1.0.0 255.255.0.0 192.168.254.2 200 ip route 10.2.0.0 255.255.0.0 192.168.254.2 200
BGP Timers Explanation:
  • Keepalive Timer: 10 seconds (how often to send keepalive messages)
  • Hold Timer: 30 seconds (how long to wait before declaring neighbor down)
  • AWS Default: 30/90 seconds - adjust based on your requirements

Routing Configuration

Transit Gateway Route Tables

# Create custom route table for segmentation aws ec2 create-transit-gateway-route-table \ --transit-gateway-id tgw-0abcd1234efgh5678 \ --tag-specifications 'ResourceType=transit-gateway-route-table,Tags=[{Key=Name,Value=Production-Routes},{Key=Environment,Value=Production}]'
Expected Output for Route Table Creation:
{ "TransitGatewayRouteTable": { "TransitGatewayRouteTableId": "tgw-rtb-0123456789abcdef0", "TransitGatewayId": "tgw-0abcd1234efgh5678", "State": "pending", "DefaultAssociationRouteTable": false, "DefaultPropagationRouteTable": false, "CreationTime": "2025-07-01T11:00:00.000Z", "Tags": [ { "Key": "Name", "Value": "Production-Routes" }, { "Key": "Environment", "Value": "Production" } ] } }
Save this ID: tgw-rtb-0123456789abcdef0 - you'll use this for associations and route creation.
# Get the route table ID from output TGW_RT_ID="tgw-rtb-0123456789abcdef0" # Associate VPC attachments with route table PROD_ATTACHMENT_ID=$(aws ec2 describe-transit-gateway-vpc-attachments \ --filters "Name=tag:Name,Values=Production-VPC-Attachment" \ --query 'TransitGatewayVpcAttachments[0].TransitGatewayAttachmentId' --output text) # Expected output: tgw-attach-0123456789abcdef0 aws ec2 associate-transit-gateway-route-table \ --transit-gateway-attachment-id $PROD_ATTACHMENT_ID \ --transit-gateway-route-table-id $TGW_RT_ID
Expected Output for Route Table Association:
{ "Association": { "TransitGatewayAttachmentId": "tgw-attach-0123456789abcdef0", "TransitGatewayRouteTableId": "tgw-rtb-0123456789abcdef0", "State": "associating" } }
# Create routes for on-premises traffic # First get the DX Gateway attachment ID DX_ATTACHMENT_ID=$(aws ec2 describe-transit-gateway-direct-connect-gateway-attachments \ --filters "Name=direct-connect-gateway-id,Values=dxgw-12345678" \ --query 'TransitGatewayDirectConnectGatewayAttachments[0].TransitGatewayAttachmentId' --output text) # Expected output: tgw-attach-0abcdef1234567890 aws ec2 create-route \ --route-table-id $TGW_RT_ID \ --destination-cidr-block 192.168.0.0/16 \ --transit-gateway-attachment-id $DX_ATTACHMENT_ID
Expected Output for Route Creation:
{ "Return": true }
Note: Simple "true" response indicates the route was created successfully.

VPC Route Table Updates

# Get VPC route table IDs PROD_RT_ID=$(aws ec2 describe-route-tables \ --filters "Name=vpc-id,Values=$PROD_VPC_ID" "Name=tag:Name,Values=*Private*" \ --query 'RouteTables[0].RouteTableId' --output text) # Expected output: rtb-0123456789abcdef0 # Add route to on-premises network via Transit Gateway aws ec2 create-route \ --route-table-id $PROD_RT_ID \ --destination-cidr-block 192.168.0.0/16 \ --transit-gateway-id tgw-0abcd1234efgh5678
Expected Output for VPC Route Creation:
{ "Return": true }
# Add route to other VPCs via Transit Gateway aws ec2 create-route \ --route-table-id $PROD_RT_ID \ --destination-cidr-block 10.2.0.0/16 \ --transit-gateway-id tgw-0abcd1234efgh5678
Expected Output:
{ "Return": true }

Route Propagation Configuration

# Enable route propagation from Direct Connect Gateway aws ec2 enable-transit-gateway-route-table-propagation \ --transit-gateway-route-table-id $TGW_RT_ID \ --transit-gateway-attachment-id $DX_ATTACHMENT_ID # Enable route propagation from VPC attachments aws ec2 enable-transit-gateway-route-table-propagation \ --transit-gateway-route-table-id $TGW_RT_ID \ --transit-gateway-attachment-id $PROD_ATTACHMENT_ID # View propagated routes aws ec2 get-transit-gateway-route-table-propagations \ --transit-gateway-route-table-id $TGW_RT_ID

Route Priority and Path Selection

graph TD subgraph "Route Selection Process" A[Packet Arrives at TGW] --> B{Destination Match?} B -->|Yes| C[Check Route Priority] B -->|No| D[Drop Packet] C --> E{Static Route?} E -->|Yes| F[Use Static Route - Highest Priority] E -->|No| G{Propagated Route?} G -->|Yes| H[Check AS Path Length] G -->|No| I[Default Route] H --> J{Shorter AS Path?} J -->|Yes| K[Select Route with Shorter Path] J -->|No| L[Use Local Preference/MED] F --> M[Forward Packet] K --> M L --> M I --> M end
Route Selection Priority (Highest to Lowest):
  1. Static Routes: Manually configured routes in TGW route tables
  2. BGP Routes: Routes learned via BGP with shortest AS path
  3. Local Preference: BGP attribute for policy-based routing
  4. MED (Multi-Exit Discriminator): BGP attribute for path preference

Troubleshooting and Verification

Verify Direct Connect Status

# Check Direct Connect connection status aws directconnect describe-connections \ --connection-id dxcon-xxxxxxxxx # Check Virtual Interface status aws directconnect describe-virtual-interfaces \ --virtual-interface-id dxvif-abcd1234 # Check BGP peer status aws directconnect describe-virtual-interfaces \ --virtual-interface-id dxvif-abcd1234 \ --query 'virtualInterfaces[0].bgpPeers'

Expected BGP Status Output

{ "bgpPeers": [ { "bgpPeerId": "dxpeer-xxxxxxxx", "asn": 65001, "authKey": "mySecretKey123", "addressFamily": "ipv4", "amazonAddress": "192.168.254.2/30", "customerAddress": "192.168.254.1/30", "bgpPeerState": "established", "bgpStatus": "up", "awsDeviceV2": "EqDC2-19y7z3w8uk" } ] }

Transit Gateway Verification

# Check Transit Gateway attachments aws ec2 describe-transit-gateway-attachments \ --transit-gateway-id tgw-0abcd1234efgh5678 # Check Transit Gateway route tables aws ec2 describe-transit-gateway-route-tables \ --transit-gateway-route-table-ids $TGW_RT_ID # Search routes in Transit Gateway route table aws ec2 search-transit-gateway-routes \ --transit-gateway-route-table-id $TGW_RT_ID \ --filters Name=state,Values=active # Check specific route aws ec2 search-transit-gateway-routes \ --transit-gateway-route-table-id $TGW_RT_ID \ --filters Name=route-search.exact-match,Values=192.168.0.0/16

Network Connectivity Testing

# Test from on-premises to AWS VPC ping 10.1.1.100 # Replace with actual EC2 private IP # Test with traceroute to verify path traceroute 10.1.1.100 # Test from AWS EC2 to on-premises # (Run this from EC2 instance) ping 192.168.10.100 # Replace with on-premises server IP # Check routing table on EC2 instance ip route show route -n

BGP Route Verification (Customer Router)

! Check BGP neighbor status show ip bgp summary ! Check received routes from AWS show ip bgp neighbors 192.168.254.2 received-routes ! Check advertised routes to AWS show ip bgp neighbors 192.168.254.2 advertised-routes ! Check BGP table show ip bgp ! Check specific route show ip route 10.1.0.0 ! Debug BGP (use carefully in production) debug ip bgp updates debug ip bgp events

Common Issues and Solutions

Issue Symptoms Solution
BGP Not Establishing bgpStatus: down, no routes learned Check VLAN config, IP addresses, ASN, auth key
Routes Not Propagating BGP up but no routes in TGW table Check allowed prefixes in DX Gateway association
Connectivity Issues Can ping AWS side IP, but not VPC resources Check VPC route tables, security groups, NACLs
Asymmetric Routing Traffic goes one way but not return path Verify route propagation in both directions
High Latency Slow response times Check for suboptimal routing, MTU issues

Monitoring and Logging

# Enable VPC Flow Logs for troubleshooting aws ec2 create-flow-logs \ --resource-type VPC \ --resource-ids $PROD_VPC_ID \ --traffic-type ALL \ --log-destination-type cloud-watch-logs \ --log-group-name VPCFlowLogs \ --deliver-logs-permission-arn arn:aws:iam::123456789012:role/flowlogsRole # Monitor Direct Connect metrics aws cloudwatch get-metric-statistics \ --namespace AWS/DX \ --metric-name ConnectionBpsEgress \ --dimensions Name=ConnectionId,Value=dxcon-xxxxxxxxx \ --start-time 2025-07-01T00:00:00Z \ --end-time 2025-07-01T23:59:59Z \ --period 300 \ --statistics Average,Maximum
Key Metrics to Monitor:
  • ConnectionBpsEgress/Ingress: Bandwidth utilization
  • ConnectionPpsEgress/Ingress: Packet rate
  • ConnectionCRCErrorCount: Physical layer errors
  • ConnectionLightLevelLow: Optical signal issues
  • VirtualInterfaceBpsEgress/Ingress: Per-VIF bandwidth

Performance Optimization

# MTU Optimization (set on customer router interface) interface GigabitEthernet0/0/1.100 ip mtu 9000 # Enable jumbo frames if supported # AWS supports up to 9000 byte MTU on Direct Connect # TCP MSS Clamping (if needed) ip tcp adjust-mss 8960 # Quality of Service (example) class-map match-all CRITICAL match dscp ef class-map match-all HIGH match dscp af31 policy-map DX-QOS class CRITICAL priority percent 20 class HIGH bandwidth percent 60 class class-default bandwidth percent 20 interface GigabitEthernet0/0/1.100 service-policy output DX-QOS