AWS Infrastructure Automation

Complete Guide to Infrastructure as Code, Event-Driven Automation, and Cost Optimization

Table of Contents

1. Infrastructure Automation Overview

graph TB A[Infrastructure Automation] --> B[Infrastructure as Code] A --> C[Event-Driven Automation] A --> D[Cost Optimization] A --> E[Testing & Validation] B --> B1[CloudFormation] B --> B2[CDK] B --> B3[CLI/SDK] C --> C1[CloudWatch Events] C --> C2[Lambda Functions] C --> C3[SNS/SQS] D --> D1[Auto Scaling] D --> D2[Scheduled Actions] D --> D3[Resource Tagging] E --> E1[CloudFormation Drift] E --> E2[Config Rules] E --> E3[Systems Manager] style A fill:#FF9500,stroke:#232F3E,stroke-width:3px,color:#fff style B fill:#232F3E,stroke:#FF9500,stroke-width:2px,color:#fff style C fill:#232F3E,stroke:#FF9500,stroke-width:2px,color:#fff style D fill:#232F3E,stroke:#FF9500,stroke-width:2px,color:#fff style E fill:#232F3E,stroke:#FF9500,stroke-width:2px,color:#fff

Infrastructure Automation Architecture Overview

This diagram illustrates the four core pillars of AWS Infrastructure Automation:

2. Infrastructure as Code (IaC)

sequenceDiagram participant Dev as Developer participant Git as Git Repository participant CI as CI/CD Pipeline participant CF as CloudFormation participant AWS as AWS Resources Dev->>Git: Push IaC Templates Git->>CI: Trigger Pipeline CI->>CI: Validate Templates CI->>CF: Deploy Stack CF->>AWS: Create/Update Resources AWS->>CF: Return Status CF->>CI: Stack Status CI->>Dev: Deployment Results Note over Dev,AWS: Infrastructure Deployment Flow

Infrastructure as Code Deployment Flow

This sequence diagram shows the typical IaC deployment workflow:

  1. Developer creates or modifies infrastructure templates
  2. Code is committed to version control (Git)
  3. CI/CD pipeline is triggered automatically
  4. Templates are validated for syntax and best practices
  5. CloudFormation deploys the infrastructure changes
  6. AWS resources are created, updated, or deleted as needed
  7. Status and results are reported back to the developer

3. CloudFormation Deep Dive

3.1 Basic Stack Creation

aws cloudformation create-stack \
    --stack-name my-vpc-stack \
    --template-body file://vpc-template.yaml \
    --parameters ParameterKey=Environment,ParameterValue=prod \
    --capabilities CAPABILITY_IAM \
    --tags Key=Project,Value=WebApp Key=Owner,Value=DevTeam

CloudFormation Stack Creation Parameters

3.2 VPC Template Example

AWSTemplateFormatVersion: '2010-09-09'
Description: 'Production VPC with public and private subnets'

Parameters:
  Environment:
    Type: String
    Default: prod
    AllowedValues: [dev, staging, prod]
    Description: Environment name for resource naming

  VpcCidr:
    Type: String
    Default: '10.0.0.0/16'
    Description: CIDR block for VPC

CloudFormation Template Structure

This template header section defines:

3.3 VPC Resource Definition

Resources:
  VPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: !Ref VpcCidr
      EnableDnsHostnames: true
      EnableDnsSupport: true
      Tags:
        - Key: Name
          Value: !Sub '${Environment}-vpc'
        - Key: Environment
          Value: !Ref Environment

VPC Resource Configuration

This creates the main VPC with the following properties:

Other VPC Options: InstanceTenancy (default|dedicated), Ipv6CidrBlock

3.4 Internet Gateway Configuration

  InternetGateway:
    Type: AWS::EC2::InternetGateway
    Properties:
      Tags:
        - Key: Name
          Value: !Sub '${Environment}-igw'

  InternetGatewayAttachment:
    Type: AWS::EC2::VPCGatewayAttachment
    Properties:
      InternetGatewayId: !Ref InternetGateway
      VpcId: !Ref VPC

Internet Gateway Setup

This configuration creates internet connectivity:

Execution Order: VPC → InternetGateway → VPCGatewayAttachment

graph LR A[VPC Creation] --> B[Internet Gateway Creation] B --> C[IGW Attachment] C --> D[Public Subnet Creation] C --> E[Private Subnet Creation] D --> F[Public Route Table] E --> G[Private Route Table] F --> H[Public Routes] G --> I[NAT Gateway Routes] style A fill:#232F3E,stroke:#FF9500,stroke-width:2px,color:#fff style B fill:#232F3E,stroke:#FF9500,stroke-width:2px,color:#fff style C fill:#232F3E,stroke:#FF9500,stroke-width:2px,color:#fff

CloudFormation Resource Creation Order

This diagram shows the dependency chain for VPC infrastructure:

  1. VPC Creation: Foundation network container
  2. Internet Gateway: Internet access component
  3. IGW Attachment: Links gateway to VPC
  4. Subnet Creation: Network segments within VPC
  5. Route Tables: Traffic routing rules
  6. Route Configuration: Specific routing entries

3.5 Stack Update Process

aws cloudformation update-stack \
    --stack-name my-vpc-stack \
    --template-body file://vpc-template-v2.yaml \
    --parameters ParameterKey=Environment,ParameterValue=prod \
                 ParameterKey=VpcCidr,ParameterValue=10.1.0.0/16 \
    --capabilities CAPABILITY_IAM

Stack Update Considerations

When updating stacks, CloudFormation determines what changes are needed:

⚠️ Important Update Considerations

Some resource changes require replacement (deletion and recreation). Always review change sets before applying updates to production environments.

3.6 Stack Monitoring and Validation

# Check stack status
aws cloudformation describe-stacks \
    --stack-name my-vpc-stack \
    --query 'Stacks[0].StackStatus'

# Validate template before deployment
aws cloudformation validate-template \
    --template-body file://vpc-template.yaml

# Check for configuration drift
aws cloudformation detect-stack-drift \
    --stack-name my-vpc-stack

Stack Monitoring Commands

4. AWS CDK Implementation

flowchart TD A[CDK App] --> B[CDK Stack] B --> C[CDK Constructs] C --> D[L1 Constructs
CloudFormation Resources] C --> E[L2 Constructs
AWS Service Patterns] C --> F[L3 Constructs
Architecture Patterns] G[CDK Synth] --> H[CloudFormation Template] H --> I[CDK Deploy] I --> J[AWS Resources] style A fill:#FF9500,stroke:#232F3E,stroke-width:3px,color:#fff style B fill:#232F3E,stroke:#FF9500,stroke-width:2px,color:#fff style G fill:#4caf50,stroke:#2e7d32,stroke-width:2px,color:#fff

CDK Architecture and Construct Levels

The CDK provides three levels of constructs:

4.1 CDK Project Initialization

# Initialize new CDK project
mkdir my-cdk-project
cd my-cdk-project
cdk init app --language=typescript

# Install additional CDK modules
npm install @aws-cdk/aws-ec2 @aws-cdk/aws-iam @aws-cdk/aws-lambda

CDK Project Setup

This creates a new CDK project with TypeScript:

4.2 VPC Stack Implementation

import * as cdk from '@aws-cdk/core';
import * as ec2 from '@aws-cdk/aws-ec2';

export class VpcStack extends cdk.Stack {
  public readonly vpc: ec2.Vpc;

  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // Create VPC with public and private subnets
    this.vpc = new ec2.Vpc(this, 'MyVpc', {
      maxAzs: 2,
      cidr: '10.0.0.0/16',
      subnetConfiguration: [
        {
          cidrMask: 24,
          name: 'public',
          subnetType: ec2.SubnetType.PUBLIC,
        },
        {
          cidrMask: 24,
          name: 'private',
          subnetType: ec2.SubnetType.PRIVATE_WITH_NAT,
        }
      ]
    });
  }
}

CDK VPC Configuration

This L2 construct automatically creates:

Other Subnet Types: PRIVATE_ISOLATED, PRIVATE_WITH_EGRESS

4.3 CDK Deployment Commands

# Synthesize CloudFormation template
cdk synth

# Deploy stack
cdk deploy VpcStack --require-approval never

# Deploy with parameters
cdk deploy VpcStack \
    --parameters VpcCidr=10.1.0.0/16 \
    --parameters Environment=prod

CDK Deployment Process

CDK Command Execution Flow

  1. cdk init: Create project structure
  2. npm install: Install CDK dependencies
  3. Code Implementation: Write CDK constructs
  4. cdk synth: Generate CloudFormation template
  5. cdk deploy: Deploy to AWS
  6. cdk diff: Compare deployed vs. current code
  7. cdk destroy: Clean up resources

4.4 Advanced CDK Features

# Cross-stack references
export class DatabaseStack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string, vpc: ec2.Vpc) {
    super(scope, id);
    
    const database = new rds.Database(this, 'Database', {
      vpc: vpc,  // Reference VPC from another stack
      engine: rds.DatabaseInstanceEngine.mysql({
        version: rds.MysqlEngineVersion.VER_8_0
      })
    });
  }
}

CDK Cross-Stack Dependencies

CDK enables sharing resources between stacks:

5. Event-Driven Network Automation

graph TB A[AWS CloudTrail] --> B[CloudWatch Events/EventBridge] C[VPC Flow Logs] --> B D[Config Changes] --> B B --> E[Lambda Function] B --> F[SNS Topic] B --> G[SQS Queue] E --> H[Auto-remediation] E --> I[Network Reconfiguration] E --> J[Security Group Updates] F --> K[Email Notifications] F --> L[Slack Integration] G --> M[Batch Processing] G --> N[Audit Logging] style B fill:#FF9500,stroke:#232F3E,stroke-width:3px,color:#fff style E fill:#232F3E,stroke:#FF9500,stroke-width:2px,color:#fff

Event-Driven Network Automation Architecture

This architecture shows how AWS events trigger automated responses:

5.1 EventBridge Rule Creation

aws events put-rule \
    --name "SecurityGroupChangeRule" \
    --event-pattern '{
        "source": ["aws.ec2"],
        "detail-type": ["AWS API Call via CloudTrail"],
        "detail": {
            "eventSource": ["ec2.amazonaws.com"],
            "eventName": [
                "AuthorizeSecurityGroupIngress",
                "RevokeSecurityGroupIngress"
            ]
        }
    }' \
    --state ENABLED

EventBridge Rule Configuration

This rule monitors security group changes:

5.2 Lambda Target Configuration

aws events put-targets \
    --rule "SecurityGroupChangeRule" \
    --targets "Id"="1","Arn"="arn:aws:lambda:us-east-1:123456789012:function:ProcessSecurityGroupChange"

# Grant EventBridge permission to invoke Lambda
aws lambda add-permission \
    --function-name ProcessSecurityGroupChange \
    --statement-id allow-eventbridge \
    --action lambda:InvokeFunction \
    --principal events.amazonaws.com \
    --source-arn arn:aws:events:us-east-1:123456789012:rule/SecurityGroupChangeRule

Event Target Setup

Connecting EventBridge to Lambda requires:

Execution Order: Rule → Target → Permission

5.3 Auto-Remediation Lambda Function

import json
import boto3

def lambda_handler(event, context):
    ec2 = boto3.client('ec2')
    
    # Extract security group ID from event
    detail = event['detail']
    sg_id = detail['requestParameters']['groupId']
    
    # Check for dangerous rules (0.0.0.0/0 on port 22)
    response = ec2.describe_security_groups(GroupIds=[sg_id])
    sg = response['SecurityGroups'][0]
    
    for rule in sg['IpPermissions']:
        if rule.get('FromPort') == 22:
            for ip_range in rule.get('IpRanges', []):
                if ip_range.get('CidrIp') == '0.0.0.0/0':
                    # Remove dangerous rule
                    ec2.revoke_security_group_ingress(
                        GroupId=sg_id,
                        IpPermissions=[rule]
                    )
                    
                    # Send notification
                    sns = boto3.client('sns')
                    sns.publish(
                        TopicArn='arn:aws:sns:us-east-1:123456789012:security-alerts',
                        Message=f'Removed dangerous SSH rule from {sg_id}',
                        Subject='Security Group Auto-Remediation'
                    )
    
    return {'statusCode': 200}

Auto-Remediation Logic

This Lambda function automatically:

sequenceDiagram participant User as User/System participant CT as CloudTrail participant EB as EventBridge participant LF as Lambda Function participant EC2 as EC2 Service participant SNS as SNS Topic User->>EC2: Modify Security Group EC2->>CT: Log API Call CT->>EB: Send Event EB->>LF: Trigger Function LF->>EC2: Analyze Security Group LF->>EC2: Remove Dangerous Rule LF->>SNS: Send Alert SNS->>User: Email Notification

Event-Driven Remediation Flow

This sequence shows the automated response to security group changes:

  1. User modifies security group (manually or via automation)
  2. CloudTrail captures the API call
  3. EventBridge receives and routes the event
  4. Lambda function analyzes the change
  5. Dangerous rules are automatically removed
  6. Administrators are notified of the action

5.4 VPC Flow Logs Automation

# Create VPC Flow Logs
aws ec2 create-flow-logs \
    --resource-type VPC \
    --resource-ids vpc-12345678 \
    --traffic-type ALL \
    --log-destination-type cloud-watch-logs \
    --log-group-name VPCFlowLogs \
    --deliver-logs-permission-arn arn:aws:iam::123456789012:role/flowlogsRole

VPC Flow Logs Configuration

6. Repeatable Network Configurations

graph LR A[Template Repository] --> B[Parameter Store] A --> C[Environment Config] B --> D[CloudFormation Stack] C --> D D --> E[Dev Environment] D --> F[Staging Environment] D --> G[Prod Environment] H[Nested Stacks] --> I[VPC Stack] H --> J[Security Stack] H --> K[Application Stack] style A fill:#FF9500,stroke:#232F3E,stroke-width:3px,color:#fff style D fill:#232F3E,stroke:#FF9500,stroke-width:2px,color:#fff

Repeatable Configuration Architecture

This approach ensures consistent deployments across environments:

6.1 Parameter Store Configuration

# Store environment-specific parameters
aws ssm put-parameter \
    --name "/myapp/dev/vpc-cidr" \
    --value "10.0.0.0/16" \
    --type "String" \
    --description "VPC CIDR for development environment"

aws ssm put-parameter \
    --name "/myapp/prod/vpc-cidr" \
    --value "10.1.0.0/16" \
    --type "String" \
    --description "VPC CIDR for production environment"

# Store encrypted parameters
aws ssm put-parameter \
    --name "/myapp/prod/database-password" \
    --value "SuperSecretPassword123!" \
    --type "SecureString" \
    --key-id "alias/parameter-store-key"

Parameter Store Benefits

6.2 Dynamic Parameter Retrieval in CloudFormation

Parameters:
  Environment:
    Type: String
    Default: dev
    AllowedValues: [dev, staging, prod]

Resources:
  VPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: !Sub '{{resolve:ssm:/myapp/${Environment}/vpc-cidr:1}}'
      EnableDnsHostnames: true
      EnableDnsSupport: true
      Tags:
        - Key: Name
          Value: !Sub '${Environment}-vpc'

Dynamic Parameter Resolution

CloudFormation can resolve parameters at deployment time:

Other Resolvers: resolve:secretsmanager, resolve:ssm-secure

6.3 Nested Stack Implementation

# Master template referencing nested stacks
Resources:
  NetworkStack:
    Type: AWS::CloudFormation::Stack
    Properties:
      TemplateURL: https://s3.amazonaws.com/my-templates/network-stack.yaml
      Parameters:
        Environment: !Ref Environment
        VpcCidr: !Sub '{{resolve:ssm:/myapp/${Environment}/vpc-cidr:1}}'
      Tags:
        - Key: StackType
          Value: Network

  SecurityStack:
    Type: AWS::CloudFormation::Stack
    DependsOn: NetworkStack
    Properties:
      TemplateURL: https://s3.amazonaws.com/my-templates/security-stack.yaml
      Parameters:
        VpcId: !GetAtt NetworkStack.Outputs.VpcId
        Environment: !Ref Environment

Nested Stack Benefits

Nested Stack Deployment Order

  1. Upload Templates: Store nested templates in S3
  2. Deploy Master Stack: References nested template URLs
  3. CloudFormation: Automatically deploys nested stacks
  4. Dependency Resolution: Respects DependsOn relationships
  5. Output Collection: Master stack aggregates nested outputs

6.4 Multi-Environment Deployment Script

#!/bin/bash

ENVIRONMENTS=("dev" "staging" "prod")
TEMPLATE_URL="https://s3.amazonaws.com/my-templates/master-template.yaml"

for ENV in "${ENVIRONMENTS[@]}"; do
    echo "Deploying to $ENV environment..."
    
    aws cloudformation deploy \
        --template-file master-template.yaml \
        --stack-name "myapp-$ENV" \
        --parameter-overrides Environment=$ENV \
        --capabilities CAPABILITY_IAM \
        --tags Environment=$ENV Project=MyApp \
        --region us-east-1
        
    # Wait for stack completion
    aws cloudformation wait stack-deploy-complete \
        --stack-name "myapp-$ENV" \
        --region us-east-1
        
    echo "$ENV deployment completed successfully!"
done

Automated Multi-Environment Deployment

This script enables consistent deployments across environments:

7. Cost Optimization Through Automation

graph TB A[Cost Optimization] --> B[Resource Scheduling] A --> C[Auto Scaling] A --> D[Resource Tagging] A --> E[Usage Monitoring] B --> B1[Lambda Scheduler] B --> B2[EventBridge Rules] B --> B3[Systems Manager] C --> C1[EC2 Auto Scaling] C --> C2[Application Auto Scaling] C --> C3[Predictive Scaling] D --> D1[Cost Allocation] D --> D2[Resource Tracking] D --> D3[Automated Tagging] E --> E1[CloudWatch Metrics] E --> E2[Cost Explorer API] E --> E3[Budgets & Alerts] style A fill:#FF9500,stroke:#232F3E,stroke-width:3px,color:#fff style B fill:#232F3E,stroke:#FF9500,stroke-width:2px,color:#fff style C fill:#232F3E,stroke:#FF9500,stroke-width:2px,color:#fff

Cost Optimization Strategy

Automated cost optimization involves multiple approaches:

7.1 Automated Resource Scheduling

# Create EventBridge rule for evening shutdown
aws events put-rule \
    --name "StopDevInstancesEvening" \
    --schedule-expression "cron(0 18 * * MON-FRI *)" \
    --state ENABLED \
    --description "Stop development instances at 6 PM weekdays"

# Create rule for morning startup
aws events put-rule \
    --name "StartDevInstancesMorning" \
    --schedule-expression "cron(0 8 * * MON-FRI *)" \
    --state ENABLED \
    --description "Start development instances at 8 AM weekdays"

Scheduled Resource Management

7.2 Instance Management Lambda Function

import boto3
import json

def lambda_handler(event, context):
    ec2 = boto3.client('ec2')
    
    # Get the action from the event source
    rule_name = event['source']
    action = 'start' if 'Start' in rule_name else 'stop'
    
    # Find instances with Environment=dev tag
    response = ec2.describe_instances(
        Filters=[
            {'Name': 'tag:Environment', 'Values': ['dev']},
            {'Name': 'instance-state-name', 
             'Values': ['stopped'] if action == 'start' else ['running']}
        ]
    )
    
    instance_ids = []
    for reservation in response['Reservations']:
        for instance in reservation['Instances']:
            instance_ids.append(instance['InstanceId'])
    
    if instance_ids:
        if action == 'start':
            ec2.start_instances(InstanceIds=instance_ids)
            message = f"Started {len(instance_ids)} development instances"
        else:
            ec2.stop_instances(InstanceIds=instance_ids)
            message = f"Stopped {len(instance_ids)} development instances"
        
        # Send notification
        sns = boto3.client('sns')
        sns.publish(
            TopicArn='arn:aws:sns:us-east-1:123456789012:cost-optimization',
            Message=message,
            Subject=f'EC2 Instance {action.title()} Automation'
        )
    
    return {
        'statusCode': 200,
        'body': json.dumps(f'Action: {action}, Instances: {len(instance_ids)}')
    }

Automated Instance Management

This Lambda function:

7.3 Auto Scaling Configuration

# Create Auto Scaling Group
aws autoscaling create-auto-scaling-group \
    --auto-scaling-group-name MyApp-ASG \
    --launch-template LaunchTemplateName=MyApp-LaunchTemplate,Version=1 \
    --min-size 1 \
    --max-size 10 \
    --desired-capacity 2 \
    --vpc-zone-identifier "subnet-12345678,subnet-87654321" \
    --target-group-arns arn:aws:elasticloadbalancing:us-east-1:123456789012:targetgroup/MyApp-TG/1234567890123456 \
    --health-check-type ELB \
    --health-check-grace-period 300 \
    --tags Key=Environment,Value=prod,PropagateAtLaunch=true

Auto Scaling Group Configuration

7.4 Cost Monitoring and Alerting

# Create CloudWatch billing alarm
aws cloudwatch put-metric-alarm \
    --alarm-name "HighBillingAlarm" \
    --alarm-description "Alert when monthly charges exceed $100" \
    --metric-name EstimatedCharges \
    --namespace AWS/Billing \
    --statistic Maximum \
    --period 86400 \
    --threshold 100 \
    --comparison-operator GreaterThanThreshold \
    --dimensions Name=Currency,Value=USD \
    --evaluation-periods 1 \
    --alarm-actions arn:aws:sns:us-east-1:123456789012:billing-alerts \
    --unit None

Billing Alarm Configuration

Prerequisite: Enable billing alerts in AWS Billing console

sequenceDiagram participant CW as CloudWatch participant EB as EventBridge participant LF as Lambda participant EC2 as EC2 participant SNS as SNS participant User as Admin Note over CW,User: Cost Optimization Workflow CW->>EB: Trigger Schedule (6 PM) EB->>LF: Execute Lambda LF->>EC2: Query Dev Instances EC2->>LF: Return Instance List LF->>EC2: Stop Instances LF->>SNS: Send Notification SNS->>User: Email Alert Note over CW,User: Morning Restart (8 AM) CW->>EB: Trigger Schedule (8 AM) EB->>LF: Execute Lambda LF->>EC2: Start Instances SNS->>User: Startup Notification

Automated Cost Optimization Flow

This sequence shows the daily cost optimization cycle:

  1. Evening Schedule: EventBridge triggers at 6 PM
  2. Instance Query: Lambda finds development instances
  3. Resource Shutdown: Non-production instances stopped
  4. Morning Restart: Instances automatically started at 8 AM
  5. Notifications: Administrators informed of all actions

8. Testing and Validation Automation

graph TD A[Infrastructure Testing] --> B[Pre-Deployment] A --> C[Post-Deployment] A --> D[Continuous Monitoring] B --> B1[Template Validation] B --> B2[Security Scanning] B --> B3[Cost Estimation] C --> C1[Connectivity Tests] C --> C2[Performance Tests] C --> C3[Security Validation] D --> D1[Drift Detection] D --> D2[Compliance Monitoring] D --> D3[Performance Monitoring] style A fill:#FF9500,stroke:#232F3E,stroke-width:3px,color:#fff style B fill:#232F3E,stroke:#FF9500,stroke-width:2px,color:#fff style C fill:#232F3E,stroke:#FF9500,stroke-width:2px,color:#fff style D fill:#232F3E,stroke:#FF9500,stroke-width:2px,color:#fff

Infrastructure Testing Strategy

Comprehensive testing covers three phases:

8.1 CloudFormation Template Validation

# Validate template syntax
aws cloudformation validate-template \
    --template-body file://infrastructure-template.yaml

# Estimate costs before deployment
aws cloudformation estimate-template-cost \
    --template-body file://infrastructure-template.yaml \
    --parameters ParameterKey=InstanceType,ParameterValue=t3.medium

# Create change set to preview changes
aws cloudformation create-change-set \
    --stack-name my-infrastructure \
    --template-body file://infrastructure-template.yaml \
    --change-set-name preview-changes-v2 \
    --capabilities CAPABILITY_IAM

Pre-Deployment Validation

8.2 Security Scanning with cfn-nag

# Install cfn-nag security scanner
gem install cfn-nag

# Scan CloudFormation template for security issues
cfn_nag_scan --input-path infrastructure-template.yaml

# Generate JSON output for CI/CD integration
cfn_nag_scan --input-path infrastructure-template.yaml --output-format json

Security Scanning Benefits

cfn-nag identifies common security issues:

8.3 Post-Deployment Testing Script

#!/bin/bash

STACK_NAME="my-infrastructure"
REGION="us-east-1"

echo "Starting post-deployment validation for $STACK_NAME"

# Get stack outputs
VPC_ID=$(aws cloudformation describe-stacks \
    --stack-name $STACK_NAME \
    --region $REGION \
    --query 'Stacks[0].Outputs[?OutputKey==`VpcId`].OutputValue' \
    --output text)

PUBLIC_SUBNET_ID=$(aws cloudformation describe-stacks \
    --stack-name $STACK_NAME \
    --region $REGION \
    --query 'Stacks[0].Outputs[?OutputKey==`PublicSubnetId`].OutputValue' \
    --output text)

# Test 1: Verify VPC exists and is available
echo "Testing VPC availability..."
VPC_STATE=$(aws ec2 describe-vpcs \
    --vpc-ids $VPC_ID \
    --region $REGION \
    --query 'Vpcs[0].State' \
    --output text)

if [ "$VPC_STATE" != "available" ]; then
    echo "❌ VPC is not available: $VPC_STATE"
    exit 1
else
    echo "✅ VPC is available"
fi

# Test 2: Verify internet connectivity from public subnet
echo "Testing internet connectivity..."
INSTANCE_ID=$(aws ec2 run-instances \
    --image-id ami-0abcdef1234567890 \
    --instance-type t3.micro \
    --subnet-id $PUBLIC_SUBNET_ID \
    --associate-public-ip-address \
    --user-data '#!/bin/bash
    yum update -y
    yum install -y awscli
    curl -s http://checkip.amazonaws.com > /tmp/public-ip.txt' \
    --query 'Instances[0].InstanceId' \
    --output text)

# Wait for instance to be running
aws ec2 wait instance-running --instance-ids $INSTANCE_ID --region $REGION

# Test connectivity (simplified - in practice, use Systems Manager)
sleep 60  # Allow time for user data script

# Cleanup test instance
aws ec2 terminate-instances --instance-ids $INSTANCE_ID --region $REGION

echo "✅ All tests passed successfully!"

Post-Deployment Test Components

8.4 CloudFormation Drift Detection

# Detect drift in stack
aws cloudformation detect-stack-drift \
    --stack-name my-infrastructure

# Get drift detection status
DRIFT_ID=$(aws cloudformation detect-stack-drift \
    --stack-name my-infrastructure \
    --query 'StackDriftDetectionId' \
    --output text)

# Wait for drift detection to complete
aws cloudformation describe-stack-drift-detection-status \
    --stack-drift-detection-id $DRIFT_ID

# Get detailed drift results
aws cloudformation describe-stack-resource-drifts \
    --stack-name my-infrastructure

Drift Detection Process

Drift detection identifies manual changes to resources:

8.5 Automated Compliance Monitoring

# Create Config rule for security group compliance
aws configservice put-config-rule \
    --config-rule '{
        "ConfigRuleName": "security-group-ssh-check",
        "Description": "Checks whether security groups allow unrestricted SSH access",
        "Source": {
            "Owner": "AWS",
            "SourceIdentifier": "INCOMING_SSH_DISABLED"
        },
        "Scope": {
            "ComplianceResourceTypes": [
                "AWS::EC2::SecurityGroup"
            ]
        }
    }'

# Create remediation configuration
aws configservice put-remediation-configurations \
    --remediation-configurations '[{
        "ConfigRuleName": "security-group-ssh-check",
        "TargetType": "SSM_DOCUMENT",
        "TargetId": "RemoveUnrestrictedSourceInSecurityGroup",
        "TargetVersion": "1",
        "Parameters": {
            "AutomationAssumeRole": {
                "StaticValue": {
                    "Values": ["arn:aws:iam::123456789012:role/ConfigRemediationRole"]
                }
            },
            "GroupId": {
                "ResourceValue": {
                    "Value": "RESOURCE_ID"
                }
            }
        },
        "Automatic": true,
        "MaximumAutomaticAttempts": 3
    }]'

AWS Config Compliance Monitoring

flowchart TD A[Code Commit] --> B[CI/CD Pipeline] B --> C[Template Validation] C --> D[Security Scanning] D --> E[Cost Estimation] E --> F{Validation Passed?} F -->|Yes| G[Create Change Set] F -->|No| H[Report Issues] G --> I[Deploy to Test] I --> J[Post-Deploy Tests] J --> K{Tests Passed?} K -->|Yes| L[Deploy to Production] K -->|No| M[Rollback] L --> N[Drift Detection] N --> O[Compliance Monitoring] O --> P[Performance Monitoring] style A fill:#FF9500,stroke:#232F3E,stroke-width:3px,color:#fff style F fill:#f44336,stroke:#d32f2f,stroke-width:2px,color:#fff style K fill:#f44336,stroke:#d32f2f,stroke-width:2px,color:#fff

Complete Testing and Deployment Pipeline

This comprehensive workflow ensures infrastructure quality:

  1. Pre-Deployment Validation: Template syntax, security, and cost checks
  2. Change Set Review: Preview exactly what will change
  3. Test Environment: Deploy to isolated environment first
  4. Automated Testing: Verify functionality and performance
  5. Production Deployment: Deploy only after all tests pass
  6. Continuous Monitoring: Ongoing drift and compliance checking

🎯 Best Practices for Infrastructure Testing

8.6 Systems Manager Automation for Testing

# Create Systems Manager automation document
aws ssm create-document \
    --name "NetworkConnectivityTest" \
    --document-type "Automation" \
    --document-format "YAML" \
    --content '{
        "schemaVersion": "0.3",
        "description": "Test network connectivity in VPC",
        "assumeRole": "arn:aws:iam::123456789012:role/SSMAutomationRole",
        "parameters": {
            "SubnetId": {
                "type": "String",
                "description": "Subnet ID to test connectivity"
            },
            "TargetUrl": {
                "type": "String",
                "default": "https://aws.amazon.com",
                "description": "URL to test connectivity"
            }
        },
        "mainSteps": [
            {
                "name": "LaunchTestInstance",
                "action": "aws:runInstances",
                "inputs": {
                    "ImageId": "ami-0abcdef1234567890",
                    "InstanceType": "t3.micro",
                    "SubnetId": "{{ SubnetId }}",
                    "UserData": "IyEvYmluL2Jhc2gKY3VybCAtcyB7eyBUYXJnZXRVcmwgfX0gPiAvdG1wL2Nvbm5lY3Rpdml0eS10ZXN0LnR4dA=="
                }
            },
            {
                "name": "WaitForInstance",
                "action": "aws:waitForAwsResourceProperty",
                "inputs": {
                    "Service": "ec2",
                    "Api": "DescribeInstances",
                    "InstanceIds": ["{{ LaunchTestInstance.InstanceIds }}"],
                    "PropertySelector": "$.Reservations[0].Instances[0].State.Name",
                    "DesiredValues": ["running"]
                }
            },
            {
                "name": "TerminateTestInstance",
                "action": "aws:changeInstanceState",
                "inputs": {
                    "InstanceIds": ["{{ LaunchTestInstance.InstanceIds }}"],
                    "DesiredState": "terminated"
                }
            }
        ]
    }'

Systems Manager Automation Benefits

8.7 Performance Testing with CloudWatch

# Create custom metric for application performance
aws cloudwatch put-metric-data \
    --namespace "MyApp/Performance" \
    --metric-data MetricName=ResponseTime,Value=245,Unit=Milliseconds,Dimensions=[{Name=Environment,Value=prod}]

# Create performance alarm
aws cloudwatch put-metric-alarm \
    --alarm-name "HighResponseTime" \
    --alarm-description "Alert when response time exceeds 500ms" \
    --metric-name ResponseTime \
    --namespace MyApp/Performance \
    --statistic Average \
    --period 300 \
    --threshold 500 \
    --comparison-operator GreaterThanThreshold \
    --dimensions Name=Environment,Value=prod \
    --evaluation-periods 2 \
    --alarm-actions arn:aws:sns:us-east-1:123456789012:performance-alerts

Performance Monitoring Setup

9. Advanced Automation Patterns

graph TB A[Advanced Patterns] --> B[GitOps] A --> C[Blue/Green Deployment] A --> D[Canary Releases] A --> E[Multi-Region Orchestration] B --> B1[Git Repository] B --> B2[CI/CD Pipeline] B --> B3[Infrastructure Sync] C --> C1[Blue Environment] C --> C2[Green Environment] C --> C3[Traffic Switch] D --> D1[Small Traffic %] D --> D2[Gradual Rollout] D --> D3[Automatic Rollback] E --> E1[Primary Region] E --> E2[Secondary Region] E --> E3[Cross-Region Replication] style A fill:#FF9500,stroke:#232F3E,stroke-width:3px,color:#fff style B fill:#232F3E,stroke:#FF9500,stroke-width:2px,color:#fff style C fill:#232F3E,stroke:#FF9500,stroke-width:2px,color:#fff

Advanced Infrastructure Automation

These patterns enable sophisticated deployment strategies:

9.1 GitOps Pipeline Implementation

# CodeBuild project for GitOps automation
aws codebuild create-project \
    --name "infrastructure-gitops" \
    --source '{
        "type": "GITHUB",
        "location": "https://github.com/myorg/infrastructure-repo",
        "buildspec": "buildspec.yml"
    }' \
    --artifacts '{"type": "NO_ARTIFACTS"}' \
    --environment '{
        "type": "LINUX_CONTAINER",
        "image": "aws/codebuild/amazonlinux2-x86_64-standard:3.0",
        "computeType": "BUILD_GENERAL1_MEDIUM",
        "privilegedMode": true
    }' \
    --service-role "arn:aws:iam::123456789012:role/CodeBuildServiceRole"

GitOps CodeBuild Configuration

9.2 Blue/Green Deployment Strategy

# Create blue environment stack
aws cloudformation create-stack \
    --stack-name "myapp-blue" \
    --template-body file://application-stack.yaml \
    --parameters ParameterKey=Environment,ParameterValue=blue \
                 ParameterKey=Color,ParameterValue=blue \
    --capabilities CAPABILITY_IAM

# Health check function
aws lambda create-function \
    --function-name "HealthChecker" \
    --runtime "python3.9" \
    --role "arn:aws:iam::123456789012:role/LambdaExecutionRole" \
    --handler "health_check.lambda_handler" \
    --zip-file "fileb://health-check-function.zip" \
    --environment Variables='{
        "BLUE_ENDPOINT": "https://blue.myapp.com",
        "GREEN_ENDPOINT": "https://green.myapp.com"
    }'

Blue/Green Infrastructure Setup

Blue/Green deployment requires:

sequenceDiagram participant Dev as Developer participant Git as Git Repository participant CB as CodeBuild participant CF as CloudFormation participant LB as Load Balancer participant Blue as Blue Environment participant Green as Green Environment Dev->>Git: Push Code Changes Git->>CB: Trigger Pipeline CB->>CF: Deploy Green Environment CF->>Green: Create New Resources CB->>CB: Run Health Checks CB->>LB: Switch Traffic to Green LB->>Green: Route 100% Traffic CB->>CF: Tear Down Blue Environment

Blue/Green Deployment Sequence

This sequence shows zero-downtime deployment:

  1. Code Change: Developer pushes new version
  2. Green Deployment: New environment created alongside existing
  3. Health Validation: Automated testing of new environment
  4. Traffic Switch: Load balancer routes to new environment
  5. Cleanup: Old environment destroyed after validation

9.3 Multi-Region Deployment Orchestration

# Deploy to multiple regions with CodePipeline
aws codepipeline create-pipeline \
    --pipeline '{
        "name": "MultiRegionDeployment",
        "roleArn": "arn:aws:iam::123456789012:role/CodePipelineRole",
        "artifactStore": {
            "us-east-1": {
                "type": "S3",
                "location": "codepipeline-artifacts-us-east-1"
            },
            "us-west-2": {
                "type": "S3", 
                "location": "codepipeline-artifacts-us-west-2"
            }
        },
        "stages": [
            {
                "name": "Source",
                "actions": [{
                    "name": "SourceAction",
                    "actionTypeId": {
                        "category": "Source",
                        "owner": "AWS",
                        "provider": "S3",
                        "version": "1"
                    },
                    "configuration": {
                        "S3Bucket": "my-source-bucket",
                        "S3ObjectKey": "infrastructure.zip"
                    },
                    "outputArtifacts": [{"name": "SourceOutput"}]
                }]
            },
            {
                "name": "DeployPrimary",
                "actions": [{
                    "name": "DeployUSEast1",
                    "actionTypeId": {
                        "category": "Deploy",
                        "owner": "AWS",
                        "provider": "CloudFormation",
                        "version": "1"
                    },
                    "configuration": {
                        "ActionMode": "CREATE_UPDATE",
                        "StackName": "MyApp-Primary",
                        "TemplatePath": "SourceOutput::template.yaml",
                        "Capabilities": "CAPABILITY_IAM",
                        "RoleArn": "arn:aws:iam::123456789012:role/CloudFormationRole"
                    },
                    "inputArtifacts": [{"name": "SourceOutput"}],
                    "region": "us-east-1"
                }]
            },
            {
                "name": "DeploySecondary",
                "actions": [{
                    "name": "DeployUSWest2",
                    "actionTypeId": {
                        "category": "Deploy",
                        "owner": "AWS",
                        "provider": "CloudFormation",
                        "version": "1"
                    },
                    "configuration": {
                        "ActionMode": "CREATE_UPDATE",
                        "StackName": "MyApp-Secondary",
                        "TemplatePath": "SourceOutput::template.yaml",
                        "Capabilities": "CAPABILITY_IAM",
                        "RoleArn": "arn:aws:iam::123456789012:role/CloudFormationRole"
                    },
                    "inputArtifacts": [{"name": "SourceOutput"}],
                    "region": "us-west-2"
                }]
            }
        ]
    }'

Multi-Region Pipeline Configuration

10. Command Reference and Execution Order

graph TD A[Infrastructure Setup Order] --> B[1. IAM Roles & Policies] B --> C[2. VPC & Networking] C --> D[3. Security Groups] D --> E[4. Application Resources] E --> F[5. Monitoring & Alerts] F --> G[6. Automation & Events] B1[Create Service Roles] --> B2[Attach Policies] B2 --> B3[Create Instance Profiles] C1[Create VPC] --> C2[Create Subnets] C2 --> C3[Create Route Tables] C3 --> C4[Create NAT/IGW] B --> B1 C --> C1 style A fill:#FF9500,stroke:#232F3E,stroke-width:3px,color:#fff style B fill:#232F3E,stroke:#FF9500,stroke-width:2px,color:#fff style C fill:#232F3E,stroke:#FF9500,stroke-width:2px,color:#fff

Infrastructure Component Dependencies

This diagram shows the proper order for creating infrastructure components. Each level depends on the completion of the previous level.

10.1 Complete Infrastructure Setup Commands

Step 1: IAM Roles and Policies (Execute First)

Create foundational security components before any other resources.

# 1a. Create CloudFormation execution role
aws iam create-role \
    --role-name CloudFormationExecutionRole \
    --assume-role-policy-document '{
        "Version": "2012-10-17",
        "Statement": [{
            "Effect": "Allow",
            "Principal": {"Service": "cloudformation.amazonaws.com"},
            "Action": "sts:AssumeRole"
        }]
    }'

# 1b. Attach administrator policy (for demo - use least privilege in production)
aws iam attach-role-policy \
    --role-name CloudFormationExecutionRole \
    --policy-arn arn:aws:iam::aws:policy/AdministratorAccess

# 1c. Create Lambda execution role for automation
aws iam create-role \
    --role-name LambdaAutomationRole \
    --assume-role-policy-document '{
        "Version": "2012-10-17",
        "Statement": [{
            "Effect": "Allow",
            "Principal": {"Service": "lambda.amazonaws.com"},
            "Action": "sts:AssumeRole"
        }]
    }'

IAM Setup Priority

IAM resources must be created first because:

Step 2: VPC and Core Networking (Execute Second)

Create the network foundation that all other resources will use.

# 2a. Deploy VPC stack (depends on IAM roles from step 1)
aws cloudformation create-stack \
    --stack-name core-networking \
    --template-body file://vpc-template.yaml \
    --parameters ParameterKey=Environment,ParameterValue=prod \
                 ParameterKey=VpcCidr,ParameterValue=10.0.0.0/16 \
    --capabilities CAPABILITY_IAM \
    --role-arn arn:aws:iam::123456789012:role/CloudFormationExecutionRole

# 2b. Wait for VPC stack completion before proceeding
aws cloudformation wait stack-create-complete \
    --stack-name core-networking

# 2c. Get VPC outputs for next steps
VPC_ID=$(aws cloudformation describe-stacks \
    --stack-name core-networking \
    --query 'Stacks[0].Outputs[?OutputKey==`VpcId`].OutputValue' \
    --output text)

Networking Setup Dependencies

Step 3: Security Groups (Execute Third)

Create security rules that reference the VPC from Step 2.

# 3a. Create application security group (depends on VPC from step 2)
aws ec2 create-security-group \
    --group-name app-security-group \
    --description "Security group for application servers" \
    --vpc-id $VPC_ID

# 3b. Get security group ID
SG_ID=$(aws ec2 describe-security-groups \
    --filters "Name=group-name,Values=app-security-group" \
              "Name=vpc-id,Values=$VPC_ID" \
    --query 'SecurityGroups[0].GroupId' \
    --output text)

# 3c. Add ingress rules
aws ec2 authorize-security-group-ingress \
    --group-id $SG_ID \
    --protocol tcp \
    --port 443 \
    --cidr 0.0.0.0/0

Security Group Dependencies

Step 4: Application Resources (Execute Fourth)

Deploy application infrastructure using networking and security from previous steps.

# 4a. Deploy application stack (depends on VPC and security groups)
aws cloudformation create-stack \
    --stack-name application-resources \
    --template-body file://application-template.yaml \
    --parameters ParameterKey=VpcId,ParameterValue=$VPC_ID \
                 ParameterKey=SecurityGroupId,ParameterValue=$SG_ID \
                 ParameterKey=Environment,ParameterValue=prod \
    --capabilities CAPABILITY_IAM \
    --role-arn arn:aws:iam::123456789012:role/CloudFormationExecutionRole

# 4b. Wait for application deployment
aws cloudformation wait stack-create-complete \
    --stack-name application-resources

Application Dependencies

Step 5: Monitoring and Alerts (Execute Fifth)

Set up monitoring for the deployed application infrastructure.

# 5a. Create SNS topic for alerts
aws sns create-topic --name infrastructure-alerts

# 5b. Get topic ARN
TOPIC_ARN=$(aws sns list-topics \
    --query 'Topics[?contains(TopicArn, `infrastructure-alerts`)].TopicArn' \
    --output text)

# 5c. Create CloudWatch alarms for application
aws cloudwatch put-metric-alarm \
    --alarm-name "ApplicationHealthCheck" \
    --alarm-description "Application health monitoring" \
    --metric-name HealthCheck \
    --namespace AWS/ApplicationELB \
    --statistic Average \
    --period 300 \
    --threshold 1 \
    --comparison-operator LessThanThreshold \
    --evaluation-periods 2 \
    --alarm-actions $TOPIC_ARN

Monitoring Setup

Step 6: Event-Driven Automation (Execute Last)

Set up automation that responds to infrastructure events.

# 6a. Create Lambda function for automation (depends on all previous resources)
aws lambda create-function \
    --function-name InfrastructureAutomation \
    --runtime python3.9 \
    --role arn:aws:iam::123456789012:role/LambdaAutomationRole \
    --handler lambda_function.lambda_handler \
    --zip-file fileb://automation-function.zip \
    --environment Variables='{
        "VPC_ID": "'$VPC_ID'",
        "SECURITY_GROUP_ID": "'$SG_ID'",
        "SNS_TOPIC_ARN": "'$TOPIC_ARN'"
    }'

# 6b. Create EventBridge rule for infrastructure changes
aws events put-rule \
    --name "InfrastructureChangeRule" \
    --event-pattern '{
        "source": ["aws.ec2", "aws.cloudformation"],
        "detail-type": ["AWS API Call via CloudTrail"],
        "detail": {
            "eventSource": ["ec2.amazonaws.com", "cloudformation.amazonaws.com"]
        }
    }'

# 6c. Add Lambda as target for the rule
aws events put-targets \
    --rule "InfrastructureChangeRule" \
    --targets "Id"="1","Arn"="arn:aws:lambda:us-east-1:123456789012:function:InfrastructureAutomation"

Automation Integration

⚠️ Critical Execution Order Notes

11. Summary and Best Practices

🎯 Key Infrastructure Automation Principles

mindmap root((Infrastructure Automation)) IaC CloudFormation CDK Terraform Events EventBridge Lambda CloudTrail Cost Scheduling Auto Scaling Tagging Testing Validation Monitoring Compliance Security IAM Config Guard Duty Operations Blue/Green Canary GitOps

Infrastructure Automation Ecosystem

This mind map shows the interconnected nature of infrastructure automation components. Success requires integration across all these areas rather than focusing on individual tools.

11.1 Production Readiness Checklist

✅ Before Deploying to Production

⚠️ Common Pitfalls to Avoid

This guide provides a comprehensive foundation for AWS Infrastructure Automation. Continue learning by implementing these patterns in your own environment and adapting them to your specific requirements.