Lesson 27-Serverless Deployment and Operations

Serverless Deployment Strategies

Single Function vs. Multi-Function Deployment

Deployment Mode Comparison:

FeatureSingle Function DeploymentMulti-Function Deployment
Architecture ComplexityLowHigh
Deployment GranularityCoarseFine
ScalabilityLowHigh
Fault IsolationPoorGood
Resource UtilizationPotentially WastefulMore Efficient
Use CasesSimple APIs, Small AppsComplex Microservices, Large Systems

Single Function Deployment Example (serverless.yml):

service: single-function-service

provider:
  name: aws
  runtime: nodejs14.x

functions:
  apiHandler:
    handler: handler.apiHandler
    events:
      - http:
          path: /api/{proxy+}
          method: any

Multi-Function Deployment Example (serverless.yml):

service: multi-function-service

provider:
  name: aws
  runtime: nodejs14.x

functions:
  userCreate:
    handler: users/create.handler
    events:
      - http:
          path: users
          method: post
  
  userGet:
    handler: users/get.handler
    events:
      - http:
          path: users/{id}
          method: get
  
  orderCreate:
    handler: orders/create.handler
    events:
      - http:
          path: orders
          method: post

Deployment Strategy Selection Recommendations:

  1. Simple Applications: Use single function deployment to reduce complexity
  2. Microservices Architecture: Use multi-function deployment for service isolation
  3. Progressive Migration: Start with single function, gradually decompose

Deployment Commands:

# Deploy entire service
serverless deploy

# Deploy single function
serverless deploy function --function userCreate

# Test function locally
serverless invoke local --function userCreate --path mock-event.json

CI/CD Pipeline Design

Typical CI/CD Workflow:

graph TD
A[Code Commit] --> B[Code Review]
B --> C{Passed?}
C -->|Yes| D[Build]
C -->|No| A
D --> E[Unit Tests]
E --> F{Passed?}
F -->|Yes| G[Integration Tests]
F -->|No| D
G --> H{Passed?}
H -->|Yes| I[Deploy to Staging]
H -->|No| D
I --> J[Manual Approval]
J --> K[Production Deployment]
K --> L[Monitoring & Alerts]

Serverless CI/CD Configuration Example (.github/workflows/deploy.yml):

name: Serverless Deployment

on:
  push:
    branches:
      - main
  pull_request:
    branches:
      - main

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      
      - name: Install dependencies
        run: npm install
      
      - name: Run tests
        run: npm test
      
      - name: Build package
        run: npm run build

  deploy-staging:
    needs: build
    if: github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      
      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: us-east-1
      
      - name: Deploy to Staging
        run: serverless deploy --stage staging --aws-profile staging-profile

  deploy-prod:
    needs: deploy-staging
    if: github.event_name == 'push' && github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      
      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: us-east-1
      
      - name: Deploy to Production
        run: serverless deploy --stage prod --aws-profile prod-profile

Advanced CI/CD Practices:

  1. Canary Deployment:
# Implement canary release with Serverless plugin
serverless deploy --stage canary --canary 10% --aws-profile prod-profile
  1. Automated Rollback:
# Rollback on deployment failure
- name: Rollback on failure
  if: failure()
  run: serverless rollback --stage prod --aws-profile prod-profile
  1. Security Scanning Integration:
- name: Security scan
  run: snyk test --all-projects

Blue-Green Deployment and Canary Release

Blue-Green Deployment Implementation:

  1. DNS Switching Approach:
# Use Route53 for DNS switching
aws route53 change-resource-record-sets --hosted-zone-id Z123456789 \
  --change-batch '{
    "Changes": [{
      "Action": "UPSERT",
      "ResourceRecordSet": {
        "Name": "api.example.com",
        "Type": "CNAME",
        "TTL": 300,
        "ResourceRecords": [{
          "Value": "blue-api.example.com"
        }]
      }
    }]
  }'
  1. Load Balancer Approach:
# Use ALB for traffic switching
Resources:
  BlueTargetGroup:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      Name: blue-target-group
      Port: 80
      Protocol: HTTP
      VpcId: vpc-12345678
  
  GreenTargetGroup:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      Name: green-target-group
      Port: 80
      Protocol: HTTP
      VpcId: vpc-12345678
  
  ApplicationLoadBalancer:
    Type: AWS::ElasticLoadBalancingV2::LoadBalancer
    Properties:
      Name: app-lb
      Scheme: internet-facing
      SecurityGroups:
        - sg-12345678
      Subnets:
        - subnet-12345678
        - subnet-87654321

Canary Release Implementation:

  1. Percentage-Based Traffic Control:
# Use API Gateway weighted routing
aws apigateway update-stage --rest-api-id abc123 --stage-name prod \
  --patch-operations '[{"op": "replace", "path": "/deploymentId", "value":"new-deployment"}]'
  1. Header-Based Routing:
# Use API Gateway custom domain and header routing
functions:
  canaryHandler:
    handler: handler.canaryHandler
    events:
      - http:
          path: /api/{proxy+}
          method: any
          request:
            parameters:
              headers:
                X-Canary: true

Deployment Strategy Selection Recommendations:

  1. Blue-Green Deployment: Ideal for critical business systems requiring zero downtime
  2. Canary Release: Suitable for fast-iterating systems with controlled risk
  3. Progressive Release: Combines benefits of both, starting with canary and moving to full rollout

Serverless Operations

Monitoring and Alerts (CloudWatch, Sentry)

CloudWatch Monitoring Configuration:

  1. Basic Metrics Monitoring:
# Enable CloudWatch logs in serverless.yml
provider:
  logging:
    http: true
    level: INFO
  1. Custom Metrics:
const AWS = require('aws-sdk');
const cloudwatch = new AWS.CloudWatch();

async function sendMetric(metricName, value) {
  await cloudwatch.putMetricData({
    Namespace: 'MyApp',
    MetricData: [{
      MetricName: metricName,
      Dimensions: [{ Name: 'Function', Value: process.env.AWS_LAMBDA_FUNCTION_NAME }],
      Unit: 'Count',
      Value: value
    }]
  }).promise();
}

Sentry Integration Example:

const Sentry = require('@sentry/node');
Sentry.init({ dsn: process.env.SENTRY_DSN });

// Capture errors
try {
  // Business logic
} catch (err) {
  Sentry.captureException(err);
  throw err;
}

// Capture uncaught exceptions and rejections
process.on('uncaughtException', (err) => {
  Sentry.captureException(err);
  process.exit(1);
});

process.on('unhandledRejection', (reason, promise) => {
  Sentry.captureException(reason);
});

Alert Rule Configuration:

# Create CloudWatch alarm
aws cloudwatch put-metric-alarm \
  --alarm-name "HighErrorRate" \
  --alarm-description "Alarm when error rate exceeds 5%" \
  --metric-name Errors \
  --namespace MyApp \
  --statistic Sum \
  --period 60 \
  --threshold 5 \
  --comparison-operator GreaterThanThreshold \
  --evaluation-periods 1 \
  --alarm-actions "arn:aws:sns:us-east-1:123456789012:ErrorAlerts"

Performance Analysis and Optimization

Performance Analysis Toolchain:

  1. AWS X-Ray:
const AWSXRay = require('aws-xray-sdk');
AWSXRay.captureAWS(require('aws-sdk'));

exports.handler = AWSXRay.captureFunc('handler', async (event) => {
  // Function logic
});
  1. Performance Monitoring Dashboard:
# CloudWatch Dashboard definition
resources:
  Resources:
    PerformanceDashboard:
      Type: AWS::CloudWatch::Dashboard
      Properties:
        DashboardName: "ServerlessPerformance"
        DashboardBody: |
          {
            "widgets": [
              {
                "type": "metric",
                "x": 0,
                "y": 0,
                "width": 12,
                "height": 6,
                "properties": {
                  "metrics": [
                    [ "AWS/Lambda", "Invocations", "FunctionName", "${AWS::StackName}-*" ],
                    [ "...", "Errors" ],
                    [ "...", "Duration" ]
                  ],
                  "period": 300,
                  "stat": "Sum",
                  "region": "${AWS::Region}",
                  "title": "Function Metrics"
                }
              }
            ]
          }

Optimization Techniques:

  1. Cold Start Optimization:
    • Use Provisioned Concurrency
    • Reduce dependency package size
    • Optimize initialization code
  2. Hot Path Optimization:
    • Cache frequently accessed data
    • Batch process requests
    • Use asynchronous non-blocking operations
  3. Resource Tuning:
    • Adjust memory configuration based on monitoring data
    • Optimize timeout settings
    • Set reasonable concurrency limits

Failure Recovery and Incident Handling

Failure Scenarios and Response Strategies:

Failure TypeSymptomsResponse Strategy
Cold Start FailureHigh latency/timeoutIncrease warm-up requests, check initialization code
Dependency UnavailableSpike in error rateImplement retry/fallback logic, set circuit breakers
Resource ExhaustionRequests rejectedScale up, optimize resource usage
Data InconsistencyBusiness logic errorsTrigger compensating transactions, manual intervention
graph TD
A[Detect Failure] --> B{Auto-Recover?}
B -->|Yes| C[Execute Recovery Script]
B -->|No| D[Notify Ops Team]
C --> E{Recovery Successful?}
E -->|Yes| F[Monitor Recovery Status]
E -->|No| D
D --> G[Execute Incident Plan]
G --> H[Restore Service]
H --> I[Post-Incident Analysis]

Incident Handling Process:

Automated Recovery Script Example:

#!/bin/bash

# Check function status
STATUS=$(serverless info --stage prod | grep "Status" | awk '{print $2}')

if [ "$STATUS" != "Active" ]; then
  echo "Function is not active, attempting to redeploy..."
  serverless deploy --stage prod --force
fi

# Check error rate
ERROR_RATE=$(aws cloudwatch get-metric-statistics \
  --namespace AWS/Lambda \
  --metric-name Errors \
  --dimensions Name=FunctionName,Value=my-function \
  --start-time $(date -d '5 minutes ago' +%s) \
  --end-time $(date +%s) \
  --period 60 \
  --statistics Average | jq -r '.Datapoints[0].Average')

if (( $(echo "$ERROR_RATE > 5" | bc -l) )); then
  echo "High error rate detected, triggering rollback..."
  serverless rollback --stage prod
fi

Serverless Automation

Automated Build and Release

CI/CD Automation Workflow:

  1. Code Commit Trigger:
# .github/workflows/deploy.yml
on:
  push:
    branches:
      - main
  pull_request:
    branches:
      - main
  1. Multi-Stage Deployment:
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Install dependencies
        run: npm install
      - name: Run tests
        run: npm test
      - name: Build package
        run: npm run build

  deploy-staging:
    needs: build
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Deploy to Staging
        run: serverless deploy --stage staging --aws-profile staging

  deploy-prod:
    needs: deploy-staging
    if: github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Deploy to Production
        run: serverless deploy --stage prod --aws-profile prod

Automated Release Strategies:

  1. Semantic Versioning:
# Automatically generate version number
NEW_VERSION=$(npm version patch -m "chore: release v%s")
  1. Changelog Auto-Generation:
# Generate CHANGELOG.md using conventional-changelog
npx conventional-changelog -p angular -i CHANGELOG.md -s

Automated Testing and Deployment

Testing Automation Workflow:

  1. Unit Testing:
# Run all unit tests
npm test
  1. Integration Testing:
# Start local service for integration tests
serverless offline start &
sleep 5
npm run integration-tests
kill %1
  1. End-to-End Testing:
# Run E2E tests using Cypress
npm run e2e-tests

Deployment Automation Strategies:

  1. Blue-Green Deployment Automation:
# Implement automated blue-green deployment with Serverless plugin
serverless deploy --stage blue --canary 20%
serverless deploy --stage green --canary 50%
serverless deploy --stage green --canary 100%
  1. Canary Release Automation:
# Automated canary release based on traffic percentage
serverless deploy --stage canary --traffic 10%
# Monitor metrics
# Increase traffic if successful
serverless deploy --stage canary --traffic 50%
# Final full rollout
serverless deploy --stage prod

Cost Monitoring and Optimization

Cost Monitoring Toolchain:

  1. CloudWatch Cost Metrics:
# Create cost monitoring dashboard
resources:
  Resources:
    CostDashboard:
      Type: AWS::CloudWatch::Dashboard
      Properties:
        DashboardName: "ServerlessCosts"
        DashboardBody: |
          {
            "widgets": [
              {
                "type": "metric",
                "x": 0,
                "y": 0,
                "width": 12,
                "height": 6,
                "properties": {
                  "metrics": [
                    [ "AWS/Lambda", "Invocations", "FunctionName", "${AWS::StackName}-*" ],
                    [ "...", "Duration" ],
                    [ "...", "BilledDuration" ]
                  ],
                  "period": 86400,
                  "stat": "Sum",
                  "region": "${AWS::Region}",
                  "title": "Daily Lambda Costs"
                }
              }
            ]
          }
  1. Cost Analysis Script:
const AWS = require('aws-sdk');
const cloudwatch = new AWS.CloudWatch();

async function getLambdaCosts(days = 7) {
  const endTime = new Date();
  const startTime = new Date(endTime.getTime() - days * 86400000);
  
  const params = {
    Namespace: 'AWS/Lambda',
    MetricName: 'BilledDuration',
    StartTime: startTime,
    EndTime: endTime,
    Period: 86400,
    Statistics: ['Sum'],
    Dimensions: [
      {
        Name: 'FunctionName',
        Value: process.env.AWS_LAMBDA_FUNCTION_NAME
      }
    ]
  };
  
  const data = await cloudwatch.getMetricStatistics(params).promise();
  // Cost calculation logic...
}

Cost Optimization Strategies:

  1. Resource Tuning:
# Adjust memory configuration based on monitoring data
serverless deploy --memory-size 512 # Test reducing from 1024 to 512
  1. Cold Start Optimization:
# Enable Provisioned Concurrency
provider:
  provisionedConcurrency: 5
  1. Idle Resource Cleanup:
# Periodically check and remove unused resources
serverless remove --stage dev
  1. Cost Alerts:
# Set CloudWatch alert
aws cloudwatch put-metric-alarm \
  --alarm-name "HighLambdaCost" \
  --alarm-description "Alarm when Lambda costs exceed $100/day" \
  --metric-name BilledDuration \
  --namespace AWS/Lambda \
  --statistic Sum \
  --period 86400 \
  --threshold 8640000 # Approx $100 (assuming $0.00001667/GB-second) \
  --comparison-operator GreaterThanThreshold \
  --evaluation-periods 1 \
  --alarm-actions "arn:aws:sns:us-east-1:123456789012:CostAlerts"

Share your love