Lesson 25-Serverless Advanced Architecture

Serverless and Microservices Architecture

Microservices Decomposition and Serverless

Microservices Decomposition Principles:

  • Single Responsibility: Each service handles one business capability
  • Independent Deployment: Services can be released and scaled independently
  • Clear Boundaries: Service contracts are defined through APIs
  • Replaceability: Individual services can be replaced without affecting the system

Serverless-Friendly Decomposition Patterns:

  1. Domain-Driven Design (DDD):
    • Divide services by bounded contexts
    • Example: An e-commerce system can be split into order, payment, and inventory services
  2. Functional Layer Decomposition:
graph TD
A[API Gateway] --> B[User Service]
A --> C[Product Service]
A --> D[Order Service]
B --> E[User Database]
C --> F[Product Database]
D --> G[Order Database]

Serverless Decomposition Practice:

# serverless.yml example - Order Service
service: order-service

provider:
  name: aws
  runtime: nodejs14.x

functions:
  createOrder:
    handler: handler.createOrder
    events:
      - http:
          path: orders
          method: post
  
  getOrder:
    handler: handler.getOrder
    events:
      - http:
          path: orders/{id}
          method: get

resources:
  Resources:
    OrdersTable:
      Type: AWS::DynamoDB::Table
      Properties:
        TableName: Orders
        AttributeDefinitions:
          - AttributeName: orderId
            AttributeType: S
        KeySchema:
          - AttributeName: orderId
            KeyType: HASH
        BillingMode: PAY_PER_REQUEST

Decomposition Benefits:

  • Independent scaling (automatic scaling on demand)
  • Fault isolation (failure in one service doesn’t affect the system)
  • Technology heterogeneity (services can use different languages/frameworks)

Service Discovery and Communication

Service Discovery Solutions:

  1. API Gateway Integration:
# Use API Gateway as a unified entry point
provider:
  name: aws
  apiGateway:
    restApiId: abc123
    restApiRootResourceId: /myresource
  1. Service Registry:
// Use AWS SSM to store service endpoints
const AWS = require('aws-sdk');
const ssm = new AWS.SSM();

async function getServiceEndpoint(serviceName) {
  const params = {
    Name: `/services/${serviceName}/endpoint`,
    WithDecryption: true
  };
  const result = await ssm.getParameter(params).promise();
  return result.Parameter.Value;
}
  1. Event-Driven Discovery:
# Use EventBridge for inter-service communication
functions:
  orderCreatedHandler:
    handler: handler.orderCreated
    events:
      - eventbridge:
          eventBusName: orders-event-bus
          pattern:
            source:
              - "order.service"
            detail-type:
              - "OrderCreated"

Communication Patterns:

PatternUse CaseExample
Synchronous APIRequires immediate responseReturn order ID after user creation
Asynchronous EventsDecouple servicesTrigger inventory check after order creation
Request/ResponseRPC-style interactionDirect calls between microservices
Publish/SubscribeBroadcast notificationsNotify all parties of order status change

Serverless Micro-Frontend Support

BFF (Backend for Frontend) Pattern:

# Provide different APIs for web and mobile
service: web-bff

functions:
  getUserData:
    handler: handler.getUserData
    events:
      - http:
          path: web/users/{id}
          method: get

service: mobile-bff

functions:
  getUserData:
    handler: handler.getUserData
    events:
      - http:
          path: mobile/users/{id}
          method: get

Micro-Frontend Architecture Integration:

  1. Independent Deployment:
# Build and deploy Web BFF
cd web-bff && serverless deploy --stage prod

# Build and deploy Mobile BFF
cd mobile-bff && serverless deploy --stage prod
  1. Frontend Integration:
// Web API calls
const webApi = axios.create({
  baseURL: 'https://web-bff-prod.execute-api.us-east-1.amazonaws.com'
});

// Mobile API calls
const mobileApi = axios.create({
  baseURL: 'https://mobile-bff-prod.execute-api.us-east-1.amazonaws.com'
});

State Synchronization Solution:

// Use EventBridge to sync state changes
function syncStateChange(event) {
  return eventbridge.putEvents({
    Entries: [{
      Source: 'microfrontend.state.change',
      DetailType: 'UserProfileUpdated',
      Detail: JSON.stringify(event),
      EventBusName: 'state-sync-bus'
    }]
  }).promise();
}

Serverless and Event Sourcing

Event Sourcing Design

Core Concepts:

  • Event: Records all state changes
  • Event Stream: A sequence of events ordered by time
  • Aggregate Root: A logical unit that reconstructs current state from events

Design Patterns:

  1. Event Storage Structure:
{
  "eventId": "evt_12345",
  "eventType": "OrderCreated",
  "aggregateId": "order_67890",
  "timestamp": "2023-01-01T12:00:00Z",
  "data": {
    "orderId": "67890",
    "customerId": "cust_123",
    "items": [...]
  },
  "metadata": {
    "userId": "user_456",
    "correlationId": "corr_789"
  }
}
  1. Event Versioning:
{
  "eventId": "evt_12345",
  "eventType": "OrderUpdated:v2",
  "aggregateId": "order_67890",
  "timestamp": "2023-01-01T12:05:00Z",
  "data": {
    "version": 2,
    "status": "shipped"
  }
}

Serverless Event Storage

Storage Solutions:

  1. Amazon DynamoDB:
resources:
  Resources:
    EventsTable:
      Type: AWS::DynamoDB::Table
      Properties:
        TableName: Events
        AttributeDefinitions:
          - AttributeName: eventId
            AttributeType: S
          - AttributeName: aggregateId
            AttributeType: S
        KeySchema:
          - AttributeName: eventId
            KeyType: HASH
          - AttributeName: aggregateId
            KeyType: RANGE
        BillingMode: PAY_PER_REQUEST
        StreamSpecification:
          StreamViewType: NEW_IMAGE
  1. Amazon S3 Event Logs:
resources:
  Resources:
    EventLogBucket:
      Type: AWS::S3::Bucket
      Properties:
        BucketName: event-log-bucket
        VersioningConfiguration:
          Status: Enabled

Event Publishing Example:

const AWS = require('aws-sdk');
const dynamodb = new AWS.DynamoDB.DocumentClient();

async function publishEvent(event) {
  const params = {
    TableName: process.env.EVENTS_TABLE,
    Item: {
      eventId: generateUUID(),
      eventType: event.type,
      aggregateId: event.aggregateId,
      timestamp: new Date().toISOString(),
      data: event.data,
      metadata: event.metadata
    }
  };
  
  await dynamodb.put(params).promise();
  
  // Trigger subsequent processing
  await triggerEventProcessors(event);
}

Event Replay and State Reconstruction

Replay Mechanism:

async function rebuildAggregate(aggregateId) {
  const params = {
    TableName: process.env.EVENTS_TABLE,
    KeyConditionExpression: 'aggregateId = :id',
    ExpressionAttributeValues: {
      ':id': aggregateId
    },
    ScanIndexForward: true // Chronological order
  };
  
  const result = await dynamodb.query(params).promise();
  
  let aggregate = { id: aggregateId, state: {} };
  
  for (const event of result.Items) {
    aggregate = applyEvent(aggregate, event);
  }
  
  return aggregate;
}

function applyEvent(aggregate, event) {
  switch(event.eventType) {
    case 'OrderCreated':
      return { ...aggregate, state: { ...event.data } };
    case 'OrderUpdated':
      return { ...aggregate, state: { ...aggregate.state, ...event.data } };
    // Additional event handling...
    default:
      return aggregate;
  }
}

Replay Optimizations:

  1. Incremental Replay: Replay only events from a specific time range
  2. Snapshot Mechanism: Periodically save aggregate state snapshots
async function rebuildWithSnapshot(aggregateId) {
  const snapshot = await getLatestSnapshot(aggregateId);
  const events = await getEventsSince(snapshot.version, aggregateId);
  
  let aggregate = snapshot.state;
  
  for (const event of events) {
    aggregate = applyEvent(aggregate, event);
  }
  
  return aggregate;
}

Serverless and Distributed Systems

Distributed Transactions and Consistency

Solutions:

  1. Saga Pattern:
// Order creation Saga
async function createOrderSaga(orderData) {
  try {
    const paymentTx = await startPaymentTransaction(orderData.amount);
    const order = await createOrder(orderData, paymentTx.id);
    
    await confirmPayment(paymentTx.id);
    return order;
  } catch (paymentError) {
    await cancelPayment(paymentTx.id);
    await cancelOrder(orderData.id);
    throw paymentError;
  }
}
  1. Eventual Consistency with Events:
functions:
  orderCreatedHandler:
    handler: handler.orderCreated
    events:
      - eventbridge:
          eventBusName: order-events
          pattern:
            source:
              - "order.service"
            detail-type:
              - "OrderCreated"
  
  inventoryUpdateHandler:
    handler: handler.inventoryUpdate
    events:
      - eventbridge:
          eventBusName: order-events
          pattern:
            source:
              - "order.service"
            detail-type:
              - "InventoryReserved"

Serverless Task Coordination

Coordination Patterns:

  1. Step Functions:
# serverless.yml with Step Functions integration
functions:
  orderWorkflow:
    handler: handler.orderWorkflow
    events:
      - http:
          path: orders
          method: post

stepFunctions:
  stateMachines:
    orderProcessing:
      name: OrderProcessingStateMachine
      definition:
        StartAt: ValidateOrder
        States:
          ValidateOrder:
            Type: Task
            Resource: arn:aws:lambda:us-east-1:123456789012:function:validateOrder
            Next: ReserveInventory
          ReserveInventory:
            Type: Task
            Resource: arn:aws:lambda:us-east-1:123456789012:function:reserveInventory
            Next: ProcessPayment
          ProcessPayment:
            Type: Task
            Resource: arn:aws:lambda:us-east-1:123456789012:function:processPayment
            End: true
  1. Custom Coordinator:
// Simple task coordinator
class TaskCoordinator {
  constructor() {
    this.tasks = [];
    this.currentTaskIndex = 0;
  }
  
  addTask(task) {
    this.tasks.push(task);
  }
  
  async execute() {
    while (this.currentTaskIndex < this.tasks.length) {
      try {
        await this.tasks[this.currentTaskIndex]();
        this.currentTaskIndex++;
      } catch (error) {
        // Error handling logic
        break;
      }
    }
  }
}

Distributed Locks and Race Conditions

Lock Implementation Solutions:

  1. DynamoDB Conditional Writes:
async function acquireLock(lockId, ownerId, ttl) {
  const params = {
    TableName: process.env.LOCKS_TABLE,
    Item: {
      lockId,
      ownerId,
      expiresAt: new Date(Date.now() + ttl).toISOString(),
      createdAt: new Date().toISOString()
    },
    ConditionExpression: 'attribute_not_exists(lockId)'
  };
  
  try {
    await dynamodb.put(params).promise();
    return true;
  } catch (error) {
    if (error.code === 'ConditionalCheckFailedException') {
      return false; // Lock already acquired
    }
    throw error;
  }
}
  1. Redis Distributed Locks:
const Redis = require('ioredis');
const redis = new Redis();

async function acquireLock(lockKey, timeout = 10000) {
  const identifier = Math.random().toString(36).substr(2, 10);
  const result = await redis.set(lockKey, identifier, 'PX', timeout, 'NX');
  
  if (result === 'OK') {
    return identifier;
  }
  return null;
}

async function releaseLock(lockKey, identifier) {
  const script = `
    if redis.call("get", KEYS[1]) == ARGV[1] then
      return redis.call("del", KEYS[1])
    else
      return 0
    end
  `;
  
  return await redis.eval(script, 1, lockKey, identifier);
}

Lock Usage Pattern:

async function processCriticalSection(resourceId) {
  const lockId = await acquireLock(`lock:${resourceId}`);
  if (!lockId) {
    throw new Error('Could not acquire lock');
  }
  
  try {
    // Execute critical section code
    await updateResource(resourceId);
  } finally {
    await releaseLock(`lock:${resourceId}`, lockId);
  }
}

Course: Micro Frontends & Serverless: Modular Web Development
Share your love

Trending now

Exploring Web 3D Core Technologies Development
Assembly Language Programming From Low-Level Logic to System Kernel
WebRTC Advanced Guide：Media, Transport & Architecture
WebSocket Deep Guide：Building High-Performance Realtime Systems