tl;dr

I have implemented a proof of concept project based on Twilio Functions, Twilio Sync, Twilio Phone Numbers and Twilio Assets. The source code is available in my serverless-twilio-dictionary GitHub repo.

Infrastructure

Given my background as a backend developer and serverless proponent I was pleasantly surprised to learn that Twilio offers serverless hosting options. Twilio Functions is a service similar to AWS Lambda or GCP Cloud Functions, thus capable of handling events so it was an easy pick. Looking further, I decided that my app should have some kind of persistent data storage for retaining data between invocations and Twilio Sync provides just that. To be fair, Sync has more features such as client subscriptions notifications when data is updated, but for me it was more important to get the integration working instead of taking full advantage of all its capabilities. Twilio Phone Numbers was used as frontend which enabled SMS communication between mobile phone and the function. Lastly, I implemented a landing page for my app using Twilio Assets.

Application

The application is pretty simple. I chose to implement a dictionary with a CRUD (create, read, update and delete) based command interface. The interaction with the Twilio Sync service is handled in the assets/dictionary.private.js file, the SMS message handler is located in the functions/sms/dictionary.protected.js and the landing page resources are available in the assets folder.

The commands and their syntax that can be used to communicate with the dictionary are explained in the usage guide in the GitHub readme, in addition to on landing page after the app has been deployed.

Build Automation

One important aspect of software development for me is build and deployment automation. Although “point and click configuration” in a web UI may work well in some cases (e.g. small project like this one), I prefer to automate and version control these aspects like any other source code. It is quicker and less error prone to execute a script than to manually read, interpret and execute steps in a runbook. Therefore, I spent some time to implement a setup script as well as a corresponding teardown script for this project. All infrastructure is provisioned by executing the former and is subsequently decommissioned when executing the latter.

References

• Source code at GitHub
• Twilio Phone Numbers for registering and managing virtual phone numbers
• Twilio Functions for backend application hosting
• Twilio Sync for data persistence
• Twilio Assets for hosting web sites

Introduction

In case you are not familiar with EventBridge, here is a paragraph copied from its product page:

Amazon EventBridge is a serverless event bus that makes it easy to connect applications together using data from your own applications, integrated Software-as-a-Service (SaaS) applications, and AWS services. EventBridge delivers a stream of real-time data from event sources, such as Zendesk, Datadog, or Pagerduty, and routes that data to targets like AWS Lambda. You can set up routing rules to determine where to send your data to build application architectures that react in real time to all of your data sources.

As can be seen, EventBridge is a versatile and powerful tool. In this particular example, we will see how it can be used for the specific purpose of detecting EC2 Spot instance interruptions.

Infrastructure

The two main components of the infrastructure is one EventBridge Rule and a Lambda function that is triggered when an event is matched by the rule. Using the AWS CDK, it is a pretty straight forward configuration:

import * as cdk from '@aws-cdk/core';
import * as lambda from '@aws-cdk/aws-lambda';
import * as logs from "@aws-cdk/aws-logs";
import * as events from '@aws-cdk/aws-events';
import * as eventsTargets from '@aws-cdk/aws-events-targets'


export class SpotInstantceEventDetectorStack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // Lambda function that will triggered
    const spotInstanceEventLambda = new lambda.Function(this, 'SpotInstanceEventLambda', {
      description: 'Handles Spot Instance events',
      runtime: lambda.Runtime.NODEJS_12_X,
      code: new lambda.AssetCode('lambdas'),
      handler: 'spot-instance-event-lambda.handler',
      logRetention: logs.RetentionDays.TWO_WEEKS,
    });

    // Events Rule that will trigger the Lambda when an EC2 interruption event occurs
    new events.Rule(this, 'SpotInstanceEventRule', {
      description: 'Rule for tracking spot instance interruptions',
      eventPattern: {
        source: ['aws.ec2'],
        detailType: ['EC2 Spot Instance Interruption Warning'],
      },
      targets: [new eventsTargets.LambdaFunction(spotInstanceEventLambda)],
    });
  }
}

Apart from the EventBridge Rule and the Lambda function, three(*) more AWS resources are generated, one IAM role that is configured with the AWSLambdaBasicExecutionRole (thus allowing the Lambda to log to CloudWatch), a Lambda Permission that allows EventBridge to invoke the Lambda and lastly a CDKMetadata resource that, as the name implies, contains metadata about the CDK.

(*) Actually, with the logRetention configuration in place there are yet another four resources created. This setting causes the CDK to configure the Lambda CloudWatch log retention policy using a custom resource with another Lambda function, an IAM Role and an IAM Policy.

Events, Event Filters and Event Targets

In the sample above, events are filtered using the eventPattern configuration part of the event rule. All AWS event have the same event structure and here we use the source and detailType as event pattern to filter out the Spot instance interruption events. Please see the EventBridge user guide for a comprehensive list of Event Examples from Supported AWS Services. Going further the targets property have been configured with the Lambda function. It should be noted that there are many more EventBridge Targets to choose from, such as Kinesis, StepFunctions, API Gateway and so on.

Application Code

In its simplest form, the Lambda function logs the important parts of the spot instance interruption events to CloudWatch:

exports.handler = (event) => {
  const { time, region, detail: { 'instance-id': instanceId, 'instance-action': instanceAction } } = event;

  console.info(JSON.stringify({ time, region, instanceId, instanceAction }));
};

Now, how would you like to be notified? What is important for your service? Perhaps logging to CloudWatch like above will suffice? Maybe it makes sense to store the events in DynamoDB? Or perhaps send a Slack notification? Different teams and different projects have different requirements, but this gives you a good starting point for further endeavours.

References

• Example Project at GitHub
• AWS Cloud Development Kit (CDK)
• Spot Instance interruptions
• AWS Events
• Event Patterns
• EventBridge Targets
• EventBridge Event Examples from Supported AWS Services.
• Taking Advantage of Amazon EC2 Spot Instance Interruption Notices

Motivation

Perhaps the most important reason why you should spend time to keep your third party libraries up to date is that every now and then a new security vulnerability is discovered and your customers, your data and your business may be at stake. By dealing with updates regularly the risk of having a daunting cascade of migrations is mitigated and the workload is distributed over time. Furthermore, with tools such as Dependabot that enables both automation of discovering and execution of dependency version updates there is no excuse why this should be overlooked.

Step by Step

Updating third party dependencies typically involves the following steps:

1. Check out the project
2. Find out which dependencies that can be updated. Typically, your project management tools can assist, the Versions Maven Plugin is helpful for Java Maven projects, yarn upgrade and yarn upgrade-interactive for JavaScript Yarn to name a few examples
3. Fix API incompatibility issues that the update incurred
4. Submit a pull request
5. Rebuild and retest the project to verify that there has been no regression
6. Deploy
7. Repeat

Assuming that you already use a build server for continuous integration that takes care of step 5 and possibly also step 6 if you also have continuous deployment in place. With Dependabot configured in the project, steps 1, 2, 4 and 7 are also managed automatically. What about step 3, API incapability issues? It turns out, this is not always a problem. If your dependencies have adopted semantic versioning, then you will have a indicator if the update will succeed or fail without code changes. If the new version is just a minor or patch release it is likely to pass. Even if it is a new major version, your code may still work if you are not using the parts of the API with the breaking changes.

Dependabot Example

This example uses GitHub’s Dependabot for dependency checks for two reasons. Firstly, it is provided out of the box for GitHub projects. Meaning, you don’t have to deploy or manage any infrastructure or build server to make use of it. Secondly, it does not require much configuration to get started:

1. Create a dependabot.yml file in the .github folder in your project root
2. specify version: 2 and an array of updates (one configuration for each package manager that you would like to check)
3. Each update have three required options (there are more, please configuration options for an exhaustive list)

Thus, a dependabot.yml file for a Docker project could look something like:

version: 2
updates:

  # Enable version updates for Docker
  - package-ecosystem: "docker"
    # Look for a `Dockerfile` in the `root` directory
    directory: "/"
    # Check for updates once a week
    schedule:
      interval: "weekly"

There is nothing that prevents Dependabot from checking two (or more) different build systems. For example, this site is built using bundler and GitHub Actions. The corresponding dependabot.yml file is in the GitHub project repo and there is also an example of a pull request generated by dependabot.

Alternatives

Two other tools similar to Dependabot are Snyk and WhiteSource Renovate

References

• Dependabot supported repositories and ecosystems
• Configuration options for dependency updates

Update

The AWS CDK version v1.60.0 was released August 19, 2020 and it partly solves some of problems mentioned in this post. One drawback with this version is that the MFA credentials are not cached, i.e. they need to entered every time a CDK CLI command is invoked. A new feature request has been created #9855 - [cli] Cache mfa credentials to address this issue. Meanwhile, keep on reading if you are interested in a MFA solution for the CDK CLI that provides caching as well.

Background

I recently had the opportunity to try the AWS CDK in a real project. The getting started guide provided a good introduction and soon it was time for the first deployment. The intended deployment account was configured to use MFA for the AWS CLI (see my previous blog AWS CLI MFA) which made deployment more complicated. That said, the paragraph about Specifying Your Credentials and Region using AWS CLI profiles seemed to provide a solution. I thought I could just pass my AWS CLI configured MFA profile and it would “just work” (spoiler: it didn’t):

$ npx cdk deploy --profile my-project-mfa
Need to perform AWS calls for account 123456789012, but no credentials found. Tried: default credentials.

Regrettably, it turns out that the AWS CDK does not currently support MFA natively. There are a couple of open issues in the AWS CDK GitHub repository that addresses this concern (#1248 and #1656). However they are more than a year old, so rather than waiting for them to be resolved, I thought I’d better look for other solutions. Preferably a solution that could leverage my MFA configuration referenced earlier. In the end, I found the cdk-multi-profile-plugin npm package and I am quite happy with the result, especially since I was already using npm as a package manager for my CDK project.

Prerequisites

• Please spend some time to read up on how MFA can be configured for AWS CLI access if you are not familiar with the topic. For example, read my AWS CLI MFA blog or the AWS CLI user guide about Using Multi-Factor Authentication.

• Install the cdk-multi-profile-plugin:

    $ npm install cdk-multi-profile-plugin --save-dev
  

Configuration

There are a few different ways to configure the cdk-multi-profile-plugin depending on how it is used. In my team, every developer is responsible for managing his or her own development environment. As a consequence, the developers also create and manage AWS CLI profiles independently of the other team members. There are a few things required in order for this to work:

cdk.json

Add the plugin to the cdk.json file, i.e. the CDK configuration file in the project root (this file was generated when by the cdk init command, see the Getting Started Guide):

{
  "app": "npx ts-node bin/YOURAPP.ts",
  "plugin": ["cdk-multi-profile-plugin"]
}

Make sure to replace the bin/YOURAPP.ts with the correct value for your CDK project before adding this file to version control.

~/.aws/config

Next, imagine that you have the following AWS CLI profiles in your ~/.aws/config file (see previous blog post for details):

[profile my-project-mfa]
source_profile = my-project
role_arn = arn:aws:iam::123456789012:role/AdminMFARole
mfa_serial = arn:aws:iam::123456789012:mfa/my-user-name

Explanation:

• source_profile name of the profile configured in the ~/.aws/config file, i.e. the profile that has the aws_access_key_id and aws_secret_access_key
• role_arn ARN of role to be assumed when accessing resources in the account (either using the AWS CLI or in this case using the CDK)
• mfa_serial ARN of your configured MFA device

The values are specific both for each developer and for each project, make sure to update them accordingly.

cdkmultiprofileplugin.json

Lastly, we must associate each AWS account in the CDK Environment with a corresponding AWS CLI profile that we can use to access that environment. For this reason, each developer creates his or her own cdkmultiprofileplugin.json configuration file and adds it to the project root, e.g.

{
  "awsProfiles": {
    "123456789012": "my-project-mfa"
  }
}

The number 123456789012 maps to the AWS Account ID used in the CDK Environment configuration. The value my-project-mfa is the name of the AWS CLI profile for the same account as outlined in the ~/.aws/config paragraph. You can add more account / CLI profile pairs if the CDK Environment has multiple deployment accounts.

Do not add this file to version control unless you have agreed to use common CLI profile names. If this is the case, keep in mind that there are other configuration options as well. The plugin README presents all options so that you can choose a suitable solution for your project.

Usage

Now that we have all pieces in place, we can build and deploy the stack!

$ npm run build && npx cdk deploy

🚀  Using profile my-project-mfa for account 123456789012 in mode ForReading
  
? MFA token for arn:aws:iam::123456789012:mfa/my-user-name: [enters MFA token]

This deployment will make potentially sensitive changes according to your current security approval level (--require-approval broadening).
Please confirm you intend to make the following modifications:

[Presents stack changes]

Do you wish to deploy these changes (y/n)?  y

TestStack: deploying...

 🚀  Using profile my-project-mfa for account 123456789012 in mode ForWriting

TestStack: creating CloudFormation changeset...

[CloudFormation stack events]

✅  TestStack

Dependencies

• aws-cdk: 1.28.0
• cdk-multi-profile-plugin: 1.1.2

Elasticsearch IAM Policy Document

Before looking at the client implementation, we need to make sure that it is allowed to access the Elasticsearch domain. As always, this requires that the client is associated with an IAM Policy Document. Adhering to the AWS guideline of principle of least privileges the policy is as strict as possible.

{
    "Version": "2012-10-17",
    "Statement": [{
        "Effect": "Allow",
        "Action": "es:ESHttp*", 
        "Resource": "arn:aws:es:eu-west-1:111122223333:my-domain/*"
    }]
}

The * character at the end of the es:ESHttp* value implies that all HTTP methods are allowed. You may choose to lock down the policy even further. One example is to use "es:ESHttpGet" for just permitting reading data from Elasticsearch. Another example is ["es:ESHttpPost", "es:ESHttpPut"] for clients that only add data to the domain. Finally, the Resource property tells us that the policy statement only affects the Elasticsearch domain with the specified ARN.

Elasticsearch Client

My first naive attempt was to use a HTTP client to make requests to the Elasticsearch HTTP API of my domain. It failed misearably, AWS requires that HTTP requests are signed with Signature Version 4 to be valid. The AWS SDK handles this internally so usually you do not need to bother. Realizing that, I took a closer look at what functionality the ES class in the AWS JavaScript SDK offers. It does indeed provide an Elasticsearch API, but it is all about domain configuration, management and it does not provide any client features. Next, when I studied the AWS Elasticsearch developer guide, I found an JavaScript client snippet. It had some limitations in my opinion (it uses global variables for request configuration and response handling just logs HTTP status code and response body). For this reason, I chose to rewrite it to a more generic elasticsearch-client.js file:

'use strict';
    
const path = require('path');
const AWS = require('aws-sdk');

const { AWS_REGION, ELASTICSEARCH_DOMAIN } = process.env;
const endpoint = new AWS.Endpoint(ELASTICSEARCH_DOMAIN);
const httpClient = new AWS.HttpClient();
const credentials = new AWS.EnvironmentCredentials('AWS');

/**
 * Sends a request to Elasticsearch
 *
 * @param {string} httpMethod - The HTTP method, e.g. 'GET', 'PUT', 'DELETE', etc
 * @param {string} requestPath - The HTTP path (relative to the Elasticsearch domain), e.g. '.kibana'
 * @param {Object} [payload] - An optional JavaScript object that will be serialized to the HTTP request body
 * @returns {Promise} Promise - object with the result of the HTTP response
 */
function sendRequest({ httpMethod, requestPath, payload }) {
    const request = new AWS.HttpRequest(endpoint, AWS_REGION);

    request.method = httpMethod;
    request.path = path.join(request.path, requestPath);
    request.body = JSON.stringify(payload);
    request.headers['Content-Type'] = 'application/json';
    request.headers['Host'] = ELASTICSEARCH_DOMAIN;

    const signer = new AWS.Signers.V4(request, 'es');
    signer.addAuthorization(credentials, new Date());

    return new Promise((resolve, reject) => {
        httpClient.handleRequest(request, null,
            response => {
                const { statusCode, statusMessage, headers } = response;
                let body = '';
                response.on('data', chunk => {
                    body += chunk;
                });
                response.on('end', () => {
                    const data = {
                        statusCode,
                        statusMessage,
                        headers
                    };
                    if (body) {
                        data.body = JSON.parse(body);
                    }
                    resolve(data);
                });
            },
            err => {
                reject(err);
            });
    });
}

module.exports = sendRequest;

Example Usage

The above implementation enables you to implement all methods in the Elasticsearch HTTP API. The only missing part is an environment variable called ELASTICSEARCH_DOMAIN that should have the value of your AWS hosted Elasticsearch domain such as my-domain-qwertyasdf.eu-west-1.es.amazonaws.com. To create a new Elasticsearch index called my-index you execute the function call by providing the required parameters in the corresponding Create Index API:

'use strict';
    
const sendElasticsearchRequest = require('./elasticsearch-client');

const params = {
    httpMethod: 'PUT',
    requestPath: 'my-index',
    payload: {
        // see link above for details
        settings: {
            index: {
                number_of_shards: 3,
                number_of_replicas: 2
            }
        }
    }
};
sendElasticsearchRequest(params)
    .then(response => {
        console.info(response);
    });

And the result may look something like:

{ 
    statusCode: 200,
    statusMessage: 'OK',
    headers: { 
        date: 'Wed, 05 Sep 2018 20:24:24 GMT',
        'content-type': 'application/json; charset=UTF-8',
        'content-length': '67',
        connection: 'keep-alive',
        'access-control-allow-origin': '*' 
    },
    body: { 
        acknowledged: true,
        shards_acknowledged: true,
        index: 'my-index'
    } 
}

Considerations

• The Elasticsearch client above returns a Promise. Timeouts and unknown domain URLs result in Promise.reject() whereas successful HTTP request/response results in Promise.resolve(). The resolved JavaScript object has three or four properties, namely the HTTP statusCode, the HTTP statusMessage, the HTTP headers and body in case there is a HTTP response body. Consequently, the promise will be resolved successfully by any 4XX client error codes (e.g. 404 - Not Found) and 5XX server errors (e.g. 503 Service Unavailable). Feel free to modify the code to reject the promise on HTTP errors if you prefer such behaviour.
• The client uses the AWS.EnvironmentCredentials class for obtaining valid credentials since it is being deployed as part of a Lambda function. This is not the only Node.js runtime environment and for this reason this is not the only credential class in the SDK. Please study the Setting Credentials in Node.js chapter in the AWS JavaScript developer guide for other alternatives.
• A different approach to connect to an AWS Elasticsearch domain is to use the official Elasticsearch JavaScript client. Like my HTTP client attempt, it cannot be used directly since it does not have the AWS Signature Version 4 capability. However, it has a pluggable architecture and there is a community extension called http-aws-es that solves this problem. I have not tried this method, but they are both available as npm dependencies. Please check elasticsearch and http-aws-es for more information.

Background

A while ago, I had the opportunity to explore AWS Glue, a serverless extract, transform and load (ETL) service from AWS. The AWS Glue service offering also includes an optional developer endpoint, a hosted Apache Zeppelin notebook, that facilitates the development and testing of AWS Glue scripts in an interactive manner. Typically, you only pay for the compute resources consumed while running your ETL job. However, in contrast to the rest of AWS Glue family, usage of developer endpoints are charged hourly, regardless if you actively use them or not. You can see this when launching a dev endpoint from the AWS Console:

Billing for a development endpoint is based on the Data Processing Unit (DPU) hours used during the entire time it remains in the READY state. To stop charges, delete the endpoint. To delete, choose the endpoint in the list, and then choose Action, Delete.

Is this something that we should care about? Each dev endpoint gets 5 DPUs by default (with a minimum of 2 DPUs), and they are currently priced at $0.44 per DPU-hour (billed per second, 10 minutes minimum). Put it differently, you pay $2.20 per hour or $52.80 per day for the default configuration (or more than $100 for a weekend if you forget to delete your endpoint before you go home for the weekend which I did). For me, this was expensive enough to motivate an automatic dev endpoint deleter.

Outline

The solution consists of the following components:

• A Lambda function that lists all Glue developer endpoints and subsequently deletes them.
• A CloudWatch Event that triggers the Lambda function at scheduled intervals (typically at the end of each workday).
• A Lambda Permission that allows the CloudWatch Event to invoke the Lambda function.
• An IAM Role that allows the Lambda function to get and delete the Glue developer endpoints.
• A CloudFormation template that comprises all resources. Thus, the stack can be re-used across AWS accounts and AWS regions.

Solution

The entire solution is presented in the CloudFormation template below. By inlining the Lambda source code into the template a single file is enough for both the infrastructure as well as the application logic. I have chosen to declare the cron expression as a parameter. I have scheduled the CloudWatch Events to trigger the Lambda when I leave the office, typically at 5PM. Since the cron expression is given in UTC the actual time will depend on daylight saving. A cron expression of 0 16 * * ? * translates to 5PM CET (Central European Time) in the winter and 6PM CEST (Central European Summer Time) in the summer.

AWSTemplateFormatVersion: 2010-09-09
Description: Stack that deletes all Glue Developer Endpoints in a region

Parameters:

  CronExpression:
    Type: String
    Description: The cron expression for triggering the Glue Dev endpoint deletion (in UTC)
    Default: 0 16 * * ? *
    ConstraintDescription: Must be a valid cron expression


Resources:

  DeleteGlueEndpointsLambda:
    Type: AWS::Lambda::Function
    Properties:
      Description: A Lambda function that gets and deletes the AWS Glue Dev endpoints
      Handler: index.handler
      Code:
        ZipFile: |
          'use strict';

          const AWS = require('aws-sdk');
          const glue = new AWS.Glue({apiVersion: '2017-03-31'});

          function deleteDevEndpoints(endpointNames) {
            console.info('Deleting Glue DevEndpoints:', JSON.stringify(endpointNames));
            const promises = endpointNames
              .map(params => {
                return glue.deleteDevEndpoint(params).promise()
                  .then(data => {
                    console.info('Deleted:', JSON.stringify(params));
                    return data;
                  });
              });
            return Promise.all(promises);
          }

          function extractEndPointNames(data) {
            return data.DevEndpoints
              .map(({ EndpointName }) => ({ EndpointName }));
          }

          exports.handler = (event, context, callback) => {
            console.info('Event:', JSON.stringify(event));
            glue.getDevEndpoints().promise()
              .then(extractEndPointNames)
              .then(deleteDevEndpoints)
              .then(data => {
                callback(null, data);
              })
              .catch(callback);
          };
      Role: !GetAtt LambdaExecutionRole.Arn
      Runtime: nodejs6.10

  TriggerRule:
    Type: AWS::Events::Rule
    Properties:
      Description: Trigger for the DeleteGlueEndpointsLambda
      ScheduleExpression: !Sub cron(${CronExpression})
      Targets:
      - Arn: !GetAtt DeleteGlueEndpointsLambda.Arn
        Id: TriggerId

  InvokeLambdaPermission:
    Type: AWS::Lambda::Permission
    Properties:
      FunctionName: !GetAtt DeleteGlueEndpointsLambda.Arn
      Action: lambda:InvokeFunction
      Principal: events.amazonaws.com
      SourceArn: !GetAtt TriggerRule.Arn

  LambdaExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service: lambda.amazonaws.com
            Action: sts:AssumeRole
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
      Policies:
        - PolicyName: GlueDeleteEndpointPolicy
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                  - glue:GetDevEndpoint
                  - glue:GetDevEndpoints
                  - glue:DeleteDevEndpoint
                Resource:
                  - '*'

Resources

The AWS documentation is comprehensive, yet it can be hard to navigate. Here are some links that you may find useful:

• Schedule Expressions for Rules
• AWS Lambda Permissions Model
• CloudFormation Resource Types Reference
• Lambda Function Handler

Motivation

If you are using the AWS platform from the command line you have configured your terminal for CLI access using an AWS Access Key ID and an AWS Secret Access Key. As a result, those values are saved in the ~/.aws/credentials file, i.e. there is a file in your computer in which the AWS account credentials are stored in plain text. Consequently, if your computer is stolen or hacked, a malicious user can use your credentials and then use the AWS CLI tool to do whatever you are privileged to do in your AWS account. Adding MFA to the CLI access will add a significant hurdle for the intruder since they also need your MFA device in addition to your AWS access keys before he or she can do any harm.

Outline

Currently, there is not easy way of mandating MFA for AWS CLI users. The trick that will be presented in this blog post is to limit the privileges for users that have not authenticated using MFA and then create a separate role with the desired privileges that requires MFA to be assumed. When the AWS CLI tool user switches to the role, the user is prompted for the TOTP (Time-based One-time Password, e.g. a six digit code that the MFA device presents) before the actual role switch occurs. As a result, the user receives temporary security credentials that are valid for 1 hour. Within that time frame multiple CLI commands can be executed without the need of re-authentication. By utilizing two different CLI profiles the role switch occurs if the user just provided the correct profile. To summarize:

• Create an IAM Role with the same privileges as the IAM Group. Make sure that the IAM Role requires MFA.
• Create an IAM Group with admin privileges to which we can add users. Make sure that the IAM Group requires MFA.
• Create a policy that allows users to manage their MFA configuration.
• Create AWS CLI profiles that handle the role switch and MFA authentication for the user.

Before You Start

I advise that you create a temporary backup user with admin privileges (and verify that it works), before you try the proposed solution. Chances are that you accidentally screw up your own user role or privileges and lock yourself out from the account accordingly. If that happens, you can use the root credentials to restore your access, but it may be easier to have a dedicated temporary user that is deleted once you have completed your IAM configuration. The source code has been published to the aws-cli-mfa repository in my GitHub account. There you will find two role / group pairs, one AdminMFARole/AdminMFAGroup with full admin privileges that I discuss in detail below. Similarly, there is one S3MFARole/S3MFaGroup that has full access to S3, but is prohibited from using other AWS services.

Admin Role

First, we need to create an AWS::IAM::Role. The privileges of this role will dictate what the AWS CLI user is allowed to do after the role switch.. In this blog post, the role will have admin privileges, but requires MFA ro be assumed, but you create other roles with more limited privileges according to your needs (the example in the GitHub repository also contains an S3 role). AWS comes with a predefined managed policy arn:aws:iam::aws:policy/AdministratorAccess that we can use, but we need to add an assume role policy document with a Condition that MFA is present using one of the AWS Global and IAM Condition Context Keys:

AdminMFARole:
  Type: AWS::IAM::Role
  Properties:
    ManagedPolicyArns:
      - arn:aws:iam::aws:policy/AdministratorAccess
    RoleName: AdminMFARole
    AssumeRolePolicyDocument:
      Version: 2012-10-17
      Statement:
        - Sid: AllowAssumeRoleIfMFAIsPresent
          # see http://docs.aws.amazon.com/cli/latest/userguide/cli-roles.html#cli-roles-mfa
          Effect: Allow
          Principal:
            AWS: !Ref AWS::AccountId
          Action: sts:AssumeRole
          Condition:
            Bool:
              aws:MultiFactorAuthPresent: true

Admin Group

Next step is to create an AWS::IAM::Group. You can assign which of your IAM users will have admin access by adding them to or removing them from this group. The group is configured with two policies. The first policy allow users in the group to assume the AdminMFARole above. Note that this policy should be without MFA requirement. Why? If the user had already been MFA authenticated, there would be no need for the role switch since its sole purpose is to trigger the MFA authentication. The second policy document has the same privileges as the previously mentioned arn:aws:iam::aws:policy/AdministratorAccess, but it adds MFA requirement. Regrettably, it is not possible to copy an existing policy and just add the MFA condition to it, hence we need to specify a full policy document. This second policy is what gives your AWS Console users administrator privileges if they use MFA as part of the login flow. If you only plan to cater for CLI users, this policy can be omitted.

AdminMFAGroup:
  Type: AWS::IAM::Group
  Properties:
    GroupName: AdminMFAGroup
    Policies:

      - PolicyName: AllowAssumeAdminMFAPolicy
        PolicyDocument:
          Version: 2012-10-17
          Statement:
            Sid: AllowUserToAssumeAdminMFARole
            Effect: Allow
            Action: sts:AssumeRole
            Resource: !GetAtt AdminMFARole.Arn

      - PolicyName: AdminMFAPolicy
        # same privileges as the arn:aws:iam::aws:policy/AdministratorAccess, but with MFA requirement
        PolicyDocument:
          Version: 2012-10-17
          Statement:
            Sid: AllowAdminUserToDoAnythingIfMFAIsPresent
            Effect: Allow
            Action: '*'
            Resource: '*'
            Condition:
              Bool:
                aws:MultiFactorAuthPresent: true

Manage MFA Policy

The third step enables MFA self service for all users in the AdminMFAGroup. Basically, this policy allows the users to create their own MFA configuration without enforcing MFA in the first place, both using the AWS CLI as well as the AWS Console.

ManageMFAPolicy:
  Type: AWS::IAM::ManagedPolicy
  Properties:
    Description: A policy that allows users to manage their personal MFA configuration
    Groups:
      - !Ref AdminMFAGroup
      # add more groups that should have MFA requirement
    ManagedPolicyName: ManageMFAPolicy
    PolicyDocument:
      Version: 2012-10-17
      Statement:

      - Sid: AllowUsersToManageTheirOwnMFADevice
        Effect: Allow
        Action:
        - iam:CreateVirtualMFADevice
        - iam:EnableMFADevice
        - iam:ResyncMFADevice
        Resource:
          # users should only manage their own resources
          - !Join ['', ['arn:aws:iam::', !Ref 'AWS::AccountId', ':mfa/${aws:username}']]
          - !Join ['', ['arn:aws:iam::', !Ref 'AWS::AccountId', ':user/${aws:username}']]

      - Sid: AllowUsersToListMFADevicesAndUsers
        Effect: Allow
        Action:
        - iam:ListMFADevices
        - iam:ListVirtualMFADevices
        - iam:ListUsers
        Resource: "*"

Enforcing MFA

Now the infrastructure is setup, and you can start enforcing MFA for your users.

Warning This is the point where you should create a temporary admin user so that you can make corrections in case you lock yourself out from the account.

• Add all administrators to the AdminMFAGroup.
• Remove the AdministratorAccess policy from all users (if any).
• Remove the AdministratorAccess policy from all groups (if any).

The result depend on whether or not the user already has an active MFA. Non-MFA users are now restricted to just enable MFA, both in the AWS Console and using the AWS CLI. As soon as they have an active MFA device (and have re-login), they will have admin privileges.

AWS CLI Profiles

All users of the account need to configure their AWS CLI environment with two profiles in order to facilitate the automatic role switch from the command line. Start by creating a named profile in your ~/.aws/credentials file in which you provide the AWS Access Key ID and the AWS Secret Access Key:

[my-project]
aws_access_key_id=AKIAI44QH8DHBEXAMPLE
aws_secret_access_key=je7MtGbClwBF/2Zp9Utk/h3yCo8nvbEXAMPLEKEY

The second profile is created in the ~.aws/config file in which you provide a reference to the profile to be use for authentication by using the source_profile, an ARN to the role which should be used for role switching and the ARN to your configured MFA device.

[profile my-project-mfa]
source_profile = my-project
role_arn = arn:aws:iam::123456789012:role/AdminMFARole
mfa_serial = arn:aws:iam::123456789012:mfa/my-user-name

Of course, you need to change the values of the profile names, AWS Access Key ID, AWS Secret Access Key, role ARN, and the MFA serial to whatever values are applicable for you. One way of getting the MFA serial is to go to the IAM Users and check in the Security Credentials. It will be presented there if the user has configured a MFA device.

Usage

Finally, all configuration is complete and you can start using your new, MFA enabled, CLI profile. To use a specific profile, you simply specify the profile name after using the --profile option. Continuing with the example profile names above, you will find that the my-project profile (i.e. the profile without MFA) is no longer allowed to for example list S3 buckets:

$ aws s3 ls --profile my-project
An error occurred (AccessDenied) when calling the ListBuckets operation: Access Denied

Using the my-project-mfa profile on the other hand yields a different behavior:

$ aws s3 ls --profile my-project-mfa
Enter MFA code:
[user enters valid MFA token]
[a list of S3 buckets is presented]

Achievement unlocked, requiring MFA for the AWS CLI! Many times you will execute multiple CLI commands to the same account. You can set the AWS_PROFILE environment variable to avoid to repeatedly key in the same profile over and over again, e.g.

$ export AWS_PROFILE=my-project-mfa

Clean-up

Do not forget to delete your temporary admin user.

Caveat

One caveat to look out for is that you may have noticed that specified the AWS::AccountId as Principal in the AdminMFARole. As a consequence, all IAM users and roles in the acconut can assume this role (given that they satisfy the MFA condition). A much better approach would be to use the AdminMFAGroup as principal, regrettably AWS does not allow you to use IAM Group as a principal. To mitigate this risk, you need to be very careful when you create policies for other groups, roles and users. For example you must make sure to restrict which roles they are allowed to switch to and they should be allowed to manage groups, roles or users. This should not be a surprise to you, if you have worked with AWS before you are probably already familiar with the principle of least privilege.

tl;dr

Claudia makes it easy to develop serverless applications that can be deployed to AWS. Execute the following steps in order to create a RESTful service that uses AWS API Gateway for HTTP request mapping and AWS Lambda for business logic:

1. Install Claudia as a global dependency by executing $ npm install claudia --global.
2. Execute $ npm init in an empty directory to create a new npm project (enter appropriate answers for your project when prompted).
3. Add Claudia API Builder as project dependency by executing $ npm install claudia-api-builder --save.

4. Create a new file called index.js in the project root folder with the following content:

   'use strict';
       
   const ApiBuilder = require('claudia-api-builder');
   const api = new ApiBuilder();
       
   module.exports = api;
       
   api.get('/greeting', (request) => {
       console.log('Event:', JSON.stringify(request));
       const name = request.queryString.name || 'World';
       return {greeting: `Hello, ${name}!`};
   });
   

5. Deploy your application to AWS $ claudia create --region eu-west-1 --api-module index (this step requires that you have an AWS account and that you have configured your access key ID and secret access key).

6. That’s it! That is all that is required to create an API Gateway backed by Lambda. To verify that it is working copy the url from the output of the previous command, append /greeting to it and make a request:

   $ curl https://[api-gateway-id].execute-api.eu-west-1.amazonaws.com/latest/greeting
   {"greeting":"Hello, World!"}
   

7. Deployed and verified, wait for customers to start using your incredible greeting service. :-)

Claudia.js

At its core, Claudia.js is a deployment tool. The following lines have been copied from the project homepage:

Claudia makes it easy to deploy Node.js projects to AWS Lambda and API Gateway. It automates all the error-prone deployment and configuration tasks, and sets everything up the way JavaScript developers expect out of the box.

In addition, there is the Claudia API Builder that takes of the API Gateway generation, request and response mapping, etc:

Claudia API Builder is an extension library for Claudia.js that helps you get started with AWS API Gateway easily, and significantly reduces the learning curve required to launch web APIs in AWS. Instead of having to learn Swagger, manually managing request and response transformation templates, manually linking gateway methods to Lambda functions, you can just write the code for your API handlers, and Claudia API Builder takes care of the rest

Behind the scenes, the AWS JavaScript SDK is used to handle the deployment. By executing just a few commands, Claudia will create an application stack that comprises not only the API Gateway and the Lambda function in addition to the Lambda Proxy Integration which glues the two together. Moreover, the required IAM role and the permission required for the API Gateway to invoke the Lambda are also created as part of the process.

CLI Tool

In order to work with Claudia, the CLI tool need to be installed. It can be installed as a global npm module, e.g. $ npm install claudia --global or you can install it as a test dependency $ npm install claudia --save-dev and then add the local node_modules/.bin directory to your $PATH. Verify by executing claudia --version. For a full list of the available command line args see the Command Args documentation or execute $ claudia --help. As an alternative (or rather as a complement) to the CLI tool, one can also write some deployment scripts by adding a few lines to the script part of the package.json file:

"scripts": {
    "create-app": "claudia create --region eu-west-1 --api-module index",
    "update": "claudia update"
}

The create-app script calls claudia create command (see link for details) and similarly the update script calls the claudia update command. To learn more about npm scripts, please read the blog Running scripts with npm authored by my colleague Anders Janmyr.

Lambda Function

Before looking at the Lambda implementation, we create a new npm project by executing $ npm init and provide some good answers. Add the claudia-api-builder to the project, e.g. $ npm install claudia-api-builder --save. Now, we can copy / paste the following lines to an index.js file in the project root:

'use strict';
    
const ApiBuilder = require('claudia-api-builder');
const api = new ApiBuilder();

module.exports = api;

api.get('/greeting', (request) => {
    console.log('Event:', JSON.stringify(request));
    const name = request.queryString.name || 'World';
    return {greeting: `Hello, ${name}!`};
});

The API builder is used to define a /greeting resource that expects a GET request. If the string contains a query string parameter called name it is used as part of the response, otherwise Hello, World! is returned as the greeting response. The request object contains all information that was passed from the client, e.g. HTTP headers, request body, the Lambda context object, the API Gateway context, etc. The return value can either be a simple JavaScript object as exemplified above, but it can also be a Promise or more complex data structures, dynamic response, custom response headers and so on, see the documentation for details. In contrast to the native CloudFormation API Gateway implementation (and the CloudFormation Swagger configuration for that matter) the Claudia API Builder maps all API Gateway resources automatically to a single Lambda function. If you would like to have a richer REST api, you simply register more http handlers with their paths by calling get, put, post or delete, very similar to how you would when you are developing a Node app that uses the Express web framework. In fact, with very little effort you can port an existing Express app to Claudia by just changing a couple of lines in the source code and importing the aws-serverless-express module, see Running Express Apps in AWS Lambda for details.

Deployment

Now we are ready to deploy. We do so by either calling the claudia create command. As a result, we will see the details of the resource that were created:

$ claudia create --region eu-west-1 --api-module index
[...]
saving configuration
{
    "lambda": {
    "role": "claudiajs-demo-executor",
    "name": "claudiajs-demo",
    "region": "eu-west-1"
    },
    "api": {
    "id": "[api-gateway-id]",
    "module": "index",
    "url": "https://[api-gateway-id].execute-api.eu-west-1.amazonaws.com/latest"
    }
}

where [api-gateway-id] is the id of your API Gateway. Alternatively we can use the npm script mentioned earlier with a similar result:

$ npm run create-app
[...]
saving configuration
[...]

After the deployment, we can verify that the API Gateway and Lambda function has been deployed successfully:

$ curl https://[api-gateway-id].execute-api.eu-west-1.amazonaws.com/latest/greeting
{"greeting":"Hello, World!"}
[...]
$ curl https://[api-gateway-id].execute-api.eu-west-1.amazonaws.com/latest/greeting?name=Superman
{"greeting":"Hello, Superman!"}

Considerations

• There are other tools that can help with API Gateway and Lambda configuration and deployment such as Serverless and Seneca that you may find interesting. Start by reading the comparison guide in the Claudia FAQ for a brief overview. There is also the aws-serverless-express library from AWS Labs.
• A claudia.json file will be created after the initial call to claudia create. This file contains your project specific Claudia configuration and should typically be added to version control so that your team mates and build server work against the same environment.
• If you need to start over (or shut down the application), there is the claudia destroy command.
• As mentioned Claudia is a deployment tool that uses the AWS JavaScript SDK. It does not use CloudFormation and thus you cannot use CloudFormation to track your resources.
• Another consequence of not using CloudFormation is that you need to think twice about how you configure the Lambda functions when calling external services such as DynamoDB. The Claudia documentation provides a checklist for integration tips, including guidelines for setting up IAM privileges and using stage variables for differentiation between environments. You can use a similar approach to detect the environment configuration without API Gateway.
• If you do not fancy the claudia-api-builder, you can still create a proxy Lambda function using Claudia. This can be useful if your clients use the Lambda API of an AWS SDK instead of a REST client when communicating with your backend.

More Resources

More information and links are available at the Claudia.js website, for example:

• Claudia Documentation.
• Hello World Lambda Tutorial.
• Hello World API Gateway Tutorial.
• Claudia Example Projects at GitHub.
• The Claudia Gitter channel.

Goal

The aim of this blog post is to re-implement the very same example as in the previous post, but this time use Swagger configuration instead of CloudFormation resources to configure the REST API. The business logic is an AWS Lambda function called GreetingLambda which has been configured with an appropriate execution role. If the event passed to the Lambda contains a name property, a JSON document with a greeting containing the name is returned. If not, the greeting Hello, World! is returned.

exports.handler = (event, context, callback) => {
  const name = event.name || 'World';
  const response = {greeting: `Hello, ${name}!`};
  callback(null, response);
};

When proxied by an API Gateway, it should be possible to perform a HTTP request with the name as a request parameter.

$ curl https://abc123.execute-api.eu-west-1.amazonaws.com/LATEST/greeting?name=Superman
{"greeting":"Hello, Superman!"}

CloudFormation Resources

Many of the CloudFormation resources will be left untouched from the previous implementation, so I will not describe them again (you will find them in the cloudformation.template). Let us focus on the differences instead.

API Gateway Rest API

The biggest change is to the AWS::ApiGateway::RestApi resource. By configuring the Body property it is possible to inline the entire REST API using Swagger’s JSON format. As a result, the CloudFormation template as a whole will be less verbose since some other CloudFormation resources can be deleted. That said, it is still a mouthful when you include the API Gateway Extensions to Swagger to handle the integration between the API Gateway and other AWS services.

"GreetingApi": {
  "Type": "AWS::ApiGateway::RestApi",
  "Properties": {
    "Name": "Greeting API",
    "Description": "API used for Greeting requests",
    "FailOnWarnings": true,
    "Body": {
      "swagger": "2.0",
      "info": {
        "version": "2016-08-18T18:08:34Z",
        "title": "Greeting API"
      },
      "basePath": "/LATEST",
      "schemes": ["https"],
      "paths": {
        "/greeting": {
          "get": {
            "parameters": [{
              "name": "name",
              "in": "query",
              "required": false,
              "type": "string"
            }],
            "produces": ["application/json"],
            "responses": {
              "200": {
                "description": "200 response"
              }
            },
            "x-amazon-apigateway-integration": {
              "requestTemplates": {
                "application/json": "{\"name\": \"$input.params('name')\"}"
              },
              "uri": {"Fn::Join": ["", [
                "arn:aws:apigateway:",
                {"Ref": "AWS::Region"},
                ":lambda:path/2015-03-31/functions/",
                {"Fn::GetAtt": ["GreetingLambda", "Arn"]},
                "/invocations"]
              ]},
              "responses": {
                "default": {
                  "statusCode": "200"
                }
              },
              "httpMethod": "POST",
              "type": "aws"
            }
          }
        }
      }
    }
  }
}

Configuration comments:

• Body Root node of the Swagger configuration. Basically a JSON document that conforms to the Swagger 2.0 Specification.
• /greeting Defines the greeting endpoint and its behavior, e.g. it accepts HTTP GET requests, it has an optional query parameter called name, it responds with 200 - OK and the response is in JSON format.
• x-amazon-apigateway-integration A big part of the code consists of the configuration that is responsible for mapping and transforming the request and response from the RESTful endpoint to the specified AWS service. It contains several sub-properties, more than the ones listed below.
  • requestTemplates This part of the configuration transforms the incoming request before passing the result downstream då the Lambda function. In this example, the value of the name query parameter is transformed into a JSON object that has the format {"name": "Superman"}.
  • uri The backend endpoint, here exemplified by the GreetingLambda ARN.
  • responses Response status patterns from the backend service. Each key has then a statusCode property, which in turn must have a matching status code in the Swagger operation responses. Put differently, this is the HTTP status code that will be returned to the client. In the sample above, you will see that default is used as status pattern and 200 as status code, meaning that the client will always get a 200 - OK as a result, regardless if the Lambda call was successful or not. This is something you likely want to change in a production project.
  • httpMethod and type defaults to HTTP POST and aws when the API Gateway proxies AWS Lambda.
• In addition to the x-amazon-apigateway-integration object the API Gateway Extensions to Swagger also have support for other properties such as custom authorization configuration.

Ref: AWS::ApiGateway::RestApi

Redundant CloudFormation Resources

By having all the RESTful endpoints declared in Swagger format, one can omit the AWS::ApiGateway::Resource and the AWS::ApiGateway::Method resources.

Test

Similarly as in the previous blog post, the API Gateway gets a root url of the https://[api-id].execute-api.[region].amazonaws.com format. If you create a stack based on the complete cloudformation.template you will see the exact url in the stack output. To verify that the stack works, you have to append the basePath and the /greeting path to it before issuing a HTTP GET request.

$ curl https://abc123.execute-api.eu-west-1.amazonaws.com/LATEST/greeting
{"greeting":"Hello, World!"}

You can also provide a request parameter:

$ curl https://abc123.execute-api.eu-west-1.amazonaws.com/LATEST/greeting?name=Superman
{"greeting":"Hello, Superman!"}

Considerations

• In this example, the Swagger configuration was inlined in the CloudFormation Template. If you prefer to keep things separate one can upload a Swagger file to S3 and point the BodyS3Location property of the AWS::ApiGateway::RestApi to it. One advantage of using this property is that the file can be in either YAML or JSON format. Another benefit of keeping the Swagger configuration separate from the CloudFormation template is that you can pass the very same Swagger file to the client developers and they can use tools such as the Swagger Code Generator to generate client APIs and documentation. A minor disadvantage is of course that you need keep two files in sync when you deploy a new API Gateway Stage.
• Swagger enables “contract first design”, i.e. you can define a RESTful API before any client or server business logic exists. It is an opinionated way of working and I usually prefer to develop the API and app incrementally. The good news is that Swagger can be used by both parties.
• It is easy to prototype a new API Gateway using “point and click” in the AWS Console. When you are finished you can choose to Export an API from API Gateway, including the API Gateway Integration, using either a HTTP Get request or the AWS Console. Once you get hold of the Swagger file, you can paste it into your CloudFormation template or save it in an S3 bucket. Preferably, you take the opportunity to do some pruning in the process, for example you can replace any string that contains an AWS region with {"Ref": "AWS::Region"}, any string that contains a reference to another CloudFormation resource with {"Ref": "<logical resource id>"}, {"Fn::GetAtt": ["<Lambda logical id>", "Arn"]} for retrieving a Lambda ARN, etc.

Goal

By revisiting the first blog post, you can see an example of how AWS Lambda Function Versioning and Aliases can be used to implement continuous delivery on Lambda. In short, two different Lambda alias, STAGE and PROD, were used as two different deployment targets for the Lambda function. Some scripts were developed that first executed unit tests of the Lambda source code, zipped the Lambda source code to a new package that was deployed a new Lambda version and updated the Lambda STAGE alias if the Lambda integration tests passed. In the second blog post, I showed how CloudFormation can be used to configure an API Gateway backed by a Lambda function. This blog post aims to combine the two previous blog posts to create two API Gateway stages, the first named stage, the second named prod, that will be mapped to each Lambda alias respectively. In addition, new integration tests should be developed that verify that the RESTful API works as expected.

CloudFormation Resources

Some changes to the CloudFormation template are required in order to make the continuous deployment work. The final result can be found in the cloudformation.template at GitHub.

Lambda Aliases

As with any other AWS resource, one must add permission before a Lambda function can be called. More to the point, in the Versioning, Aliases, and Resource Policies section of the AWS Lambda Developer Guide, it is stated that one must add a permission based on the Lambda alias ARN in order to invoke the Lambda using an alias name. Any attempt to use a permission based on the unqualified Lambda ARN will result in a permission error. For this reason two AWS::Lambda::Alias resources are created up front that will be used later when creating the Lambda permissions.

"GreetingStageLambdaAlias": {
  "Type" : "AWS::Lambda::Alias",
  "Properties" : {
    "FunctionName" : {"Ref": "GreetingLambda"},
    "FunctionVersion" : "$LATEST",
    "Name" : "STAGE"
  }
},

"GreetingProdLambdaAlias": {
  "Type" : "AWS::Lambda::Alias",
  "Properties" : {
    "FunctionName" : {"Ref": "GreetingLambda"},
    "FunctionVersion" : "$LATEST",
    "Name" : "PROD"
  }
}

Both aliases reference the same Lambda FunctionName, but they have different alias Names. The FunctionVersion has been set to $LATEST version which means that both alias will point to the latest Lambda version after the initial stack deployment. Later, the aliases will be updated by the continuous deployment scripts to point to new versions. Ref: AWS::Lambda::Alias

Lambda Permissions

After the aliases have been created, two AWS::Lambda::Permissions are created to allow the API Gateway to call both GreetingLambda:STAGE and GreetingLambda:PROD alias.

"GreetingLambdaStagePermission": {
  "Type": "AWS::Lambda::Permission",
  "Properties": {
    "Action": "lambda:invokeFunction",
    "FunctionName": {"Ref": "GreetingLambdaStageAlias"},
    "Principal": "apigateway.amazonaws.com",
    "SourceArn": {"Fn::Join": ["",
      ["arn:aws:execute-api:", {"Ref": "AWS::Region"}, ":", {"Ref": "AWS::AccountId"}, ":", {"Ref": "GreetingApi"}, "/*"]
    ]}
  }
},

"GreetingLambdaProdPermission": {
  "Type": "AWS::Lambda::Permission",
  "Properties": {
    "Action": "lambda:invokeFunction",
    "FunctionName": {"Ref": "GreetingLambdaProdAlias"},
    "Principal": "apigateway.amazonaws.com",
    "SourceArn": {"Fn::Join": ["",
      ["arn:aws:execute-api:", {"Ref": "AWS::Region"}, ":", {"Ref": "AWS::AccountId"}, ":", {"Ref": "GreetingApi"}, "/*"]
    ]}
  }
}

Note that we pass references to the two Lambda aliases instead of the Lambda resource as FunctionName values. Ref: AWS::Lambda::Permission

API Gateway Stages

Now we have all required components to create the API Gateway stages. They are very similar to the example developed in the previous blog post, but with one important distinction. By defining a Stage Variables called LambdaAlias we can add a value for mapping the Lambda alias later on. Stage variables act as environment variables in your API Gateway configuration. You can use whatever variable name and value that you like, as long as you use the allowed characters. Please see API Gateway Stage Variables Reference for details and use cases of how stage variables can be used.

"GreetingApiStageStage": {
  "DependsOn" : ["ApiGatewayAccount"],
  "Type": "AWS::ApiGateway::Stage",
  "Properties": {
    "DeploymentId": {"Ref": "ApiDeployment"},
    "MethodSettings": [{
      "DataTraceEnabled": true,
      "HttpMethod": "*",
      "LoggingLevel": "INFO",
      "ResourcePath": "/*"
    }],
    "RestApiId": {"Ref": "GreetingApi"},
    "StageName": "stage",
    "Variables": {
      "LambdaAlias": "STAGE"
    }
  }
},

"GreetingApiProdStage": {
  "DependsOn" : ["ApiGatewayAccount"],
  "Type": "AWS::ApiGateway::Stage",
  "Properties": {
    "DeploymentId": {"Ref": "ApiDeployment"},
    "MethodSettings": [{
      "DataTraceEnabled": true,
      "HttpMethod": "*",
      "LoggingLevel": "INFO",
      "ResourcePath": "/*"
    }],
    "RestApiId": {"Ref": "GreetingApi"},
    "StageName": "prod",
    "Variables": {
      "LambdaAlias": "PROD"
    }
  }
},

API Gateway Method

The last change of the CloudFormation template compared to the previous blog post is in the API Gateway method resource where the HTTP method is mapped to the Lambda function. Previously, the API Gateway Uri was “just” a magic string including the Lambda ARN. With the addition of the ${stageVariables.LambdaAlias} stage variable to provide the Lambda alias the string becomes even more magic. Depending on which API Gateway stage is being called, the stage variable will be evaluated to STAGE and PROD respectively, thus directing the request to the matching Lambda alias. Please see the AWS Integration URIs (Lambda Functions) for more details on how this works.

"GreetingRequest": {
  "DependsOn": "LambdaPermission",
  "Type": "AWS::ApiGateway::Method",
  "Properties": {
    "AuthorizationType": "NONE",
    "HttpMethod": "GET",
    "Integration": {
      "Type": "AWS",
      "IntegrationHttpMethod": "POST",
      "Uri": {"Fn::Join" : ["",
        ["arn:aws:apigateway:",
        {"Ref": "AWS::Region"},
        ":lambda:path/2015-03-31/functions/",
        {"Fn::GetAtt": ["GreetingLambda", "Arn"]},
        ":${stageVariables.LambdaAlias}",
        "/invocations"]
      ]},
      "IntegrationResponses": [{
        "StatusCode": 200
      }],
      "RequestTemplates": {
        "application/json": {"Fn::Join" : ["", [
          "{",
          "  \"name\": \"$input.params('name')\"",
          "}"
        ]]}
      }
    },
    "RequestParameters": {
      "method.request.querystring.name": false
    },
    "ResourceId": {"Ref": "GreetingResource"},
    "RestApiId": {"Ref": "GreetingApi"},
    "MethodResponses": [{
      "StatusCode": 200
    }]
  }
}

Tests

After the CloudFormation template, the application can be tested in three different levels. First of all, there is the unit test at the JavaScript level (executor 1-test.sh). Next level tests the Lambda function using the AWS JavaScript SDK after a new Lambda function has been updated (executor 4-lambda-itest.sh) and acts as a gatekeeper whether or not the Lambda alias should be updated. The last level of integration tests uses a REST client to send requests to the API Gateway and validates the responses (executor 7-api-gateway-itest.sh).

Consideration

The continuous integration flow has only been configured to update the Lambda implementation and not the API Gateway stage. Consequently, if you update your Lambda function in a way that requires changes to the API Gateway integration request or response mapping, the REST client integration tests will not work, despite appropriate updates. The cause of this error is that once the API Gateway stage has been been deployed, it cannot be updated. If you find the need for such update, you must either deploy a new stage, or delete the existing stage, make the necessary changes and re-deploy it.

Update

On the November 18th, 2016 AWS introduced the Serverless Application Model (or SAM for short) that provides an alternative solution to the one described in this blog post. Please read the AWS blog post and study the related project at the AWS Labs GitHub account for more information.