Going Serverless with Amazon S3 and Lambda

It has been possible to host static web sites on Amazon S3 for quite some time. Combined with the CloudFront CDN, this provides a fast and efficient way to reach global audiences. In addition, using S3 and CloudFront is typically cheaper than running your own web server (with or without CloudFront in front).

The basic problem with this setup is the lack of dynamic content. If you want to integrate typical web features, such as login and saving data, to a website hosted on S3, you still need to have a separate server running your API. And running your own server to host your API includes all the typical problems of running a server. You need make sure that the operating system is patched up, the firewall is secure and so on.

Amazon Lambda allows you to run simple scripts or programs in response to events. These programs should be small, stateless and only serve a singular purpose. Lambda runs these programs in response to events. The events can be triggered by actions like a file being uploaded to S3 or Amazon Kinesis stream. One interesting option is to trigger Lambda functions in response to REST API events. This is discussed in more detail later.

Running code on Lambda is billed by the time and RAM used. Billing is done based on actual resources that have been used and there is no need to pay for reserved or provisioned capacity. This makes it especially cost efficient to run seldom used code in Lambda. On the other hand, Lambda can also be used for high-throughput processing, as AWS automatically provisions capacity for Lambda functions as required. The only hard limitation is that a single function execution may not last longer than 300 seconds. Amazon has pricing examples on their Lambda billing page.

Amazon API Gateway allows you to publish and proxy REST APIs. These APIs can be point to any HTTP endpoint such as servers running on EC2 or public APIs on the internet. API Gateway also allows calling Lambda functions. This allows publishing Lambda functions as a REST API. Amazon API Gateway is priced $3.50 for 1 million requests and $0.09 per gigabyte of data transfer (as of 15Dec15).

So by combining API Gateway and Lambda, we can implement a fully functioning REST API without any servers that we need to manage. In addition, these Lambda functions are fully capable computer programs and may for example persist data on RDS or DynamoDB. Therefore we may combine CloudFront/S3 with API Gateway and Lambda to implement fully serverless web site or application. The basic architecture is illustrated in the following picture.

serverless (1)

 

The benefits of this setup include that there is only need to pay for used resources, not provisioned capacity. The code and content is fully hosted on different services and there is no need to manage individual servers and handle issues like security updates. In addition, API Gateway forces HTTPS connections and CORS headers, so your data should be secure in transit.

Drawbacks include lack of access to the servers running the code and, in the case of custom domains, the need to obtain SSL certificates. In addition, there is no way to control how Lambda provisions your code and you just need to trust that there are enough resources available for running. It should be noted that Lambda only allows you to adjust the amount of allocated RAM but this will also affect your CPU allocation. The more RAM you have, the more CPU you are given.

Managing Lambda applications and API Gateway routes is currently challenging. Pretty much everything needs to be explicitly mapped and allocated either manually or through some kind of automation. There are some tools such as Serverless to help setting this up but the tooling can still be called rudimentary.

I wrote a simple Lambda function that implements a counter using DynamoDB for storage. This is mapped using API Gateway at https://urqaoa4bd0.execute-api.eu-west-1.amazonaws.com/prod. This is a simple node.js script that makes an API call to DynamoDB and returns the answer to caller. It should be noted that no username or password is needed for DynamoDB as the rights are given through IAM. The source code is included below for reference.

A small web site and JavaScript snippets hosted on S3 implement the client-side functionality. This site can be found at http://serverless.goforecrew.com/.

To sum up, I think that hosting a simple website on S3 and Lambda is fully possible. This is especially viable for small websites with limited interactive functionality which can be implemented with a small amount of Lambda functions. However implementing larger applications on top of API Gateway and Lambda might be challenging as all routing needs to be handled on API Gateway and all functions need to be managed separately on Lambda.