serverless-application-model VS aws-lambda-runtime-interface-emulator

The complete solution with SAM is available here.

There are many options to deploy Serverless Applications in AWS and one of them is SAM, the Serverless Application Model. I chose to use it here, because it doesn't add too many layers of abstraction between what's being deployed and the code we write and our infrastructure is quite simple.

The solution uses AWS SAM with the global configuration for Lambda functions and the public API you can use to access Apache Zeppelin. The stack deployment provides the URL as an output value.

When you combine this with the AWS Serverless Application Model you can also very easily include your dependencies. Or use a compiled language like golang for your Lambda functions. You simply run sam build before you run the aws cloudformation package and aws cloudformation deploy commands. SAM will build the binary and update the template to point to the newly built binary. Package will then upload it to S3 and replace the local reference to the S3 location. Deploy can then create or update the stack or you can use the CloudFormation integration in CodePipeline.

I'm a big fan of using an Infrastructure as Code (IaC) approach for any project. My go to tools for this are the Servlerless Application Model (SAM) and it's associated CLI (SAM CLI). For more official use cases and for cross platform apps I typically use Terraform.

AWS SAM

Kicking off the tour and not starting a war, but I'm going to be using the Serverless Application Model.

The diagram here is super simple. I'm going to write something a little later that shows how this code could fit into a bigger workflow, but for now, I'm keeping it basic. And yes, that's the SAM Squirrel in there.

Interestingly, AWS CDK and SAM are both explicitly mentioned. While CDK broadly addresses Infrastructure as Code, SAM is highlighted for its role in developing serverless data pipelines - a hugely underrated concept.

Naturally, there are several options available to declare your cloud resources. The options with the most popularity are the CDK, AWS CloudFormation, SST, Serverless framework, Terraform, and AWS SAM. There are others, but when talking about Infrastructure as Code (IaC), these are the ones you hear about most often.

Enter the aws-lambda-runtime-interface-emulator. This tool will allow us to emulate the AWS Lambda Runtime API locally (I suspect that the AWS SAM uses this tool under the hood as well). The aws-lambda-runtime-interface-emulator is designed to be used with Docker. Still, nothing stops us from containerizing our code for the sake of development and then, whenever we are ready, proceeding with deployment how we wish to. This local workflow is a bit more involved than the previous one, but it might be a valid alternative for those not using AWS SAM or are already using containers to deploy their Lambdas.

There is a special tool to test AWS Lambda images locally. It's called AWS Lambda Runtime Interface Emulator (RIE). You have two options: include RIE in your image or install it locally. We don't need it in the production image, so let's choose the second option. We will download binary locally and mount it to our image if we need to test it.

Suggestions:

1. If you are building APIs and using Lambda functions as targets from an API Gateway API, look into libraries like serverless-wsgi (Python) or wai-handler-hal (Haskell) that translate between API Gateway request/response payloads and some kind of ecosystem-native representation. Then as long as you're writing code where all state gets persisted outside of the request/response cycle, you can develop locally as if you were writing for a more normal deploy environment.

2. Look into the lambda runtime interface emulator ( https://github.com/aws/aws-lambda-runtime-interface-emulator... ). This lets you send invoke requests to a fake listener and locally test the lambda more easily. While the emulator is provided in the AWS container base images, you don't need to run it inside a container if you're deploying with zip files. (AWS-provided container images automatically enable the emulator if not running in a lambda runtime environment, and using docker for port remapping is nice. But not at all required.)

3. Get really good at capturing all requests to external services, and mocking them out for local testing. Whether this is with free monads, effect systems, gateway classes will depend on your language and library choices.