fl-aws VS docker-lambda

Living in my team's bubble i thought everyone runs or tries to run parallel environments: prod, staging, dev, but also an individual (person) or feature env. Why? Because there's no emulator or documentation that will teach you real behavior. Like others have said, AWS seems out of this world. Just like GCP and Azure i might add. Some things you don't expect and they mesmerize you how smart they are. Some you expect and you can't fathom how come you're the "only" one screaming. Random thought: this is how i ended up logging all I bumped into into "Fl-aws" https://github.com/andreineculau/fl-aws

Back to the point: reality is that many build their AWS environment (prod) manually, maybe they duplicate once (dev) also manually, maybe they use some automation for their "code" (lambda) but that's it. This implies it's practically impossible to run end-to-end tests. You can't do that in prod for obvious reasons and you can't do it in dev either - you have many devs queueing, maybe dev is not in sync with prod etc.

My team ran cloudformation end-to-end. We actually orchestrated and wrapped cloudformation (this is yet another topic for not using terraform etc) so that if smth couldn't be done in CFN, it would still be automated and reproducible. Long story short, in 30 minutes (it was this long because we had to wait for cloudfront etc) we had a new environment, ready to play with. A total sandbox. Every dev had their own and it was easy to deploy from a release artifact or a git branch to this environment. Similarly you could create a separate env for more elaborate changes to the architecture. And test in a live environment.

Finally to your question: how do you test end-to-end?

If we talk about lambdas because that's where the business logic lies in a "serverless" architecture, then the answer is by calling the system which will eventually call your lambda/s along the way. If your lambda ia sitting behind AWS gateway, then fire an http request. Is it triggered when objects land on S3? Then push some object to S3. How do you assert? Just the same - http response, S3 changes etc. Not to mention you can also check cloudwatch for specific log entries (though they are not instant).

With this type of a setup, which sounds complex, but it is not since it is 100% reproducible (also from project to project - I had several), adding this proxy-to-my-dev-machine lambda would mean I can make local changes and then fire unit AND end-to-end tests without any changes pushed to AWS, which is the main time/energy consumer imo.

PS: sorry for the wall of text. Like i said i recently realized that the development realities have huge discrepancies, so i tried to summarize my reality :)

https://github.com/dherault/serverless-offline https://github.com/lambci/docker-lambda

Step 2: Push a container from your local machine to ECR. Currently, I use sample python lambda docker from https://github.com/lambci/docker-lambda.git

You can see the following projects for some examples of using this plugin to build a layer. They all leverage Docker and the docker-lambda images to compile for AWS’s Lambda environment on any operating system:

I end up using Lambda a lot so lambci is very useful for local testing: https://github.com/lambci/docker-lambda

There are kind souls in this world who created this: https://github.com/lambci/docker-lambda

If all you need is the ability to run a lambda function's code locally you might interested in docker-lambda[1]. I haven't really used localstack or SAM but a couple of years ago when we needed to run some lambda functions locally for development docker-lambda worked well enough.

[1] https://github.com/lambci/docker-lambda

Docker Lambda — Docker Lambda is a tool which tries to replicate the AWS Lambda environment as closely as possible, but within a Docker container. Docker Lambda has support for most of the AWS Lambda run times.

Want to emulate AWS environments? Try docker-lambda.