firecracker-container VS aws-codebuild-docker-images

There is this project, which I have never used, but seems promising. https://github.com/firecracker-microvm/firecracker-container...

How does that compare to firecracker-containerd?

https://github.com/firecracker-microvm/firecracker-container...

This repository enables the use of a container runtime, containerd, to manage Firecracker microVMs. Like traditional containers, Firecracker microVMs offer fast start-up and shut-down and minimal overhead. Unlike traditional containers, however, they can provide an additional layer of isolation via the KVM hypervisor.

I'm really impressed by fly.io, and the candidness with which they share some of their really awesome technology. Being container-first is the next step for PaaS IMO and they are ahead of the pack.

I aim to build a platform like theirs someday (probably not any time soon) but I don't think I'd do any of what they're doing -- it feels unnecessary. Bear with me as I recently learned that they use nomad[0] and some of these suggestions are kubernetes projects but I'd love to hear why the following technologies were decided against (if they were):

- kata-containers[1] (it does the whole container -> VM flow for you, automatically, nemu, firecracker) with multiple VMM options[2]

- linuxkit[3] (let's say you didn't go with kata-containers, this is another container->VM path)

- firecracker-containerd[4] (very minimal keep-your-container-but-run-it-as-a-VM)

- kubevirt[5] (if you just want to actually run VMs, regardless of how you built them)

- Ceph[6] for storage -- make LVM pools and just give them to Ceph, you'll get blocks, distributed filesystems (CephFS), and object gateways (S3/Swift) out of it (in the k8s space Rook manages this)

As an aside to all this, there's also LXD, which supports running "system" (user namespace isolated) containers, VMs (somewhat recent[7][8]), live migration via criu[9], management/migration of underlying filesystems, runs on LVM or zfs[10], it's basically all-in-one, but does fall behind in terms of ecosystem since everyone else is aboard the "cloud native"/"works-with-kubernetes" train.

I've basically how I plan to run a service like fly.io if I ever did -- so maybe my secret is out, but I sure would like to know just how much of this fly.io got built on (if any of it), and/or what was turned down.

[0]: https://news.ycombinator.com/item?id=26745514

[1]: https://github.com/kata-containers/kata-containers

[2]: https://github.com/kata-containers/kata-containers/blob/2fc7...

[3]: https://github.com/linuxkit/linuxkit

[4]: https://github.com/firecracker-microvm/firecracker-container...

[5]: https://github.com/kubevirt/kubevirt

[6]: https://docs.ceph.com/

[7]: https://discuss.linuxcontainers.org/t/running-virtual-machin...

[8]: https://github.com/lxc/lxd/issues/6205

[9]: https://criu.org/Main_Page

[10]: https://linuxcontainers.org/lxd/docs/master/storage

Based on public repository for Codebuild Image, the image base will be the Ubuntu standard 7.0.

This is basically what CodeBuild does.

The default Docker containers that CodeBuild uses (you can create your own) and the shell script it uses to parse the yaml configuration file (mostly a list of shell scripts) are all open source and the entire process can be run locally.

https://github.com/aws/aws-codebuild-docker-images

https://docs.aws.amazon.com/codebuild/latest/userguide/use-c...

Disclaimer: I work for AWS. But nowhere near the team that developed Firecracker

Disclaimer: I work for AWS in Professional Services. All opinions are my own.

The beauty about CodeBuild is that there is no “lock-in”. All it is fundamentally is a Linux or Windows Docker container with popular language runtimes and a shell script that processes a yaml file or you can supply your own Docker container.

You just put a bunch of bash commands or PowerShell commands in the yaml file and it runs anything.

The Docker container and the shell scripts are all open source and you can quite easily run them locally.

I could see outside of AWS keeping your Docker containers for your specific build environments in a local repository and doing all of your builds inside them using Jenkins.

https://github.com/aws/aws-codebuild-docker-images

https://docs.aws.amazon.com/codebuild/latest/userguide/use-c...

For a “batteries included” approach though, I really like Azure DevOps Pipelines.

I’ve even done a couple of integrations between Azure DevOps and AWS when we had clients that are Microsoft shops.

https://aws.amazon.com/vsts/

For AWS, if you use CodeCommit (AWS git service), all access is via IAM and granular permissions. If you integrate with Azure DevOps, the AWS credentials do have to be stored in a separate MS hosted credential storage.

CodeBuild also supports at least Github natively.

I’m not shilling for AWS. I have an MS development background (.Net) and only have “DevOps” experience using AWS and Microsoft tooling.

Docker Images provided by the CodeBuild team should be updated regularly and should support all "modern" toolkits. The open source project has some activity, but an issue for supporting newer Android versions is now open for some time...

The runtimes available and exposed by CodePipeline support Android runtime 29 – and the Docker images are provisioned using Java 8. Unfortunately, as of July 2021, the Android gradle tools (used by Flutter) require Java 11. I have created an issue in the corresponding Github (see here) but needed to find a workaround to move on – I think I’ve found one, but I hope that anyone reading this might have a better way or idea?

$ git clone https://github.com/aws/aws-codebuild-docker-images.git $ cd aws-codebuild-docker-images/al2/aarch64/standard/2.0 $ docker build -t codebuild/amazonlinux2-aarch64-standard:2.0 .

This also is the place where we specify this is an AArch64 build. The managed image indicates to use a standard image provided by AWS. The source of the Graviton2 image can be found on GitHub.

🏗 AWS CodeBuild Local Builds - Simulate a CodeBuild environment locally to quickly troubleshoot the commands and settings located in the BuildSpec file.