cloud-run-faq VS kata-containers

I kind of miss the point of this. So if I'm reading this right, fly.io practically only exposes the Pods API, but Kubernetes is really much more than that. I'm not very familiar with any serious company that directly uses Pods API to launch containers, so if their reimplementation of Pods API is just a shim, and they're not going to be able to implement ever-growing set of features in Kubernetes Pod lifecycle/configuration (starting from /logs, /exec, /proxy...) why even bother branding it Kubernetes? Instead they could do what Google does with Cloud Run (https://cloud.run/) which Fly.io is already doing?

I don't know why would anyone would be like "here's a container execution platform, let me go ahead and use their fake Pods API instead of their official API".

I agree with this sentiment. If your model is available as a file, yes you can use GCS and have your Cloud Function fetch it from its bucket upon start-up, but if performance matters, you should consider bundling your function into a container and running with Cloud Run instead, because you have filesystem access there (no need to make an API call to GCS if you can read it as a file).

I am building a HTTP Service running in Google Cloud Run in Go and wanted an easy way to log stuff to Google Cloud Logging.

Datastore mode had its start in App Engine's early days (launched in 2008), where its Datastore was the original scalable NoSQL database provided for all App Engine apps. In 2013, Datastore was made available all developers outside of App Engine, and "re-launched" as Cloud Datastore. In 2014, Google acquired Firebase for its RTDB (real-time database). Both teams worked together for the next 4 years, and in 2017, the next generation of Cloud Datastore was released, having merged in some of the Firebase RTDB features, and was re-branded as Cloud Firestore (in Datastore mode). This mode targets cloud compute as its users, whether serverless (App Engine, Cloud Functions, Cloud Run) or "serverful" (Compute Engine VMs, GKE/Kubernetes/Knative-compliant systems). If you provide a service via compute, use this mode.

You don't need to learn about containers unless that's something you wish to explicitly use to put together your app in a consistent, reproducible manner. Cloud Run is the service that can host your containerized app. If you are in this camp and have learned Docker, you can use that if you wish, but it's optional. Cloud Run (well, Cloud Build, the tool that builds your container for Cloud Run) can build your app by detecting what's in your app so a Dockerfile isn't needed. So like App Engine, Cloud Run can host your full-on web apps if desired. You don't even need to build the container image yourself. Both App Engine and Cloud Run deploy source code directly from the command-line, and along w/Cloud Functions, your app is generally deployed and available globally in less than 60 seconds.

Next to the docs, this page also is quite useful: https://github.com/ahmetb/cloud-run-faq

Actually meant to link this "unofficial" one.. Hope it helps a bit: https://github.com/ahmetb/cloud-run-faq

https://github.com/ahmetb/cloud-run-faq#can-i-mount-storage-volumes-or-disks-on-cloud-run

Seems like Fly.io Machines are trying reimplement Kata Containers with the Firecracker backend [0].

Kata has a guest image and guest agent to run multiple isolated containers [1].

[0] https://katacontainers.io/

[1] https://github.com/kata-containers/kata-containers/blob/main...

> Last time I looked (a few months ago), the documentation was pretty sparse or outdated.

It still is, though it works somewhat seamlessly when installing with https://github.com/kata-containers/kata-containers/blob/main...

Though only one of the hypervisors works well.

Better to use a secure VM, can even get container-like VMs with kata-containers

As I understand,Kata Containers

Kubernetes using Kata containers as a containerd backend

https://github.com/kata-containers/kata-containers/blob/main...

My home k8s cluster is now "locked down" using micro-vms (kata-containers[0]), pod level firewalling (cilium[1]), permission-limited container users, and mostly immutable environments. Given how quickly I rolled this out; the tools to enhance cluster environment security seem more accessible now than my previous research a few years ago.

I know it's not exactly a production setup, but I really do feel that it's the most secure runtime environment I've ever had accessible at home. Probably more so than my desktops, which you could argue undermines most of my effort, but I like to think I'm pretty careful.

In the beginning I was very skeptical, but being able to just build a docker/OCI image and then manage its relationships with other services with "one pane of glass" that I can commit to git is so much simpler to me than my previous workflows. My previous setup involved messing with a bunch of tools like packer, cloud-init, terraform, ansible, libvirt, whatever firewall frontend was on the OS, and occasionally sshing in for anything not covered.

[0] https://github.com/kata-containers/kata-containers

I'm really impressed by fly.io, and the candidness with which they share some of their really awesome technology. Being container-first is the next step for PaaS IMO and they are ahead of the pack.

I aim to build a platform like theirs someday (probably not any time soon) but I don't think I'd do any of what they're doing -- it feels unnecessary. Bear with me as I recently learned that they use nomad[0] and some of these suggestions are kubernetes projects but I'd love to hear why the following technologies were decided against (if they were):

- kata-containers[1] (it does the whole container -> VM flow for you, automatically, nemu, firecracker) with multiple VMM options[2]

- linuxkit[3] (let's say you didn't go with kata-containers, this is another container->VM path)

- firecracker-containerd[4] (very minimal keep-your-container-but-run-it-as-a-VM)

- kubevirt[5] (if you just want to actually run VMs, regardless of how you built them)

- Ceph[6] for storage -- make LVM pools and just give them to Ceph, you'll get blocks, distributed filesystems (CephFS), and object gateways (S3/Swift) out of it (in the k8s space Rook manages this)

As an aside to all this, there's also LXD, which supports running "system" (user namespace isolated) containers, VMs (somewhat recent[7][8]), live migration via criu[9], management/migration of underlying filesystems, runs on LVM or zfs[10], it's basically all-in-one, but does fall behind in terms of ecosystem since everyone else is aboard the "cloud native"/"works-with-kubernetes" train.

I've basically how I plan to run a service like fly.io if I ever did -- so maybe my secret is out, but I sure would like to know just how much of this fly.io got built on (if any of it), and/or what was turned down.

[0]: https://news.ycombinator.com/item?id=26745514

[1]: https://github.com/kata-containers/kata-containers

[2]: https://github.com/kata-containers/kata-containers/blob/2fc7...

[3]: https://github.com/linuxkit/linuxkit

[4]: https://github.com/firecracker-microvm/firecracker-container...

[5]: https://github.com/kubevirt/kubevirt

[6]: https://docs.ceph.com/

[7]: https://discuss.linuxcontainers.org/t/running-virtual-machin...

[8]: https://github.com/lxc/lxd/issues/6205

[9]: https://criu.org/Main_Page

[10]: https://linuxcontainers.org/lxd/docs/master/storage

Kata Containers https://github.com/kata-containers/kata-containers