plane
nix2container
plane | nix2container | |
---|---|---|
23 | 9 | |
1,583 | 453 | |
2.3% | - | |
9.4 | 6.3 | |
8 days ago | 2 days ago | |
Rust | Go | |
MIT License | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
plane
- Plane: A distributed system for running WebSocket services at scale
-
Pingora: HTTP Server and Proxy Library, in Rust, by Cloudflare, Released
One reason I'm excited about this is that it appears to let you write arbitrary routing logic into a layer 7 proxy. This is something we had to build for https://plane.dev and it would have been nicer to use something like this, but we couldn't find anything like it at the time.
- Plane: A distributed system for running stateful WebSocket services at scale
-
"VMWare rewritten in Rust
Of course, as a user, I would prefer as much as possible to be under MIT or Apache. See for instance https://plane.dev/ for a similar project which is MIT licensed and https://jamsocket.com/ for the commercial version.
- Session back end orchestrator for ambitious browser-based apps
- Plane – open-source Jira alternative
- Plane: A container orchestrator for ambitious browser-based applications
-
The growing pains of database architecture
From an earlier blog post[1], the system for synchronizing file/document state lives independently for their database. As I understand it, their database is for metadata rather than actual document content.
> It’s worth noting that we only use multiplayer for syncing changes to Figma documents. We also sync changes to a lot of other data (comments, users, teams, projects, etc.) but that is stored in Postgres, not our multiplayer system, and is synced with clients using a completely separate system that won’t be discussed in this article. Although these two systems are similar, they have separate implementations because of different tradeoffs around certain properties such as performance, offline availability, and security.
This post[2] goes into more detail on how they spin up backend processes to serve as the source of truth while documents are open (we took heavy inspiration from it when building https://plane.dev, which aims to be an open-source implementation of that architecture.)
[1] https://www.figma.com/blog/how-figmas-multiplayer-technology...
-
Show HN: Accelerated Docker builds on your local machine with Depot (YC W23)
We have been happily using Depot for months now to build https://plane.dev. Prior to finding Depot, we basically gave up on building an M1 image from a GitHub action.
(btw, I always get suspicious when a Show HN post has a lot of praise in the comments, but I swear the Depot folks did not ask me to post anything and I only saw the post because I was checking HN)
-
Launch HN: Depot (YC W23) – Fast Docker Images in the Cloud
Congrats on the launch!
We've been using Depot with Plane (https://plane.dev/). Prior to depot, I had to disable arm64 builds because they slowed the build down so much (30m+) on GitHub's machines. With Depot, we get arm64 and amd64 images in ~2m.
nix2container
-
Nix is a better Docker image builder than Docker's image builder
No discussion about Nix-built containers is complete with nix2container:
https://github.com/nlewo/nix2container
It is truly magical for handling large, multi-layered containers. Instead of building the container archives themselves and storing them in the nix store, it builds a JSON manifest that is consumed by a lightly patched version of skopeo that streams the layers directly to either your local container engine or the registry.
This means you never rebuild or reupload a container layer that is unchanged.
-
Way to get NVM working in CI/CD systems
- Container images are built with https://github.com/nlewo/nix2container instead of Nix's built-in docker tools, because nix2container allows configuration of layers and thus makes it easier to share layers between images and improve layer caching.
As for integrating it into our CI pipeline, this is pretty much CI system agnostic as all you have to do is run `nix build ...` inside the pipeline, generating a `result` directory or file as an artifact that, in the case of nix2container, can be pushed to your container image registry.
-
How do I create a docker image for postgres with nix?
dockerTools is the canonical way, but https://github.com/nlewo/nix2container seems to be gaining support as it's faster and doesn't use so much disk space while building.
-
We deploy 5X faster with warm Docker containers
> The key factor behind our decision was the realization that while Docker images are industry standard, moving around 100s of megabytes of images seems unnecessarily heavy-handed when we just need to synchronize a small change.
I think the culprit is more the GitHub Actions cache than Docker since it seems to be hard to get a clean cache management. I'm not sure about caching Docker image layers, but caching the Nix store with GitHub Actions is pretty complicated (not even sure it's possible): this means we have to download all required Nix store paths on each run, but i consider this is because of a GitHub Action cache limitation.
So, did you consider using another CI, which offers better caching mechanisms?
With a CI able to preserve the Nix store (Hydra[1] or Hercules[2] for instance), I think nix2container (author here) could also fit almost all of your requirements ("composability", reproducibility, isolation) and maybe provide better performances because it is able to split your application into several layers [2][3].
Note i'm pretty sure a lot of Docker CI also allows to efficiently build Docker images.
[1] https://hercules-ci.com/
[2] https://grahamc.com/blog/nix-and-layered-docker-images
[3] https://github.com/nlewo/nix2container/blob/85670cab354f7df6...
-
Launch HN: Depot (YC W23) – Fast Docker Images in the Cloud
FYI, the nix2container [1] project (author here) aims to speedup the standard Nix container workflow (dockerTools.buildImage) by basically skipping the tarball step: it directly streams non already pushed layers.
[1] https://github.com/nlewo/nix2container
-
Crafting container images without Dockerfiles
To get Rust incremental builds, did you consider using something such as crane https://github.com/ipetkov/crane ?
And regarding OCI images, i built nix2container (https://github.com/nlewo/nix2container) to speed up image build and push times.
-
Optimising Docker Layers for Better Caching with Nix (2018)
For a more modern iteration of that idea, check out https://github.com/nlewo/nix2container
It has all the benefits of OP but produces the images much faster, and with less on-disk usage.
-
Nixpacks takes a source directory and produces an OCI compliant image
I've been using nix2container[1] for awhile now. It looks like this depends on Docker, have you guys considered removing that dependency? It shouldn't be necessary to create an OCI compliant image with Nix.
[1]: https://github.com/nlewo/nix2container
-
Nixery – Docker images on the fly with Nix
With https://github.com/nlewo/nix2container, I'm trying to make a more standalone tool. Basically, a Go binary takes a reference graph and produces a JSON file describing a container image. This JSON file is then ingested by a Skopeo fork (it adds a new `transport`) to produce images (to file, registries,...).
Currently, it supports the dockerTools layering algorithm and is designed to work with Guix [1] as well;)
[1] https://github.com/nlewo/nix2container/blob/065e5b108650ee4c...
What are some alternatives?
Centrifugo - Scalable real-time messaging server in a language-agnostic way. Self-hosted alternative to Pubnub, Pusher, Ably. Set up once and forever.
naersk - Build Rust projects in Nix - no configuration, no code generation, no IFD, sandbox friendly.
NATS - High-Performance server for NATS.io, the cloud and edge native messaging system.
crate2nix - rebuild only changed crates in CI with crate2nix and nix
pingora - A library for building fast, reliable and evolvable network services.
nixpacks - App source + Nix packages + Docker = Image
aper - A Rust data structure library built on state machines.
Moby - The Moby Project - a collaborative project for the container ecosystem to assemble container-based systems
cli - 🖥️ Depot CLI, build your Docker images in the cloud
cargo2nix - Granular builds of Rust projects for Nix
statebox_riak - Convenience library that makes it easier to use statebox with riak, extracted from best practices in our production code at Mochi Media.
buildah - A tool that facilitates building OCI images.