cog VS nixpacks

View on GitHub

I'm in the same situation. I found this cog project to dockerise ML https://github.com/replicate/cog : you write just one python class and a yaml file, and it takes care of the "CUDA hell" and deps. It even creates a flask app in front of your model.

That helps keep your system clean, but someone with big $s please rewrite pytorch to golang or rust or even nodejs / typescript.

https://github.com/replicate/cog

Our thinking was just that a bunch of folks will want to fine-tune right away, then deploy the fine-tunes, so trying to make that easy... Or even just deploy the models-as-is on their own infra without dealing with CUDA insanity!

I have used this tool before: https://github.com/replicate/cog/tree/main

Then I discovered Cog: Containers for Machine Learning. Looks like a way more flexible solution to plug in the existing infrastructure: you write your custom code and Cog plugs it in a Docker image with FastAPI, no extra ecosystem complexity added.

Highly recommend using cog https://github.com/replicate/cog

Dokku Maintainer here.

Dokku also supports Dockerfiles, Docker Images, Tarballs (similar to heroku slugs), and Cloud Native Buildpacks. I'm also actively working on AWS Lambda support (both for simple usage without much config as well as SAM-based usage) and investigating Replicate's Cog[1] and Railways Nixpacks[2] functionalities for building apps.

There are quite a few options in the OSS space (as well as Commercial offerings from new startups and popular incumbents). It's an interesting space to be in, and its always fun to see how new offerings innovate on existing solutions.

  [1] https://github.com/replicate/cog

Example with Nixpacks:

We have an upcoming nixpacks builder that should work on ARM/ARM64, but there are issues in the nixpacks codebase I'm still trying to sort out before that lands.

I think this is something the writer of the article would be delighted to find: https://github.com/railwayapp/nixpacks

Yes, it builds the Dockerfile if it exists otherwise it uses nixpacks to create an image from source files for any language/framework. This image is then run using docker run .

I think this is true for Nix in the deployment/ops space, where debugging a broken build can be very frustrating. Language improvements are going to be less useful for app developers, the Flake learning curve is not going to get better with a type system.

Perhaps something like heroku buildpaks (https://github.com/railwayapp/nixpacks ?) would help devs get on the Nix train.

Hey! Any plans on integrating nixpacks? I'm one of the core maintainers & would love to help you out. No worries if not.

Uses NixPacks to build the images;