cog VS piku

View on GitHub

I'm in the same situation. I found this cog project to dockerise ML https://github.com/replicate/cog : you write just one python class and a yaml file, and it takes care of the "CUDA hell" and deps. It even creates a flask app in front of your model.

That helps keep your system clean, but someone with big $s please rewrite pytorch to golang or rust or even nodejs / typescript.

https://github.com/replicate/cog

Our thinking was just that a bunch of folks will want to fine-tune right away, then deploy the fine-tunes, so trying to make that easy... Or even just deploy the models-as-is on their own infra without dealing with CUDA insanity!

I have used this tool before: https://github.com/replicate/cog/tree/main

Then I discovered Cog: Containers for Machine Learning. Looks like a way more flexible solution to plug in the existing infrastructure: you write your custom code and Cog plugs it in a Docker image with FastAPI, no extra ecosystem complexity added.

Highly recommend using cog https://github.com/replicate/cog

Dokku Maintainer here.

Dokku also supports Dockerfiles, Docker Images, Tarballs (similar to heroku slugs), and Cloud Native Buildpacks. I'm also actively working on AWS Lambda support (both for simple usage without much config as well as SAM-based usage) and investigating Replicate's Cog[1] and Railways Nixpacks[2] functionalities for building apps.

There are quite a few options in the OSS space (as well as Commercial offerings from new startups and popular incumbents). It's an interesting space to be in, and its always fun to see how new offerings innovate on existing solutions.

  [1] https://github.com/replicate/cog

I should add one to https://piku.github.io (spoiler - this doesn't use Docker at all)

I like it. I wrote Piku (https://github.com/piku/piku) with much the same interest in fixing some of my pains, so I get where you're coming from with this. Will drop it into one of my current projects to build ESP32 binaries :)

I never had any serious issue with SD cards since the Pi 2B (and I've kept Pis running for years).

Anyway, for those wanting to deploy more generic apps, that is why I initially wrote https://github.com/piku/piku - you still have to flash the OS (and rpi-imager does that with sane defaults these days), but once you're done you have Heroku-like deployments for any language runtime you install on the Pi.

I also have https://github.com/rcarmo/ground-init, a cloud-init like shim that simplifies setting up machines (I'm a big fan of cloud-init, but since Raspbian doesn't support it and Ubuntu on ARM requires some fiddling to make it work I decided it wasn't too hard to roll my own).

(I probably should look into glueing that into rpi-imager, but there is are only so many hours in the day...)

I automated that away a long time ago: https://github.com/piku/piku/blob/master/piku.py#L814

Piku https://github.com/piku/piku

So, stupid question: why couldn't I just use the 'baremetal' OS provided by Hetzner, install Postgres, Redis & node, create a separate db for each app, and run each app with https://github.com/piku/piku on a different port? For backups, I'll setup crontab to dump dbs locally and to S3.

piku?