distributed-transcode VS rslurp

Back when I still cared about saving disk space, I made a cluster of NVidia Jetson Nanos running in a docker swarm configuration [1], but honestly even when you have six computers working at once, H264 on a single computer is still often faster.

On the Jetson Nanos I was lucky to get maybe 1fps in ffmpeg using VP9. Multiply that by six boards and that's about 6fps in total; ffmpeg running x264 in software mode was getting around 11fps on a single board, not even counting using the onboard encoder chip, meaning that I was getting better performance from one board using x264 than all six using VP9.

Now obviously this is a single anecdote on specific hardware, so I'm not saying that this applies to every single case, but it's a big reason why I personally have not used VP9 for anything substantial yet.

[1] https://gitlab.com/tombert/distributed-transcode

I have a fairly large blu-ray collection (~300 movies, ~15 complete TV series). I rip them and serve them with Jellyfin, which works, but due to codec annoyances, I need to transcode them to run on web browsers, and the SBC I'm running Jellyfin + ZFS on is not really fast enough to transcode in real time.

Since I have a ton of little SBCs sitting around my house, I decided to write a clojure app the queues up and transcodes my movies to H264. It uses Docker Swarm to handle distribution of nodes, RabbitMQ to queue up the movies, and core.async to handle local queuing within the application, and uses the Java NIO filesystem stuff to handle any kind of atomicity.

It's hardly the "first" or the "best" at what it does, but the advantage of writing your own is of course that you can tailor it exactly to your setup, and of course it was fun to write.

https://gitlab.com/tombert/distributed-transcode

Plenty.

Of the ones I use every day:

* Parallel downloader (https://github.com/ThomasHabets/rslurp)

Also became a fun learning experience about terminals.

https://github.com/ThomasHabets/cmdg

I wanted to use GMail from a fast cli that used the native gmail API.

https://github.com/ThomasHabets/rslurp

I wanted to download concurrently and according to patterns. Ok, so honestly this one probably exists somewhere in a form that I would like, but I couldn't find it.

https://github.com/ThomasHabets/sim

I wanted multi-party authorization for sudo, and couldn't find one.

https://github.com/ThomasHabets/monotonic_clock

People kept using gettimeofday, so this is part of my compaign against it. (see https://blog.habets.se/2010/09/gettimeofday-should-never-be-...)

https://github.com/ThomasHabets/gtping

I worked in mobile core networks, and wanted a "ping" that used the GTP protocol since that won't be firewalled.

https://github.com/ThomasHabets/ind

I wanted my bash scripts to have automatic indentation, while not sacrificing buffering latency and such.

https://github.com/ThomasHabets/tlscheck

I wanted a simple tool to audit my TLS certificates for expiry.

https://github.com/google/huproxy

I was travelling to China on vacation and wanted a VPN out that would be unlikely to be blocked by the great firewall. Ok, so there are many VPN-like tools for getting through the GFW. Maybe it was just an excuse for me to write it. Honestly ssh -D would have likely worked just fine. It's being used by the keymaster project now though, so maybe it did something right: https://github.com/Cloud-Foundations/keymaster/blob/master/d...

https://github.com/google/tcpauth

I wanted to lock down SSH to anyone who doesn't have a secret key (and portknocking is usually ridiculous). Why not use TCP MD5 for it? https://github.com/google/tcpauth