DistorteD VS Video Transcoding

> If you have any resources on this topic off the top of your head I'd appreciate it if you shared them

I wrote a Ruby library that attempts to be good at this https://github.com/okeeblow/DistorteD/tree/NEW%E2%80%85SENSA...

Just a style thing. I find it less visually overwhelming when every statement that contributes a true/false is visually contiguous. You can see it in context here if curious: https://github.com/okeeblow/DistorteD/blob/NEW%E2%80%85SENSA...

I experienced a heisenbug exactly like this in Ruby when trying to `while case Ractor::receive`: https://github.com/okeeblow/DistorteD/blob/dd2a99285072982d3...

Here's one that sounds like exactly the sort of example you had in mind: https://github.com/okeeblow/DistorteD/tree/NEW%E2%80%85SENSA...

Disclaimer: mine :)

This resulted in a huge memory-usage win since I can load data on the fly without blocking other queries. The last pre-Ractorized version of CYO allocated around ~200k objects in 7MiB and retained ~17k in 2MiB of memory. The first Ractorized version allocated ~20k objects in 2MiB and retained ~2.5k objects in ~260KiB of memory. See revision 8c88844b9d256ecc447f6818ab427284b5636cb9 for the initial conversion.

This is the use-case for me. Here's an actual example of a Struct I will probably convert to Data in the file-identification library I've been working on. Right now they just have their `#to_a` overridden to disable some of their annoying automatic Enumerable behavior: https://github.com/okeeblow/DistorteD/blob/dd2a99285072982d3...

In addition to this resource and UK's equivalent (PRONOM/DROID, also mentioned in the linked post), I've found ArchiveTeam's wiki to be very useful for obscure file format details: http://fileformats.archiveteam.org/

The `shared-mime-info` database from freedesktop-dot-org is probably more worthy of contribution than these government-backed databases, at least in terms of number-of-end-users. New type definitions in their database will improve the entire Linux/BSD ecosystem (both desktop and server!) because it's consumed not only by fd.o's own `update-mime-database` utility but by many language-specific type-identification libraries too https://gitlab.freedesktop.org/xdg/shared-mime-info/-/blob/m...

…including (shameless plug) the new Ractor-based Ruby type library I've been working on in the wake of the `mimemagic` drama earlier this year: https://github.com/okeeblow/DistorteD/tree/NEW%E2%80%85SENSA...

> for example it’s asynchronous

Ruby also lets you build programs like this as of 3.0, using Ractor#send / Ractor::receive https://docs.ruby-lang.org/en/master/doc/ractor_md.html#labe...

It’s still pretty new so a lot of the Gem ecosystem hasn’t caught up yet (e.g. C extensions need to be explicitly opted-in as Ractor-safe), but I built a new “MIME::Types” library replacement with it recently and have enjoyed very few teething issues: https://github.com/okeeblow/DistorteD/blob/NEW%E2%80%85SENSA...

I considered switching to Hugo a while back but ended up sticking with Jekyll for the extensibility, like a few others have said. I'm particularly interested in image thumbnailing and format conversion since so many of my posts are image-heavy. I often found that to be my biggest and most frequent barrier to writing since plain text only goes so far on the modern web.

In Hugo, every solution I've seen uses a custom shortcode or custom Markdown image template-rendering hook along with Hugo's build-in image resizing. Many posts even suggest converting image resources to different formats with an external tool. It does seem like the image handling situation in Hugo is improving since it just gained WebP processing support in addition to JPEG: https://gohugo.io/news/0.83.0-relnotes/

Jekyll plugins offer way more power than a shortcode ('include' in Jekyll-speak) can, like how the author of the OP is using Jekyll-Picture-Tag. I've been working on my own similar plugin to handle converting and embedding my site's images, videos, SVGs, PDFs, text files, fonts and weird retro computer formats, etc. For example I can embed an SVG using standard Markdown syntax like `` and get a tag with the SVG plus rasterized JPG+WebP+AVIF+whatever at multiple sizes all totally seamlessly: https://github.com/okeeblow/DistorteD

Very happy the existence of Hugo lit a fire under the Jekyll team to work on speed though :)

> It works at first, but you end up wanting to design your own custom SSG once you run up against something that goes against your mental model of how things should work.

There is a middle ground. I hit this point in Jekyll when I wanted Insanely Great image thumbnailing that no extant Jekyll plugin could provide, ended up writing my own tool to do that, but didn't want to duplicate the rest of Jekyll's functionality too. It's kiiinda hacky and I probably should propose the interface changes upstream if I keep doing this, but a very light monkey-patch lets my tool pretend to be a Jekyll::StaticFile that just happens to write out many separate files: https://github.com/okeeblow/DistorteD/blob/master/DistorteD-...

I use this project by Don Melton to get a Blu-ray video down to an 8 - 10 GB file size: https://github.com/donmelton/video_transcoding

It uses HandBrake, FFmpeg, MKVToolNix, and MP4v2 with some custom tuned settings and has really good results from my experience.

I have primarily used Plex and pretty much everything you said is accurate for Plex as well. Limited transcoding based on the machine it is running on. As disc has become cheaper, I have pretty much stopped doing batch transcodes, which is great for the most part. But there are definitely negatives when you want to watch something offline, or remotely. Biggest pain point is subtitles though. Since they aren't ripped as text and then sent to a client, they have to be burned in to the video itself and transcoded on the fly. Which means losing out on 'forced' ones if it can't transcode fast enough.

Plex has definitely started to try and commercialize itself more and offer other stuff, when all I want is access to my own media. So I may look into Jellyfin more soon.

As for batch transcode jobs, I had a system that I was able to set up as essentially a black box. Drop a rip into a folder and out the other side comes a smaller one at a reasonable quality. With forced subs burned right into the actual video. Mostly based on https://github.com/donmelton/video_transcoding

I use Don Melton’s tools to transcode videos to mp4 files. His tools makes use of Handbrake but he has it tuned to produce very small video files of very high quality. You are unlikely to notice the difference when watching the videos.

Checkout https://github.com/donmelton/video_transcoding. In my experience produces higher quality and smaller files than handbrake alone.

When I was ripping my disc collection, I used Don Melton’s library. Don originally started the Safari and Webkit project at Apple but after he retired, he spent some significant time trying to create an easy way of compressing video but resting quality. Great collection of tools in my opinion that leverage handbrake for encoding. Good luck!

Check out https://github.com/donmelton/video_transcoding

I've been using Don Melton's Video Transcoding tool for my whole library. I upgraded my NAS to a Synology DS1019+ a few months ago, so now I have the space to store the untranscoded MakeMKV files. I've had issues with playback in my system, so I don't mind manually transcoding.

Check out https://github.com/donmelton/video_transcoding – I use it to turn raw Blu-ray tips from 30+ gigs down to 5-7 with no noticeable loss of quality.