llm.f90 VS JPEGDEC

I'd like to think he took the name from my llm.f90 project https://github.com/rbitr/llm.f90

It was originally based off of Karpathy's llama2.c but I renamed it when I added support for other architectures.

Probable a coincidence :)

I'm a Fortran hobbyist. I'm working (unfortunately less frequently now) on a LLM framework in Fortan: https://github.com/rbitr/llm.f90

Also, I should promote the code I wrote for running this. It runs models in ggml format, the one I made available is an older checkpoint though. It's easy to convert the newer one. And it's in Fortran but it should be easy to get gfortran if you don't have it installed.

https://github.com/rbitr/llm.f90/tree/optimize16/purefortran

The original mamba code has a lot of speed optimizations and other stuff that make it difficult to immediately get so this will help with learning.

I can't help but also plug my own Mamba inference implementation. https://github.com/rbitr/llm.f90/tree/master/ssm

You may also be interested in https://github.com/rbitr/llm.f90/tree/master/ssm it's my inference only implementation of mamba which ends up being much simpler than the training code in the original repo

It takes some time to decode, but I can grab- for example- this 259k JPEG: http://placekitten.com/3840/2160

And display it from the microSD card with this code: https://gist.github.com/Gadgetoid/0b8e352e377135d743338c9483...

As more or less demonstrated by this example: https://github.com/pimoroni/pimoroni-pico/blob/main/micropyt...

Decoding is done via Larry Bank's JPEGDEC: https://github.com/bitbank2/JPEGDEC

It uses roughly 20k RAM to provide the necessary buffers both for decoding JPEGs into blocks which are passed to a drawing routine that handles copying the data into the display RAM buffer (on our larger Inky display this is backed by PSRAM). I don't believe there's a hard limit to the size of the JPEGs you can display. The main bottlenecks are decoding time and the relatively limited scaling options: FULL, HALF, QUARTER, EIGHTH.

This aside, the authors solution is actually quite elegant. As long as you have control over the image pipeline there's no real reason to encumber the device with handling large (both in bytes and pixel dimensions) files. You'll also get much better dithering results writing your own routine to convert files from JPG to the raw 4-bits per pixel format for the display. Our built-in dithering is just a plain ordered dither matrix and, while quaint and retro, it leaves much to be desired visually.

If you're trying to use a public API you can also make GitHub actions automate the whole image conversion process and publish the results to GitHub pages. This works great for, for example, the daily XKCD, serving both to reformat the strip for the display, credit the author, extract the "alt" text and avoid excess requests to the origin. Eg: https://pimoroni.github.io/feed2image/xkcd-800x480-daily.jpg. Though the astute will notice I still opted for jpeg in this case.

I'm doing the same thing with MJPEG and the JPEGDEC library https://github.com/bitbank2/JPEGDEC. I preprocess the video using ffmpeg to scale the video down to 240x240 and remove the compression.