llama2.c VS libsql

FOr inference, less than 1KLOC of pure, dependency-free C is enough (if you include the tokenizer and command line parsing)[1]. This was a non-obvious fact for me, in principle, you could run a modern LLM 20 years ago with just 1000 lines of code, assuming you're fine with things potentially taking days to run of course.

Training wouldn't be that much harder, Micrograd[2] is 200LOC of pure Python, 1000 lines would probably be enough for training an (extremely slow) LLM. By "extremely slow", I mean that a training run that normally takes hours could probably take dozens of years, but the results would, in principle, be the same.

If you were writing in C instead of Python and used something like Llama CPP's optimization tricks, you could probably get somewhat acceptable training performance in 2 or 3 KLOC. You'd still be off by one or two orders of magnitude when compared to a GPU cluster, but a lot better than naive, loopy Python.

[1] https://github.com/karpathy/llama2.c

[2] https://github.com/karpathy/micrograd

A bit off topic but ML-guru Mr Karpathy has implemented a state-of-art Llama2 model in a plain C with no dependencies on 3rd party/freeware libraries. See repo.

Related. I built karpathy’s llama2.c (https://github.com/karpathy/llama2.c) without modifications to WASM and run it in the browser. It was a fun exercise to directly compare native vs. Web perf. Getting 80% of native performance on my M1 Macbook Air and haven’t spent anytime optimizing the WASM side.

Demo: https://diegomarcos.com/llama2.c-web/

Code:

Would be cool for there to be a `llama2.f`, similar to https://github.com/karpathy/llama2.c, to demo it's capabilities

This is a fork of https://github.com/karpathy/llama2.c

karpathy's llama2.c is like llama.cpp but it is written in C and the python training code is available in that same repo. llama2.c's goal is to be a elegant single file C implementation of the inference and an elegant python implementation for training.

His goal is for people to understand how llama 2 and LLM's work, so he keeps it simple and sweet. As the project progresses, so will features and performance improvements added.

Currently it can infer baby (small) Story models trained by Karpathy at a fast pace. It can also infer Meta LLAMA 2 7b models, but at a very slow rate such as 1 token per second.

So currently this can be used for learning or as a tech preview.

Our friendly fork tries to make it portable, performant and more usable (bells and whistles) over time. Since we mirror upstream closely, the inference capabilities of our fork is similar but slightly faster if compiled with acceleration. What we try to do different is that we try to make this bootable (not there yet) and portable. Right now you can get binary portablity - use the same run.com on any x86_64 machine running on any OS, it will work (possible due to cosmopolitan toolchain). The other part that works is unikernels - boot this as unikernel in VM's (possible due unikraft unikernel & toolchain).

See our fork currently as a release early and release often toy tech demo. We plan to build it out into a useful product.

In my latest release of ggml.js, I've added support for Karapathy's llama2.c model.

I really enjoyed Anrej Kaparthy's llama2.c project (https://github.com/karpathy/llama2.c), which runs through creating and running a miniature Llama2 architecture model from scratch.

- https://github.com/karpathy/llama2.c

I think there may be some applications in this limited space that are worth looking into. You won’t replicate GPT-anything but it may be possible to solve some nice problems very much more efficiently that one would expect at first.

It does now: https://github.com/karpathy/llama2.c#metas-llama-2-models

> PS: I've got nothing against Turso, or libSQL. In fact I spent the last year perusing their virtual WAL API. The problem is that I found no documentation, nor any useful open source implementations of it. If there any I'd be very interested. So, thus far, I also don't have anything that drives towards libSQL.

Hey, this is v and I am an engineer at Turso. We do have some documentation and an example implementation of Virtual WAL

docs: https://github.com/tursodatabase/libsql/blob/ef44612/libsql-...

example: https://github.com/tursodatabase/libsql/blob/ef44612/libsql-...

for an open source implementation, you may check how Bottomless works. Bottomless is another project which does back up like litestream and it internally implements a Virtual WAL.

Bottomless - https://github.com/tursodatabase/libsql/tree/main/bottomless

I am sure we can improve our docs, make it more discover-able and easy to find. I am open to feedback and suggestions!

Astro DB is powered by LibSQL, an open source fork of SQLite that was created by Turso. You can use Astro DB's drop-in database to build features like blogs, comment functionality, forums, feedback systems, and user authentication.

sqlite already had an active community fork started by Turso called libsql. They are fixing longstanding API gaps the upstream team isn’t interested in supporting. For example, they added a native write-ahead log API, so you can plug directly into the WAL for streaming replication. This is possible-ish with upstream sqlite + LiteFs but litefs has to implement a whole FUSE file system and can’t run on Mac for that reason.

It’s more risky to run libsql because new features mean new bugs, but it seems worth it to me.

Libsql: https://github.com/tursodatabase/libsql

A bit of a tangent but for those who’d like to use SQLite for a backend, running it as a separate daemon could be an interesting choice, which would also remove there need of Cgo for the build and maybe make things like separate background job processes easier to accomplish. See [1], [2].

—-

1: https://github.com/tursodatabase/libsql/tree/main/libsql-ser...

2: https://news.ycombinator.com/item?id=38602175

https://www.sqlite.org/copyright.html

To summarize, instead of using one of the OSS licenses, the copyright holders simply declare the source to be in the public domain. In order to preserve that status they don't accept patches unless you submit some signed document that you agree with that.

To make things more complicated, they also use their a relatively niche version management system instead of git. Which would complicate making contributions (if they accepted them).

There's a popular fork that fixes all of these issues: https://github.com/libsql/libsql It is MIT licensed, on Github, and open for contributions.

Kind of a weird legal situation for a popular project like this that so many people depend on to have. Not judging; but it is odd. Seems like a lot of wasted efforts between users, would be contributors, and the people that forked this thing to address all that.

You could try making feature requests for https://github.com/libsql/libsql , which is a community fork of SQLite that aims to speed-up the development of long-wanted features.

For a comprehensive view, check out the issues list for libSQL core and sqld. But mostly, I want libSQL to be a home for all builders who believe there is room to take a lean, mean, and SQLite-compatible embedded database to new heights. I’d love to see your contribution!