data_jd VS tinygrad

I suspect the J package Jd is probably the most non-trivial public codebase. I don’t love the coding style (functions are long and scripted) and it doesn’t make use of newer lambda functions (“direct definitions”) which are easier to read. https://github.com/jsoftware/data_jd

An example J file because this link doesn't say much:

https://github.com/jsoftware/data_jd/blob/master/csv/csv.ijs

Here's one of the more central files that ties into how a Jd database is laid out:

https://github.com/jsoftware/data_jd/blob/master/base/common...

Not that I claim anyone in particular can read it of course. Jd uses a hierarchy of folder, database, table, column that's handled with an object system to share code between them. A folder is just a place to put databases and hardly needs to add anything, while the other levels have a lot of extra functionality. As an inverted database, Jd stores each column in a file, and accesses it using memory mapping.

https://github.com/jsoftware/data_jd/blob/master/base/folder...

https://github.com/jsoftware/data_jd/blob/master/base/table....

(I designed this system when I did some of the early work to turn JDB into Jd as a summer intern)

I found this license for jd itself. It is free only for non-commercial use:

https://github.com/jsoftware/data_jd/blob/master/doc/License...

The link you mentioned only applies to the jsource folder: the jengine code.

A longer piece from GitHub: csvreportsummary=: 3 : 0 t=. <;.2 fread PATHLOGLOGFILE b=. (<,LF)=t b=. b+.(<'!')={.each t b=. b+.(<'src: ')=5{.each t b=. b+.(<'snk: ')=5{.each t b=. b+.(<'elapsed: ')=9{.each t b=. b+.(<'rows: ')=6{.each t b=. b+.(<'error: ')=7{.each t ;b#t )

Also in that realm is tinygrad by geohot, which has an open PR for integrating LLaMA support.

https://github.com/geohot/tinygrad/pull/666/commits

George Hotz already implemented LLaMA 7B and 15B on Twitch yesterday on GPU in Tunygrad llama branch:

https://github.com/geohot/tinygrad/tree/llama

The only problem is that it's swapping on 16GB Macbook, so you need at least 24GB in practice.

While PyTorch is obviously the future in the short term, it will be interesting to see how this space evolves.

Before Tensorflow, people (myself included) were largely coding all of this stuff pretty manually, or with the zoo of incredibly clucky homemade libs.

Tensorflow and PyTorch made the whole situation far more accessible and sane. You can get a basic neural network working in a few lines of code. Magical.

But it's still early days. George Hotz, author of tinygrad[0], a PyTorch "competitor", made a really insightful comment -- we will look back on PyTorch & friends like we look back on FORTRAN and COBOL. Yes, they were far better than assembly. But they are really clunky compared to what we have today.

What will we have in 20 years?

[0] https://github.com/geohot/tinygrad, https://tinygrad.org

Read all the leading papers, many times, to get a deep understanding, the writing quality is usually pretty low, but the information density can be very high, you'll probably miss the important details the first time.

Most medium and low-quality papers are full of errors and noise, but you can still learn from them.

Get your hands dirty with real code.

I would take a look at those:

https://github.com/geohot/tinygrad

George Hotz is the founder of comma.ai, and an open source god: https://github.com/geohot/tinygrad

git clone https://github.com/geohot/tinygrad.git