data_jd VS BQNoise

I suspect the J package Jd is probably the most non-trivial public codebase. I don’t love the coding style (functions are long and scripted) and it doesn’t make use of newer lambda functions (“direct definitions”) which are easier to read. https://github.com/jsoftware/data_jd

An example J file because this link doesn't say much:

https://github.com/jsoftware/data_jd/blob/master/csv/csv.ijs

Here's one of the more central files that ties into how a Jd database is laid out:

https://github.com/jsoftware/data_jd/blob/master/base/common...

Not that I claim anyone in particular can read it of course. Jd uses a hierarchy of folder, database, table, column that's handled with an object system to share code between them. A folder is just a place to put databases and hardly needs to add anything, while the other levels have a lot of extra functionality. As an inverted database, Jd stores each column in a file, and accesses it using memory mapping.

https://github.com/jsoftware/data_jd/blob/master/base/folder...

https://github.com/jsoftware/data_jd/blob/master/base/table....

(I designed this system when I did some of the early work to turn JDB into Jd as a summer intern)

I found this license for jd itself. It is free only for non-commercial use:

https://github.com/jsoftware/data_jd/blob/master/doc/License...

The link you mentioned only applies to the jsource folder: the jengine code.

A longer piece from GitHub: csvreportsummary=: 3 : 0 t=. <;.2 fread PATHLOGLOGFILE b=. (<,LF)=t b=. b+.(<'!')={.each t b=. b+.(<'src: ')=5{.each t b=. b+.(<'snk: ')=5{.each t b=. b+.(<'elapsed: ')=9{.each t b=. b+.(<'rows: ')=6{.each t b=. b+.(<'error: ')=7{.each t ;b#t )