tiny_sqlite VS crystal

I did some work on Nim's hash tables back in 2020, specifically with OrderedTable, comparable to a Python dict where insertion order is preserved. I stumbled on this table module in a roundabout way, via Nim's database module, db_sqlite. The db_sqlite module was much slower than Python for simple tests, and on investigation, I found that it didn't automatically handled prepared statement caching like Python's sqlite3 module. There were some other issues with db_sqlite, like blob handling and null handling, which led me to a different SQLite interface, tiny_sqlite. This was a big improvement, handling both nulls and blobs, and the developer was great to work with. But it also didn't support prepared statement caching. I filed an issue and he implemented it, using Nim's OrderedTable to simulate an LRU cache by adding a new prepared statement and deleting the oldest one if the cache was too big:

https://github.com/GULPF/tiny_sqlite/issues/3

Performance was hugely improved. There was another LRUCache implementation I played with, and when using that for the statement cache, performance was 25% faster than OrderedTable. That didn't make much sense to me for a 100-entry hash table, so I started running some tests comparing LRUCache and OrderedTable. What I discovered is that OrderedTable delete operations created an entirely new copy of the table, minus the entry being deleted, on every delete. That seemed pretty crazy, especially since it was already showing up as performance problems in a 100-entry table.

The tiny_sqlite developer switched to LRUCache, and I did some work on the OrderedTable implementation to make deletes O(1) as expected with hash table operations:

https://github.com/nim-lang/Nim/pull/14995

After spending a lot of time on this, I finally gave up. The problems were:

- the JSON implementation used OrderedTables and never did deletes. JSON benchmark performance was rather sacred, so changing OrderedTables to be slightly slower/larger (I used a doubly-linked list) was not desirable, even if it changed delete performance from O(n) to O(1)

- the Nim compiler also used OrderedTables and never did deletes

- Nim tables allowed multiple values for the same key (I did help get that deprecated).

- alternatives were proposed by others that maintained insertion order until a deleted occurred, but then it could become unordered. That made no sense to me.

The TLDR is, if you use Nim tables, don't use OrderedTable unless you can afford to make an copy of the table on every deleted.

Current Nim OrderedTable delete code: https://github.com/nim-lang/Nim/blob/15bffc20ed8da26e68c88bb...

Issue for db_sqlite not handling nulls, blobs, statement cache: https://github.com/nim-lang/Nim/issues/13559

27. Crystal - $77,104

I like the first code example on https://crystal-lang.org

    # A very basic HTTP server

Obviously as an interpreted language, it's never going to be as fast as something like C, Rust, or Go. Traditionally the ruby maintainers have not designed or optimized for pure speed, but that is changing, and the language is definitely faster these days compared to a decade ago.

If you like the ruby syntax/language but want the speed of a compiled language, it's also worth checking out Crystal[^1]. It's mostly ruby-like in syntax, style, and developer ergonomics.[^2] Although it's an entirely different language. Also a tiny community.

[1]: https://crystal-lang.org/

Crystal is a nice language that's not only simple to read and write but performs very well too. And the documentation is amazing as well.

Ruby is a super fun scripting language. I much prefer it to python when I need something with a little more "ooomph" than bash. It's just...nice...to write in. Ruby performance has come a long way in the last decade as well. There's libraries for pretty much everything.

My modern programming toolkit is basically golang + ruby + bash and I am never left wanting.

I do find Crystal (https://crystal-lang.org/) really interesting and am hoping it has its own "ruby on rails" moment that helps the language reach a tipping point in popularity. All the beauty of ruby with all of the speed of Go (and then some, it often compares favorably to languages like rust in benchmarks).