The search for a better persistent cache (SQLite)

• duckdb

  52 16,356 10.0 C++

  DuckDB is an in-process SQL OLAP Database Management System

  

  I'm not sure if it would help in your case, but could you process all categories at once with a larger SQL query?

  If so, DuckDB can process bulk queries about 20x faster than SQLite per CPU core because it is vectorized and column oriented. Then with multiple cores you can easily reach 100x SQLite speed. DuckDB has node bindings and is an in-process DB like SQLite.

  If reading from disk is your bottleneck, I would recommend storing your data in compressed parquet files and reading them with DuckDB's parquet reader.

  One drawback is that indexes are not persistent to the filesystem in DuckDB yet, but full table scans are much faster than SQLite since it is columnar.

  https://github.com/duckdb/duckdb/tree/master/tools/nodejs

  https://duckdb.org/

• sqlite-worker

  1 168 0.0 JavaScript

  A simple, and persistent, SQLite database for Web and Workers.

  I have recently started using https://github.com/WebReflection/sqlite-worker which works pretty well

• WorkOS

  workos.com

  sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

Suggest a related project