Many Small Queries Are Efficient in SQLite

• litestream

  165 9,997 7.5 Go

  Streaming replication for SQLite.

I feel like the sibling comments here are basically just saying "yep, that's the main challenge!" without providing useful tips. I personally haven't used it, but I'm aware that this library exists to help resolve this challenge. https://litestream.io

• rqlite

  112 14,862 9.9 Go

  The lightweight, distributed relational database built on SQLite.

  

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• memlite

  1 3 0.0 TypeScript

  SQLite-based state management JS library. (by Ziinc)

I just started work on a simple state management tool based on SQLite [0], utilising SQL.js under the hood. Just wrote it last week too! It intends to be much easier to learn than redux and mobx, but more powerful than a react's context.

In theory, it should support most of redux's ecosystem too, such as reselect, though I have yet to create examples for it.

[0]: https://github.com/ziinc/memlite

• s2geometry

  26 2,181 5.8 C++

  Computational geometry and spatial indexing on the sphere

  

> store and index it by tiles

Options include the https://sqlite.org/rtree.html module, and building an application-specific mapping from geometric indexes to an integer keyspace (https://github.com/google/s2geometry or similar).

We're using SQLite archives of many GB successfully without issue. As long as the primary keyspace is well-designed (see also https://sqlite.org/withoutrowid.html), ranged queries are extremely fast.

> I'm never sure how to properly copy raw struct binary data from sql, directly in ram.

BLOB columns and an application-specific serialization/deserialization step work well. memcpy to a struct works if you are absolutely certain that you know what the layout will be. All of the standard perils apply - alignment, internal padding, platform-specific layout, endianness, etc.

We're using Protobuf with success. I imagine Flatbuffers would also work well. I'd put Protobuf/Flatbuf and their competitors on the front of the tool shelf.

• WSL

  406 16,652 8.6 PowerShell

  Issues found on WSL

  

Interesting. Did you tested any performance by non-NTFS?

I remembered this post. https://github.com/microsoft/WSL/issues/873#issuecomment-425...

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

Suggest a related project