Consider SQLite

• sql.js

  43 12,250 6.5 JavaScript

  A javascript library to run SQLite on the web.

  

I believe SQLite is about to explode in usage into areas it’s not be used before.

SQL.js[0] and the incredible “Absurd SQL”[1] are making it possible to build PWAs and hybrid mobile apps with a local SQL db.

Also I recently discovered the Session Extension[2] which would potentially enable offline distributed updates with eventual consistency!

I can imagine building a SAAS app where each customer has a “workspace” each as a single SQLite db, and a hybrid/PWA app which either uses a local copy of the SQLite db synced with the session extension or uses a serveless backend (like CloudFlare workers) where a lightweight function performs the db operations. I haven’t yet found a nice way to run SQLite on CloudFlare workers, it need some sort of block storage, but it can’t be far off.

0: https://sql.js.org/

1: https://github.com/jlongster/absurd-sql

2: https://www.sqlite.org/sessionintro.html

• absurd-sql

  24 4,063 2.5 JavaScript

  sqlite3 in ur indexeddb (hopefully a better backend soon)

I believe SQLite is about to explode in usage into areas it’s not be used before.

SQL.js[0] and the incredible “Absurd SQL”[1] are making it possible to build PWAs and hybrid mobile apps with a local SQL db.

Also I recently discovered the Session Extension[2] which would potentially enable offline distributed updates with eventual consistency!

I can imagine building a SAAS app where each customer has a “workspace” each as a single SQLite db, and a hybrid/PWA app which either uses a local copy of the SQLite db synced with the session extension or uses a serveless backend (like CloudFlare workers) where a lightweight function performs the db operations. I haven’t yet found a nice way to run SQLite on CloudFlare workers, it need some sort of block storage, but it can’t be far off.

0: https://sql.js.org/

1: https://github.com/jlongster/absurd-sql

2: https://www.sqlite.org/sessionintro.html

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• timeliner

  5 3,551 4.0 Go

  All your digital life on a single timeline, stored locally -- DEPRECATED, SEE TIMELINIZE (link below)

Not a "big project/service" but a Go project that uses Sqlite is one of my own, Timeliner[1] and its successor, Timelinize[2] (still in development). Yeah the cgo dependency kinda sucks but you don't feel it in code, just compilation. And it easily manages Timeline databases of a million and more entries just fine.

[1]: https://github.com/mholt/timeliner

[2]: https://twitter.com/timelinize

• Gravitational Teleport

  62 16,647 10.0 Go

  The easiest, and most secure way to access and protect all of your infrastructure.

  

https://github.com/gravitational/teleport/ has the option to use it, but it only uses it as a key value store.

CGO isnt too big a problem and if it is a real dealbreaker something like https://pkg.go.dev/modernc.org/sqlite will work as it transpiled the c into go and passes the sqlite test suite. I think there is performance degradation with writes but reads are still pretty quick.

• boringproxy

  10 1,114 2.8 Go

  Simple tunneling reverse proxy with a fast web UI and auto HTTPS. Designed for self-hosters.

  

Am I the only one who thinks SQLite is still too complicated for many programs? Maybe it's just the particular type of software I normally work on, which tends towards small, self-hosted networking services[0] that would often have a single user, or maybe federated with <100 users. These programs need a small amount of state for things like tokens, users accounts, and maybe a bit of domain-specific things. This can all live in memory, but needs to be persisted to disk on writes. I've reached for SQLite several times, and always come back to just keeping a struct of hashmaps[1] in memory and dumping JSON to disk. It's worked great for my needs.

Now obviously if I wanted to scale up, at some point you would have too many users to fit in memory. But do programs at that scale actually need to exist? Why can't everyone be on a federated server with state that fits in memory/JSON? I guess that's more of a philosophical question about big tech. But I think it's interesting that most of our tech stack choices are driven by projects designed to work at a scale most of us will never need, and maybe nobody needs.

[0]: https://boringproxy.io/

[1]: https://github.com/boringproxy/boringproxy/blob/master/datab...

• dqlite

  33 3,721 9.5 C

  Embeddable, replicated and fault-tolerant SQL engine.

  

>it's not easy to inspect or fix data in production the way we would with postgres.

I assume because you're using a remote socket connection from the client?

I haven't tried it in a serious setting yet, but I did play around with dqlite and was impressed. Canonical uses it as the backing for lxd. Basically sqlite with raft clustering and the ability for clients to connect remotely via a wire protocol. https://dqlite.io/

• rqlite

  112 14,927 9.9 Go

  The lightweight, distributed relational database built on SQLite.

  

I've always thought it interesting that there was a time when large(ish) websites were hosted using servers that would struggle to outperform a modern smart toaster or wristwatch, and yet modern web applications tend to demand a dramatic distributed architecture. I like the examples in this article showing what a single modern server can do when you're not scaling to Google's level.

As an aside, what about distributed derivatives of sqlite, like rqlite, as a response to the criticism that sqlite requires your database server to also be your web server. Could something like rqlite also provide a way for an sqlite database to grow into a distributed cluster at a later point?

https://github.com/rqlite/rqlite

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

• dbeaver

  27 37,575 9.9 Java

  Free universal database tool and SQL client

  

I think the usual approach would be to SSH into the server and run sqlite3 there. This issue [0] mentions some workarounds for connecting from DBeaver, which I assume would work for other graphical client software. I haven't tried those approaches, and they seem pretty hacky, but I guess that's to be expected given that SQLite isn't designed for that type of access.

[0] https://github.com/dbeaver/dbeaver/issues/6876

• yjs

  53 15,280 8.6 JavaScript

  Shared data types for building collaborative software

  

You have to handle the merge conflicts yourself, see https://www.sqlite.org/session/sqlite3changeset_apply.html

So you need to be carful how you design your schema, but very possible.

One option is to use something like Yjs and a JSON column to get proper CRDTs for merging.

https://github.com/yjs/yjs

• Lunar

  192 4,380 9.0 Swift

  Intelligent adaptive brightness for your external monitors

I'm exactly at a point where I'm considering SQLite for its single file db advantage, but I'm struggling to find solutions for my use case.

I need to import some 30k JSONs of external monitor data from Lunar (https://lunar.fyi) into a normalized form so that everyone can query it.

I'd love to get this into a single SQLite file that can be served and cached through CDN and local browser cache.

But is there something akin to Metabase that could be used to query the db file after it was downloaded?

I know I could have a Metabase server that could query the SQLite DB on my server, but I'd like the db and the queries to run locally for faster iteration and less load on my server.

Besides, I'm reluctant to run a public Metabase instance given the log4j vulnerabilities that keep coming.

• go-sqlite3

  40 7,488 6.2 C

  sqlite3 driver for go using database/sql

Using it for a pet project right now running https://github.com/mattn/go-sqlite3

So far not a single issue with it, I never had to think about any CGO specifics.

• datasette

  187 8,963 9.3 Python

  An open source multi-tool for exploring and publishing data

Check out datasette: https://datasette.io/

• LevelDB

  27 35,186 0.0 C++

  LevelDB is a fast key-value storage library written at Google that provides an ordered mapping from string keys to string values.

  

Suggest a related project