sql.js
litestream
sql.js | litestream | |
---|---|---|
43 | 165 | |
12,234 | 9,997 | |
0.6% | - | |
6.5 | 7.5 | |
11 days ago | 12 days ago | |
JavaScript | Go | |
GNU General Public License v3.0 or later | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
sql.js
-
Show HN: Appendable – Index JSONL data and query via CDN
Hi HN! A friend and I were inspired by projects like https://github.com/sql-js/sql.js and the idea of querying files served over CDN with HTTP range requests. We started thinking: what would a database that was specifically designed for this type of use case look like? So we started building one, and we landed on a functional prototype that we're pretty proud of!
With our prototype, Appendable, we're able to serve and query large (GB+) datasets by hosting them on a static file host like Amazon S3 or Cloudflare R2 without running a separate server and worrying about things like tail latency, replication, and connection pooling -- all that is handled for us by the file hoster.
Additionally, one tenet that we have been following is Appendable won't touch your underlying data, so your jsonl file is preserved and we point at that data instead of consuming it into an Appendable-specific file format. This keeps your data yours and makes it easy to introspect the data: just open it up with your favorite editor aka vim.
We're curious what you think, we're excited to build this out further to get the performance even better and add features like pubsub. Everything is open source at https://github.com/kevmo314/appendable.
Kevin and Matthew
- How to show CRUD projects on Github?
-
I made a website where you can use SQLite in your browser
My project is powered by sql.js, I recommend checking that out if you're interested - https://github.com/sql-js/sql.js/
-
How to build interactive way to learn SQL using Next.js and database?
Maybe you can try to use some SQL database compiled as Web Assembly Modules? Like this one for example: https://github.com/sql-js/sql.js
-
Recommendations for data structure and storage
If you want to have persistence, then I would go with a database like Dexie, as it uses IndexedDB and has transactions. If you just want something that's in memory, you could look at Sql.js or something simple like lowdb.
-
I have a large JSON object (~2GB), what's the best way to make a site that lets you search through it and display the results without crashing?
not necessarily. you can host an html/js/sqlite site on github pages for free. json -> sqlite3 js -> sql
-
new release of : https://sql.js.org/
Link: https://sql.js.org
- Web-Projekt - Hilfe, weil ich nicht weiß, was ich benötige :S
- Learn Postgres at the Playground
-
Show HN: CSVFiddle – Query CSV files with DuckDB in the browser
Does it work with really large files? Like, >100mb or so. I was considering making something similar but with sqlite.js [1], but the problem with it is that it loads everything in memory, so I wasn't entirely sure how it will deal with larger workloads.
[1]: https://sql.js.org/#/
litestream
-
Ask HN: SQLite in Production?
I have not, but I keep meaning to collate everything I've learned into a set of useful defaults just to remind myself what settings I should be enabling and why.
Regarding Litestream, I learned pretty much all I know from their documentation: https://litestream.io/
-
How (and why) to run SQLite in production
This presentation is focused on the use-case of vertically scaling a single server and driving everything through that app server, which is running SQLite embedded within your application process.
This is the sweet-spot for SQLite applications, but there have been explorations and advances to running SQLite across a network of app servers. LiteFS (https://fly.io/docs/litefs/), the sibling to Litestream for backups (https://litestream.io), is aimed at precisely this use-case. Similarly, Turso (https://turso.tech) is a new-ish managed database company for running SQLite in a more traditional client-server distribution.
-
SQLite3 Replication: A Wizard's Guide🧙🏽
This post intends to help you setup replication for SQLite using Litestream.
-
Ask HN: Time travel" into a SQLite database using the WAL files?
I've been messing around with litestream. It is so cool. And, I either found a bug in the -timestamp switch or don't understand it correctly.
What I want to do is time travel into my sqlite database. I'm trying to do some forensics on why my web service returned the wrong data during a production event. Unfortunately, after the event, someone deleted records from the database and I'm unsure what the data looked like and am having trouble recreating the production issue.
Litestream has this great switch: -timestamp. If you use it (AFAICT) you can time travel into your database and go back to the database state at that moment. However, it does not seem to work as I expect it to:
https://github.com/benbjohnson/litestream/issues/564
I have the entirety of the sqlite database from the production event as well. Is there a way I could cycle through the WAL files and restore the database to the point in time before the records I need were deleted?
Will someone take sqlite and compile it into the browser using WASM so I can drag a sqlite database and WAL files into it and then using a timeline slider see all the states of the database over time? :)
-
Ask HN: Are you using SQLite and Litestream in production?
We're using SQLite in production very heavily with millions of databases and fairly high operations throughput.
But we did run into some scariness around trying to use Litestream that put me off it for the time being. Litestream is really cool but it is also very much a cool hack and the risk of database corruption issues feels very real.
The scariness I ran into was related to this issue https://github.com/benbjohnson/litestream/issues/510
-
Pocketbase: Open-source back end in 1 file
Litestream is a library that allows you to easily create backups. You can probably just do analytic queries on the backup data and reduce load on your server.
https://litestream.io/
- Litestream – Disaster recovery and continuous replication for SQLite
- Litestream: Replicated SQLite with no main and little cost
-
Why you should probably be using SQLite
One possible strategy is to have one directory/file per customer which is one SQLite file. But then as the user logs in, you have to look up first what database they should be connected to.
OR somehow derive it from the user ID/username. Keeping all the customer databases in a single directory/disk and then constantly "lite streaming" to S3.
Because each user is isolated, they'll be writing to their own database. But migrations would be a pain. They will have to be rolled out to each database separately.
One upside is, you can give users the ability to take their data with them, any time. It is just a single file.
[0]. https://litestream.io/
-
Monitor your Websites and Apps using Uptime Kuma
Upstream Kuma uses a local SQLite database to store account data, configuration for services to monitor, notification settings, and more. To make sure that our data is available across redeploys, we will bundle Uptime Kuma with Litestream, a project that implements streaming replication for SQLite databases to a remote object storage provider. Effectively, this allows us to treat the local SQLite database as if it were securely stored in a remote database.
What are some alternatives?
localForage - 💾 Offline storage, improved. Wraps IndexedDB, WebSQL, or localStorage using a simple but powerful API.
rqlite - The lightweight, distributed relational database built on SQLite.
LokiJS - javascript embeddable / in-memory database
pocketbase - Open Source realtime backend in 1 file
PouchDB - :koala: - PouchDB is a pocket-sized database.
realtime - Broadcast, Presence, and Postgres Changes via WebSockets
WatermelonDB - 🍉 Reactive & asynchronous database for powerful React and React Native apps ⚡️
k8s-mediaserver-operator - Repository for k8s Mediaserver Operator project
DB.js - db.js is a wrapper for IndexedDB to make it easier to work against
sqlcipher - SQLCipher is a standalone fork of SQLite that adds 256 bit AES encryption of database files and other security features.
basket.js - A script and resource loader for caching & loading files with localStorage
litefs - FUSE-based file system for replicating SQLite databases across a cluster of machines