spl.js
datasette-lite
Our great sponsors
spl.js | datasette-lite | |
---|---|---|
10 | 10 | |
158 | 308 | |
- | - | |
4.6 | 5.4 | |
about 1 month ago | about 1 month ago | |
JavaScript | HTML | |
GNU General Public License v3.0 only | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
spl.js
- SpatiaLite (SQLite extension) for browser and node
-
Exploring SQLite Implementations for the Web in 2023
Use spatialite WASM https://github.com/jvail/spl.js Can read geopackage and Shapefiles and GeoJSON and KML and GPX and perform operations
-
Manipulate CSV files in your browser using SQL
I use SPATIALITE WEBASSEMBLY let's you use CSV or excel or JSON or sqlite or gpkg or shapefiles or KML in the browser https://github.com/jvail/spl.js
-
Online tool to process GeoJSON data using JavaScript
I recommend you add spatialite webassembly so you can run spatial functions operations https://github.com/jvail/spl.js Support geopackage vector features and esri mobile geodatabase (sqlite) and shapefiles and kml and output GEOJSON for map view (ExportGeojson2)
-
geopackage sqlite
https://github.com/jvail/spl.js The geopackage pub.dev package could not work for me reading vector geometry from. Geopackage
-
Cascading slider filters and exposing SQL query builder on leaflet
Take a look at spatialite web assembly you can do full spatialite functions and operations on GeoJSON data or Shapefile or geopackage and even KNN NEAREST And point in polygon. No server needed run in the browser https://github.com/jvail/spl.js
-
If I have a GeoJSON file of a certain city's wards, how would I figure out which ward I'm in based on my lat/long coordinates?
We've recently been using more client-side (browser based approaches for this type of work) that have more horsepower and functionality than TurfJS) we've successfully used both https://github.com/jvail/spl.js Web Assembly Version
- A SQLite extension for reading large files line-by-line
- SpatiaLite: Extends SQLite core to support Spatial SQL capabilities
-
Combining several feature layers into 1 layer using ArcGIS JavaScript
https://github.com/jvail/spl.js Your use case could be good for doing this in spatialite Create a new geopackage Import GeoJSON from featureserver or mapserver layers -f=GeoJSON for each layer or use virtualgeojson You can make the import happen for new records or changes And then create new table and append the records Then you can execute SQL queries Counts or Distinct or where clauses
datasette-lite
-
Sqlime: Online SQLite Playground
Also see: https://github.com/simonw/datasette-lite
- Use SQL Without Databases
-
GitHub – GSA/code-gov: An informative repo for all Code.gov repos
https://github.com/simonw/datasette-lite :
> You can use this tool to open any SQLite database file that is hosted online and served with a `access-control-allow-origin: ` CORS header. Files served by GitHub Pages automatically include this header, as do database files that have been published online using `datasette publish`.*
> [...] You can paste in the "raw" URL to a file, but Datasette Lite also has a shortcut: if you paste in the URL to a page on GitHub or a Gist it will automatically convert it to the "raw" URL for you
> To load a Parquet file, pass a URL to `?parquet=`
> [...] https://lite.datasette.io/?parquet=https://github.com/Terada...*
There are various *-to-sqlite utilities that load data into a SQLite database for use with e.g. datasette. E.g. Pandas with `dtype_backend='arrow'` saves to Parquet.
datasette plugins are written in Python and/or JS w/ pluggy:
-
[SQLlite] Is there any online SQL editor I can host on my website? Maybe something in JS or php
Datasette Lite might be even better for this - you can construct URLs that link directly to examples: https://github.com/simonw/datasette-lite
-
SQLite WASM Official
There are some amazing things for SQLite in the browser especially if you're looking for ways to host queryable data for cheap.
I have a hacked up POC experimental version of datasette-lite to be able to look at multi-GB databases at https://github.com/simonw/datasette-lite/pull/49. It uses a hacked up chunk'd lazyFile implementation from emscripten and others to grap pages from Cloudflare R2.
It's a test with california's unclaimed property records (https://www.sco.ca.gov/upd_download_property_records.html) of a 28GB searching up that guy who owns Twitter: https://datasette-lite-lab.mindflakes.com/index.html?url=htt...
I think there may be a space for super-large multi-GB files served from static storage being accessible from SQlite as well. Another one would be this full-text search of a 43GB SQLite database of Wikipedia's full text search: http://static.wiki/ . Hearing there's official support for this is awesome and I hope they also might add some provisions for those sticking with POSIX/Emscripten as well.
-
Hosting SQLite Databases on GitHub Pages
I grafted the enhanced lazyFile implementation of this to datasette-lite relatively recently. Threw in a 18GB CSV from
https://www.sco.ca.gov/upd_download_property_records.html
into a FTS5 Sqlite Database which came out to about 28GB after processing:
POC, non-merging Draft PR for the hack:
https://github.com/simonw/datasette-lite/pull/49
You can run queries through it if you URL hack into it and just get to the query dialog, browsing is kind of a dud at the moment since datasette runs a count(*) which downloads everything.
- Learn Postgres at the Playground
-
A SQLite extension for reading large files line-by-line
Oh wow! I wonder how hard it would be to load that module into https://github.com/simonw/datasette-lite
-
This Week in Python
datasette-lite – Datasette running in your browser using WebAssembly and Pyodide
-
Datasette Lite: a server-side Python web application running in a browser
I have an open issue for that here: https://github.com/simonw/datasette-lite/issues/28
My initial hunch is that this will be really difficult - probably require a fork of something like https://github.com/coleifer/pysqlite3 then compiled for WebAssembly.
I'm confident it's feasible, but I don't have the skills to figure it out myself.
What are some alternatives?
rtree.c - An R-tree implementation in C
pyscript - Try PyScript: https://pyscript.com Examples: https://tinyurl.com/pyscript-examples Community: https://discord.gg/HxvBtukrg2
sqlite-lines - A SQLite extension for reading large files line-by-line (NDJSON, logs, txt, etc.)
sqlite-plus - The ultimate set of SQLite extensions
geojsonscript - GeoJSON scripting environment in the web browser
file-system-access - Expose the file system on the user’s device, so Web apps can interoperate with the user’s native applications.
datastation - App to easily query, script, and visualize data from every database, file, and API.
pyodide - Pyodide is a Python distribution for the browser and Node.js based on WebAssembly
mergestat-lite - Query git repositories with SQL. Generate reports, perform status checks, analyze codebases. 🔍 📊
pysqlite3 - SQLite3 DB-API 2.0 driver from Python 3, packaged separately, with improvements
v86 - x86 PC emulator and x86-to-wasm JIT, running in the browser
slipcover - Near Zero-Overhead Python Code Coverage