sqlite-utils
just
sqlite-utils | just | |
---|---|---|
35 | 169 | |
1,531 | 17,780 | |
- | - | |
8.1 | 9.0 | |
16 days ago | about 9 hours ago | |
Python | Rust | |
Apache License 2.0 | Creative Commons Zero v1.0 Universal |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
sqlite-utils
-
Ask HN: High quality Python scripts or small libraries to learn from
https://github.com/simonw/sqlite-utils
So, his code might not be a good place to find best patterns (for ex, I don't think they are fully typed), but his repos are very pragmatic, and his development process is super insightful (well documented PRs for personal repos!). Best part, he blogs about every non-trivial update, so you get all the context!
-
Why you should probably be using SQLite
Sounds like your problem is with SQLAlchemy, not with SQLite.
My https://sqlite-utils.datasette.io library might be a better fit for you. It's a much thinner abstraction than SQLAlchemy.
-
Welcome to Datasette Cloud
There are a few things you can do here.
SQLite is great at JSON - so I often dump JSON structures in a TEXT column and query them using https://www.sqlite.org/json1.html
I also have plugins for running jq() functions directly in SQL queries - https://datasette.io/plugins/datasette-jq and https://github.com/simonw/sqlite-utils-jq
I've been trying to drive the cost of turning semi-structured data into structured SQL queries down as much as possible with https://sqlite-utils.datasette.io - see this tutorial for more: https://datasette.io/tutorials/clean-data
This is also an area that I'm starting to explore with LLMs. I love the idea that you could take a bunch of messy data, tell Datasette Cloud "I want this imported into a table with this schema"... and it does that.
I have a prototype of this working now, I hope to turn it into an open source plugin (and Datasette Cloud feature) pretty soon. It's using this trick: https://til.simonwillison.net/gpt3/openai-python-functions-d...
-
SQLite Functions for Working with JSON
I've baked a ton of different SQLite tricks - including things like full-text indexing support and advanced alter table methods - into my sqlite-utils CLI tool and Python library: https://sqlite-utils.datasette.io
My Datasette project provides tools for exploring, analyzing and publishing SQLite databases, plus ways to expose them via a JSON API: https://datasette.io
I've also written a ton of stuff about SQLite on my two blogs:
- https://simonwillison.net/tags/sqlite/
- https://til.simonwillison.net/sqlite
-
Show HN: Trogon – An automatic TUI for command line apps
This is really fun. I have an experimental branch of my sqlite-utils CLI tool (which has dozens of sub-commands) running with this now and it really did only take 4 lines of code - I'm treating Trogon as an optional dependency because people using my package as a Python library rather than a CLI tool may not want the extra installed components:
https://github.com/simonw/sqlite-utils/commit/ec12b780d5dcd6...
There's an animated GIF demo of the result here: https://github.com/simonw/sqlite-utils/issues/545#issuecomme...
-
I'm sure I'm being stupid.. Copying data from an API and making a database
My project https://datasette.io/ is ideal for this kind of thing. You can use https://sqlite-utils.datasette.io/ to load JSON data into a SQLite database, then publish it with Datasette.
-
Just: A Command Runner
I've been using this for about six months now and I absolutely love it.
Make never stuck for me - I couldn't quite get it to fit inside my head.
Just has the exact set of features I want.
Here's one example of one of my Justfiles: https://github.com/simonw/sqlite-utils/blob/fc221f9b62ed8624... - documented here: https://sqlite-utils.datasette.io/en/stable/contributing.htm...
I also wrote about using Just with Django in this TIL: https://til.simonwillison.net/django/just-with-django
-
Ask HN: What Do You Use for a Personal Database
SQLite with the open source toolchain I've been building over the past five years:
https://datasette.io as the interface for running queries against (and visualizing) my data.
https://sqlite-utils.datasette.io/ as a set of tools for creating and modifying my databases (inserting JSON or CSV data, enabling full text search text)
https://dogsheep.github.io as a suite of tools for importing my personal data - see also this talk I gave about that project: https://simonwillison.net/2020/Nov/14/personal-data-warehous...
-
The Perfect Commit
Here's an example: https://github.com/simonw/sqlite-utils/pull/468
> After identifying about 7 commits (with pretty basic/useless messages, and no PR link!), I then had to find the corresponding PRs based on timestamps, and search the PR history for PRs merged around those timestamps.
Not sure if this would save any time, but it is possible to search PRs by commit. For example, say git blame led me to this commit: https://github.com/simonw/sqlite-utils/commit/129141572f249e...
I could have found PR #373 via this search: https://github.com/simonw/sqlite-utils/pulls?q=bb16f52681b6d...
> I thus treat PRs as ephemeral
I think I see what you're saying but as others have pointed out, sometimes you want to add screenshots etc to the context, and you can't capture this kind of info in commit messages. So then you have two choices: issues or PRs.
> Then any review comments are preferably not addressed directly in the PR
I would think that sometimes you really do want to have a back and forth conversation in the PR, rather than just a "make this change" -> "ok done" type of feedback loop.
I view the PR as an decent place for all of this because it's basically a commit of commits, capturing the related changes/conversation/context all in a single place at the point of merge.
just
-
I stopped worrying and loved Makefiles
I don't like makefiles, but I've been enjoying justfiles: https://github.com/casey/just
- Just a Command Runner
-
Ask HN: Any tool for managing large and variable command lines?
I started using just [0] on my projects and have been very happy so far. It is very similar to make but focused on commands rather than build outputs.
Define your recipes and then you can compose them as needed.
[0] https://github.com/casey/just
-
Ask HN: What software sparks joy when using?
just - https://github.com/casey/just
-
GitHub switched to Docker Compose v2, action needed
Welp there is absolute chaos in that thread -- guess it's not an April Fools joke.
I wonder if relying on CI for anything other than provisioning machines is a mistake -- maybe we should have never moved from doing things from local scripts written in $LANGUAGE.
That said, I'm probably biased since I'm a massive fan of things like `make` and more appropriately for the current age, `just`[0]
[0]: https://github.com/casey/just
-
Which command did you run 1731 days ago?
> When a command has some cognitive requirements I create a script with some ${1:-default} values and I store them all in $PATH enabled local/bin
I would consider using just for this:
https://github.com/casey/just
-
Using Make – writing less Makefile
Your coworker's experience is more principled: Make is a mediocre tool for executing commands. It wasn't ever designed for that. Although it is pretty common to see what you are mentioning in projects because it doesn't require installing a dependency.
For a repo where an easy to install (single binary) dependency is a non-issue, consider using just. [1] You get `just -l` where you can see all the command available, the ability to use different languages, and overall simpler command writing.
[1] https://github.com/casey/just
-
Show HN: Just.sh – compiler that turns Justfiles into portable shell scripts
This is fantastic, but I'd say that this solution is somewhat in response to this open issue from 2019:
https://github.com/casey/just/issues/429
I really wish just was included as a package in distributions.
-
Sharing Saturday #496
So far, I didn't work on new features at all but on stabilizing the ground for further development: 1. CMake lists and modules were rewritten a lot, now managing builds and their configurations is much lesser pain. 2. Brought in Justfile for regular tasks, and it's great, no less. 3. Linters, formatters, analyzers for almost all the code (except for Janet for now, as because of it being a niche and young technology, it didn't get enough attention yet). 4. ECS stub. Now runtime class doesn't look like a god object. 5. Started writing unit tests which didn't happen with my personal projects before and maybe indicates how serious am I about this one :D 6. Some of previously hardcoded data has been moved to INI files. Now, if I release the game in 10 years, and in 10 more years some eccentric person decides to make a variant of it, it will be slightly simpler.
-
What’s with DevOps engineers using `make` of all things?
i've grown to like this for my personal projects. https://github.com/casey/just
What are some alternatives?
sqlmodel - SQL databases in Python, designed for simplicity, compatibility, and robustness.
Task - A task runner / simpler Make alternative written in Go
sqliteviz - Instant offline SQL-powered data visualisation in your browser
cargo-make - Rust task runner and build tool.
ImportExcel - PowerShell module to import/export Excel spreadsheets, without Excel
cargo-xtask
octosql - OctoSQL is a query tool that allows you to join, analyse and transform data from multiple databases and file formats using SQL.
Taskfile - Repository for the Taskfile template.
q - q - Run SQL directly on delimited files and multi-file sqlite databases
CodeLLDB - A native debugger extension for VSCode based on LLDB
Scoop - A command-line installer for Windows.
cargo-release - Cargo subcommand `release`: everything about releasing a rust crate.