simonw
github-to-sqlite
simonw | github-to-sqlite | |
---|---|---|
9 | 7 | |
380 | 376 | |
- | 0.8% | |
9.9 | 6.0 | |
4 days ago | 4 months ago | |
Python | Python | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
simonw
-
Ask HN: High quality Python scripts or small libraries to learn from
Everything @simonw has worked on, honestly: https://github.com/simonw
- Datasette is my data hammer
-
How to add coding projects to Github portfolio?
You can create a repo at https://github.com/simonw/simonw (only using your username twice) and the README.md file in there will be used to populate your profile page.
-
Should i keep my forked repos after contribution if i want to use github as resume ?
I'd leave the forked repos there, and then use pinned repositories on your profile to highlight the repositories you are most proud of. You can also use a personal README to customize your profile - I have one here for example: https://github.com/simonw
-
how to run a github action every 6 hours
Here's one of mine that runs three times an hour: https://github.com/simonw/simonw/blob/main/.github/workflows/build.yml
-
How to Build a Dynamic GitHub Profile with GitHub Actions and PHP
As I was browsing examples for some inspiration, I stumbled upon Simon Willison's version, which features some dynamic content like recent work and blog publications. He explained how he used a combination of GitHub Actions and Python to achieve this in a blog post, and I decided to do something similar with PHP.
-
sqlite-utils - my Python library and CLI tool for manipulating SQLite databases
I have GitHub sponsors setup: https://github.com/simonw
-
CALISHOT 2021-08: Find ebooks among 403 Calibre sites
If you really want to please me, consider sponsoring Simon Willinson the author of the framework I'm relying on .
-
Ask HN: What are some tools / libraries you built yourself?
I have 112 mostly-maintained released project listed on https://github.com/simonw/simonw/blob/main/releases.md now - most of which are tools for loading data into SQLite database files (a surprisingly under-served use-case given how ubiquitous SQLite is) and plugins for my https://datasette.io/ project for reading data back out of SQLite again.
I realized a few years ago that SQLite was the perfect tool for doing data analysis at the small-data scale, where small data is less that 10GB which is pretty much everything you might want to analyze - especially for personal projects.
So I've been building tools to fill that niche!
github-to-sqlite
-
Getting things done (in small increments)
Yeah, I worry about that too. I have SO much of my stuff dependent on my GitHub account now.
I'm slightly reassured by how useful their APIs are. I have automated exports of a lot of my GitHub issues, though I really should shore those up and make sure I'm capturing everything.
That's one of the reasons I built https://github.com/dogsheep/github-to-sqlite
-
Automating screenshots for the Datasette documentation using shot-scraper
I have trouble answering this question myself, and I created it!
The problem I have is that it can be applied to too many different problems.
I personally have used it for the following (a truncated summary):
- Publishing data online to allow other people to explore it, for example https://scotrail.datasette.io and https://russian-ira-facebook-ads.datasettes.com/
- Building websites, by combining it with custom templates. https://datasette.io and https://www.niche-museums.com and https://til.simonwillison.net are three examples
- Building my own combined search engine over a bunch of different data. https://github-to-sqlite.dogsheep.net is this for my GitHub issues and commits and issue comments across 100+ projects
- Similarly, building a code search engine across multiple repos (partly to demonstrate how far you can go with custom plugins): https://ripgrep.datasette.io
- Any time I have a CSV file I open it in the Datasette Desktop macOS app first to start exploring it: https://datasette.io/desktop
- As a prototyping tool. It's the fastest way I know of to get from some data files (CSV or JSON) to a working JSON API - and a GraphQL API too using this plugin: https://datasette.io/plugins/datasette-graphql
- Messing around with geospatial data - here's a write-up of my favourite experiment with that so far: https://simonwillison.net/2021/Jan/24/drawing-shapes-spatial...
This is a bewilderingly wide array of things! And I keep on finding new problems I can apply it to:
Of course, if all you have is a hammer, everything looks like a nail. But thanks to the plugin system (and the amazing flexibility of SQLite under the good) I can reshape my hammer into all sorts of interesting shapes!
I've been trying to capture some of this at https://datasette.io/for
This is one of my biggest marketing challenges for the project though. If someone asks you for an elevator pitch you need to do better than spending 15 minutes talking through a wide ranging bulleted list!
-
Any tools for cloning github repos with revision?
You might also want to look at https://github.com/dogsheep/github-to-sqlite for saving issues, pull-requests, etc
-
I created Scrumdog – a program to download Jira Issues to a local database
Thanks for letting me know - fixed that here: https://github.com/dogsheep/github-to-sqlite/issues/74
-
Store SQLite in Cloudflare Durable Objects
My application Datasette can serve this need: https://datasette.io/
Lots of live demos - two of my favourites are https://github-to-sqlite.dogsheep.net/ and https://global-power-plants.datasettes.com/
-
sqlite-utils - my Python library and CLI tool for manipulating SQLite databases
I've been using the sqlite_utils Python library to build a whole suite of tools for importing data from various different sources into SQLite, including healthkit-to-sqlite, swarm-to-sqlite, twitter-to-sqlite, inaturalist-to-sqlite, google-takeout-to-sqlite, github-to-sqlite, genome-to-sqlite, pocket-to-sqlite, hacker-news-to-sqlite, evernote-to-sqlite.
What are some alternatives?
rupy - HTTP App. Server and JSON DB - Shared Parallel (Atomic) & Distributed
adama-lang - A headless spreadsheet document container service.
datasette - An open source multi-tool for exploring and publishing data
pocket-to-sqlite - Create a SQLite database containing data from your Pocket account
kondo - Cleans dependencies and build artifacts from your projects.
litestream - Streaming replication for SQLite.
google-takeout-to-sqlite - Save data from Google Takeout to a SQLite database
genome-to-sqlite - Import your genome into a SQLite database
hacker-news-to-sqlite - Create a SQLite database containing data pulled from Hacker News
healthkit-to-sqlite - Convert an Apple Healthkit export zip to a SQLite database
tera - A template engine for Rust based on Jinja2/Django
inaturalist-to-sqlite - Create a SQLite database containing your observation history from iNaturalist