Automating screenshots for the Datasette documentation using shot-scraper

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

Our great sponsors
  • WorkOS - The modern identity platform for B2B SaaS
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • SaaSHub - Software Alternatives and Reviews
  • datasette

    An open source multi-tool for exploring and publishing data

  • I have trouble answering this question myself, and I created it!

    The problem I have is that it can be applied to too many different problems.

    I personally have used it for the following (a truncated summary):

    - Publishing data online to allow other people to explore it, for example https://scotrail.datasette.io and https://russian-ira-facebook-ads.datasettes.com/

    - Building websites, by combining it with custom templates. https://datasette.io and https://www.niche-museums.com and https://til.simonwillison.net are three examples

    - Building my own combined search engine over a bunch of different data. https://github-to-sqlite.dogsheep.net is this for my GitHub issues and commits and issue comments across 100+ projects

    - Similarly, building a code search engine across multiple repos (partly to demonstrate how far you can go with custom plugins): https://ripgrep.datasette.io

    - Any time I have a CSV file I open it in the Datasette Desktop macOS app first to start exploring it: https://datasette.io/desktop

    - As a prototyping tool. It's the fastest way I know of to get from some data files (CSV or JSON) to a working JSON API - and a GraphQL API too using this plugin: https://datasette.io/plugins/datasette-graphql

    - Messing around with geospatial data - here's a write-up of my favourite experiment with that so far: https://simonwillison.net/2021/Jan/24/drawing-shapes-spatial...

    This is a bewilderingly wide array of things! And I keep on finding new problems I can apply it to:

    Of course, if all you have is a hammer, everything looks like a nail. But thanks to the plugin system (and the amazing flexibility of SQLite under the good) I can reshape my hammer into all sorts of interesting shapes!

    I've been trying to capture some of this at https://datasette.io/for

    This is one of my biggest marketing challenges for the project though. If someone asks you for an elevator pitch you need to do better than spending 15 minutes talking through a wide ranging bulleted list!

  • fusionauth-site

    Website and documentation for FusionAuth

  • I had a colleague (thanks Sanjay!) make a similar tool at a hackfest (you can see the source code here: https://github.com/FusionAuth/fusionauth-site/blob/master/sr... ).

    I've modified it a bit to take a URL, but haven't yet set it up to read a config file to make a large number of screenshots easy to do.

    We do outline certain fields or other areas in the doc to highlight a point. That's caused some hesitation on my part. However, it looks like I could use imagemagick to automatically put a red box or similar on an image (with a `-draw` command).

    We have a ton of screenshots (600+) throughout our doco, and a way to initialize our product to a known state, so the pieces are all there.

    One of these days it'll be worthwhile to do this.

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

    WorkOS logo
  • til

    Today I Learned (by simonw)

  • I have trouble answering this question myself, and I created it!

    The problem I have is that it can be applied to too many different problems.

    I personally have used it for the following (a truncated summary):

    - Publishing data online to allow other people to explore it, for example https://scotrail.datasette.io and https://russian-ira-facebook-ads.datasettes.com/

    - Building websites, by combining it with custom templates. https://datasette.io and https://www.niche-museums.com and https://til.simonwillison.net are three examples

    - Building my own combined search engine over a bunch of different data. https://github-to-sqlite.dogsheep.net is this for my GitHub issues and commits and issue comments across 100+ projects

    - Similarly, building a code search engine across multiple repos (partly to demonstrate how far you can go with custom plugins): https://ripgrep.datasette.io

    - Any time I have a CSV file I open it in the Datasette Desktop macOS app first to start exploring it: https://datasette.io/desktop

    - As a prototyping tool. It's the fastest way I know of to get from some data files (CSV or JSON) to a working JSON API - and a GraphQL API too using this plugin: https://datasette.io/plugins/datasette-graphql

    - Messing around with geospatial data - here's a write-up of my favourite experiment with that so far: https://simonwillison.net/2021/Jan/24/drawing-shapes-spatial...

    This is a bewilderingly wide array of things! And I keep on finding new problems I can apply it to:

    Of course, if all you have is a hammer, everything looks like a nail. But thanks to the plugin system (and the amazing flexibility of SQLite under the good) I can reshape my hammer into all sorts of interesting shapes!

    I've been trying to capture some of this at https://datasette.io/for

    This is one of my biggest marketing challenges for the project though. If someone asks you for an elevator pitch you need to do better than spending 15 minutes talking through a wide ranging bulleted list!

  • github-to-sqlite

    Save data from GitHub to a SQLite database

  • I have trouble answering this question myself, and I created it!

    The problem I have is that it can be applied to too many different problems.

    I personally have used it for the following (a truncated summary):

    - Publishing data online to allow other people to explore it, for example https://scotrail.datasette.io and https://russian-ira-facebook-ads.datasettes.com/

    - Building websites, by combining it with custom templates. https://datasette.io and https://www.niche-museums.com and https://til.simonwillison.net are three examples

    - Building my own combined search engine over a bunch of different data. https://github-to-sqlite.dogsheep.net is this for my GitHub issues and commits and issue comments across 100+ projects

    - Similarly, building a code search engine across multiple repos (partly to demonstrate how far you can go with custom plugins): https://ripgrep.datasette.io

    - Any time I have a CSV file I open it in the Datasette Desktop macOS app first to start exploring it: https://datasette.io/desktop

    - As a prototyping tool. It's the fastest way I know of to get from some data files (CSV or JSON) to a working JSON API - and a GraphQL API too using this plugin: https://datasette.io/plugins/datasette-graphql

    - Messing around with geospatial data - here's a write-up of my favourite experiment with that so far: https://simonwillison.net/2021/Jan/24/drawing-shapes-spatial...

    This is a bewilderingly wide array of things! And I keep on finding new problems I can apply it to:

    Of course, if all you have is a hammer, everything looks like a nail. But thanks to the plugin system (and the amazing flexibility of SQLite under the good) I can reshape my hammer into all sorts of interesting shapes!

    I've been trying to capture some of this at https://datasette.io/for

    This is one of my biggest marketing challenges for the project though. If someone asks you for an elevator pitch you need to do better than spending 15 minutes talking through a wide ranging bulleted list!

  • datasette-app

    The Datasette macOS application

  • I have trouble answering this question myself, and I created it!

    The problem I have is that it can be applied to too many different problems.

    I personally have used it for the following (a truncated summary):

    - Publishing data online to allow other people to explore it, for example https://scotrail.datasette.io and https://russian-ira-facebook-ads.datasettes.com/

    - Building websites, by combining it with custom templates. https://datasette.io and https://www.niche-museums.com and https://til.simonwillison.net are three examples

    - Building my own combined search engine over a bunch of different data. https://github-to-sqlite.dogsheep.net is this for my GitHub issues and commits and issue comments across 100+ projects

    - Similarly, building a code search engine across multiple repos (partly to demonstrate how far you can go with custom plugins): https://ripgrep.datasette.io

    - Any time I have a CSV file I open it in the Datasette Desktop macOS app first to start exploring it: https://datasette.io/desktop

    - As a prototyping tool. It's the fastest way I know of to get from some data files (CSV or JSON) to a working JSON API - and a GraphQL API too using this plugin: https://datasette.io/plugins/datasette-graphql

    - Messing around with geospatial data - here's a write-up of my favourite experiment with that so far: https://simonwillison.net/2021/Jan/24/drawing-shapes-spatial...

    This is a bewilderingly wide array of things! And I keep on finding new problems I can apply it to:

    Of course, if all you have is a hammer, everything looks like a nail. But thanks to the plugin system (and the amazing flexibility of SQLite under the good) I can reshape my hammer into all sorts of interesting shapes!

    I've been trying to capture some of this at https://datasette.io/for

    This is one of my biggest marketing challenges for the project though. If someone asks you for an elevator pitch you need to do better than spending 15 minutes talking through a wide ranging bulleted list!

  • shot-scraper

    A command-line utility for taking automated screenshots of websites

  • I just released shot-scraper 1.0, which promises CLI interface stability (until 2.0) and introduces a couple of small new features:

    - https://github.com/simonw/shot-scraper/releases/tag/1.0

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts