extruct
datasette-scraper
extruct | datasette-scraper | |
---|---|---|
3 | 1 | |
821 | 57 | |
1.3% | - | |
3.8 | 2.5 | |
12 days ago | about 1 year ago | |
Python | Python | |
BSD 3-clause "New" or "Revised" License | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
extruct
-
GitHub – GSA/code-gov: An informative repo for all Code.gov repos
https://github.com/rushter/selectolax#simple-benchmark )
(Apache Nutch is a Java-based web crawler which supports e.g. CommonCrawl (which backs various foundational LLMs)) https://en.wikipedia.org/wiki/Apache_Nutch#Search_engines_bu... . But extruct extracts more types of metadata and data than Nutch AFAIU: https://github.com/scrapinghub/extruct )
datasette-graphql adds a GraphQL HTTP API to a SQLite database:
-
Alternative to extruct python library ? (scraping schema.org, jsonld, twitter and fb card)
Is there an alternative for extruct python library in golang ?
-
Scraping MMA fighter stats from a list of names
Seems like sherdog.com supports schema.org data markup - which is really easy to scrape! There's a brilliant python parser for https://github.com/scrapinghub/extruct.
datasette-scraper
-
GitHub – GSA/code-gov: An informative repo for all Code.gov repos
https://github.com/cldellow/datasette-scraper/#architecture
(TIL datasette-scraper parses HTML with selectolax; and Selectolax with Modest or Lexbor is ~25x faster at HTML parsing than BeautifulSoup in the selectolax benchmark:
What are some alternatives?
rdflib - RDFLib is a Python library for working with RDF, a simple yet powerful language for representing information.
kylo - Kylo is a data lake management software platform and framework for enabling scalable enterprise-class data lakes on big data technologies such as Teradata, Apache Spark and/or Hadoop. Kylo is licensed under Apache 2.0. Contributed by Teradata Inc.
PyLD - JSON-LD processor written in Python
code-json-generator - Automation that scrapes USEPA github and provides that metadata for code.gov
contextualise - Contextualise is an effective tool particularly suited for organising information-heavy projects and activities consisting of unstructured and widely diverse data and information resources
code-gov - An informative repo for all Code.gov repos
hugo-obsidian - simple GitHub action to parse Markdown Links into a .json file for Hugo
metatron - A Python 3.x HTML Meta tag parser, with emphasis on OpenGraph and complex meta tag schemes
PheKnowLator - PheKnowLator: Heterogeneous Biomedical Knowledge Graphs and Benchmarks Constructed Under Alternative Semantic Models
topic-db - TopicDB is a topic maps-based semantic graph store (using SQLite for persistence)