selectolax
xmltodict
Our great sponsors
selectolax | xmltodict | |
---|---|---|
6 | 7 | |
967 | 5,380 | |
- | - | |
7.7 | 0.6 | |
about 2 months ago | 3 months ago | |
Cython | Python | |
MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
selectolax
-
GitHub – GSA/code-gov: An informative repo for all Code.gov repos
https://github.com/rushter/selectolax#simple-benchmark )
(Apache Nutch is a Java-based web crawler which supports e.g. CommonCrawl (which backs various foundational LLMs)) https://en.wikipedia.org/wiki/Apache_Nutch#Search_engines_bu... . But extruct extracts more types of metadata and data than Nutch AFAIU: https://github.com/scrapinghub/extruct )
datasette-graphql adds a GraphQL HTTP API to a SQLite database:
-
8 Most Popular Python HTML Web Scraping Packages with Benchmarks
selectolax
- High performance code in Python
-
Web Scraping with Python: Everything you need to know to get started (2022)
try this... https://github.com/rushter/selectolax
-
The State of Web Scraping in 2021
Lazyweb link: https://github.com/rushter/selectolax
although I don't follow the need to have what appears to be two completely separate HTML parsing C libraries as dependencies; seeing this in the readme for Modest gives me the shivers because lxml has _seen some shit_
> Modest is a fast HTML renderer implemented as a pure C99 library with no outside dependencies.
although its other dep seems much more cognizant about the HTML5 standard, for whatever that's worth: https://github.com/lexbor/lexbor#lexbor
---
> It looks like the author of the article just googled some libraries for each language and didn't research the topic
Heh, oh, new to the Internet, are you? :-D
- Show HN: Fast HTML5 parser for Python with multiple backends
xmltodict
-
XML to CSV or JSON using Cloud Function
Your Cloud Function would be written in Node.js, Python, Go, Java, C#, Ruby, or PHP; pick the one you're most comfortable with. It would get the name and bucket of the newly uploaded XML file as an input parameter. It would then load the file and call a library that makes the conversion. Example libraries: xml-js (for Node), xmltodict (for Python).
-
Did I reinvent a wheel?
Go with xmltodict. Works pretty fine, and you just have to drop any key begining with @ or # (if there is not already an option for that).
-
Top python libraries/ frameworks that you suggest every one
Nope, sorry, it's just an XML generator. The Python stdlib offers https://docs.python.org/3/library/xml.etree.elementtree.html and PyPI offers https://github.com/martinblech/xmltodict for parsing, and you could write CSV with csvwriter or pandas.
- Dict or List to store table like data
-
Like JQ, but for HTML
xmlstarlet is really nothing like jq, as a language. But yes, I use it because it is the best commandline xml processor I'd found. That's the only similarity to jq.
Is this the yq? https://kislyuk.github.io/yq/ It does contain an 'xq', as a literal wrapper for jq, piping output into it after transcoding XML to JSON using xmltodict https://github.com/martinblech/xmltodict (which explodes xml into separate JSON data structures).
This is a bash one-liner! But TBF it really is a 'jq for xml'. I think it would be horrible for some things, but you could also do a lot of useful things painlessly.
- Parsing unknown XML file with Python?
-
I used raw data from my watch (and Python) to make a map of all the NH48 hikes from this year. I hiked Liberty and Flume before I got the watch in June, so I need to do those again! Color-coded by altitude.
Super-easy, take a look at xmltodict https://github.com/martinblech/xmltodict xmltodict.parse(xml_str) gets you a dictionary
What are some alternatives?
lxml - The lxml XML toolkit for Python
lexbor - Lexbor is development of an open source HTML Renderer library. https://lexbor.com
untangle - Converts XML to Python objects
html5lib - Standards-compliant library for parsing and serializing HTML documents and fragments in Python
MarkupSafe - Safely add untrusted strings to HTML/XML markup.
pyppeteer - Headless chrome/chromium automation library (unofficial port of puppeteer)
pyquery - A jquery-like library for python
xhtml2pdf - A library for converting HTML into PDFs using ReportLab
gazpacho - 🥫 The simple, fast, and modern web scraping library
xmldataset - xmldataset: xml parsing made easy 🗃️