gather-cli
python-readability
gather-cli | python-readability | |
---|---|---|
1 | 5 | |
118 | 2,584 | |
- | - | |
5.5 | 3.4 | |
4 months ago | 12 months ago | |
Swift | Python | |
MIT License | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
gather-cli
-
Show HN: I made a tool to clean and convert any webpage to Markdown
I've been using gather-cli[0]for this, built by the venerable Brett Terpstra.
[0] https://github.com/ttscoff/gather-cli
python-readability
-
Show HN: I made a tool to clean and convert any webpage to Markdown
One of the cases when AI not needed. There is very good working algorithm to extract content from the pages, one of implementations: https://github.com/buriy/python-readability
- python-readability – extract and clean up HTML main body text and title
-
Is it possible scrape different websites with a single script
Is it mostly text articles? If yes, check this: https://github.com/buriy/python-readability
- O secretário de Estado que aprovou a Lei dos Eucaliptos tornou-se diretor-geral das celuloses
- Scrapy for financial news
What are some alternatives?
newspaper - newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:
python-goose - Html Content / Article Extractor, web scrapping lib in Python
html2text - Convert HTML to Markdown-formatted text.
textract - extract text from any document. no muss. no fuss.
sumy - Module for automatic summarization of text documents and HTML pages.
trafilatura - Python & command-line tool to gather text on the Web: web crawling/scraping, extraction of text, metadata, comments
sanitize - Bringing sanity to world of messed-up data
opengraph - A python module to parse the Open Graph Protocol
htmldate - Fast and robust date extraction from web pages, with Python or on the command-line
toapi - Every web site provides APIs.
JSONPATH - A query expression for extracting data from JSON.
Goose3 - A Python 3 compatible version of goose http://goose3.readthedocs.io/en/latest/index.html