Data-extraction-and-text-analysis
tarsier
| Data-extraction-and-text-analysis | tarsier | |
|---|---|---|
| 1 | 7 | |
| 11 | 504 | |
| - | 7.3% | |
| 1.7 | 9.1 | |
| about 1 year ago | 9 days ago | |
| Jupyter Notebook | Jupyter Notebook | |
| - | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Data-extraction-and-text-analysis
-
Need help wrt NLP
Something like this solutionGitHub Ipynb file
tarsier
-
ScrapeGraphAI: Web scraping using LLM and direct graph logic
Agreed!
Apify's Website Content Crawler[0] does a decent job of this for most websites in my experience. It allows you to "extract" content via different built-in methods (e.g. Extractus [1]).
We currently use this at Magic Loops[2] and it works _most_ of the time.
The long-tail is difficult though, and it's not uncommon for users to back out to raw HTML, and then have our tool write some custom logic to parse the content they want from the scraped results (fun fact: before GPT-4 Turbo, the HTML page was often too large for the context window... and sometimes it still is!).
Would love a dedicated tool for this. I know the folks at Reworkd[3] are working on something similar, but not sure how much is public yet.
[0] https://apify.com/apify/website-content-crawler
[1] https://github.com/extractus/article-extractor
[2] https://magicloops.dev/
[3] https://reworkd.ai/
- Control the browser using GPT-4 vision by AgentGPT team
- Show HN: GPT-4 vision utilities to browse the web
What are some alternatives?
datadoubleconfirm - Simple datasets and notebooks for data visualization, statistical analysis and modelling - with write-ups here: http://projectosyo.wix.com/datadoubleconfirm.
python-awesome - Learn Python, Easy to learn, Awesome
yellowpage-scraper - It is create to scrape yellowpages.com
article-extractor - To extract main article from given URL with Node.js
HackerNEWS-Simplified - A more simplified, straightforward, and plain version of Hacker News.
html2text - Convert HTML to Markdown-formatted text.
udemyscraper - A Udemy Course Scraper built with bs4 and selenium, that fetches udemy course information. Get udemy course information and convert it to json, csv or xml file, without authentication!
acgn-bot - Telegram bot: Check anime/comic/game/novel websites update
AutoCrawler - Official implement of paper "AutoCrawler: A Progressive Understanding Web Agent for Web Crawler Generation"
substituicoes_br2021 - Scraping do fbref.com e análise das substituições no Brasileirão 2021.
dude - dude uncomplicated data extraction: A simple framework for writing web scrapers using Python decorators