Managing outdated pull requests is time-consuming. Mergify's Merge Queue automates your pull request management & merging. It's fully integrated to GitHub & coordinated with any CI. Start focusing on code. Try Mergify for free. Learn more →
Top 23 Python News Projects
-
-
Stream-Framework
Stream Framework is a Python library, which allows you to build news feed, activity streams and notification systems using Cassandra and/or Redis. The authors of Stream-Framework also provide a cloud service for feed technology:
An ultralight social media app with no dependencies that can run on shared web hosting. It's an API like Getstream, so F/E is up to you. I've had a fork of it in production for 2.5 years on a subscription site that generates a small income.
-
Sonar
Write Clean Python Code. Always.. Sonar helps you commit clean code every time. With over 225 unique rules to find Python bugs, code smells & vulnerabilities, Sonar finds the issues while you focus on the work.
-
trafilatura
Python & command-line tool to gather text on the Web: web crawling/scraping, extraction of text, metadata, comments
Project mention: Trafilatura: Python tool to gather text on the Web | news.ycombinator.com | 2023-08-14The feature list answers that question pretty well: https://github.com/adbar/trafilatura#features
Basically: you could implement all of this on top of BeautifulSoup - polite crawling policies, sitemap and feed parsing, URL de-duplication, parallel processing, download queues, heuristics for extracting just the main article content, metadata extraction, language detection... but it would require writing an enormous amount of extra code.
-
news-please
news-please - an integrated web crawler and information extractor for news that just works
Look at news-please, you can find it in GitHub. I did something similar and it was very helpful. You can hit me up with if you have any questions.
-
-
GNews
A Happy and lightweight Python Package that Provides an API to search for articles on Google News and returns a JSON response.
Project mention: Need some help with my personal project (interactive world map with real-time data) | /r/datascience | 2023-05-15The web crawling part wasn't much of an issue - I am using an existing API (https://pypi.org/project/gnews/) which does what I needed. The issue lies in, well, pretty much the rest of the task described above. I need to create an interactive world map with real-time data (news articles) - more specifically, maintaining the data server, figuring out the data mapping part, etc. Since I pretty much have no experience in this, I would like to ask you guys for some directions. What tool would I need to use and how would I store/load the data? Is it possible to do so without writing some Javascript code myself?
-
Project mention: Python script that opens my bookmarks and returns only links posted in the last 14 days | /r/learnpython | 2023-05-07
Another option you could consider would be using a wrapper library around google news if you struggle with implementing the scarping logic yourself. The downside is that you'll still have to be careful so your IP doesn't get blocked. Make sure you limit the amount of requests per second/minute...
-
Mergify
Tired of breaking your main and manually rebasing outdated pull requests?. Managing outdated pull requests is time-consuming. Mergify's Merge Queue automates your pull request management & merging. It's fully integrated to GitHub & coordinated with any CI. Start focusing on code. Try Mergify for free.
-
You could try django-newsfeed but this might make things harder in the long run if you want to make modifications to it.
-
Project mention: Ask HN: Why paywalled content is not banned yet? | news.ycombinator.com | 2023-08-26
Paywalled content is annoying. But it can be bypassed most of the time by cut'n'pasting the URL into the search fields of either
https://archive.is/ -
savepagenow
A simple Python wrapper and command-line interface for archive.org’s "Save Page Now" capturing service
-
-
Project mention: Speaking of AI image and text storytelling, there's now an AI-powered news website... | /r/behindthebastards | 2023-06-23
NewsNotFound website
-
JARVIS-GUI
Jarvis is a simple Chatbot with a GUI capable of chatting and retrieving information and daily news from the internet for it's user using python.
-
-
-
nepstonks
An automated bot that scrapes the latest upcoming issues, news, and investment opportunities that are announced inside Nepal and sends them to a telegram channel.
-
-
Project mention: Enhancing AI Capabilities with External API Integration and Assistance | /r/GPT3 | 2023-03-20
I am looking to do the same! My idea was a bit different but we share the vision. Happy to collaborate at open source project https://github.com/iamaliveai1/ailive
-
-
Project mention: Hacker News API VS python-client - a user suggested alternative | libhunt.com/r/API | 2023-08-04
NewsData.io is the best news API to get real-time and historical news data from the world. NewsData.io gathers news from more than 20,000+ news sources which cover around 154 countries in 77 languages. As of now, NewsData.io has access to over 100 million news articles which are gathered from 2018 till today. It is also the best alternative to Google News API.
-
-
Project mention: 68k.news: Basic HTML Google News for Vintage Computers | news.ycombinator.com | 2023-06-16
I share the frustration with the major online news portals, and have in fact built my own portal powered by Wikipedia[1].
But eventually I realized that my biggest gripe with news today isn't the presentation but the content. And I'm not talking about biases or sensationalism – I'm talking about the news items themselves.
Much of what passes as news today is stuff like "15 people die when a copper mine collapses in Chile". I'm trying to get a big picture view of the world, and I don't believe that such stories are at all conducive to that endeavor. News as we know it is just an endless stream of random events, apparently selected according to a handful of crude criteria, the most important one being dead people. I've been a keen follower of global news for many years, and I don't feel that I'm understanding anything.
Where are the truly novel approaches to painting a picture of what the world is today? Where are the quantitative news portals, the event pattern search engines, the automatically derived trends? I'm still looking.
-
-
InfluxDB
Collect and Analyze Billions of Data Points in Real Time. Manage all types of time series data in a single, purpose-built database. Run at any scale in any environment in the cloud, on-premises, or at the edge.
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
The latest post mention was on 2023-09-10.
Python News related posts
- Ask HN: Why paywalled content is not banned yet?
- archive.is reddit problem
- Show HN: Bookmarklet to open any web page in archive.is
- Ask HN: Do you pay to read news articles online?
- MITM on wan or facebook issue ?
- Birds Build Nests from Anti-Bird Spikes
- Article about more shows at Folsom field.
-
A note from our sponsor - Mergify
blog.mergify.com | 22 Sep 2023
Index
What are some of the best open-source News projects in Python? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | newspaper | 13,119 |
| 2 | Stream-Framework | 4,695 |
| 3 | trafilatura | 2,039 |
| 4 | news-please | 1,742 |
| 5 | pygooglenews | 1,179 |
| 6 | GNews | 384 |
| 7 | GoogleNews | 248 |
| 8 | django-newsfeed | 187 |
| 9 | archiveis | 166 |
| 10 | savepagenow | 152 |
| 11 | news-fetch | 151 |
| 12 | newsnotfound | 116 |
| 13 | JARVIS-GUI | 61 |
| 14 | newsemble | 44 |
| 15 | wallabag-kindle-consumer | 40 |
| 16 | nepstonks | 19 |
| 17 | JapanDailyNews | 12 |
| 18 | ailive | 10 |
| 19 | pressReadMePlease | 8 |
| 20 | python-client | 6 |
| 21 | YourDailyRundownBackend | 3 |
| 22 | pastevents | 3 |
| 23 | cabbage_news | 2 |
Collect and Analyze Billions of Data Points in Real Time
Manage all types of time series data in a single, purpose-built database. Run at any scale in any environment in the cloud, on-premises, or at the edge.
www.influxdata.com