Our great sponsors
-
SCrawler
🏳️🌈 Media downloader from any sites, including Twitter, Reddit, Instagram, Threads, Facebook, OnlyFans, YouTube, Pinterest, PornHub, XHamster, XVIDEOS, ThisVid etc.
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
gallery-dl
Discontinued Command-line program to download image galleries and collections from several image hosting sites
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
Any of these: ripme, wfdownloader, gallery-dl would also work on twitter.
Any of these: ripme, wfdownloader, gallery-dl would also work on twitter.
Here you go. I have used this to archive hundreds of subreddits in their entity, even bypassing the 1000 limit.
URL = u'https://twitter.com/{username}'.format(username=username) request = requests.get(URL) page_source = request.text soup = BeautifulSoup(page_source, 'lxml') code_tweets_content = soup('p', {'class': 'js-tweet-text'}) code_tweets_time = soup('span', {'class': '_timestamp'}) code_tweets_ID = soup('a', {'tweet-timestamp'})