TWINT VS gallery-dl

twint is a project that can scrape twitter data via the webpages rather than the twitter API, which means that it can get more than the last 3200 tweets of an account. Unfortunately it seems that the repo was archived and is no longer in development, so I'm not sure if it even still works. It's also a bit heavy on dependencies and is written in Python, neither of which make it easier to install and use.

The tool doesn't work anymore since Twitter changed its APIs, but a good example is twint. Most people in OSINT are not highly technical and don't know their way around a CLI. On the other hand, a CLI tool is one of the quickest, lowest (dev) cost ways to release a tool to the public, and many developers who build tools for the OSINT community do so for free (open source).

You can try twint which is extensive and should be able to do that. Another is via this twitter downloader but might require multiple runs depending on what you want to archive.

They do have examples that don't specify a username such as number 3 on this page or this one on the main page: "twint -g="48.880048,2.385939,1km" -o file.csv --csv - Scrape Tweets from a radius of 1km around a place in Paris and export them to a csv file."

This will motivate researchers to web scrape to circumvent these restrictions. Twint can scrape tweets and it supports proxies. It can also be multi threaded. A huge hassle and it's prone to breaking when the site changes.

The comment I saw said they used this: https://github.com/twintproject/twint

If you want to grab all the images and videos take a look at gallery-dl, found here https://github.com/mikf/gallery-dl

My question is about the docs. Here is the link to gallery-dl on github. from what i see it's kept up-to-date. I'm trying to download a post from ig through any method. Also thinking of creating a configuration-file if necessary. My OS: windows 10, i have worked with a cli before but on linux and my knowledge is very superficial. I have managed to use this tool on other websites like pinterest but can't think of any solutions for instagram as of now. The errors i get are about using the authentication method and when trying to set the cookies-from-browser option.

I could set up a redirect to the '/raw/' pages but then the syntax highlighting is gone.

The same page is perfectly viewable over plain html on gothub[2] though.

Github also seems to be hiding their "Assets" (binaries et al) on the "/releases" page for some projects behind javascript(especially older versions).[3] Something else that wasn't the case about ~1.5 years ago.

Would be great if gothub could unshackle the links to those as well[4], but that appears to not be working at the moment[5] .

This project appears to be a more performant(measurably so), more privacy friendly(as Microsoft won't have a record of your interest in certain projects) alternative front-end for "non logged in" github users.

I like it, but it still needs work.

[1] https://github.com/mackyle/sqlite/blob/18cf47156abe94255ae14...

[2] https://gh.bloatcat.tk/mackyle/sqlite/blob/18cf47156abe94255...

[3] https://github.com/mikf/gallery-dl/releases

[4] https://gh.bloatcat.tk/mikf/gallery-dl/releases

[5] https://codeberg.org/gothub/gothub/issues/74

since it's primarily a picture/video platform, you can use gallery-dl to scrape the feeds you want to follow if you provide authentication, and you should be able to write the captions and stuff to metadata files, but not sure how you'd display it in a useful way.

you could use gallery-dl to do this as well.

gallery-dl (CLI based) focuses on images.

Maybe there is a way to set specific configs only for reddit gallery images, but I'm not sure how it works. Gallery-dl's documentation did not help me, and I also tried to find information from the source code.

I use https://github.com/mikf/gallery-dl

You can try image scraping with something like gallery-dl or grabber. I use grabber for stuff off of booru style sites (anime weeb shit), and grabber for everything else. The challenge with these tools is: