webcrystal VS wikiteam

While it lacks a search feature last I checked there's always https://github.com/davidfstr/webcrystal

One .py file. Only one dependency (urllib3).n with a little love the concept could become a full transparent proxy.

From the the project page it says:

> The offline content is stored in ~/.cache/offpunk/ as plain .gmi/.html files. The structure of the Gemini-space is tentatively recreated. One key element of the design is to avoid any database. The cache can thus be modified by hand, content can be removed, used or added by software other than offpunk.

One ambition I have it to setup

https://github.com/davidfstr/webcrystal

> An archiving HTTP proxy and on-disk archival format for websites.

so that all my regular web browsing is auto archived at some level.

It would sure be neat if the archive formats could be compatible. It would allow for a setup where everything I’ve seen with my eyes is then immediately accessible programmatically or in a terminal. I feel that could open some significant productive advantages, especially in the age of LLMs also in the terminal.

Webcrystal?

WikiTeam is working on the archival, with the usual XML dumps and image dumps. You can follow updates and see how to help:

https://github.com/WikiTeam/wikiteam/issues/465#issuecomment...

https://wiki.archiveteam.org/index.php/Miraheze

Already before the announcement we had XML dumps for thousands of Miraheze wikis.

The thread linked has the information you need. Read me on the Github page for WikiTeam's dump generator.

ArchiveTeam WikiTeam has download tooling: https://github.com/WikiTeam/wikiteam

Hi all - I'm trying to archive a number of fandom pages. Upon checking out this subreddit, I've found a few ways of doing so, and am currently working with the WikiTeam python tool (https://github.com/WikiTeam/wikiteam)

Httrack is a good choice for having a local copy of the wiki you can browse personally, but note that if you ever have to back up a wiki in a formal suitable for migrating to another wiki site, something like ArchiveTeam's WikiTeam tool would be suitable. It also has a built-in tool to upload the resulting backup to archive.org, like how someone has done so with the MGQ wiki here.

I found ArchiveTeam's WikiTeam tool relatively easy to use. I just had to download the repository from github (from the Code: Download Zip in the top right), have Python installed, open a command prompt in the folder, copy-paste the commands from their front page, have it fail complaining about missing modules, look up the command to install Python modules, and install the modules it needs. Their tutorial has additional instructions for uploading the resulting archives to archive.org and for downloading lists of wikis.

If anyone has used this app please help me. I have followed the instruction in the readme.txt https://github.com/WikiTeam/wikiteam and I have the dumpgenerator.py but, when I run it with this commands: