wikiteam
mwparserfromhell
| wikiteam | mwparserfromhell | |
|---|---|---|
| 23 | 5 | |
| 688 | 703 | |
| 1.3% | - | |
| 3.8 | 7.2 | |
| about 1 month ago | 4 months ago | |
| Python | Python | |
| GNU General Public License v3.0 only | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
wikiteam
-
Miraheze to Shut Down
WikiTeam is working on the archival, with the usual XML dumps and image dumps. You can follow updates and see how to help:
https://github.com/WikiTeam/wikiteam/issues/465#issuecomment...
https://wiki.archiveteam.org/index.php/Miraheze
Already before the announcement we had XML dumps for thousands of Miraheze wikis.
- Dan Parker has accidentally deleted Yugipedia without recent backup
-
Questions about mirroring fandom/wiki sites
The thread linked has the information you need. Read me on the Github page for WikiTeam's dump generator.
- WikiTeam: We archive wikis, from Wikipedia to tiniest wikis
- PSA: Fandom has acquired GameSpot, GameFAQ’s, metacritic and more.
-
Best way to archive a wiki "Powered by MediaWiki"
ArchiveTeam WikiTeam has download tooling: https://github.com/WikiTeam/wikiteam
-
Archiving Wiki (Fandom) Pages
Hi all - I'm trying to archive a number of fandom pages. Upon checking out this subreddit, I've found a few ways of doing so, and am currently working with the WikiTeam python tool (https://github.com/WikiTeam/wikiteam)
-
[Censorship] Fandom Wiki (formerly Wikia) is deleting wikis on sexual topics November 24, such as the Monster Girl Encyclopedia wiki
Httrack is a good choice for having a local copy of the wiki you can browse personally, but note that if you ever have to back up a wiki in a formal suitable for migrating to another wiki site, something like ArchiveTeam's WikiTeam tool would be suitable. It also has a built-in tool to upload the resulting backup to archive.org, like how someone has done so with the MGQ wiki here.
-
Fandom Wiki (formerly Wikia) is deleting wikis on sexual topics in 2 weeks
I found ArchiveTeam's WikiTeam tool relatively easy to use. I just had to download the repository from github (from the Code: Download Zip in the top right), have Python installed, open a command prompt in the folder, copy-paste the commands from their front page, have it fail complaining about missing modules, look up the command to install Python modules, and install the modules it needs. Their tutorial has additional instructions for uploading the resulting archives to archive.org and for downloading lists of wikis.
-
I need help with WikiTeam
If anyone has used this app please help me. I have followed the instruction in the readme.txt https://github.com/WikiTeam/wikiteam and I have the dumpgenerator.py but, when I run it with this commands:
mwparserfromhell
- FLaNK AI Weekly for 29 April 2024
-
Processing Wikipedia Dumps With Python
There's also https://github.com/earwig/mwparserfromhell, if you don't want to roll your own.
-
[Python] How can I clean up Wikipedia's XML backup dump to create dictionaries of commonly used words for multiple languages?
In particular what you're looking at is not XML but wikitext. I found a discussion on stackoverflow about solving the same problem of getting text from wikitext. Seems like the most promising solution in Python since you already have the dump is to run each page through mwparserfromhell. According to the top stackoverflow answer you could use something like
-
How can I clean up Wikipedia's XML backup dump to create dictionaries of commonly used words for multiple languages?
Thank you so much! I was actually talking about the markup language within the text. Turns out it's proprietary to WikiMedia and user lowerthansound kindly suggested I use this: https://github.com/earwig/mwparserfromhell
What are some alternatives?
ArchiveBox - 🗃 Open source self-hosted web archiving. Takes URLs/browser history/bookmarks/Pocket/Pinboard/etc., saves HTML, JS, PDFs, media, and more...
wikitextparser - A Python library to parse MediaWiki WikiText
webscrapbook - A browser extension that captures web pages to local device or backend server for future retrieval, organization, annotation, and edit. This project inherits from legacy Firefox add-on ScrapBook X.
archwiki - MediaWiki used on Arch Linux websites (read-only mirror)
reddit-save - A Python tool for backing up your saved and upvoted posts on reddit to your computer.
WiktionaryParser - A Python Wiktionary Parser
diskimageprocessor - Tool for automated processing of disk images in BitCurator
pywikibot - A Python library that interfaces with the MediaWiki API. This is a mirror from gerrit.wikimedia.org. Do not submit any patches here. See https://www.mediawiki.org/wiki/Developer_account for contributing.
rexport - Reddit takeout: export your account data as JSON: comments, submissions, upvotes etc. 🦖
isbntools - python app/framework for 'all things ISBN' including metadata, descriptions, covers...
bitwarden-to-keepass - Export (most of) your Bitwarden items into KeePass (kdbx) database. That includes logins - with TOTP seeds, URIs, custom fields, attachments and secure notes
wiki_dump - A library that assists in traversing and downloading from Wikimedia Data Dumps and their mirrors.