wikiextractor

A tool for extracting plain text from Wikipedia dumps (by attardi)

Wikiextractor Alternatives

Similar projects and alternatives to wikiextractor

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a better wikiextractor alternative or higher similarity.

wikiextractor reviews and mentions

Posts with mentions or reviews of wikiextractor. We have used some of these posts to build our list of alternatives and similar projects. The last one was on 2022-06-13.
  • Letter and next letter frequencies for 24 languages (see comments for non-English plots) [OC]
    3 projects | /r/dataisbeautiful | 13 Jun 2022
    Larger text corpora: each plot is generated from around 450MB of Wikipedia article text (or as much as is available), extracted using wikiextractor.
  • Most similar language to each European language, based purely on letter distribution [OC]
    1 project | /r/dataisbeautiful | 8 Jun 2022
    Methodology: extracted 100MB of article texts from each of the different Wikipedias using https://github.com/attardi/wikiextractor, and counted the character prevalences using Python. The similarity measure is just the sum of the absolute differences in character prevalences (so a lower score means more similar): e.g. if language A has distribution {A: 0.5, B: 0.3, C: 0.2} and language B has distribution {A: 0.8, B: 0.2} then their similarity is |0.5-0.8|+|0.3-0.2|+|0.2-0.0|=0.6. The final chart was generated using graphviz and pillar.
  • Finding a English Wikipedia dump
    1 project | /r/LanguageTechnology | 5 Aug 2021
    With the help of wikiextractor, i was able to query it and process the dump. However, when i start inspecting it, some articles are empty. For example AccessibleComputing should not be empty, but the dump gave: ``` AccessibleComputing 0 10 854851586 2021-01-23T15:15:01Z Elli shel wikitext text/x-wiki #REDIRECT [[Computer accessibility]]
  • A note from our sponsor - InfluxDB
    www.influxdata.com | 8 May 2024
    Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality. Learn more →

Stats

Basic wikiextractor repo stats
3
3,637
0.0
4 months ago

attardi/wikiextractor is an open source project licensed under GNU Affero General Public License v3.0 which is an OSI approved license.

The primary programming language of wikiextractor is Python.

Popular Comparisons


Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com