Updated: I've saved all of Wikipedia into a SQLITE database!

• PlainTextWikipedia

  6 261 1.2 Python

  Discontinued Convert Wikipedia database dumps into plaintext files

• wikitextparser

  1 268 9.0 Python

  A Python library to parse MediaWiki WikiText

The use of regex seems inefficient, is there any reason why you didn't start with lxml or a purpose built parser like wikitextparser?

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• DrQA

  1 4,467 0.0 Python

  Reading Wikipedia to Answer Open-Domain Questions

  

Nice work! That AI "framework" (to summarize the RAVEN acronym somehow) of yours reminds me of an old project of myself years ago, using prolog and first order logic to build a QA engine and pulling data from wikipedia. Something I eventually abandoned due to changing philosophical views on human consciousness... - yet it was still a fun learning exercise mixing compiler theory and logical inference. Facebook once open sourced code for something similar https://github.com/facebookresearch/DrQA - also pulling raw data from wikipedia.

• python-libzim

  2 54 7.2 Python

  Libzim binding for Python: read/write ZIM files in Python

  

https://github.com/openzim/python-libzim is the official one

Suggest a related project