witokit

A Python toolkit to generate a tokenized dump of Wikipedia for NLP (by akb89)
Review Wikipedia wikipedia-dump Dump NLP tokenize Multilingual
Source Code
Suggest alternative
Edit details
Save 47% on cloud hosting with autoscaling that just works
Judoscale integrates with Django, FastAPI, Celery, and RQ to make autoscaling easy and reliable. Save big, and say goodbye to request timeouts and backed-up task queues.
judoscale.com
featured

Witokit Alternatives

Similar projects and alternatives to witokit

akb89
witokit

  1. wp2git

    3 witokit VS wp2git

    Downloads and imports Wikipedia page histories to a git repository

  2. Judoscale

    Save 47% on cloud hosting with autoscaling that just works. Judoscale integrates with Django, FastAPI, Celery, and RQ to make autoscaling easy and reliable. Save big, and say goodbye to request timeouts and backed-up task queues.

    Judoscale logo

  3. wiki_dump

    1 witokit VS wiki_dump

    A library that assists in traversing and downloading from Wikimedia Data Dumps and their mirrors.

  4. Sentimentanalysis

    2 witokit VS Sentimentanalysis

    Language independent sentiment analysis

  5. wit

    5 witokit VS wit

    WIT (Wikipedia-based Image Text) Dataset is a large multimodal multilingual dataset comprising 37M+ image-text sets with 11M+ unique images across 100+ languages. (by google-research-datasets)

  6. codex

    1 witokit VS codex

    CoDEx: A set of knowledge graph Completion Datasets Extracted from Wikidata and Wikipedia (by tsafavi)

  7. trankit

    1 witokit VS trankit

    Trankit is a Light-Weight Transformer-based Python Toolkit for Multilingual Natural Language Processing

  8. NLP-Model-for-Corpus-Similarity

    1 witokit VS NLP-Model-for-Corpus-Similarity

    A NLP algorithm I developed to determine the similarity or relation between two documents/Wikipedia articles. Inspired by the cosine similarity algorithm and built from WordNet.

  9. InfluxDB

    InfluxDB high-performance time series database. Collect, organize, and act on massive volumes of high-resolution data to power real-time intelligent systems.

    InfluxDB logo

  10. wikiteam

    23 witokit VS wikiteam

    Tools for downloading and preserving wikis. We archive wikis, from Wikipedia to tiniest wikis. As of 2025, WikiTeam has preserved more than 600,000 wikis.

  11. laserembeddings

    2 witokit VS laserembeddings

    LASER multilingual sentence embeddings as a pip package

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a better witokit alternative or higher similarity.
Suggest an alternative to witokit

witokit discussion

Log in or Post with

witokit reviews and mentions

Posts with mentions or reviews of witokit. We have used some of these posts to build our list of alternatives and similar projects.

Stats

Basic witokit repo stats
1
11
0.0
12 months ago

akb89/witokit is an open source project licensed under MIT License which is an OSI approved license.

The primary programming language of witokit is Python.

Popular Comparisons

  1. witokit VS wp2git
  2. witokit VS wiki_dump
  3. witokit VS Sentimentanalysis
  4. witokit VS wit
  5. witokit VS codex
  6. witokit VS trankit
  7. witokit VS NLP-Model-for-Corpus-Similarity
  8. witokit VS wikiteam
  9. witokit VS laserembeddings

Sponsored
Save 47% on cloud hosting with autoscaling that just works
Judoscale integrates with Django, FastAPI, Celery, and RQ to make autoscaling easy and reliable. Save big, and say goodbye to request timeouts and backed-up task queues.
judoscale.com

Did you know that Python is
the 2nd most popular programming language
based on number of references?

Loading...