tika-python
tldrstory
Our great sponsors
tika-python | tldrstory | |
---|---|---|
4 | 3 | |
1,411 | 344 | |
- | 0.6% | |
2.2 | 3.8 | |
10 days ago | 7 months ago | |
Python | Python | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
tika-python
-
Document Parsing - an unsolved problem?
At my previous job we had the same problem which we solved by using Tika. We called it on the server along with other stuff, but there is also a Python binding.
-
Extract text from PDF
Tika is from Apache so yes its original code base is Java but it has bindings in other languages. Checkout Tika-Python!
-
Extract text from documents
The Textractor instance is the main entrypoint for extracting text. This method is backed by Apache Tika, a robust text extraction library written in Java. Apache Tika has support for a large number of file formats: PDF, Word, Excel, HTML and others. The Python Tika package automatically installs Tika and starts a local REST API instance used to read extracted data.
tldrstory
-
Extract text from documents
['Introducing txtai, an AI-powered search engine built on Transformers Add Natural Language Understanding to any application Search is the base of many applications.', 'Once data starts to pile up, users want to be able to find it.', 'Itβs the foundation of the internet and an ever-growing challenge that is never solved or done.', 'The field of Natural Language Processing (NLP) is rapidly evolving with a number of new developments.', 'Large-scale general language models are an exciting new capability allowing us to add amazing functionality quickly with limited compute and people.', 'Innovation continues with new models and advancements coming in at what seems a weekly basis.', 'This article introduces txtai, an AI-powered search engine that enables Natural Language Understanding (NLU) based search in any application.', 'Introducing txtai txtai builds an AI-powered index over sections of text.', 'txtai supports building text indices to perform similarity searches and create extractive question-answering based systems.', 'txtai also has functionality for zero-shot classification.', 'txtai is open source and available on GitHub.', 'txtai and/or the concepts behind it has already been used to power the Natural Language Processing (NLP) applications listed below: β’ paperai β AI-powered literature discovery and review engine for medical/scientific papers β’ tldrstory β AI-powered understanding of headlines and story text β’ neuspo β Fact-driven, real-time sports event and news site β’ codequestion β Ask coding questions directly from the terminal Build an Embeddings index For small lists of texts, the method above works.', 'But for larger repositories of documents, it doesnβt make sense to tokenize and convert all embeddings for each query.', 'txtai supports building pre- computed indices which significantly improves performance.', 'Building on the previous example, the following example runs an index method to build and store the text embeddings.', 'In this case, only the query is converted to an embeddings vector each search.', 'https://github.com/neuml/codequestion https://neuspo.com/ https://github.com/neuml/tldrstory https://github.com/neuml/paperai Introducing txtai, an AI-powered search engine built on Transformers Add Natural Language Understanding to any application Introducing txtai Build an Embeddings index']
-
Tutorial series on txtai
tldrstory - AI-powered understanding of headlines and story text
-
Apply labels with zero-shot classification
tldrstory has full-stack implementation of a zero-shot classification system using Streamlit, FastAPI and Hugging Face Transformers. There is also a Medium article describing tldrstory and zero-shot classification.
What are some alternatives?
EasyOCR - Ready-to-use OCR with 80+ supported languages and all popular writing scripts including Latin, Chinese, Arabic, Devanagari, Cyrillic and etc.
txtai - π‘ All-in-one open-source embeddings database for semantic search, LLM orchestration and language model workflows
codequestion - π Semantic search for developers
layout-parser - A Unified Toolkit for Deep Learning Based Document Image Analysis
faiss - A library for efficient similarity search and clustering of dense vectors.
py-pdf-parser - A Python tool to help extracting information from structured PDFs.
paperai - π π€ Semantic search and workflows for medical/scientific papers
contextualized-topic-models - A python package to run contextualized topic modeling. CTMs combine contextualized embeddings (e.g., BERT) with topic models to get coherent topics. Published at EACL and ACL 2021.
sentence-transformers - Multilingual Sentence & Image Embeddings with BERT
magnitude - A fast, efficient universal vector embedding utility package.