Extract text from documents

This page summarizes the projects mentioned and recommended in the original post on dev.to

Our great sponsors
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • WorkOS - The modern identity platform for B2B SaaS
  • SaaSHub - Software Alternatives and Reviews
  • txtai

    💡 All-in-one open-source embeddings database for semantic search, LLM orchestration and language model workflows

    pip install txtai # Get test data wget -N https://github.com/neuml/txtai/releases/download/v2.0.0/tests.tar.gz tar -xvzf tests.tar.gz

  • tika-python

    Tika-Python is a Python binding to the Apache Tika™ REST services allowing Tika to be called natively in the Python community.

    The Textractor instance is the main entrypoint for extracting text. This method is backed by Apache Tika, a robust text extraction library written in Java. Apache Tika has support for a large number of file formats: PDF, Word, Excel, HTML and others. The Python Tika package automatically installs Tika and starts a local REST API instance used to read extracted data.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  • codequestion

    🔎 Semantic search for developers

    ['Introducing txtai, an AI-powered search engine built on Transformers Add Natural Language Understanding to any application Search is the base of many applications.', 'Once data starts to pile up, users want to be able to find it.', 'It’s the foundation of the internet and an ever-growing challenge that is never solved or done.', 'The field of Natural Language Processing (NLP) is rapidly evolving with a number of new developments.', 'Large-scale general language models are an exciting new capability allowing us to add amazing functionality quickly with limited compute and people.', 'Innovation continues with new models and advancements coming in at what seems a weekly basis.', 'This article introduces txtai, an AI-powered search engine that enables Natural Language Understanding (NLU) based search in any application.', 'Introducing txtai txtai builds an AI-powered index over sections of text.', 'txtai supports building text indices to perform similarity searches and create extractive question-answering based systems.', 'txtai also has functionality for zero-shot classification.', 'txtai is open source and available on GitHub.', 'txtai and/or the concepts behind it has already been used to power the Natural Language Processing (NLP) applications listed below: • paperai — AI-powered literature discovery and review engine for medical/scientific papers • tldrstory — AI-powered understanding of headlines and story text • neuspo — Fact-driven, real-time sports event and news site • codequestion — Ask coding questions directly from the terminal Build an Embeddings index For small lists of texts, the method above works.', 'But for larger repositories of documents, it doesn’t make sense to tokenize and convert all embeddings for each query.', 'txtai supports building pre- computed indices which significantly improves performance.', 'Building on the previous example, the following example runs an index method to build and store the text embeddings.', 'In this case, only the query is converted to an embeddings vector each search.', 'https://github.com/neuml/codequestion https://neuspo.com/ https://github.com/neuml/tldrstory https://github.com/neuml/paperai Introducing txtai, an AI-powered search engine built on Transformers Add Natural Language Understanding to any application Introducing txtai Build an Embeddings index']

  • tldrstory

    📊 Semantic search for headlines and story text

    ['Introducing txtai, an AI-powered search engine built on Transformers Add Natural Language Understanding to any application Search is the base of many applications.', 'Once data starts to pile up, users want to be able to find it.', 'It’s the foundation of the internet and an ever-growing challenge that is never solved or done.', 'The field of Natural Language Processing (NLP) is rapidly evolving with a number of new developments.', 'Large-scale general language models are an exciting new capability allowing us to add amazing functionality quickly with limited compute and people.', 'Innovation continues with new models and advancements coming in at what seems a weekly basis.', 'This article introduces txtai, an AI-powered search engine that enables Natural Language Understanding (NLU) based search in any application.', 'Introducing txtai txtai builds an AI-powered index over sections of text.', 'txtai supports building text indices to perform similarity searches and create extractive question-answering based systems.', 'txtai also has functionality for zero-shot classification.', 'txtai is open source and available on GitHub.', 'txtai and/or the concepts behind it has already been used to power the Natural Language Processing (NLP) applications listed below: • paperai — AI-powered literature discovery and review engine for medical/scientific papers • tldrstory — AI-powered understanding of headlines and story text • neuspo — Fact-driven, real-time sports event and news site • codequestion — Ask coding questions directly from the terminal Build an Embeddings index For small lists of texts, the method above works.', 'But for larger repositories of documents, it doesn’t make sense to tokenize and convert all embeddings for each query.', 'txtai supports building pre- computed indices which significantly improves performance.', 'Building on the previous example, the following example runs an index method to build and store the text embeddings.', 'In this case, only the query is converted to an embeddings vector each search.', 'https://github.com/neuml/codequestion https://neuspo.com/ https://github.com/neuml/tldrstory https://github.com/neuml/paperai Introducing txtai, an AI-powered search engine built on Transformers Add Natural Language Understanding to any application Introducing txtai Build an Embeddings index']

  • paperai

    📄 🤖 Semantic search and workflows for medical/scientific papers

    ['Introducing txtai, an AI-powered search engine built on Transformers Add Natural Language Understanding to any application Search is the base of many applications.', 'Once data starts to pile up, users want to be able to find it.', 'It’s the foundation of the internet and an ever-growing challenge that is never solved or done.', 'The field of Natural Language Processing (NLP) is rapidly evolving with a number of new developments.', 'Large-scale general language models are an exciting new capability allowing us to add amazing functionality quickly with limited compute and people.', 'Innovation continues with new models and advancements coming in at what seems a weekly basis.', 'This article introduces txtai, an AI-powered search engine that enables Natural Language Understanding (NLU) based search in any application.', 'Introducing txtai txtai builds an AI-powered index over sections of text.', 'txtai supports building text indices to perform similarity searches and create extractive question-answering based systems.', 'txtai also has functionality for zero-shot classification.', 'txtai is open source and available on GitHub.', 'txtai and/or the concepts behind it has already been used to power the Natural Language Processing (NLP) applications listed below: • paperai — AI-powered literature discovery and review engine for medical/scientific papers • tldrstory — AI-powered understanding of headlines and story text • neuspo — Fact-driven, real-time sports event and news site • codequestion — Ask coding questions directly from the terminal Build an Embeddings index For small lists of texts, the method above works.', 'But for larger repositories of documents, it doesn’t make sense to tokenize and convert all embeddings for each query.', 'txtai supports building pre- computed indices which significantly improves performance.', 'Building on the previous example, the following example runs an index method to build and store the text embeddings.', 'In this case, only the query is converted to an embeddings vector each search.', 'https://github.com/neuml/codequestion https://neuspo.com/ https://github.com/neuml/tldrstory https://github.com/neuml/paperai Introducing txtai, an AI-powered search engine built on Transformers Add Natural Language Understanding to any application Introducing txtai Build an Embeddings index']

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts