Indexing

Top 23 Indexing Open-Source Projects

  • google-indexing-script

    Script to get your site indexed on Google in less than 48 hours

  • Project mention: Google Indexing Script | news.ycombinator.com | 2024-01-28
  • FASTER

    Fast persistent recoverable log and key-value store + cache, in C# and C++.

  • Project mention: A MySQL compatible database engine written in pure Go | news.ycombinator.com | 2024-04-09

    You would be surprised by performance of modern .NET :)

    Writing no-alloc is oftentimes done by reducing complexity and not doing "stupid" tricks that actually work against JIT and CoreLib features.

    For databases specifically, .NET is actually positioned very well with its low-level features (intrisics incl. SIMD, FFI, struct generics though not entirely low-level) and high-throughput GC.

    Interesting example of this applied in practice is Garnet[0]/FASTER[1]. Keep in mind that its codebase still consist of un-idiomatic C# and you can do way better by further simplification, but it already does the job well enough.

    [0] https://github.com/microsoft/garnet

    [1] https://github.com/microsoft/FASTER

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
  • Toshi

    A full-text search engine in rust

  • Project mention: Tantivy 0.20 is released: Schemaless column store, Schemaless aggregations, Phrase prefix queries, Percentiles, and more... | /r/rust | 2023-06-20

    I don't think you have an active project that addresses all those use cases. There was an attempt in Rust with Toshi that is built on top of tantivy, but the project seems to have stalled.

  • ugrep

    NEW ugrep 5.1: an ultra fast, user-friendly, compatible grep. Ugrep combines the best features of other grep, adds new features, and searches fast. Includes a TUI and adds Google-like search, fuzzy search, hexdumps, searches nested archives (zip, 7z, tar, pax, cpio), compressed files (gz, Z, bz2, lzma, xz, lz4, zstd, brotli), pdfs, docs, and more

  • Project mention: Ugrep – a more powerful, ultra fast, user-friendly, compatible grep | news.ycombinator.com | 2023-12-30
  • roaring

    Roaring bitmaps in Go (golang), used by InfluxDB, Bleve, DataDog

  • column

    High-performance, columnar, in-memory store with bitmap indexing in Go

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

    WorkOS logo
  • PumpkinDB

    Immutable Ordered Key-Value Database Engine

  • kernel-memory

    Index and query any data using LLM and natural language, tracking sources and showing citations.

  • Project mention: Open source alternative to ChatGPT and ChatPDF-like AI tools | news.ycombinator.com | 2023-12-09

    about #3 I’ll recommend https://github.com/microsoft/kernel-memory :)

  • hypopg

    Hypothetical Indexes for PostgreSQL

  • Project mention: Show HN: PostgreSQL Index Advisor | news.ycombinator.com | 2024-04-13

    It works particularly well with pg_stat_statements[0] which tracks execution statistics of all SQL statements executed on your Postgres database.

    It leans heavily on HypoPG[1], an excellent extension to determine if PostgreSQL will use a given index without spending resources to create them.

    [0] pg_stat_statements: https://www.postgresql.org/docs/current/pgstatstatements.htm...

    [1] https://github.com/HypoPG/hypopg

  • trueblocks-core

    The main repository for the TrueBlocks system

  • Project mention: How to store large amounts of blockchain data for analysis and low-latency querying? | /r/dataengineering | 2023-06-03

    search for trueblocks https://github.com/TrueBlocks/trueblocks-core

  • PGM-index

    🏅State-of-the-art learned data structure that enables fast lookup, predecessor, range searches and updates in arrays of billions of items using orders of magnitude less space than traditional indexes

  • Project mention: Self-indexing RDBMS? Could AI help? | /r/Database | 2023-04-26

    PGM Index

  • AlphabetIndex Fast Scroll RecyclerView

    A Powerful AlphabetIndex FastScroller Library for Android's RecyclerView!

  • spotweb

    Decentralized community

  • Project mention: Stopweb not indexing old spots on Truenas Scale | /r/usenet | 2023-07-06

    Have you tried this? https://github.com/spotweb/spotweb/issues/805

  • SeqAn

    SeqAn's official repository.

  • pyfaidx

    Efficient pythonic random access to fasta subsequences

  • tg-index

    Python web app to index telegram chats and serve its files for download over HTTP.

  • summa

    Full-text IPFS-friendly and WASM-compatible Search in Rust

  • substrate-archive

    Blockchain Indexing Engine

  • StarWarsArrays.jl

    Arrays indexed as the order of Star Wars movies

  • Project mention: Star Wars Arrays | news.ycombinator.com | 2023-05-17
  • idx

    maps, sets and vectors with on-demand secondary indexes.

  • sdsl-lite

    Succinct Data Structure Library 3.0 (by xxsds)

  • orient

    CLI/GUI file searcher combining find, locate and Everything, plus content searching. Works on Linux, Windows and macOS. This repo is for CLI/Library.

  • Project mention: `orient` File Searcher - Drop-in Cross-platform Replacement for `Everything`, as Fast as `plocate`, plus Content Searches. Has both CLI and GUI. | /r/coolgithubprojects | 2023-05-11
  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2024-04-13.

Indexing related posts

Index

What are some of the best open-source Indexing projects? This list will help you:

Project Stars
1 google-indexing-script 6,481
2 FASTER 6,194
3 Toshi 4,110
4 ugrep 2,422
5 roaring 2,340
6 js-search 2,182
7 column 1,373
8 PumpkinDB 1,366
9 kernel-memory 1,150
10 hypopg 1,106
11 trueblocks-core 1,017
12 PGM-index 747
13 AlphabetIndex Fast Scroll RecyclerView 596
14 spotweb 521
15 SeqAn 454
16 pyfaidx 438
17 tg-index 361
18 summa 206
19 substrate-archive 194
20 StarWarsArrays.jl 122
21 idx 78
22 sdsl-lite 76
23 orient 61

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com