FileTrove: A file indexer

This page summarizes the projects mentioned and recommended in the original post on /r/DataHoarder

InfluxDB - Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
featured
  • FileTrove

    FileTrove indexes files and creates metadata from them.

  • The most recent version, documentation and the source code can be found on Github: https://github.com/steffenfritz/FileTrove

  • filedriller

  • The 5.3GB is an old value in the documentation from the web site, I have to correct that, thanks. I got it down to 3.5GB. However, you are right, this is a huge number of SHA1s. I once had another tool, filedriller (https://github.com/steffenfritz/filedriller), that I presented at iPres and used a Redis db for that with less than 1GB. Redis handles that better than Bolt. On the other hand, NIST changed the NSRL format and range (in the sense of years and systems) of the RDS sets. So I think the overhead is not too big. We have over 62.500.000 hashes in it. I think that's ok :)

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
  • puremagic

    Pure python implementation of identifying files based off their magic numbers

  • My tool is focused on media and it has a few different scanning modes. It only uses exiftool with fsadd --image, ffprobe with either fsadd --video or fsadd --audio, and filetype via magic numbers using puremagic.

  • library

    70+ CLI tools to build, browse, and blend your media library. An index for your archive. (by chapmanjacobd)

  • okay https://github.com/chapmanjacobd/library

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts

  • Show HN: Find similar folders based on folder name, folder size, and count

    1 project | news.ycombinator.com | 29 Apr 2024
  • CurlyQ: Command line helper for curl and web scraping

    2 projects | news.ycombinator.com | 11 Jan 2024
  • Show HN: Merge folders and simulate merging–count of conflicts, trumps, and new

    2 projects | news.ycombinator.com | 7 Dec 2023
  • CLI HackerNews TV

    2 projects | /r/Python | 2 Nov 2022
  • Show HN: I built an open-source data copy tool called ingestr

    3 projects | news.ycombinator.com | 27 Feb 2024