Effectively monitoring very large number of files

• notify

  10 2,515 8.0 Rust

  🔭 Cross-platform filesystem notification library for Rust. (by notify-rs)

  

I'm most well versed in Python so I started with that and eventually ended up with a POC using watchdog, but the program took more than a day and a half to traverse everything and register all of the watches. I've been trying to learn Rust for a while, and decided to perhaps use this as an excuse to try something 'real' and not just a learning exercise. I found the notify crate (https://github.com/notify-rs/notify) and basically copied their example listed on their GitHub, but even this takes about 16-18 hours to place all the watches before it starts processing events. I did not see any obvious ways to enable asynchronous or async/await code in notify, so I don't know of a way to parallelize the disk I/O with this approach.

• redbpf

  27 1,689 6.1 Rust

  Discontinued Rust library for building and running BPF/eBPF modules

  

[0] https://github.com/foniod/redbpf

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project