Our great sponsors
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
You can probably do a good subset it in bash, it's just a nicer interface with a lot of configurability and several convenience features.
I'm generally a big fan of showing alternatives: https://github.com/Nukesor/pueue/?tab=readme-ov-file#similar...
Would you be willing to write a proper guide on how to do all of these things in bash? It would be great to have this as guide an alternative inside the Pueue wiki and link to it. It'll help people to make a more informed decision on whether they need this tool or not.
A similar tool I highly recommend: https://github.com/justanhduc/task-spooler
At first I thought it would just be a one-off tool I used for one of my projects, not until I discovered later that it has everything I need and became my daily driver ever since.
Pueue dumps the state of the queue to the disk as JSON every time the state changes, so when you have a lot of queued jobs this results in considerable disk io. I actually changed it to compress the state file via zstd which helped quite a bit but then eventually just moved on to running NATS [1] locally.
[1] https://nats.io/
I was running a lot of small tasks. With 3000 queued tasks my state file was around 25 MB.
zstd compressed that down to 5% of the size. I have the code still if you want to look at it but it was just a quick experiment so I didn't add any tests. I did add it to the config, disabled by default, though.
https://github.com/veyh/pueue/commit/e9dcf52227304b4b4a2ded4...