quantum VS agenda

In addition, here's a good project for you to look at for cron (Crontab) type functionality you might want to borrow from: https://github.com/quantum-elixir/quantum-core

I'm using Quantum as my job runner in Elixir. Doesn't require even Postgres. Can't due that in Rails or other stacks:

https://github.com/quantum-elixir/quantum-core

^ obviously it won't fit every use case but for that there's Oban which is awesome:

https://getoban.pro

In some cases, we might want to run our code at specific times. Not necessarily every 3 minutes, but every day at 08.00AM. While this is perfectly achievable leveraging the same tools, I'll be pragmatic and recommend Quantum. It allows you to schedule the execution of functions with a cron syntax and takes away all the complexity of managing processes. It's a seasoned library, widely-adopted within the community, very lightweight... And extremely simple.

While many ways to write scalable Node.js applications exist, all narrow down to the underlying architecture. However, common hurdles in Node.js include serving static files, task scheduling, and utilizing available resources. To combat these, you can use a tactical approach. Because serving static assets with Node.js takes CPU time, a proxy content delivery network like CloudFront should help fix it. Recurring tasks need to fire events, which isn’t reliable with setTimeout or setInterval. As a solution, you can use a task scheduler framework like agenda.js. To handle resource utilization, you can use the cluster feature in Node.js, which listens to ports, accepts new connections, and distributes them across workers with built-in smarts to avoid overloading the process.

Now we have the materialized view and we need to update it daily. For this, we'll use Agenda, a light-weight job scheduling library for Node.js with Mongo backed persistence layer.

6. agenda

There are many job queue implementations for Node.js, including Bull, Bee, and Agenda.

Agenda JS is an alternative to bull that uses MongoDB rather than redis. Not saying one is better than the other, just giving alternatives.

If processing the file is going to take a relatively long time, you should send it to some asynchronous background process and immediately send a response to the user with status 202, then notify them later somehow if processing the file failed. Agenda is a good candidate for scheduling/managing more complex background tasks like this.

So I thought of 2 approaches, one using [agenda](https://github.com/agenda/agenda] and other using setTimeouts and storing the unique id for particular user in database. I am not sure which of the approach is scalable and maintainable in long run for thousands of users having daily activities.

i think it would be helpful if you provided some links to what you're talking about. How on earth are we supposed to help without knowing which agenda package you're referring to? I went through the agenda github repo and not one of those files mentioned in your post even exists in that repo........

For more advanced use cases, you might want to look into using a full-featured job queue instead like bree, bull, or agenda.