redis VS Bee-Queue

services: - type: web name: myapp-web env: ruby region: frankfurt # or oregon plan: starter numInstances: 1 buildCommand: ./bin/render-build.sh startCommand: REDIS_URL="redis://${REDIS_HOST}" bundle exec puma -C config/puma.rb domains: - example.com # replace with your domain name envVars: - key: DATABASE_URL fromDatabase: name: myapp-db property: connectionString - key: REDIS_HOST fromService: name: myapp-redis type: pserv property: hostport - key: RAILS_MASTER_KEY sync: false - type: worker name: myapp-sidekiq env: ruby region: frankfurt # or oregon plan: starter buildCommand: bundle install && bundle exec rake assets:precompile startCommand: REDIS_URL="redis://${REDIS_HOST}" bundle exec sidekiq -e production envVars: - key: DATABASE_URL fromDatabase: name: myapp-db property: connectionString - key: REDIS_HOST fromService: name: myapp-redis type: pserv property: hostport - key: RAILS_MASTER_KEY sync: false - type: pserv name: myapp-redis env: docker region: frankfurt # or oregon repo: https://github.com/render-examples/redis.git numInstances: 1 disk: name: myapp-redis-data mountPath: /var/lib/redis sizeGB: 1 databases: - name: myapp-db plan: starter region: frankfurt # or oregon

You could use a Redis based job system like https://github.com/bee-queue/bee-queue for maintenance and email. The jobs could be populated by the API service, some cron scripts, and whatever else, while you'd have dedicated worker processes (written in Node.js?) pulling the jobs and running them.

The Node.js libraries BullMQ and Bee Queue are also infrastructure and interface. However, you typically have to use their interface.

I'm a fan of bee-queue for this when working in Node, but it's just a wrapper around reliable queue patterns in Redis; you might want to investigate how it works (the readme is quite good) for your own use case, or find an equivalent for your environment. The key is it's designed for short tasks (seconds long, not minutes or hours), and you can receive a result back from the task unlike some fire-and-forget queueing solutions.

I'd also consider something like bee-queue, which would enable you to scale across processes or even across multiple server machines.

For NodeJS, there are BullMQ(successor of Bull), Bull and Bee Queue

bee-queue for offloading tasks onto non-web-server machines