For those running Go in production at scale, what do you use for distributed task queues?

• work

  1 2,207 0.0 Go

  Process background jobs in Go

  

  One of the gocraft/work maintainers here. Gocraft/work is not abandoned; we are just thinking what v2 could become, and working on v2: https://github.com/gocraft/work/issues/120#issuecomment-530615442

• nsq

  4 23,776 0.0 Go

  A realtime distributed messaging platform

  

  I've been using NSQ in production for a number of years now. It's also written in Go and has a first class client library maintained by the devs

• SonarQube

  www.sonarqube.org

  sponsored

  Static code analysis for 29 languages.. Your projects are multi-language. So is SonarQube analysis. Find Bugs, Vulnerabilities, Security Hotspots, and Code Smells so you can release quality code every time. Get started analyzing your projects today for free.

  

• machinery

  4 7,014 0.0 Go

  Machinery is an asynchronous task queue/job queue based on distributed message passing.

  Machinery (https://github.com/RichardKnop/machinery) seems to be the most popular out of the box solution. I've played with this in the past and it's worked pretty well but I'm not sure how it handles under serious volume.

• Asynq

  15 6,854 0.0 Go

  Simple, reliable, and efficient distributed task queue in Go

  Asynq (https://github.com/hibiken/asynq): Looks promising but not quite at v1.0 yet.

• gocelery

  0 2,218 0.0 Go

  Celery Distributed Task Queue in Go

  

  go-celery (https://github.com/gocelery/gocelery): Celery port in Go. Not much recent activity so a bit worried it's abandoned.

• cadence

  6 7,311 0.0 Go

  Cadence is a distributed, scalable, durable, and highly available orchestration engine to execute asynchronous long-running business logic in a scalable and resilient way.

  

  We use cadence https://github.com/uber/cadence

• gq

  0 24 0.0 Go

  gq is a lightweight scalable message queue backed by the most popular SQL DBs. Add message queue semantics to your application without integrating extra infrastructure.

  If you want something simple https://github.com/mattbonnell/gq

• InfluxDB

  www.influxdata.com

  sponsored

  Collect and Analyze Billions of Data Points in Real Time. Manage all types of time series data in a single, purpose-built database. Run at any scale in any environment in the cloud, on-premises, or at the edge.

  

• rabbit-hole

  0 383 0.0 Go

  RabbitMQ HTTP API client in Go

  We use Rabbitmq with streadway for the client and https://github.com/michaelklishin/rabbit-hole for some management and monitoring tasks.

• temporal

  6 8,112 0.0 Go

  Temporal service

  

  Use ScyllaDB if you need fast inserts but eventual consistency (just remember to set larger batch_size_fail_threshold) or CockroachDB for strong consistency. I assure you, the hardware requirements will be much smaller then with Cassandra.

• miniqueue

  0 169 0.0 Go

  A simple, single binary, message queue. Supports HTTP/2 and Redis Protocol.

  Have been working on https://github.com/tomarrell/miniqueue, intentionally non distributed though for simplicity

• docker-compose

  0 184 0.0 Shell

  Temporal docker-compose files (by temporalio)

  

Suggest a related project