provisioning-backend VS taskq

I wrote a makefile which installs tools into PROJDIR/bin which is also in the gitignore.

Extremely simple in go, you just implement what is called Doer interface (one method). Here is an example from one of my projects, it is a simple decorator with logger in this case. Then you just initialize the client with this instance, in my project it is slightly more complex because I also setup OpenTelemetry but you get the idea.

Authentication and authorization is typically just a middleware which is essentially a function. Recently I implemented a RBAC functionality into our microservice which does not use any big framework. On our platform we have a RBAC service we need to call via REST. As you can see the whole patch is small if you exclude the generated OpenAPI client.

We use DAO/DAL in our app and I ran into the same problem - DAO does not work well with transactions. We have our own WithTransaction function which is currently only used within one model and that works fine. But the problem appears when we want to do a transaction across several models - that needs to be done in the business (service) layer.

Our project does have extensive makefile broken down into individual files so it is more readable. As you can see, we have targets for database, code quality, modules, testing, client generation, OpenAPI etc. It also has a trivial help:

Here is our code: https://github.com/RHEnVision/provisioning-backend/tree/main/pkg/worker

Our solution is simply one Redis queue, jobs have "type" (string) and are marshalled via Gob (for type safety) and there is no return value from job or error. This makes things extremely easy. We keep statistics (metrics) of job queue size and "in flight" jobs. Here is our implementation, just for inspiration. I suggest to write this on your own: https://github.com/RHEnVision/provisioning-backend/tree/main/pkg/worker

Hi, I'm starting a new project that involves distributing task on different (and possibly distributed) workers in Golang. So I'm looking for the best task queue library to use; for now the ones I like are the following (in no particular order): - asynq - machinery - taskq

+ taskq

I had a look at: 1. machinery - https://github.com/RichardKnop/machinery 2. go-celery - https://github.com/gocelery/gocelery 3. asynq - https://github.com/hibiken/asynq 3. taskq - https://github.com/vmihailenco/taskq

Oh my god, I have to deal with a similar system at work and I wish we hadn't tried to handroll this. If its possible you should look into something off-the-shelf. Could Celery work for 1000*10000 tasks? But if you must do it yourself, https://github.com/vmihailenco/taskq looks good

I was looking at https://github.com/vmihailenco/taskq to use potentially to manage tasks being picked up by a set of workers. Since we already have a Kakfa expertise I was looking to potentially use that instead.