workgroup VS infra

I still think my worker pool is the best: https://github.com/carlmjohnson/workgroup

I have one that I’ve been using for a while and just updated to use Go 1.20 multierrors: https://github.com/carlmjohnson/workgroup

Read https://blog.carlmjohnson.net/post/share-memory-by-communicating/ and look at the code in https://github.com/carlmjohnson/workgroup for reference.

So far, just refactoring. I made a concurrency manager to simplify some stuff, and unified some pagination code. It’s like we said before generics: there are places you miss it, but not having it was never a total blocker. I think going working the x/slices x/maps packages will be the biggest time savers.

Yeah, I wrote my own generic concurrency thing, and it has two functions. There's a high level Process(nWorkers, task, manager, initial) and a lower level Start(nWorkers, task) (in chan<- Input, out <-chan Result[Input, Output]). The high level function is built off of the low level one and just queues jobs for the task workers by giving job results to the manager to see if it produces any more work. Context isn't involved in either because the idea is that the tasks should have their own internal contexts. You don't want to cancel the task runner, you want to cancel the tasks!

I've had to make some changes to our linter because of this. golangci-lint has disabled many of the linters, which helps until they're all updated. We've still had to force golangci-lint to use the correct Go version.