serviceq VS Benthos

https://github.com/gptankit/serviceq - a load balancer and queue (need http2 support) https://github.com/gptankit/go-wasm - wasm experiments in go (if you want to help community adapt to wasm using go)

If interested in load balancers/proxies and networking in general, you may want to look at https://github.com/gptankit/serviceq.

When you have so many metrics to consider, ideal choice is to assign weights to each metric and let the load balancer distribute load probabilistically (think weighted random). Implemented a load balancer in Go on similar lines (https://github.com/gptankit/serviceq) by calculating 'effective error' as weight from each node and inversely distributing load to the cluster. This ensures that a) one single node doesn't get overwhelmed with all the requests and b) even if some node is down, it keeps receiving a small percentage of requests (albeit lesser and lesser as time progresses).

I’ve used https://www.benthos.dev/ and it’s really easy and well implemented. The author is also very responsive

Looks cool. What is the difference between this tools and benthos (https://www.benthos.dev/)?

You might want to gradually replace that one with https://github.com/twmb/franz-go because Shopify is looking to find a new owner for Sarama and, until or if they do, it seems to be falling behind with maintenance: https://github.com/Shopify/sarama/issues/2461 For example, they still haven’t addressed this breaking change https://github.com/Shopify/sarama/issues/2358. franz-go has worked well so far in Benthos https://github.com/benthosdev/benthos/tree/main/internal/impl/kafka and it will likely end up as the only implementation once the Sarama-based one will be deprecated

I upload a mix of code reviews and live streams on https://www.youtube.com/@Jeffail, mostly building https://www.benthos.dev out in the open so the content ranges from beginner friendly stuff to more advanced things like stream processing, parser combinators, etc.