aperture VS Echo

Anyone interested in load shedding and graceful degradation with request prioritization should check out the Aperture OSS project.

https://github.com/fluxninja/aperture

I agree that queues can problem especially when misconfigured. But some amount of queuing is necessary, to absorb short spikes in demand vs capacity. Also, queues can be helpful to re-order requests based on criticality which won't be possible with zero queue size - in which case we have to immediately drop a request or admit it without considering it's priority.

I think it is beneficial to re-think how we tune queues. Instead of setting a queue size, we should be tuning the max permissible latency in the queue which is what a request timeout actually is. That way, you stay within the acceptable response time SLA while keeping only the serve-able requests in the queue.

Aperture, an open-source load management platform took this approach. Each request specifies a timeout for which it is willing to stay in the queue. And weighted fair queuing scheduler then allocates the capacity (a request quota or max number of in-flight request) across requests based on the priority and tokens (request heaviness) of each request.

Read more about the WFQ scheduler in Aperture: https://docs.fluxninja.com/concepts/scheduler

Link to Aperture's GitHub: https://github.com/fluxninja/aperture

Would love to hear your thoughts on our approach!

For those keen to dive deeper, I highly recommend exploring both the Twitter Space and Aperture: [Twitter Spaces]: https://twitter.com/kelseyhightower/status/1689355284802629633?s=20 [GitHub repo]: https://github.com/fluxninja/aperture

Very interesting blog post! Our team has been working intensively in this area for the last couple of years - flow control, load shedding, controllability (PID control), and so on.

We have open-sourced our work at - https://github.com/fluxninja/aperture

We would love feedback from folks reading this blog post!

Disclaimer: I am one of the co-authors of the Aperture project. There are several interesting ideas we have built into this project and I will be happy to dive into the technical details as well.

It's a blog on an open-source project that precisely tells you how to implement adaptive rate limiting.

Just click around a bit:

- https://github.com/fluxninja/aperture

- https://docs.fluxninja.com/use-cases/adaptive-service-protec...

Note: I am one of the authors' of this project.

At the time of writing, the first sample image on that page is this:

https://coderabbit.ai/assets/section-1-f9a48066.png

which recommends adding a "maxIterations" counter to the "for len(executedComponents) ..." loop here:

https://github.com/fluxninja/aperture/blob/26e00ea818c7c28da...

HOWEVER

- the review has failed to notice the logic using "numExecutedBefore" (around line 377) that already prevents the specific bug it is suggesting a fix for

- the suggested change decrements "maxIterations" inside the "for ... range circuit.components {" loop which means it isn't counting iterations, it's counting components

This kind of suggestion is particularly nasty because it's unlikely that the test suite populates enough components to hit "maxIterations" - so an inattentive reader could accept it, get a green build, and then deploy a production bug!

The need an adaptive protection system like Aperture[0] to mitigate overloads.

[0]: https://github.com/fluxninja/aperture

It’s customized to our policy spec. But you can learn from this and adapt it to your spec.

https://github.com/fluxninja/aperture/blob/main/scripts/json...

Echo - web framework for Go

The three behaviors I've described that we want all depend on two things, the first of which is "idiomatic error handling". We need to be able to simply return err in our handlers. Unfortunately, the standard libray doesn't give us this. But some third-party frameworks do. The most popular one I'm familiar with is labstack echo, whose HandlerFunc looks like this:

In this tutorial, I will be using the Echo framework to build the backend. You can learn more about Echo here.

____ __ / __/___/ / ___ / _// __/ _ \/ _ \ /___/\__/_//_/\___/ v4.11.1 High performance, minimalist Go web framework https://echo.labstack.com ____________________________________O/_______ O\ ⇨ http server started on [::]:8080

Some of the features: - ✅ Using Vertical Slice Architecture as a high level architecture - ✅ Using Event Driven Architecture on top of RabbitMQ Message Broker with a custom [Event Bus](pkg/messaging/bus/) - ✅ Using Event Sourcing in Audit Based services like [Orders Service](services/orders/) - ✅ Using CQRS Pattern and Mediator Patternon top of Go-MediatR library - ✅ Using Dependency Injection and Inversion of Controlon top of uber-go/fx library - ✅ Using RESTFul api with Echo framework and using swagger with swaggo/swag library - ✅ Using Postgres and EventStoreDB to write databases with fully supports transactions(ACID) - ✅ Using MongoDB and Elastic Search for read databases (NOSQL) - ✅ Using OpenTelemetry for collection Distributed Tracing with using Jaeger and Zipkin - ✅ Using OpenTelemetry for collection Metrics with using Prometheus and Grafana - ✅ Using Unit Test for testing small units with mocking dependent classes and using Mockery for mocking dependencies - ✅ Using End2End Test and Integration Test for testing features with all of their real dependeinces using docker containers (cleanup tests) and testcontainers-go library

If you come from NodeJS background, you may find Echo (https://echo.labstack.com) most similar to express.

Can I try to rewrite it using the following? I'll just hand you the code I don't care about credit, I just enjoy cleaning things up. - https://github.com/spf13/cobra - https://echo.labstack.com/ - SQLite - and not a bunch of if statements

Use a library for HTTP serving, such as Gin, Chi, or Echo. I personally use Chi, as it's just the right level of abstraction for how I like to work. Despite what others say here, don't try to re-implement everything in a modern serving library using the standard library.

Here I'm not really sure what you're referring to: * You can set request timeout and it has nothing to do with whether you handled your error or not. * In most cases you either bubble it up the callstack or do something with error in place you o received it i.e. you switch to default value, retry or sth along those lines. In some cases frameworks like echo will translate error into 5XX response for you if you don't do anything with it in top level handler. * Panics are recoverable. Also in case your handler panics it won't crash entire server -> stdlib HTTP server just closes connection, frameworks might even provide panic handler which will return 5XX instead of nothing. * try/catch doesn't really solve anything I mentioned here ¯_(ツ)_/¯. You just hope somebody caught your exception somewhere else.