flowbase VS Echo

I see a lot of potential in pipeline concurrency, as seen in dataflow (DF) and flow-based programming (FBP). That is, modeling computation as pipelines where one component sends data to the next component via message passing. As long as there is enough data it will be possible for multiple components in the chain to work concurrently.

The benefits are that no other synchronization is needed than the data sent between processes, and race conditions are ruled out as long as only one process is allowed to process a data item at a time (this is the rule in FBP).

The main blockers I think is that it requires quite a rethink of the architecture of software. I see this rethink happening in larger, especially distributed systems, which are modeled a lot around these principles already, using systems such as Kafka and message queues to communicate, which more or less forces people to model computations around the data flow.

I think the same could happen inside monolithic applications too, with the right tooling. The concurrency primitives in Go are superbly suited to this in my experience, given that you work with the right paradigm, which I've been writing about before [1, 2], and started making a micro-unframework for [3] (though the latter one will be possible to make so much nicer after we get generics in Go).

But then, I also think there are some lessons to be learned about the right granularity for processes and data in the pipeline. Due to the overhead of message passing, it will not make sense performance-wise to use dataflow for the very finest-grain data.

Perhaps this in a sense parallels what we see with distributed computing, where there is a certain breaking point before which it isn't really worth it to go with distributed computing, because of all the overhead, both performance-wise and complexity-wise.

[1] https://blog.gopheracademy.com/composable-pipelines-pattern/

[2] https://blog.gopheracademy.com/advent-2015/composable-pipeli...

[3] https://flowbase.org

Echo - web framework for Go

The three behaviors I've described that we want all depend on two things, the first of which is "idiomatic error handling". We need to be able to simply return err in our handlers. Unfortunately, the standard libray doesn't give us this. But some third-party frameworks do. The most popular one I'm familiar with is labstack echo, whose HandlerFunc looks like this:

In this tutorial, I will be using the Echo framework to build the backend. You can learn more about Echo here.

____ __ / __/___/ / ___ / _// __/ _ \/ _ \ /___/\__/_//_/\___/ v4.11.1 High performance, minimalist Go web framework https://echo.labstack.com ____________________________________O/_______ O\ ⇨ http server started on [::]:8080

Some of the features: - ✅ Using Vertical Slice Architecture as a high level architecture - ✅ Using Event Driven Architecture on top of RabbitMQ Message Broker with a custom [Event Bus](pkg/messaging/bus/) - ✅ Using Event Sourcing in Audit Based services like [Orders Service](services/orders/) - ✅ Using CQRS Pattern and Mediator Patternon top of Go-MediatR library - ✅ Using Dependency Injection and Inversion of Controlon top of uber-go/fx library - ✅ Using RESTFul api with Echo framework and using swagger with swaggo/swag library - ✅ Using Postgres and EventStoreDB to write databases with fully supports transactions(ACID) - ✅ Using MongoDB and Elastic Search for read databases (NOSQL) - ✅ Using OpenTelemetry for collection Distributed Tracing with using Jaeger and Zipkin - ✅ Using OpenTelemetry for collection Metrics with using Prometheus and Grafana - ✅ Using Unit Test for testing small units with mocking dependent classes and using Mockery for mocking dependencies - ✅ Using End2End Test and Integration Test for testing features with all of their real dependeinces using docker containers (cleanup tests) and testcontainers-go library

If you come from NodeJS background, you may find Echo (https://echo.labstack.com) most similar to express.

Can I try to rewrite it using the following? I'll just hand you the code I don't care about credit, I just enjoy cleaning things up. - https://github.com/spf13/cobra - https://echo.labstack.com/ - SQLite - and not a bunch of if statements

Use a library for HTTP serving, such as Gin, Chi, or Echo. I personally use Chi, as it's just the right level of abstraction for how I like to work. Despite what others say here, don't try to re-implement everything in a modern serving library using the standard library.

Here I'm not really sure what you're referring to: * You can set request timeout and it has nothing to do with whether you handled your error or not. * In most cases you either bubble it up the callstack or do something with error in place you o received it i.e. you switch to default value, retry or sth along those lines. In some cases frameworks like echo will translate error into 5XX response for you if you don't do anything with it in top level handler. * Panics are recoverable. Also in case your handler panics it won't crash entire server -> stdlib HTTP server just closes connection, frameworks might even provide panic handler which will return 5XX instead of nothing. * try/catch doesn't really solve anything I mentioned here ¯_(ツ)_/¯. You just hope somebody caught your exception somewhere else.