optimus
Benthos
| optimus | Benthos | |
|---|---|---|
| 5 | 76 | |
| 737 | 7,617 | |
| 0.1% | 2.7% | |
| 4.4 | 9.7 | |
| 6 months ago | 7 days ago | |
| Go | Go | |
| Apache License 2.0 | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
optimus
-
Data Engineering Tools in Go
You can check odpf github, they created some dataops tools using go, one of the example is optimus (https://github.com/odpf/optimus) which is a data pipeline orchestrator
-
Modern open-source data platform that empowers organizations to discover, transform, analyse and secure data faster and efficiently.
Optimus is an easy-to-use, reliable, and performant workflow orchestrator for data transformation, data modeling, pipelines, and data quality management.
-
Seeking opinions on workflow/DAG orchestration with Go
this may help: https://github.com/odpf/optimus
- Optimus - An easy-to-use, reliable and performant workflow orchestrator for data transformation, data modeling, pipelines and data quality management
- GitHub - odpf/optimus: Optimus is an easy-to-use, reliable, and performant workflow orchestrator for data transformation, data modeling, pipelines, and data quality management.
Benthos
- Ask HN: Who is hiring? (December 2023)
- Structured Logging with Slog
- Fancy stream processing made operationally mundane
- Benthos: Fancy stream processing made operationally mundane
-
Any golang library to batch process a queue ?
I’ve used https://www.benthos.dev/ and it’s really easy and well implemented. The author is also very responsive
-
Show HN: Arroyo – Write SQL on streaming data
Looks cool. What is the difference between this tools and benthos (https://www.benthos.dev/)?
- Benthos: Open-source stream processing tool
-
book about golang and kafka
You might want to gradually replace that one with https://github.com/twmb/franz-go because Shopify is looking to find a new owner for Sarama and, until or if they do, it seems to be falling behind with maintenance: https://github.com/Shopify/sarama/issues/2461 For example, they still haven’t addressed this breaking change https://github.com/Shopify/sarama/issues/2358. franz-go has worked well so far in Benthos https://github.com/benthosdev/benthos/tree/main/internal/impl/kafka and it will likely end up as the only implementation once the Sarama-based one will be deprecated
- Show HN: Open-source Auth0 alternative Ory Kratos v0.13 released – nearing v1.0
-
Go in depth youtube channels?
I upload a mix of code reviews and live streams on https://www.youtube.com/@Jeffail, mostly building https://www.benthos.dev out in the open so the content ranges from beginner friendly stuff to more advanced things like stream processing, parser combinators, etc.
What are some alternatives?
cadence - Cadence is a distributed, scalable, durable, and highly available orchestration engine to execute asynchronous long-running business logic in a scalable and resilient way.
Confluent Kafka Golang Client - Confluent's Apache Kafka Golang client
goflow - A Golang based high performance, scalable and distributed workflow framework
appsmith - Platform to build admin panels, internal tools, and dashboards. Integrates with 25+ databases and any API.
protobuf-bigquery-go - Seamlessly save and load protocol buffers to and from BigQuery using Go.
watermill - Building event-driven applications the easy way in Go.
beneath - Beneath is a serverless real-time data platform ⚡️
sarama - Sarama is a Go library for Apache Kafka. [Moved to: https://github.com/IBM/sarama]
dagu - Yet another cron alternative with a Web UI, but with much more capabilities. It aims to solve greater problems.
salsa - A generic framework for on-demand, incrementalized computation. Inspired by adapton, glimmer, and rustc's query system.
flyte - Scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks.
grpcurl - Like cURL, but for gRPC: Command-line tool for interacting with gRPC servers