kafka-go VS elastic

There are two main libraries that people use to write clients Confluent Kafka and segment io kafka

Unlike the standard scripts, and many binary clients, it's a native, statically-compiled, binary. It uses segmentio/go-kafka internally, which means CGO can be disabled during compilation.

Scaling up our producers/consumers proved to be more cost-effective than adding more CPU/brokers to the Kafka cluster. To accomplish this, we used the segmentio/kafka-go client library which provides an excellent abstraction for interacting with the cluster and will handle data compression completely transparently.

PostgeSQL as event store database Kafka as messages broker gRPC Go implementation of gRPC Jaeger open source, end-to-end distributed tracing Prometheus monitoring and alerting Grafana for to compose observability dashboards with everything from Prometheus MongoDB MongoDB database Elasticsearch Elasticsearch client for Go. Echo web framework Kibana Kibana is data visualization dashboard software for Elasticsearch Migrate for migrations

I want to announce the first release of a kafkagosaur, a new Kafka client. It's built using WebAssembly and binds to the kafka-go library. The first release includes functionality to read and write Kafka messages and SASL support.

Use the Segment Kafka library, not Sarama, it’s much easier to use https://github.com/segmentio/kafka-go

Kafaka - kafka-go, Debezium Outbox

Segment learned quite some time ago that librdkafka-go has problems like these (and doesn’t support Contexts either), so they wrote a pure Go replacement instead. https://github.com/segmentio/kafka-go

In this article let's try to create closer to real world CQRS microservices with tracing and monitoring using: 🚀 Kafka as messages broker gRPC Go implementation of gRPC PostgreSQL as database Jaeger open source, end-to-end distributed tracing Prometheus monitoring and alerting Grafana for to compose observability dashboards with everything from Prometheus MongoDB Web and API based SMTP testing Redis Type-safe Redis client for Golang swag Swagger for Go Echo web framework

i am trying to generate the following query using github.com/olivere/elastic/v7

I think that https://olivere.github.io/elastic/ is a lot better as an API client

For Go available two good libraries for elasticsearch, the official Elasticsearch client and another one from community olivere elastic, both is good, but at this moment only the official client supports 8 version of elasticsearch and for serious production think it's the choice.

You are using the Query DSL, which is documented here, and has examples for boolean combination queries, so if you want it built up in a more strongly-typed fashion, check the documentation for it: https://github.com/olivere/elastic/wiki/QueryDSL

ElasticSearch repository implementation uses go-elasticsearch official library, another good one is olivere elastic but here it's not support 8 version which used for this project.

In this project we have microservice working with EventStoreDB using oficial go client, for [projections (https://zimarev.com/blog/event-sourcing/projections/) used MongoDB and Elasticsearch for search, and communicate by gRPC and REST. Did not implement here any interesting business logic and didn't cover tests, because don't have enough time, the events list is very simple: create a new order, update shopping cart, pay, submit, cancel, change the delivery address, complete order, and of course in real-world better use more concrete and meaningfully events, but the target here is to show the idea and how it works. Event Sourcing can be implemented in different ways, used here EventStoreDB, but we can do it with PostgreSQL and Kafka for example. After trying both approaches, found EventStoreDB is a better solution because all required features are implemented out of the box, it is optimized and really very good engineers developing it.

Relevant Discussion from the maintainer

I have only used the olivere library and I think it is as good as a Go library gets.