DataflowTemplates VS debezium-examples

Google provide a Dataflow template for copying from BigQuery to Datastore, see this stack overflow answer.

Ah, looks like the template default appends new rows. If I want to overwrite the table, looks like I might be able to just replace this line in the template code to WRITE_TRUNCATE (see here). Cool!

However, learning to use Apache Beam, which is the open source framework behind Dataflow, is no bed of roses: The official documentation is sparse, GCP-provided templates don't work out-of-the-box, and the Javadoc is, well, a javadoc.

There are a lot of great example here -> https://github.com/debezium/debezium-examples

I'm thinking about open sourcing the framework that I wrote to implement a managed saga inspired to the framework package in Debezium saga order-service: https://github.com/debezium/debezium-examples/tree/main/saga

Check out their examples repo: https://github.com/debezium/debezium-examples/tree/main/debezium-server-postgres-pubsub

Hi, I am struggling with debezium and elastic search the problem seems to be on the elasticsearch side or docker. I am using this example and when I try to create an index with mappings the return is (52) Empty reply from server. Even when I run es container according to es documentation and make curl -X GET "localhost:9200/_cat/nodes?v=true&pretty" request server returns an empty reply.

We have some Kafka Streams related examples in the debezium-examples repo, e.g. for FK joins, aggregations, and using a state store. They show usage of Kafka Streams in the context of CDC, but the same patterns are more generically applicable of course.

Well, it works with Postgres, but apparently not CloudSQL for PostgreSQL.

The original code for the connector context can be found here: https://github.com/debezium/debezium-examples/tree/master/end-to-end-demo/debezium-jdbc