pgsink VS xact

This is a really awesome project. I’ve used it on https://github.com/lawrencejones/pgsink to generate type safe bindings to the Postgres catalog tables, along with a few of the tables the project maintains itself.

pgx is really good: it's what I used to write logical decoders in https://github.com/lawrencejones/pgsink

It used to be that companies would write their own hacky scripts to perform this extraction - I've had terrible incidents caused by ETL database triggers in the past, and even built a few generic ETL tools myself.

Postgres change-capture device that supports high-throughput and low-latency capture to a variety of sinks (at first release, just Google BigQuery):

https://github.com/lawrencejones/pgsink

I know there's debezium and Netflix's dblog, but this project aims to be much simpler.

Forget about kafka and any other dependency: just point it at Postgres, and your data will be pushed into BigQuery. And for people with highly-performance-sensitive databases, the read workload has been designed with Postgres efficiency in mind.

I'm hoping pgsink could be a gateway drug to get small companies up and running with a data warehouse. If your datastore of choice is Postgres, it's a huge help to replicate everything into an analytics datastore. A similar tool has helped my company extract expensive work out of our primary database, which is super useful for scaling.

The project is 90% there, about 10hrs and some testing away from being useable. Once there, I'll be hitting up some start-up friends and seeing if they want to give it a whirl.

https://github.com/wtpayne/xact - Model Based Systems/Software Engineering tool with support for machine learning and synthetic data.

MBSE is a vastly underappreciated technique, and one that deserves much more exposure in the mainstream tech community.

Sadly, I have found myself exceedingly disappointed at the tooling available to support MBSE, having mainly used Simulink, and also experimented with a few others like Rhapsody, Capella and EA.

Constant mouse usage with Simulink gave me really bad RSI, and merging models was a pain, due to the way that layout and structural information were mixed together in the xml-based .mdl file format.

So (naturally) I made a text-based alternative. It's a bit like TensorFlow in that you create a compute graph (computational model), and then run it.

The model itself can be generated dynamically in a script, using JQuery-like syntax to add or change nodes, or alternatively it can be stored and version controlled in one or more text files, using YAML, XML, JSON or TOML (or some mixture of those) to serialise the structure in an easy-to-diff-and-merge form.

https://github.com/wtpayne/xact