xact
pgsink
xact | pgsink | |
---|---|---|
2 | 5 | |
1 | 76 | |
- | - | |
0.0 | 0.0 | |
almost 3 years ago | about 1 year ago | |
Python | Go | |
GNU Affero General Public License v3.0 | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
xact
-
Ask HN: Show me your Half Baked project
https://github.com/wtpayne/xact - Model Based Systems/Software Engineering tool with support for machine learning and synthetic data.
-
Ask HN: Does Your Company Practice Model Based Systems Engineering? (MBSE)
MBSE is a vastly underappreciated technique, and one that deserves much more exposure in the mainstream tech community.
Sadly, I have found myself exceedingly disappointed at the tooling available to support MBSE, having mainly used Simulink, and also experimented with a few others like Rhapsody, Capella and EA.
Constant mouse usage with Simulink gave me really bad RSI, and merging models was a pain, due to the way that layout and structural information were mixed together in the xml-based .mdl file format.
So (naturally) I made a text-based alternative. It's a bit like TensorFlow in that you create a compute graph (computational model), and then run it.
The model itself can be generated dynamically in a script, using JQuery-like syntax to add or change nodes, or alternatively it can be stored and version controlled in one or more text files, using YAML, XML, JSON or TOML (or some mixture of those) to serialise the structure in an easy-to-diff-and-merge form.
https://github.com/wtpayne/xact
pgsink
-
GitHub - go-jet/jet: Type safe SQL builder with code generation and automatic query result data mapping
This is a really awesome project. I’ve used it on https://github.com/lawrencejones/pgsink to generate type safe bindings to the Postgres catalog tables, along with a few of the tables the project maintains itself.
-
Trade-offs from using ULIDs at incident.io
pgx is really good: it's what I used to write logical decoders in https://github.com/lawrencejones/pgsink
-
A modern data stack for startups
It used to be that companies would write their own hacky scripts to perform this extraction - I've had terrible incidents caused by ETL database triggers in the past, and even built a few generic ETL tools myself.
- Sync Postgres to BigQuery, possible? How?
-
Ask HN: Show me your Half Baked project
Postgres change-capture device that supports high-throughput and low-latency capture to a variety of sinks (at first release, just Google BigQuery):
https://github.com/lawrencejones/pgsink
I know there's debezium and Netflix's dblog, but this project aims to be much simpler.
Forget about kafka and any other dependency: just point it at Postgres, and your data will be pushed into BigQuery. And for people with highly-performance-sensitive databases, the read workload has been designed with Postgres efficiency in mind.
I'm hoping pgsink could be a gateway drug to get small companies up and running with a data warehouse. If your datastore of choice is Postgres, it's a huge help to replicate everything into an analytics datastore. A similar tool has helped my company extract expensive work out of our primary database, which is super useful for scaling.
The project is 90% there, about 10hrs and some testing away from being useable. Once there, I'll be hitting up some start-up friends and seeing if they want to give it a whirl.
What are some alternatives?
pegao - Pegao is a community about lists of links on topics of interest.
pastty - Copy and paste across devices
UsTaxes - Tax filing web application
dupver - Deduplicating VCS for large binary files in Go
wcp
DataflowTemplates - Cloud Dataflow Google-provided templates for solving in-Cloud data tasks
tinyjam - A radically simple, zero-configuration static site generator in JavaScript
debezium-examples - Examples for running Debezium (Configuration, Docker Compose files etc.)
jarvis - An AI sidekick that helps you control your computer.
dbt-metabase - dbt + Metabase integration
langmap - A complete map of all the languages and their dependencies
thgtoa - The Hitchhiker’s Guide to Online Anonymity