Our great sponsors
-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
-
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
merkle-tree-solidity
JS - Solidity sha3 merkle tree bridge. Generate proofs in JS; verify in Solidity.
If the issue happen a lot, there is also: https://github.com/datafold/data-diff
That is a nice tool to do it cross database as well.
I think it's based on checksum method.
Slightly related: I recently needed to compare several large databases for schema differences. The output from `pg_dump` isn't very diff-friendly, so I built this[1].
1. https://github.com/orf/diffable-sql
This is part of why I don't use MINUS for table value comparisons... All you need is just GROUP BY/UNION ALL/HAVING, using the following technique:
https://github.com/gregw2hn/handy_sql_queries/blob/main/sql_...
Not sure if Go code is your kind of thing, but if it is my colleague wrote a SQLite "diff" capability for our online hosting operation:
https://github.com/sqlitebrowser/dbhub.io/blob/5c9e1ab1cfe0f...
The code there can also output a "merge" object out of the differences too, in order to merge the differences from one database object into another.
In case you haven't tried dbt (www.getdbt.com / "Data Build Tool") - there's a whole package ecosystem that solves for things like this. The one that came to mind is "dbt-audit-helper": https://github.com/dbt-labs/dbt-audit-helper#compare_relatio...
It's kind of like PyPI/DockerHub for SQL. Lots of cool stuff in there...here's link to the package hub: https://hub.getdbt.com/
Why not hashed merkle tree approach? [0]
[0] : https://github.com/ameensol/merkle-tree-solidity/blob/master...