wal2json
PaperTrail
Our great sponsors
wal2json | PaperTrail | |
---|---|---|
13 | 18 | |
1,234 | 6,693 | |
- | 0.4% | |
2.7 | 5.9 | |
11 months ago | 3 months ago | |
C | Ruby | |
BSD 3-clause "New" or "Revised" License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
wal2json
-
On the performance of REPLICA IDENTITY FULL in Postgres
To illustrate the above, let’s look at an example, using wal2json for convenience. An update event looks something like this:
-
What is the recommended way to synchronize data between Postgres and Elasticsearch?
You could also use tools in conjunction with logical replication to stream changes, like debezium or wal2json.
-
DMS or Debezium for CDC for legacy system
I've done CDC before for mysql and postgres, in both cases, rolled my own 200~ loc python scripts. No experience with DMS. Debezium, as I understand, needs a kafka topic to stream to, so wasn't workable for us. Rolling our own proved to be fairly trivial. For postgres, there's wal2json https://github.com/eulerto/wal2json
-
Sync databases on command
I was researching for the whole day and found out https://github.com/eulerto/wal2json and postgres bult-in `pgoutput` plugin help me to catch the changes.
-
Looking for a Rails Gem that Audits Manual Database Changes
I don't think this is something for "rails" to do per se, and more the underlying DB. replication tools to capture a "change log". I have use postgres Write Ahead Logs (WAL) shipped to s3 in json format https://github.com/eulerto/wal2json
- How Retool upgraded its 4 TB main application PostgreSQL database
-
Multiplayer Demo Built with Elixir
[0] https://supabase.com [1] https://github.com/supabase/realtime [2] https://github.com/eulerto/wal2json [3] https://github.com/supabase/walrus [4] https://gsd.di.uminho.pt/members/cbm/ps/delta-crdt-draft16ma... [5] https://supabase.com/blog/2022/04/01/supabase-realtime-with-... [6] https://fly.io [7] https://github.com/supabase/realtime/tree/multiplayer [8] https://github.com/supabase/supabase/tree/master/docker
- Show HN: Multiplayer Demo Built with Elixir
-
Hacking PostgreSQL Internals to Deliver Push Notifications
wal2json Output Plugin that converts WAL output to JSON objects [Open Source] pg_recvlogical Postgres app that can consume update stream [Out-of-the-box with Postgres] decoderbufs Output Plugin that delivers data as protobuf [Open Source, Used in Debezium]
- eulerto/wal2json: JSON output plugin for changeset extraction
PaperTrail
-
historical data and "point in time" data modeling techniques, advice.
if the source (web) application makes their own audit tables. ex: our ruby on rails application uses the paper-trail gem
-
Best rails tools to automatically handle logging of things like all a user's actions, or changes to a record in a module - primarily for audit purposes.
Start with https://github.com/paper-trail-gem/paper_trail
-
Inventory/Sales Management module built on a Rails app - what would be the best way to "version" updates made against an SKU.
We use paper_trail for this
-
is there a gem for tracking adhoc rails console changes
I think you could use that in conjunction with the paper_trail gem, as /u/GreenCalligrapher571 mentioned, which is also a good suggestion. As an additional note, when changing records in production while using the paper_trail gem, I suggest wrapping your database-mutating statements executed in the rails console within a whodunnit block, so PaperTrail.request(whodunnit: 'Dorian Marié') { widget.update name: 'Wibble' } or something rather than just widget.update name: 'Wibble'. Or, if you have some sort of issue-tracking / ticketing system, you could set the whodunnit value to the ticket number or whatever, and then anyone who wants to know why the records are in the state they're in can consult that ticket, which hopefully has additional relevant context.
-
History Tracking With Postgres
For a while we did this using the paper-trail gem. This was a very simple way to add a few lines of code to keep track of all of the changes made to an ActiveRecord model. But it came with one drawback. Every change to the data had to be done through ActiveRecord. There are often times when this makes an app vulnerable to a race condition. I’ll use a contrived example so as not to share any real code from our client’s app.
-
Adding soft delete to a Phoenix Commanded (CQRS) API
In most designs, this would probably not be possible unless a table tracking extension is being used in an ORM. Even with change tracking enabled through extensions like paper trail or Django simple history, it can be tricky to restore deleted entities. Object tracking would need to have been enabled before it is needed to ensure the data is still around to be restored.
- Looking for a Rails Gem that Audits Manual Database Changes
-
Temporality/time-travelling in DB with ActiveRecord?
Maybe you are looking for the papertrail gem? https://github.com/paper-trail-gem/paper_trail
- Looking for an observer gem
What are some alternatives?
debezium - Change data capture for a variety of databases. Please log issues at https://issues.redhat.com/browse/DBZ.
Audited - Audited (formerly acts_as_audited) is an ORM extension that logs all changes to your Rails models.
supabase - The open source Firebase alternative.
Paranoia - acts_as_paranoid for Rails 5, 6 and 7
walrus - Applying RLS to PostgreSQL WAL
Logidze - Database changes log for Rails
pg-logical-replication - PostgreSQL Logical Replication client for node.js
mongoid-history - Multi-user non-linear history tracking, auditing, undo, redo for mongoid.
realtime - Broadcast, Presence, and Postgres Changes via WebSockets
ActsAsParanoid - ActiveRecord plugin allowing you to hide and restore records without actually deleting them.
container-images - Docker images for Debezium. Please log issues in our JIRA at https://issues.redhat.com/projects/DBZ/summary
Discard - 🃏🗑 Soft deletes for ActiveRecord done right