pachyderm VS openreplay

20. Pachyderm | Github | tutorial

Pachyderm specializes in creating compliance-focused pipelines that integrate with enterprise-level storage solutions.

There are a couple of other contenders in this space. DVC (https://dvc.org/) seems most similar.

If you're interested in something you can self-host... I work on Pachyderm (https://github.com/pachyderm/pachyderm), which doesn't have a Git-like interface, but also implements data versioning. Our approach de-duplicates between files (even very small files), and our storage algorithm doesn't create objects proportional to O(n) directory nesting depth as Xet appears to. (Xet is very much like Git in that respect.)

The data versioning system enables us to run pipelines based on changes to your data; the pipelines declare what files they read, and that allows us to schedule processing jobs that only reprocess new or changed data, while still giving you a full view of what "would" have happened if all the data had been reprocessed. This, to me, is the key advantage of data versioning; you can save hundreds of thousands of dollars on compute. Being able to undo an oopsie is just icing on the cake.

Xet's system for mounting a remote repo as a filesystem is a good idea. We do that too :)

Pachyderm - License prevents competition.

I was at Airbnb when we open-sourced Airflow, it was a great solution to the problems we had at the time. It's amazing how many more use cases people have found for it since then. At the time it was pretty focused on solving our problem of orchestrating a largely static DAG of SQL jobs. It could do other stuff even then, but that was mostly what we were using it for. Airflow has become a victim of its success as it's expanded to meet every problem which could ever be considered a data workflow. The flaws and horror stories in the post and comments here definitely resonate with me. Around the time Airflow was opensource I starting working on data-centric approach to workflow management called Pachyderm[0]. By data-centric I mean that it's focused around the data itself, and its storage, versioning, orchestration and lineage. This leads to a system that feels radically different from a job focused system like Airflow. In a data-centric system your spaghetti nest of DAGs is greatly simplified as the data itself is used to describe most of the complexity. The benefit is that data is a lot simpler to reason about, it's not a living thing that needs to run in a certain way, it just exists, and because it's versioned you have strong guarantees about how it can change.

[0] https://github.com/pachyderm/pachyderm

Here is a demo run of lgtm on pachyderm

Uncover frustrations, understand bugs and fix slowdowns like never before with OpenReplay — an open-source session replay tool for developers. Self-host it in minutes, and have complete control over your customer data. Check our GitHub repo and join the thousands of developers in our community.

- Optimize user experience: use insights to refine and improve your app’s usability.

Interested? For more details, you can check out the GitHub repo [0] or documentation [1]:

[0] https://github.com/openreplay/openreplay or documentation

[1] https://docs.openreplay.com/en/rn-sdk/

Hey HN,

For anyone who uses session replay tools to debug, monitor, or analyze user journeys, this feature might come in handy. OpenReplay — the self-hosted session replay tool that helps developers troubleshoot web apps faster — now supports tabbed browsing.

*Here's how:*

1. Capture and replay user sessions that span across multiple browser tabs, all within a single recording.

2. OpenReplay's tracker communicates across browser tabs, ensuring accurate tracking of each tab, even when duplicated or opened with `window.open` without `_blank`.

3. With "co-browsing", you can watch and support users in real-time and see how they navigate across multiple tabs of your app.

*Why it’s important?*

1. This feature is convenient for developers as it allows them to easily identify bugs by reviewing a complete session in one replay, rather than having to reference multiple recordings.

2. It provides a deeper understanding of the user journey and interactions across multiple tabs in your web app, helping in the comprehension of complex user behaviors and paths.

3. It provides accurate feedback on user tab actions such as opening, switching, and closing tabs, assisting in UX/UI improvements.

Interested? For more details, you can check out the GitHub repo at https://github.com/openreplay/openreplay.

You could maybe get in a scenario where a company rather sneakily changes their license without proper announcement to make usage non-open and billable, but that's pretty rare and bad behavior and that underhandedness may bite them back if it goes to court (I am not a lawyer). That said, it can happen. As an example, see this discussion I opened where the project switched licenses and the maintainers thought it was enough to only really announce this within a 1 hour YouTube video (Rather than their blog or update notes).

OpenReplay would not be typically be considered open source. (Which I have raised with them and have since been blocked from their repos).

How does it compare to OpenReplay ?

> Before deciding on ClickHouse, we were planning to use OpenSearch

You should have tried Quickwit :)

Anyway, sounds like a great project, best of luck!

[1] https://github.com/openreplay/openreplay

[2] https://github.com/quickwit-oss/quickwit