Sgr Alternatives

sgr reviews and mentions

• dbhub.io alternatives - csvbase and sgr
  3 projects | 21 Aug 2024
• Show HN: Loofi – Our AI-Powered SQL Query Builder
  1 project | news.ycombinator.com | 21 May 2023
• Release engineering is exhausting so here's cargo-dist
  12 projects | news.ycombinator.com | 1 Feb 2023

  I wrote up the details of this in a PR [0] where I last dealt with it.

  [0] https://github.com/splitgraph/sgr/pull/656

• Ask HN: Serverless SQLite or Closest DX to Cloudflare D1?
  2 projects | news.ycombinator.com | 2 Jan 2023

  This is the vision of what we're building at Splitgraph. [0] You might be most interested in our recent project Seafowl [1] which is an open-source analytical database optimized for running "at the edge," with cache-friendly semantics making it ideal for querying from Web applications. It's built in Rust using DataFusion and incorporates many of the lessons we've learned building the Data Delivery Network [2] for Splitgraph.

  [0] https://www.splitgraph.com

  [1] https://seafowl.io

  [2] https://www.splitgraph.com/connect

• Postgres Auditing in 150 lines of SQL
  10 projects | news.ycombinator.com | 9 Mar 2022

  You might like what we're doing with Splitgraph. Our command line tool (sgr) installs an audit log into Postgres to track changes [0]. Then `sgr commit` can write these changes to delta-compressed objects [1], where each object is a columnar fragment of data, addressable by the LTHash of rows added/deleted by the fragment, and attached to metadata describing its index [2].

  I haven't explored sirix before, but at first glance it looks like we have some similar ideas — thanks for sharing, I'm excited to learn more, especially about its application of ZFS.

  [0] https://www.splitgraph.com/docs/working-with-data/tracking-c...

  [1] https://www.splitgraph.com/docs/concepts/objects

  [2] https://github.com/splitgraph/splitgraph/blob/master/splitgr...

• The world of PostgreSQL wire compatibility
  3 projects | news.ycombinator.com | 10 Feb 2022
• Scalable PostgreSQL Connection Pooler
  11 projects | news.ycombinator.com | 12 Nov 2021

  We are building a solution for this problem at Splitgraph [0] – it sounds like we could probably help with your use case. You can get it to work yourself with our open source code [1], but our (private beta, upcoming public) SaaS service will put all your schemas on a more scalable “data delivery network,” which incidentally, happens to be implemented with PgBouncer + rewriting + ephemeral instances. In a local engine (just a Postgres DB managed by Splitgraph client to add extra stuff), there is no PgBouncer, but we use Foreign Data Wrappers to accomplish the same.

  On Splitgraph, every dataset – and every version of every dataset – has an address. Think of it like tagged Docker images. The address either points to an immutable “data image” (in which case we can optionally download objects required to resolve a query on-the-fly, although loading up-front is possible too) or to a live data source (in which case we proxy directly to it via FDW translation). This simple idea of _addressable data products_ goes a long way – for example, it means that computing a diff is now as simple as joining across two tables (one with the previous version, one with the new).

  Please excuse the Frankenstein marketing site – we’re in the midst of redesign / rework of info architecture while we build out our SaaS product.

  Feel free to reach out if you’ve got questions. And if you have a business case, we have spots available in our private pilot. My email is in my profile – mention HN :)

  [0] https://www.splitgraph.com/connect

  [1] examples: https://github.com/splitgraph/splitgraph/tree/master/example...

• Ask HN: How to get compeitors to use our open source interop-prototcol?
  4 projects | news.ycombinator.com | 4 Oct 2021

  Federated data sharing is the core use case of the magic Postgres database we’re building at Splitgraph [0]. We’d love to help you solve these problems! The ideas you’re describing are exactly what we want to achieve – data sharing should be as easy as changing a connection string in a SQL client. It sounds like your use case would be a good fit for what we’re building. If you’d like to learn more, please send me a note – email in profile.

  [0] https://www.splitgraph.com

• Cloudera taken private for $5.3b, acquires Datacoral and Cazena
  2 projects | news.ycombinator.com | 1 Jun 2021

  The data industry continues to hype this idea of “multi-cloud,” but then the “modern data stack” is centralized around a single warehouse and nobody sees any irony in that.

  The big bet we’re making at Splitgraph [0] is that the next wave of data engineering will take a more decentralized, “data mesh” type approach to enterprise architecture. “Data gravity” really does exist -expensive to move, in terms of both cost and operational complexity. So instead of bringing the data to the query, why not bring the query to the data? All we need for that is a set of read only credentials.

  Cloudera mentions they bought DataCoral to help with data integration and connectors. They’ve correctly identified the problem - data sprawl and fragmentation will inevitably grow - but I’m not sure they have the right solution.

  Data integration is important, but it’s a moving target, which is why it calls for a collaborative open source solution. This is why so many new startups, like AirByte most recently, are coalescing around the Singer taps that Stitch left behind after its acquisition by Talend.

  We also support using Singer taps to ingest data into versioned Splitgraph images [1], so we’re excited to see more collaboration on maintenance of taps. For us it’s a useful feature, but it should be just that — a feature. Is there really a need to replicate all of your data before you can even query it? Or would you rather experiment by directly querying its source?

  [0] https://www.splitgraph.com

  [1] unreleased and undocumented atm, but it does work. We’re hiring, especially on the frontend if you want to help build the web UI. See profile.

• Google Dataset Search
  1 project | news.ycombinator.com | 6 May 2021

  On the public DDN (data.splitgraph.com:5432), we enforce a (currently arbitrary) 10k row limit on responses. You can construct multiple queries using LIMIT and OFFSET, or you can run a local Splitgraph engine without a limit. We also have a private beta program if you want a managed or self-hosted deployment. And we are planning to ship some features for "export to csv" type use cases (potentially other output formats too).

  For live/external data, we proxy the query to the data source, so there is no theoretical data size limit except for any defined by the upstream.

  For snapshotted data, we store the data as fragments in object storage. Any size limit depends on the machine where Splitgraph's Postgres engine is running, and how you choose to materialize the data when downloading it from object storage. You can "check out" an entire image to materialize it locally, at which point it will be like any other Postgres schema. Or you can use "layered querying" which will return a result set while only materializing the fragments necessary to answer the query.

  Regarding ClickHouse, you could watch this presentation [0] my co-founder Artjoms gave at a recent ClickHouse meet-up on the topic of your question. We also have specific documentation for using the ClickHouse ODBC client with the DDN [1], as well as an example reference implementation. [2]

  [0] https://www.youtube.com/watch?v=44CDs7hJTho

  [1] https://www.splitgraph.com/connect

  [2] https://github.com/splitgraph/splitgraph/tree/master/example...

• A note from our sponsor - CodeRabbit
  coderabbit.ai | 19 Mar 2025

  Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR. Learn more →