timescale-analytics
pgx
timescale-analytics | pgx | |
---|---|---|
8 | 19 | |
336 | 2,376 | |
1.8% | - | |
6.0 | 9.6 | |
10 days ago | about 1 year ago | |
Rust | Rust | |
GNU General Public License v3.0 or later | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
timescale-analytics
-
Timescale raises $110M Series C
Hi! So the team is over 100 at this point, but engineering effort is spread across multiple products at this point.
The core timescaledb repo [0] has 10-15 primary engineers (although we are aggressively hiring for database internal engineers), with a few others working on DB hyperfunctions and our function pipelining [1] in a separate extension [2]. I think generally the set of folks who contribute to low-level database internals in C is just smaller than other type of projects.
We also have our promscale product [3], which is our observability backend powered by SQL & TimescaleDB.
And then there is Timescale Cloud, which is obviously a large engineering effort (most of which does not happen in public repos).
And we are hiring. Fully remote & global.
https://www.timescale.com/careers
[0] https://github.com/timescale/timescaledb
[1] https://www.timescale.com/blog/function-pipelines-building-f...
[2] https://github.com/timescale/timescaledb-toolkit
[3] https://github.com/timescale/promscale ; https://github.com/timescale/tobs
-
Function pipelines: Building functional programming into PostgreSQL
(NB: Post author here)
This is in the TimescaleDB Toolkit extension [1] which is licensed under our community license for now and it's not available on DO. It is available on our cloud service fully managed. You can also install it and run it for free yourself.
[1]: https://github.com/timescale/timescaledb-toolkit
- How percentile approximation works (and why it's more useful than averages)
-
How PostgreSQL aggregation works and how it inspired our hyperfunctions’ design
Absolutely! We're actually developing a lot of that: https://github.com/timescale/timescaledb-toolkit/tree/main/d...
A number of the things you're looking for we've done experimentally and we'll be stabilizing over the next few releases. So we'd love some feedback while we're still able to futz with the API without making breaking changes.
But the two you're asking about are, I think, going to be covered by hyperloglog (we just reimplemented the internals with HLL++) and stats_agg family of functions, which have both 1D (which will give you avg, stddev, variance, etc) and 2D (co-variance, slope, intercept, x-intercept etc as well as all the 1D functions).
Would also love issues if you think we're missing other stuff, going to be generalizing this and want to make it useful for folks.
(NB: Post author here.)
-
Postgres downsampling performance
If you know that you're going to be doing downsampling at the hourly level then a continuous aggregate on the hour is probably a good idea. We're also building some functions to make some of the continuous aggregate stuff for these sorts of cases easier/more accurate in more cases, especially if you need things like exact averages when you don't have the same number of points in an hour and want to re-aggregate on top of the continuous agg. See: https://github.com/timescale/timescale-analytics/pull/141/files
-
TimescaleDB Raises $40M
Fair point about adaptive chunking. You sound like a long-term user!
There is always a trade-off between getting features to users quickly to experiment and incrementally improve, versus doing it always very conservatively.
When we launched adaptive chunking (introduced in 0.11, deprecated in 1.2), we explicitly marked it as beta and default off, to hopefully reflect that. [1]
The approach we are now taking with Timescale Analytics [2] is to have an explicit distinction between experimental features (which will be part of a distinct"experimental" schema in the database, and must be expressly turned on with appropriate warnings) and stable features. Hopefully this can help find a good balance between stability and velocity, but feedback welcome!
[1] https://github.com/timescale/timescaledb/releases/tag/0.11.0
[2] https://github.com/timescale/timescale-analytics/tree/main/e...
pgx
-
Write Postgres functions in Rust
It uses pgx (https://github.com/tcdi/pgx) which is our more generalized framework for developing Postgres extensions with Rust.
-
Why not Rust for Omnigres?
It's a great question, considering I've been using Rust for a number of years now, and I generally advocate its use for its rich ecosystem, safety and tooling. I actively contribute to pgx, a library for building Postgres extensions in Rust. Yet, Omnigres appears to be all done in C.
-
Supabase Wrappers: A Framework for Building Postgres Foreign Data Wrappers
Our release today is a framework which extends this functionality to other databases/systems. If you’re familiar with Multicorn[1] or Steampipe[2], then it’s very similar. The framework is written in Rust, using the excellent pgx[3].
We have developed FDWs for Stripe, Firebase, BigQuery, Clickhouse, and Airtable (all in various “pre-release” states). The plan is to focus on the tools we’re using internally while we stabalize the framework.
There’s a lot in the blog post into our goals for this release. It’s early, but one of the things I’m most excited about.
[0] Postgres FDW: [https://www.postgresql.org/docs/current/sql-createforeigndat...
[1] Multicorn: https://multicorn.org/
[2] Steampipe: https://steampipe.io/
[2] pgx: [https://github.com/tcdi/pgx](https://github.com/tcdi/pgx)
- Apache Age, a PostgreSQL Extension with Graph Database Functionality
-
Postgres FTS vs the new wave of search engines
BTW one nice easter egg is that with pgx there is actually no reason that we can't build even better search solutions inside the database itself.
-
Postgres Full Text Search vs. the Rest
> That thread led me to a project/product idea where you take an existing Postgres instance used for normal products or whatever, replicate it to various read only clusters with a custom search extension loaded and some orchestrator sitting on top (I’ve written most of one in rust that uses 0mq to communicate with it’s nodes) and create drop in search from existing databases with a nice guided web gui for automatic tuning suitable for most business use cases.
Very interesting idea -- just want to add one thing, write it in rust (with pgx?[0]) :)
[0]: https://github.com/tcdi/pgx
-
Show HN: pg_idkit, a Postgres extension for generating exotic UUIDs
Hey HN,
It turns out choosing a good database-optimized UUID (and deciding whether to use serial, etc) isn't quite so simple, and I finally got a chance to do some exploration, write about it[0].
One of the reasons Postgres is the best open source database out there is it's extensibility -- so I hacked up a small extension for generating some of the more exotic (but crucially, lexicograhically sortable) UUID generation mechanisms:
https://github.com/t3hmrman/pg_idkit
This idea has been bumbling around my head for a while, but I finally got a chance to build it while working with Supabase on a post about IDs[0]!
Most of the heavy lifting is done by pgx[1] which is an amazing framework for building Postgres extensions in Rust. I think we are very early to the trend of amazing postgres extensions built in Rust, and it's yet another reason that it's an exciting time to be all-in on Postgres.
[0]: https://supabase.com/blog/choosing-a-postgres-primary-key
[1]: https://github.com/tcdi/pgx
[0]: https://supabase.com/blog/choosing-a-postgres-primary-key
-
Introducing pg_idkit: an extension for generating lexicographically sortable UUIDs (UUIDv6-8, CUID, Timeflake) in Postgres
The extension is still WIP but for those of ya'll that like Rust it's built on pgx which has excellent DX. The rust involved isn't complicated -- I'm basically laundering the functionality from other crates that are listed in the README.md.
-
GitHub - supabase/pg_jsonschema: PostgreSQL extension providing JSON Schema validation
Seems to be using this: https://github.com/tcdi/pgx
-
Show HN: Pg_jsonschema – A Postgres extension for JSON validation
- https://github.com/furstenheim/is_jsonb_valid
pgx[0] is going to be pretty revolutionary for the postgres ecosystem I think -- there is so much functionality that can be utilized at the database level and I can't think of a language I want to do it with more than Rust.
[0]: https://github.com/tcdi/pgx
What are some alternatives?
orioledb - OrioleDB – building a modern cloud-native storage engine (... and solving some PostgreSQL wicked problems) 🇺🇦
tauri - Build smaller, faster, and more secure desktop applications with a web frontend.
TimescaleDB - An open-source time-series SQL database optimized for fast ingest and complex queries. Packaged as a PostgreSQL extension.
code - Source code for the book Rust in Action
Telegraf - The plugin-driven server agent for collecting & reporting metrics.
bevy - A refreshingly simple data-driven game engine built in Rust
promscale - [DEPRECATED] Promscale is a unified metric and trace observability backend for Prometheus, Jaeger and OpenTelemetry built on PostgreSQL and TimescaleDB.
postgrest - REST API for any Postgres database
t-digest - A new data structure for accurate on-line accumulation of rank-based statistics such as quantiles and trimmed means
supabase-graphql-example - A HackerNews-like clone built with Supabase and pg_graphql
tsbs - Time Series Benchmark Suite, a tool for comparing and evaluating databases for time series data
feophant - A PostgreSQL inspired SQL database written in Rust.