Our great sponsors
-
dbt-core
dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
yeah, this issue has been open for years:
https://github.com/dbt-labs/dbt-core/issues/1162
I wanted incremental refresh in Postgres as well and found that you can manage your own table to get something close.
Basically you create a regular table in place of a materialised one, only aggregate data newer than what's currently in the table then store the new aggregates in table. Repeat this an interval.
https://github.com/cadbox1/prawn-stack/blob/master/src/backe...
I use this to show page view data aggregated by hour without calculating it on each request using Lambda
https://prawn.cadell.dev/