core
Build platforms that flexibly mix SQL, batch, and stream processing paradigms (by gazette)
pg_cron
Run periodic jobs in PostgreSQL (by citusdata)
Our great sponsors
core | pg_cron | |
---|---|---|
6 | 20 | |
516 | 2,545 | |
1.4% | 3.9% | |
8.1 | 6.1 | |
5 days ago | 3 days ago | |
Go | C | |
MIT License | PostgreSQL License |
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
core
Posts with mentions or reviews of core.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2022-08-09.
-
Lakehouse - event data
If you happen to be putting together a platform then look at Gazette. We use it to read/write streaming data into nice and orderly "fragment files" in cloud storage. It provides excellent durability guarantees and turns out to be an extremely efficient way to handle both reads and writes of streaming data. Readers get handed out pre-signed cloud storage URLs that are fetched by the client, and writes are batched into time and/or size bounded "fragments" that are durably replicated until being uploaded into cloud storage.
-
Request for example Real Life programs in Go?
this is a real time data streaming tool built with Go - https://github.com/gazette/core
-
Hacker News top posts: Jul 16, 2022
Gazette – Build platforms that flexibly mix SQL, batch, and stream processing\ (0 comments)
- Gazette – Build platforms that flexibly mix SQL, batch, and stream processing
-
Issues we've encountered while building a Kafka based data processing pipeline
If you're in the Go ecosystem, Gazette [0] offers transactional integrations [1] with remote DB's for stateful processing pipelines, as well as local stores for embedded in-process state management.
It also natively stores data as files in cloud storage. Brokers are ephemeral, you don't need to migrate data between them, and you're not constrained by their disk size. Gazette defaults to exactly-once semantics, and has stronger replication guarantees (your R factor is your R factor, period -- no "in sync replicas").
Estuary Flow [2] is building on Gazette as an implementation detail to offer end-to-end integrations with external SaaS & DB's for building real-time dataflows, as a managed service.
[0]: https://github.com/gazette/core
pg_cron
Posts with mentions or reviews of pg_cron.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2023-06-09.
-
Supabase Beta May 2023
[Postgres Extensions] pg_cron 1.5.2 (new projects only) now supports sub-minute schedules. PR
-
🏺Database Architecture - History Over State
(Your PostgreSQL installation might already have pg_cron available, but if not, you would need to install it)
- Edge Functions or Database Functions?
-
Upgrading to PostgreSQL 15 on Mac OS
I’d set up citusdata/pg_cron in 14, so I’ll need to set that up in 15 as well.
-
PostgreSQL Functions in Typescript
Functions can be executed via a database query or on a regular schedule. Trigger functions are a special type of function that are covered in a previous article. They provide a way to execute an action based on a database event.
-
Pulling Cloud API data to PostgreSQL server (Supabase)
* extraction and loading: the psql-http PostgreSQL extension which is available in Supabase (but not all services like AWS RDS). Not sure how it works with OAuth but something to look into. * automation: use pg_cron to automate the ingest. * transformation: leverage Postgresql JSON functions.
- How to : trigger a MV refresh after another MV refreshed ?
-
Is it conventional to use Redis in an authentication service?
A low-level alternative is using a PostgreSQL-side facility like https://github.com/citusdata/pg_cron - this, however, requires installing a PostgreSQL extension.
- Como agendar execução de consultas e comandos pelo PostgreSQL no RDS
-
Do you know of a robust library that handles persistent job scheduling and queuing using PostgreSQL
If you just want scheduling queries, you can use this https://github.com/citusdata/pg_cron