client_python
hatchet
client_python | hatchet | |
---|---|---|
15 | 16 | |
3,778 | 3,167 | |
1.3% | 18.7% | |
7.2 | 9.7 | |
7 days ago | 7 days ago | |
Python | Go | |
Apache License 2.0 | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
client_python
-
Show HN: Hatchet – Open-source distributed task queue
Here you go: https://stackoverflow.com/questions/75652326/celery-spawn-si...
Plus some adjacent discussion on GitHub: https://github.com/prometheus/client_python/issues/902
Hope that helps!
-
How to monitor Python application performance
Prometheus, which is also a CNCF open source project, collects metrics data by scraping HTTP endpoints and then stores that data in a time series database that uses a multidimensional model. It’s a powerful tool for gathering metrics about your application and it also includes alerting functionality that you can use to notify your teams when issues come up. Prometheus includes a client library for Python.
-
Kafka-Python metric reporters
We have a java one but the principle is the same. Install the Prometheus client ( https://github.com/prometheus/client_python) ,create the metrics you want, then push jmx settings to Prometheus.
- Observabilidade com Prometheus
-
Setup Grafana with Prometheus for Python projects using Docker
The code above is copied from the official documentation of prometheus_client which simply creates a new metric named request_processing_seconds that measures the time spent on that particular request. We'll cover other types of metrics later in this post.
-
Prometheus histogram with python
Just use the client? https://github.com/prometheus/client_python
-
Monitoring Latency with Python
I've experimented with the official Prometheus python client, i really really like the way they use decorators to instrument. I've tried to measure latency with multiple types of metrics (histogram, & summary), i see the value in both of them, but the one that between fits my objective is the histogram metric type. Great!
-
Best way to handle several python script plugins for a service? Create an image + container for each one? Create one for them all? Running them as microservices?
Now is a good time to expand your event loop by adding metrics collection of the event handler functions and also use that endpoint as a liveness probe. E.g. https://github.com/prometheus/client_python just add the event handled, success/error and the duration as a histogram (look for examples of tracking http requests served)
-
Why is Prometheus generating duplicate data (while using python client)?
I've spent along time trying to figure out a bug that I'm facing while using Prometheus from its python client.
-
Python node exporter *Help
The official Prometheus Python client library makes this easy, no need to worry about the export file format.
hatchet
-
Ask HN: Who is hiring? (April 2024)
Hatchet (https://hatchet.run) | New York City | Full-time
We're hiring a founding engineer to help us with development on our open-source, distributed task queue: https://github.com/hatchet-dev/hatchet.
We recently launched on HN, you can check out our launch here: https://news.ycombinator.com/item?id=39643136. We're two second-time YC founders in this for the long haul and we are just wrapping up the YC W24 batch.
As a founding engineer, you'll be responsible for contributing across the entire codebase. We'll compensate accordingly and with high equity. It's currently just the two founders + a part-time contractor. We're all technical and contribute code.
Stack: Typescript/React, Go and PostgreSQL.
To apply, email alexander [at] hatchet [dot] run, and include the following:
1. Tell us about something impressive you've built.
2. Ask a question or write a comment about the state of the project. For example: a file that stood out to you in the codebase, a Github issue or discussion that piqued your interest, a general comment on distributed systems/task queues, or why our code is bad and how you could improve it.
-
Show HN: Hatchet – Open-source distributed task queue
Can you explain why you chose every function to take in context? https://github.com/hatchet-dev/hatchet/blob/main/python-sdk/...
This seems like a lot of boiler plate to write functions with to me (context I created http://github.com/DAGWorks-Inc/hamilton).
Hello HN, we're Gabe and Alexander from Hatchet (https://hatchet.run), we're working on an open-source, distributed task queue. It's an alternative to tools like Celery for Python and BullMQ for Node.js, primarily focused on reliability and observability. It uses Postgres for the underlying queue.
Why build another managed queue? We wanted to build something with the benefits of full transactional enqueueing - particularly for dependent, DAG-style execution - and felt strongly that Postgres solves for 99.9% of queueing use-cases better than most alternatives (Celery uses Redis or RabbitMQ as a broker, BullMQ uses Redis). Since the introduction of SKIP LOCKED and the milestones of recent PG releases (like active-active replication), it's becoming more feasible to horizontally scale Postgres across multiple regions and vertically scale to 10k TPS or more. Many queues (like BullMQ) are built on Redis and data loss can occur when suffering OOM if you're not careful, and using PG helps avoid an entire class of problems.
We also wanted something that was significantly easier to use and debug for application developers. A lot of times the burden of building task observability falls on the infra/platform team (for example, asking the infra team to build a Grafana view for their tasks based on exported prom metrics). We're building this type of observability directly into Hatchet.
What do we mean by "distributed"? You can run workers (the instances which run tasks) across multiple VMs, clusters and regions - they are remotely invoked via a long-lived gRPC connection with the Hatchet queue. We've attempted to optimize our latency to get our task start times down to 25-50ms and much more optimization is on the roadmap.
We also support a number of extra features that you'd expect, like retries, timeouts, cron schedules, dependent tasks. A few things we're currently working on - we use RabbitMQ (confusing, yes) for pub/sub between engine components and would prefer to just use Postgres, but didn't want to spend additional time on the exchange logic until we built a stable underlying queue. We are also considering the use of NATS for engine-engine and engine-worker connections.
We'd greatly appreciate any feedback you have and hope you get the chance to try out Hatchet.
-
Show HN: R2R – Open-source framework for production-grade RAG
This is a great question, thanks for asking.
We are testing workflows internally that use orchestration software like Hatchet/Temporal to allow the framework to robustly handle 100s of GBs of upload data from parsing to chunking to embedding to storing [1][2]. The goal is to build durable execution at each step, because even steps like PDF extraction can be expensive / time consuming. We are targeting an prelim. release of these features in < 1 month.
Logging is built natively into the framework with postgres or sqlite options. We ship a GUI that leverages these logs and the application flow to allow developers to see queries, search results, and RAG completions in realtime.
We are planning on adding more features here to help with evaluation / insight as we get further feedback.
On the A/B, slow rollout, and analytics side, we are still early but suspect there is a lot of value to be had here, particularly because human feedback is pretty crucial in optimizing any RAG system. Developer feedback will be particularly important here since there are a lot of paths to choose between.
[1] https://hatchet.run/
- Show HN: Hatchet – open-source, event-based workflow engine
- Hatchet – open-source workflow engine for Go applications
-
Hatchet — yet another TFC/TFE open-source alternative
Absolutely -- just created an issue if you'd like to follow along or provide feedback!
What are some alternatives?
prometheus-fastapi-instrumentator - Instrument your FastAPI with Prometheus metrics.
otf - An open source alternative to terraform enterprise.
django-prometheus - Export Django monitoring metrics for Prometheus.io
conductor - Conductor is an event driven orchestration platform
netbox-plugin-prometheus-sd - Provide Prometheus url_sd compatible API Endpoint with data from Netbox
hn-search - Hacker News Search
pushgateway - Push acceptor for ephemeral and batch jobs.
terrakube - Open source IaC Automation and Collaboration Software.
node_exporter - Exporter for machine metrics
wakaq-ts - Background task queue for TypeScript backed by Redis, a super minimal Celery
statsd_exporter - StatsD to Prometheus metrics exporter
gue - Golang queue on top of PostgreSQL