dagster
fastapi
Our great sponsors
dagster | fastapi | |
---|---|---|
46 | 466 | |
10,215 | 70,779 | |
4.8% | - | |
10.0 | 9.8 | |
about 8 hours ago | 6 days ago | |
Python | Python | |
Apache License 2.0 | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
dagster
- Experience with Dagster.io?
-
Dagster tutorials
My recommendation is to continue on with the tutorial, then look at one of the larger example projects especially the ones named “project_”, and you should understand most of it. Of what you don't understand and you're curious about, look into the relevant concept page for the functions in the docs.
-
The Dagster Master Plan
I found this example that helped me - https://github.com/dagster-io/dagster/tree/master/examples/project_fully_featured/project_fully_featured
-
What are some open-source ML pipeline managers that are easy to use?
I would recommend the following: - https://www.mage.ai/ - https://dagster.io/ - https://www.prefect.io/ - https://metaflow.org/ - https://zenml.io/home
-
The Why and How of Dagster User Code Deployment Automation
In Helm terms: there are 2 charts, namely the system: dagster/dagster (values.yaml), and the user code: dagster/dagster-user-deployments (values.yaml). Note that you have to set dagster-user-deployments.enabled: true in the dagster/dagster values-yaml to enable this.
-
Best Orchestration Tool to run dbt projects?
Dagster seemed really cool when I looked into it as an alternative to airflow. I especially like the software defined assets and built-in lineage which I haven't seen in any other tool. However it seems it does not support RBAC which is a pretty big issue if you want a self-service type of architecture, see https://github.com/dagster-io/dagster/issues/2219. It does seem like it's available in their hosted version, but I wanted to run it myself on k8s.
-
dbt Cloud Alternatives?
Dagster? https://dagster.io
-
What's the best thing/library you learned this year ?
One that I haven't seen on here yet: dagster
- Anyone have an example of a project where a handful of the more popular Python tools are used? (E.g. airbyte, airflow, dbt, and pandas)
- Can we take a moment to appreciate how much of dataengineering is open source?
fastapi
-
Building an Email Assistant Application with Burr
In this tutorial, I will demonstrate how to use Burr, an open source framework (disclosure: I helped create it), using simple OpenAI client calls to GPT4, and FastAPI to create a custom email assistant agent. We’ll describe the challenge one faces and then how you can solve for them. For the application frontend we provide a reference implementation but won’t dive into details for it.
-
FastAPI Got Me an OpenAPI Spec Really... Fast
That’s when I found FastAPI.
-
How to Deploy a Fast API Application to a Kubernetes Cluster using Podman and Minikube
FastAPI & Uvicorn
-
Analysing FastAPI Middleware Performance
Discussion at FastAPI GitHub: https://github.com/tiangolo/fastapi/issues/2696
-
LangChain, Python, and Heroku
An API application framework (such as FastAPI)
-
Litestar – powerful, flexible, and highly performant Python ASGI framework
It’s been my experience that async Python frameworks tend to turn IO bound problems into CPU bound problems with a high enough request rate, because due to their nature they act as unbounded queues.
This ends up made worse if you’re using sync routes.
If you’re constrained on a resource such as a database connection pool, your framework will continue to pull http requests off the wire that a sane client will cancel and retry due to timeouts because it takes too long to get a connection out of the pool. Since there isn’t a straightforward way to cancel the execution of a route handler in every Python http framework I’ve seen exhibit this problem, the problem quickly snowballs.
This is an issue with fastapi, too- https://github.com/tiangolo/fastapi/issues/5759
-
AI-Powered Image Search with CLIP, pgvector, and Fast API
Fast API.
- Ask HN: What is your go-to stack for the web?
-
Fun with Avatars: Crafting the core engine | Part. 1
We will create our API using FastAPI, a modern high-performance web framework for building fast APIs with Python. It is designed to be easy to use, efficient, and highly scalable. Some key features of FastAPI include:
-
Building Fast APIs with FastAPI: A Comprehensive Guide
FastAPI is a modern, fast, web framework for building APIs with Python 3.7+ based on standard Python type hints. It is designed to be easy to use, fast to run, and secure. In this blog post, we’ll explore the key features of FastAPI and walk through the process of creating a simple API using this powerful framework.
What are some alternatives?
Prefect - The easiest way to build, run, and monitor data pipelines at scale.
AIOHTTP - Asynchronous HTTP client/server framework for asyncio and Python
Airflow - Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
HS-Sanic - Async Python 3.6+ web server/framework | Build fast. Run fast. [Moved to: https://github.com/sanic-org/sanic]
Mage - 🧙 The modern replacement for Airflow. Mage is an open-source data pipeline tool for transforming and integrating data. https://github.com/mage-ai/mage-ai
Tornado - Tornado is a Python web framework and asynchronous networking library, originally developed at FriendFeed.
airbyte - The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
django-ninja - 💨 Fast, Async-ready, Openapi, type hints based framework for building APIs
MLflow - Open source platform for the machine learning lifecycle
Flask - The Python micro framework for building web applications.
meltano
swagger-ui - Swagger UI is a collection of HTML, JavaScript, and CSS assets that dynamically generate beautiful documentation from a Swagger-compliant API.