feast
Poetry
feast | Poetry | |
---|---|---|
8 | 378 | |
5,303 | 29,745 | |
1.6% | 1.7% | |
9.5 | 9.7 | |
about 22 hours ago | 2 days ago | |
Python | Python | |
Apache License 2.0 | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
feast
- What's Happening with Feast?
-
Running The Feast Feature Store With Dragonfly
Feast stands as an exceptional open-source feature store, revolutionizing the efficient management and uninterrupted serving of machine learning (ML) features for real-time applications. At its core, Feast offers a sophisticated interface for storing, discovering, and accessing features—the individual measurable properties or characteristics of data essential for ML modeling. Operating on a distributed architecture, Feast harmoniously integrates several pivotal components, including the Feast Registry, Stream Processor, Batch Materialization Engine, and Stores.
-
Ask HN: How to Break into AI Engineering
AI Engineering is basically Data Engineering focused on AI. When in "traditional" Data Engineering you create pipelines that store processed data in something like a Data Lake, in AI Eng. your end storage might be a specialized Feature Storage (like Feast or GCP Vertex AI).
There are some AI Engineers with strong scientific/mathematical background, but that's rare. Usually, you're paired with these ML people that actually develop and evaluate the models.
So my advice is to start with Data Engineering and then find a specialization AI. You should have a VERY solid foundation on scripting and programming, specially Python. Also, a lot of concepts of "data wrangling". Understanding how data flows from point A to point B, how the intermediate storages and streaming engines work, etc. Functional programming is key here.
[0] https://github.com/feast-dev/feast
-
In Need of Guidance: Implementing MLOps in a Complex Organization as a Junior Data Engineer
A feature store usually stores features which are used for training ML model. It is a centralized place for collaboration between data engineer, ML engineer, and data scientist, so that data engineer can write to the feature store while ML engineer and data scientist read from it. Hopsworks https://www.hopsworks.ai and feast https://github.com/feast-dev/feast are examples of open source feature store.
- [D] Your 🫵 Preferred Feature Stores?
-
[P] Announcing Feast 0.10: The simplest way to serve features in production
Github: https://github.com/feast-dev/feast
-
[D] What’s the simplest, most lightweight but complete and 100% open source MLOps toolkit? -> MY OWN CONCLUSIONS
Have you looked at Feats as a Feature Store solution? It seems promising but I haven't really looked into it yet though.
- Feast: OSS Feature Store for Production ML
Poetry
-
Understanding Dependencies in Programming
You can manage dependencies in Python with the package manager pip, which comes pre-installed with Python. Pip allows you to install and uninstall Python packages, and it uses a requirements.txt file to keep track of which packages your project depends on. However, pip does not have robust dependency resolution features or isolate dependencies for different projects; this is where tools like pipenv and poetry come in. These tools create a virtual environment for each project, separating the project's dependencies from the system-wide Python environment and other projects.
-
Implementing semantic image search with Amazon Titan and Supabase Vector
Poetry provides packaging and dependency management for Python. If you haven't already, install poetry via pip:
-
From Kotlin Scripting to Python
Poetry
-
How to Enhance Content with Semantify
The Semantify repository provides an example Astro.js project. Ensure you have poetry installed, then build the project from the root of the repository:
-
Uv: Python Packaging in Rust
Has anyone else been paying attention to how hilariously hard it is to package PyTorch in poetry?
https://github.com/python-poetry/poetry/issues/6409
-
Boring Python: dependency management (2022)
Based on this comment 5 days ago[0], it's working? I'm not sure didn't dig in too far but based on that comment it seems fair to say that it's not fully Poetry's fault because torch removed hashes (which poetry needs to be effective) for a while only recently adding it back in.
Not sure where I would stand if I fully investigated it tho.
[0] https://github.com/python-poetry/poetry/issues/6409#issuecom...
-
Fun with Avatars: Crafting the core engine | Part. 1
We will be running this project in Python 3.10 on Mac/Linux, and we will use Poetry to manage our dependencies. Later, we will bundle our app into a container using docker for deployment.
-
Python Packaging, One Year Later: A Look Back at 2023 in Python Packaging
Here are the two main packaging issues I run into, specifically when using Poetry:
1) Lack of support for building extension modules (as mentioned by the article). There is a workaround using an undocumented feature [0], which I've tried, but ultimately decided it was not the right approach. I still use Poetry, but build the extension as a separate step in CI, rather than kludging it into Poetry.
2) Lack of support for offline installs [1], e.g. being able to download the dependencies, copy them to another machine, and perform the install from the downloaded dependencies (similar to using "pip --no-index --find-links=."). Again, you can work around this (by using "poetry export --with-credentials" and "pip download" for fetching the dependencies, then firing up pypiserver [2] to run a local PyPI server on the offline machine), but ideally this would all be a first class feature of Poetry, similar to how it is in pip.
I don't have the capacity to create Pull Requests for addressing these issues with Poetry, and I'm very grateful for the maintainers and those who do contribute. Instead, on the linked issues I share my notes on the matter, in the hope that it may at least help others and potentially get us closer to a solution.
Regardless, I'm sticking with Poetry for now. Though to be fair, the only other Python packaging tools I've used extensively are Pipenv and pip/setuptools. It's time consuming to thoroughly try out these other packaging tools, and is generally lower priority than developing features/fixing bugs, so it's helpful to read about the author's experience with these other tools, such as PDM and Hatch.
[0] https://github.com/python-poetry/poetry/issues/2740
[1] https://github.com/python-poetry/poetry/issues/2184
[2] https://pypi.org/project/pypiserver/
-
Introducing Flama for Robust Machine Learning APIs
We believe that poetry is currently the best tool for this purpose, besides of being the most popular one at the moment. This is why we will use poetry to manage the dependencies of our project throughout this series of posts. Poetry allows you to declare the libraries your project depends on, and it will manage (install/update) them for you. Poetry also allows you to package your project into a distributable format and publish it to a repository, such as PyPI. We strongly recommend you to learn more about this tool by reading the official documentation.
-
How do you resolve dependency conflicts?
I started using poetry. The problem is poetry will not install if there is dependency conflict and there is no way to ignore: github
What are some alternatives?
kedro-great - The easiest way to integrate Kedro and Great Expectations
Pipenv - Python Development Workflow for Humans.
featureform - The Virtual Feature Store. Turn your existing data infrastructure into a feature store.
PDM - A modern Python package and dependency manager supporting the latest PEP standards
Milvus - A cloud-native vector database, storage for next generation AI applications
hatch - Modern, extensible Python project management
metaflow - :rocket: Build and manage real-life ML, AI, and data science projects with ease!
pyenv - Simple Python version management
great_expectations - Always know what to expect from your data.
pip-tools - A set of tools to keep your pinned Python dependencies fresh.
mlrun - MLRun is an open source MLOps platform for quickly building and managing continuous ML applications across their lifecycle. MLRun integrates into your development and CI/CD environment and automates the delivery of production data, ML pipelines, and online applications.
virtualenv - Virtual Python Environment builder