hopsworks
feast
Our great sponsors
| hopsworks | feast | |
|---|---|---|
| 4 | 8 | |
| 1,074 | 5,255 | |
| 1.4% | 1.9% | |
| 9.2 | 9.3 | |
| 5 days ago | 5 days ago | |
| Java | Python | |
| GNU Affero General Public License v3.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
hopsworks
- Hopworks: MLOps platform with Python-centric Feature Store
- Show HN: Feature Store and Model Registry; Hopsworks 3.0
-
[D] Your 🫵 Preferred Feature Stores?
Anyways -> https://github.com/logicalclocks/hopsworks
-
Reflections on the Lack of Adoption of Domain Specific Languages [pdf]
We built the first open-source feature store for ML, https://github.com/logicalclocks/hopsworks , when every existing proprietary feature store (Uber Michelangelo and Bighead at AirBnb) were shouting about how their DSL for feature engineering was the future.
Fast-forward 2 years and it is clear that Data Scientists want to work with Python, not with a DSL. We based our Feature Store on a Dataframe API for Python/PySpark. The DSL can never evolve at the same rate as libraries in a general-purpose programming language. So, your DSL is great for show-casing a Feature Store, but when you need to compute embeddings or train a GAN or done any type of feature engineering that is not a simple time-window aggregation, you pull out Python (or Scala/Java). I am old enough to have seen many DSLs in different domains (GUIs, aspect-oriented programming, feature engineering) have their day in the sun only to be replaced by general-purpose programming languages due to their unmatched utility.
feast
- What's Happening with Feast?
-
Running The Feast Feature Store With Dragonfly
Feast stands as an exceptional open-source feature store, revolutionizing the efficient management and uninterrupted serving of machine learning (ML) features for real-time applications. At its core, Feast offers a sophisticated interface for storing, discovering, and accessing features—the individual measurable properties or characteristics of data essential for ML modeling. Operating on a distributed architecture, Feast harmoniously integrates several pivotal components, including the Feast Registry, Stream Processor, Batch Materialization Engine, and Stores.
-
Ask HN: How to Break into AI Engineering
AI Engineering is basically Data Engineering focused on AI. When in "traditional" Data Engineering you create pipelines that store processed data in something like a Data Lake, in AI Eng. your end storage might be a specialized Feature Storage (like Feast or GCP Vertex AI).
There are some AI Engineers with strong scientific/mathematical background, but that's rare. Usually, you're paired with these ML people that actually develop and evaluate the models.
So my advice is to start with Data Engineering and then find a specialization AI. You should have a VERY solid foundation on scripting and programming, specially Python. Also, a lot of concepts of "data wrangling". Understanding how data flows from point A to point B, how the intermediate storages and streaming engines work, etc. Functional programming is key here.
[0] https://github.com/feast-dev/feast
-
In Need of Guidance: Implementing MLOps in a Complex Organization as a Junior Data Engineer
A feature store usually stores features which are used for training ML model. It is a centralized place for collaboration between data engineer, ML engineer, and data scientist, so that data engineer can write to the feature store while ML engineer and data scientist read from it. Hopsworks https://www.hopsworks.ai and feast https://github.com/feast-dev/feast are examples of open source feature store.
- [D] Your 🫵 Preferred Feature Stores?
-
[P] Announcing Feast 0.10: The simplest way to serve features in production
Github: https://github.com/feast-dev/feast
-
[D] What’s the simplest, most lightweight but complete and 100% open source MLOps toolkit? -> MY OWN CONCLUSIONS
Have you looked at Feats as a Feature Store solution? It seems promising but I haven't really looked into it yet though.
- Feast: OSS Feature Store for Production ML
What are some alternatives?
feathr - Feathr – A scalable, unified data and AI engineering platform for enterprise
kedro-great - The easiest way to integrate Kedro and Great Expectations
featureform - The Virtual Feature Store. Turn your existing data infrastructure into a feature store.
textX - Domain-Specific Languages and parsers in Python made easy http://textx.github.io/textX/
Milvus - A cloud-native vector database, storage for next generation AI applications
OpenMLDB - OpenMLDB is an open-source machine learning database that provides a feature platform computing consistent features for training and inference.
metaflow - :rocket: Build and manage real-life ML, AI, and data science projects with ease!
iwlearn - "Production First" Machine Learning Framework
great_expectations - Always know what to expect from your data.
serverless-ml-course - Serverless Machine Learning Course for building AI-enabled Prediction Services from models and features
mlrun - MLRun is an open source MLOps platform for quickly building and managing continuous ML applications across their lifecycle. MLRun integrates into your development and CI/CD environment and automates the delivery of production data, ML pipelines, and online applications.