Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality. Learn more →
Top 12 experimentation Open-Source Projects
-
-
uptrain
UpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding use-cases), perform root cause analysis on failure cases and give insights on how to resolve them.
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
-
shadow
Shadow is a discrete-event network simulator that directly executes real application code, enabling you to simulate distributed systems with thousands of network-connected processes in realistic and scalable private network experiments using your laptop, desktop, or server running Linux. (by shadow)
-
go-feature-flag
GO Feature Flag is a simple, complete and lightweight self-hosted feature flag solution 100% Open Source. 🎛️
-
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
-
mab
Library for multi-armed bandit selection strategies, including efficient deterministic implementations of Thompson sampling and epsilon-greedy.
-
sigopt-server
Open Source version of SigOpt API, performing hyperparameter optimization and visualization
-
Project mention: GrowthBook: Open-source feature flagging and A/B testing platform | /r/opensource | 2023-10-20
Currently seeking feedback for the developed tool. Would love it if you can check it out on: https://github.com/uptrain-ai/uptrain/blob/main/examples/assistants/assistant_evaluator.ipynb
Feature flags, also known as feature toggles, have become an essential tool for modern software development. They enable developers to control the release of new features and updates in a flexible and controlled manner. In this blog post, we'll explore what feature flags are, their classical and modern use cases, and how to implement them effectively in front-end, mobile, and server-side applications. We'll also delve into the concept of overlapping design A/B testing and how feature flags play a crucial role in building robust experimentation.
Project mention: Turmoil, a framework for developing and testing distributed systems | news.ycombinator.com | 2023-08-17Cool, will be interested to see how this develops! tokio's loom framework has been a big help in testing some tricky concurrency code I've worked on.
Folks interested in this space might also be interested in the system I spend most of my time working on: Shadow. It also performs deterministic simulation of a network of hosts, but it intercepts network and system interactions at the syscall level via seccomp. As such it can work with binaries compiled from ~any language, usually without any code modification or special compilation. https://shadow.github.io/
Project mention: Train to 94% on CIFAR-10 in 3.29 seconds on a single A100 | news.ycombinator.com | 2024-04-04A training speed project building on https://github.com/tysam-code/hlb-CIFAR10 to reach faster times
Project mention: GO Feature Flag is a simple, complete, and lightweight open source self-hosted feature flag solution alternative to Launchdarkly | /r/selfhosted | 2023-07-26
Project mention: Show HN: Open-source LaunchDarkly alternative with GitOps | news.ycombinator.com | 2023-10-30
Project mention: Show HN: MLtraq – Track and Collaborate on AI Experiments (Open-Source) | news.ycombinator.com | 2024-02-06- Source code: https://github.com/elehcimd/mltraq
Thoughts? Looking forward to your feedback. I hope you enjoy it! Thank You!
Project mention: When deployments are easy, code becomes simpler | news.ycombinator.com | 2023-09-05I wanted a very simple feature flagging solution, so I wrote my own with Google Sheets as the backend: https://github.com/stillmatic/flagsheet
It's slightly more complex than a code push but still very easy to use.
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
experimentation related posts
-
Evaluation of OpenAI Assistants
-
Show HN: Open-source LaunchDarkly alternative with GitOps
-
GrowthBook: Open-source feature flagging and A/B testing platform
-
When deployments are easy, code becomes simpler
-
Turmoil, a framework for developing and testing distributed systems
-
Feature Flags: Theory vs. Reality
-
Featurevisor: Git-based feature flags management
-
A note from our sponsor - InfluxDB
www.influxdata.com | 5 May 2024
Index
What are some of the best open-source experimentation projects? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | growthbook | 5,549 |
| 2 | uptrain | 1,999 |
| 3 | featbit | 1,355 |
| 4 | shadow | 1,349 |
| 5 | hlb-CIFAR10 | 1,187 |
| 6 | go-feature-flag | 1,075 |
| 7 | featurevisor | 513 |
| 8 | mab | 46 |
| 9 | sigopt-server | 33 |
| 10 | mltraq | 32 |
| 11 | dead-salmon-brain | 11 |
| 12 | flagsheet | 10 |
Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com