Show HN: Open-source A/B Testing framework

1. lamed

   1 2 43 0.0 Python

   an A/B testing backend using AWS Lambda/API Gateway + Redis.

   

   > Open source A/B testing is a graveyard of abandoned projects

   Quite true, sadly. FWIW, we keep maintaining Alephbet, even though honestly I have no clue who's actively using it besides us :) the codebase is simple enough that it doesn't require a lot of work luckily.

   Re stats: I did implement a Bayesian dashboard of sorts with Alephbet, but I'm not sure it prevents the peeking problem on its own. It requires some discipline when planning the tests to decide ahead when to look at results. Disclaimer: my stats chops are virtually non-existent, but that's what I learned over the years. Georgi's platform really helps structure this process of planning and when to stop the experiment (either when successful or when it's failed).

   Another small (but in my experience important) thing that sets Alephbet apart from other A/B testing platforms: ad blockers. Mixpanel, GA, Amplitude etc frequently and trivially get blocked by ad blockers on the client. For client-side A/B tests this can reduce the data quality (even though typically A/B tests are not privacy invasive). Alephbet's Lamed[0] backend allows you to create a custom AWS url that's far less likely to get blocked. The "data quality" with Alephbet is higher in my experience than the data we see, e.g. in Amplitude.

   [0] https://github.com/Alephbet/lamed

2. Sevalla

   sevalla.com featured

   Deploy and host your apps and databases, now with $50 credit! Sevalla is the PaaS you have been looking for! Advanced deployment pipelines, usage-based pricing, preview apps, templates, human support by developers, and much more!

   

3. alephbet

   2 2 285 2.8 JavaScript

   AlephBet is a pure-javascript A/B (multivariate) testing framework for developers.

   

   Wow this looks very polished. (Out of frustration with Optimizely) I created and maintain a couple of A/B test open source projects[0][1] but the statistical analysis was always the hardest part so I’m keen to see what you are doing. We’re currently relying on a commercial tool called Analytics Toolkit[2] for this part alone and have been quite happy with it though. The owner is very knowledgable and responsive (no affiliation just happy customers). I wonder if you can adopt similar ideas/algorithms into the open source tool. That can be useful I imagine.

   [0] https://github.com/Alephbet/alephbet

   [1] https://github.com/Alephbet/lamed

   [2] https://www.analytics-toolkit.com/

4. sixpack

   3 2 1,763 0.0 Python

   Sixpack is a language-agnostic a/b-testing framework

   

   This reminds me of https://github.com/sixpack/sixpack which I've been eyeballing for years.

Suggest a related project