MixtralKit
ez-openai
MixtralKit | ez-openai | |
---|---|---|
4 | 3 | |
758 | 20 | |
2.2% | - | |
8.1 | 7.0 | |
5 months ago | 8 days ago | |
Python | Python | |
Apache License 2.0 | GNU Affero General Public License v3.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
MixtralKit
-
Paris-Based Startup and OpenAI Competitor Mistral AI Valued at $2B
> Mistral's latest just released model is well below GPT-3 out of the box
The early information I see implies it is above. Mind you, that is mostly because GPT-3 was comparatively low: for instance its 5-shot MMLU score was 43.9%, while Llama2 70B 5-shot was 68.9%[0]. Early benchmarks[1] give Mixtral scores above Llama2 70B on MMLU (and other benchmarks), thus transitively, it seems likely to be above GPT-3.
Of course, GPT-3.5 has a 5-shot score of 70, and it is unclear yet whether Mixtral is above or below, and clearly it is below GPT-4’s 86.5. The dust needs to settle, and the official inference code needs to be released, before there is certainty on its exact strength.
[0]: https://paperswithcode.com/sota/multi-task-language-understa...
[1]: https://github.com/open-compass/MixtralKit#comparison-with-o...
- Benchmarks results reveal Mixtral-8x7B BEATS LLaMA-2-70b
- Inference and Evaluation of Mistral AI's MoE model(Mixtral-8x7b-32kseqlen)
-
Mixtral 8x7B is a scaled-down GPT-4
Inference code: https://github.com/open-compass/MixtralKit Evaluation results will be updated soon
ez-openai
-
Building a local AI smart Home Assistant
I did the same thing, but I went the easy way and used OpenAI's API. Half way through, I got fed up with all the boilerplate, so I wrote a really simple (but very Pythonic) wrapper around function calling with Python functions:
https://github.com/skorokithakis/ez-openai
Then my assistant is just a bunch of Python functions and a prompt. Very very simple.
-
Paris-Based Startup and OpenAI Competitor Mistral AI Valued at $2B
This is just tangential, but I wouldn't call their APIs "nice", I'd be far less charitable. I spent a few hours (because that's how long it took to figure out the API, due to almost zero documentation) and wrote a nicer Python layer:
https://github.com/skorokithakis/ez-openai/
With all that money, I would have thought they'd be able to design more user-friendly APIs. Maybe they could even ask an LLM for help.