shady.ai
more-ane-transformers
| shady.ai | more-ane-transformers | |
|---|---|---|
| 1 | 4 | |
| 107 | 35 | |
| - | - | |
| 7.6 | 7.0 | |
| 3 months ago | 6 months ago | |
| Dart | Python | |
| GNU Affero General Public License v3.0 | - |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
shady.ai
-
The Coming of Local LLMs
I’ve got some of their smaller Raven models running locally on my M1 (only 16GB of RAM).
I’m also in the middle of making it user friendly to run these models on all platforms (built with Flutter). First MacOS release will be out before this weekend: https://github.com/BrutalCoding/shady.ai
more-ane-transformers
- M2 Ultra can run 128 streams of Llama 2 7B in parallel
- Is it possible to use ANE(Apple Neural Engine) to run those models?
-
The Coming of Local LLMs
Apple should get working on a version of the Neural Engine that is useful for these models, and remove the 3GB size limit [1] to take full advantage of the 'unified' memory architecture. Game changer.
Waste of die space currently
[1] https://github.com/smpanaro/more-ane-transformers/blob/main/...
- Anthropic’s $5B, 4-year plan to take on OpenAI
What are some alternatives?
StudentAI - StudentAI is an prompt-less AI chatbot app that uses OpenAI's large language model to help students learn more effectively. StudentAI can answer questions, provide explanations, and even generate creative content. This makes it a powerful tool for students of all ages and levels of learning.
pyllms - Minimal Python library to connect to LLMs (OpenAI, Anthropic, AI21, Cohere, Aleph Alpha, HuggingfaceHub, Google PaLM2, with a built-in model performance benchmark.
flutter_ci_cd - CI/CD & branching template for flutter apps
neural-engine - Everything we actually know about the Apple Neural Engine (ANE)
Flutter-AssetsAudioPlayer - Play simultaneously music/audio from assets/network/file directly from Flutter, compatible with android / ios / web / macos, displays notifications
whisper.coreml - Robust Speech Recognition via Large-Scale Weak Supervision
rwkv.cpp - INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model
tinygrad - You like pytorch? You like micrograd? You love tinygrad! ❤️ [Moved to: https://github.com/tinygrad/tinygrad]
duckduckgo-locales - Translation files for <a href="https://duckduckgo.com"> </a>
experiments-coreml-ane-distilbert - Experimenting with https://github.com/apple/ml-ane-transformers
ollama - Get up and running with Llama 3, Mistral, Gemma, and other large language models.
llama.cpp - LLM inference in C/C++