termux-ndk
llama.cpp
| termux-ndk | llama.cpp | |
|---|---|---|
| 12 | 773 | |
| 465 | 57,463 | |
| - | - | |
| 3.5 | 10.0 | |
| 4 months ago | about 3 hours ago | |
| C | C++ | |
| GNU General Public License v3.0 or later | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
termux-ndk
-
Hosting Your Own AI Chatbot on Android Devices
Next, we need to set up the Android NDK (Native Development Kit) to compile the llama.cpp project. Visit the Termux-NDK repository and download the latest NDK release. Extract the ZIP file, then set the NDK path in Termux:
-
How install Koboldcpp via Termux (Guide)
10 - Install the Android SDK by copying and pasting the following command in the console: wget https://github.com/lzhiyong/termux-ndk/releases/download/ndk-r23/android-ndk-r23c-aarch64.zip unzip android-ndk-r23c-aarch64.zip export NDK=~/android-ndk-r23c-aarch64
-
Pain
You can use termux. r/termux. For ndk you can use https://github.com/Lzhiyong/termux-ndk for cmake below 3.14 (I think) you have to use a prooted distro. For build-tools you cant use sdkmanager unless you use android studio in GUI mode . you have to download manually . there is a website for that . and that's how you build android apps in Android.
- I need a help 🙏
- basic xfce session
-
Android Studio ARM build
And for ndk I guess you can use this if there is no arm official build. https://github.com/Lzhiyong/termux-ndk
-
Documentation of building Android apps in termux.
And the docs : https://github.com/Lzhiyong/termux-ndk/tree/master/build-app
-
Static binaries clang termux .
Found this https://github.com/Lzhiyong/termux-ndk/blob/master/patches/align_fix.py to patch the binary. But why do I need to patch it?
-
Terminal emulator for X
Did you install ndk from here? https://github.com/Lzhiyong/termux-ndk
-
Flutter with code-server, looking for aarch64 based Android SDK.
You can check out this repo
llama.cpp
-
Better and Faster Large Language Models via Multi-Token Prediction
For anyone interested in exploring this, llama.cpp has an example implementation here:
https://github.com/ggerganov/llama.cpp/tree/master/examples/...
- Llama.cpp Bfloat16 Support
-
Fine-tune your first large language model (LLM) with LoRA, llama.cpp, and KitOps in 5 easy steps
Getting started with LLMs can be intimidating. In this tutorial we will show you how to fine-tune a large language model using LoRA, facilitated by tools like llama.cpp and KitOps.
- GGML Flash Attention support merged into llama.cpp
-
Phi-3 Weights Released
well https://github.com/ggerganov/llama.cpp/issues/6849
- Lossless Acceleration of LLM via Adaptive N-Gram Parallel Decoding
- Llama.cpp Working on Support for Llama3
-
Embeddings are a good starting point for the AI curious app developer
Have just done this recently for local chat with pdf feature in https://recurse.chat. (It's a macOS app that has built-in llama.cpp server and local vector database)
Running an embedding server locally is pretty straightforward:
- Get llama.cpp release binary: https://github.com/ggerganov/llama.cpp/releases
- Mixtral 8x22B
- Llama.cpp: Improve CPU prompt eval speed
What are some alternatives?
android-iperf - Pre-compiled iperf/iperf3 binaries for Android + Dockerfile with SDK and NDK for manual build
ollama - Get up and running with Llama 3, Mistral, Gemma, and other large language models.
llvm-project - The LLVM Project is a collection of modular and reusable compiler and toolchain technologies. Note: the repository does not accept github pull requests at this moment. Please submit your patches at http://reviews.llvm.org.
gpt4all - gpt4all: run open-source LLMs anywhere
cling - The cling C++ interpreter
text-generation-webui - A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.
play-audio - Command-line tool to play audio using OpenSL ES.
GPTQ-for-LLaMa - 4 bits quantization of LLaMA using GPTQ
zapcc - zapcc is a caching C++ compiler based on clang, designed to perform faster compilations
ggml - Tensor library for machine learning
Termux-Java - Install Java (Open-JDK-8) in Termux without root!
alpaca.cpp - Locally run an Instruction-Tuned Chat-Style LLM