ctransformers
kendryte-standalone-sdk
| ctransformers | kendryte-standalone-sdk | |
|---|---|---|
| 4 | 4 | |
| 1,718 | 421 | |
| - | 0.0% | |
| 8.6 | 0.0 | |
| 4 months ago | over 2 years ago | |
| C | C | |
| MIT License | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
ctransformers
-
Refact LLM: New 1.6B code model reaches 32% HumanEval and is SOTA for the size
Does ctransformer (https://github.com/marella/ctransformers#supported-models) support running refact?
I see that model type "gpt_refact" in https://huggingface.co/smallcloudai/Refact-1_6B-fim/blob/mai...
-
How do I utilize these quantized models being uploaded?
You can also use ctransformers with the ggml models if you want to use python rather than c++.
-
Langchain and self hosted LLaMA hosted API
For ggml https://github.com/marella/ctransformers/ and https://github.com/abetlen/llama-cpp-python has a decent server. https://github.com/go-skynet/LocalAI is very active too.
- Also reconnecting with Scala. Interested in LLMs
kendryte-standalone-sdk
-
Canaan K510 CRB RISC-V AI development kit ships with dual-camera module and LCD display
Here is link number 1 - Previous text "SDK"
-
What is happening with K210?
What is happening with Canaan and K210? They stopped developing the FreeRTOS SDK. Last release for the Standalone SDK was in 2019. Documentation is almost nonexistent.
-
Kendryte K210. Is GPKPU possible?
See kpu_conv2d() in https://github.com/kendryte/kendryte-standalone-sdk/blob/develop/lib/nncase/include/kernels/k210/k210_kernels.h for a SW implementation of the KPU.
What are some alternatives?
llama-cpp-python - Python bindings for llama.cpp
MaixPy - Easily create AI projects with Python on edge device
LangChain_PDFChat_Oobabooga - oobaboga -text-generation-webui implementation of wafflecomposite - langchain-ask-pdf-local
artificial-nose - Instructions, source code, and misc. resources needed for building a Tiny ML-powered artificial nose.
text-generation-inference - Large Language Model Text Generation Inference
kendryte-freertos-sdk - This project is no longer maintained Not recommended for product development.
risc-v-examples - RISC-V examples for GD32V, K210, and QEMU
text-generation-webui - A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.
riscv-gnu-toolchain - GNU toolchain for RISC-V, including GCC
lamp - deep learning and scientific computing framework with native CPU and GPU backend for the Scala programming language
kendryte-gnu-toolchain - Kendryte GNU Toolchain