kafka-streams-dashboards
showcases Grafana dashboards for Kafka Stream applications leveraging client JMX metrics. (by kineticedge)
llm-awq
[MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration (by mit-han-lab)
| kafka-streams-dashboards | llm-awq | |
|---|---|---|
| 2 | 7 | |
| 33 | 1,902 | |
| - | 10.9% | |
| 6.9 | 8.0 | |
| 4 days ago | 8 days ago | |
| Java | Python | |
| Apache License 2.0 | MIT License |
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
kafka-streams-dashboards
Posts with mentions or reviews of kafka-streams-dashboards.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2023-10-23.
llm-awq
Posts with mentions or reviews of llm-awq.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2023-10-23.
-
TinyChat: Large Language Model on the Edge
TinyChat is an efficient, lightweight, Python-native serving framework for 4-bit LLMs by AWQ. It delivers 2.3x generation speed up on RTX4090.
Code: https://github.com/mit-han-lab/llm-awq/tree/main/tinychat
- FLaNK Stack Weekly 23 Oct 2023
-
New base model InternLM 7B weights released, with 8k context window.
I am having trouble finding any 8bit GPTQ models at all, there don't seem to be any on HF it's almost all 4bit with the odd 3bit of the big ones. Suspect I will have to make my own for eval purposes but it's lower priority on my list then finding a 4bit that's GPU friendly but doesn't have such a performance penalty... Looking at AWQ they have 3 and 4bit versions.
-
Llama33B vs Falcon40B vs MPT30B
Using the currently popular gptq the 3bit quantization hurts performance much more than 4bit, but there's also awq (https://github.com/mit-han-lab/llm-awq) and squishllm (https://github.com/SqueezeAILab/SqueezeLLM) which are able to manage 3bit without as much performance drop - I hope to see them used more commonly.
- New hardware-friendly quantization method
-
Activation-Aware Weight Quantization for LLM Compression Outperforms GPTQ
Better quantization would have a direct and meaningful impact for everyone running local LLMs. The technique has already been applied to both Vicuna and the multimodal LLaMA variant LLaVA.
https://github.com/mit-han-lab/llm-awq
-
New quantization method AWQ outperforms GPTQ in 4-bit and 3-bit with 1.45x speedup and works with multimodal LLMs
GitHub: https://github.com/mit-han-lab/llm-awq
What are some alternatives?
When comparing kafka-streams-dashboards and llm-awq you can also consider the following projects:
langchain4j-examples
SqueezeLLM - [ICML 2024] SqueezeLLM: Dense-and-Sparse Quantization
lang2sql - A tutorial for setting an SQL code generator with the OpenAI API
GPTQ-for-LLaMa - 4 bits quantization of LLaMA using GPTQ
data-in-motion - This is repository for tutorials of Data In Motion starting with Data Distribution
Voyager - An Open-Ended Embodied Agent with Large Language Models
optiagent - autonomous agents for competitive intelligence!
kpow-local - Run Local Kafka and Kpow with Docker Compose
CML_AMP_AI_Text_Summarization_with_Amazon_Bedrock - CML_AMP_AI_Text_Summarization_with_Amazon_Bedrock
stable-audio-tools - Generative models for conditional audio generation
kafka-streams-dashboards vs langchain4j-examples
llm-awq vs SqueezeLLM
kafka-streams-dashboards vs lang2sql
llm-awq vs GPTQ-for-LLaMa
kafka-streams-dashboards vs data-in-motion
llm-awq vs Voyager
kafka-streams-dashboards vs optiagent
llm-awq vs langchain4j-examples
kafka-streams-dashboards vs kpow-local
llm-awq vs CML_AMP_AI_Text_Summarization_with_Amazon_Bedrock
kafka-streams-dashboards vs stable-audio-tools
llm-awq vs data-in-motion