AutoAWQ Alternatives
Similar projects and alternatives to AutoAWQ
-
-
text-generation-webui
A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
-
sglang
SGLang is a structured generation language designed for large language models (LLMs). It makes your interaction with models faster and more controllable.
AutoAWQ reviews and mentions
-
AMD ROCm Software Blogs
Thanks! Again, partnerships over customers. If you're experienced and have the technical chops to make a MI300x sing, we want to work with you. Our model is that we are the capex/opex investor for businesses. As much as I love software, Hot Aisle is more of a hardware business. Running super high end large scale compute is an extreme challenge in itself. We are less interested in building the software side of things and want to foster those who can focus on that side.
https://github.com/unslothai/unsloth/issues/160
https://github.com/search?q=repo%3Apredibase%2Florax+rocm&ty...
https://github.com/sgl-project/sglang/issues/157
https://github.com/casper-hansen/AutoAWQ (supports rocm)
-
1,200 tokens per second for Llama 2 7B on H100!
Apparently you can reach about 800 t/s with rtx 4090 using autoawq with batch size 8. https://github.com/casper-hansen/AutoAWQ
Stats
casper-hansen/AutoAWQ is an open source project licensed under MIT License which is an OSI approved license.
The primary programming language of AutoAWQ is Python.
Popular Comparisons
Sponsored