LLMs up to 4x Faster With latest Nvidia drivers on Windows

• code-llama-for-vscode

  5 512 4.6 Python

  Use Code Llama with Visual Studio Code and the Continue extension. A local LLM alternative to GitHub Copilot.

Do you use https://github.com/xNul/code-llama-for-vscode or something else?

Haven’t found any good setup instructions for Linux or my Google skills are failing me.

• text-generation-webui

  876 36,552 9.9 Python

  A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• llama.cpp

  774 57,463 10.0 C++

  LLM inference in C/C++

I dont understand, are you claiming non-Apple devices cannot run LLMs?

https://github.com/ggerganov/llama.cpp/issues/34

If you meant eGPU support, IIRC that is beta for everyone right now.

Suggest a related project