i need some help guys

• koboldcpp

  180 4,187 10.0 C++

  A simple one-file way to run various GGML and GGUF models with KoboldAI's UI

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• runpodctl

  8 223 9.4 Go

  🧰 | RunPod CLI for pod management

  

Another option is to use a service like www.runpod.io to rent time on more powerful systems. A few times a week I’ll load up whatever the latest 13B or 20/24B (and even low bpw 70Bs with EXL2) model is on a system with an rtx 3090 for $0.44/hr, and sometimes I’ll treat myself to an A6000 system to run 4bit 70Bs for $0.79/hr. They also offer A4000 systems with 16GB VRAM which is plenty to run a 4bpw 13B EXL2 model or an 8bpw 7B model, and those systems are just $0.36/hr.

Suggest a related project