-
Basic-UI-for-GPT-J-6B-with-low-vram
A repository to run gpt-j-6b on low vram machines (4.2 gb minimum vram for 2000 token context, 3.5 gb for 1000 token context). Model loading takes 12gb free ram.
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
You need a lot of VRAM to run the AI models, scaling somewhat with the amount of parameters a model uses. The most advanced model Pygmalion has is 6 billion parameters, which requires a minimum of 16GB of VRAM to run locally at decent speeds. There are methods of running 6b locally on low VRAM machines as listed here: https://github.com/arrmansa/Basic-UI-for-GPT-J-6B-with-low-vram but even then, the generations would be excruciatingly slow, and the lowest VRAM card used with this method has 6GB of VRAM.