-
Basic-UI-for-GPT-J-6B-with-low-vram
A repository to run gpt-j-6b on low vram machines (4.2 gb minimum vram for 2000 token context, 3.5 gb for 1000 token context). Model loading takes 12gb free ram.
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
While waiting however I came across https://github.com/arrmansa/Basic-UI-for-GPT-J-6B-with-low-vram which allows you to use some of system memory to run the model. I was able to get a version working with 2.7B on my 2060 6GB with KoboldAI. The github above has an error that prevents it from working (https://github.com/arrmansa/Basic-UI-for-GPT-J-6B-with-low-vram/issues/1), but other than that it works.