Basic-UI-for-GPT-J-6B-with-low-vram

A repository to run gpt-j-6b on low vram machines (4.2 gb minimum vram for 2000 token context, 3.5 gb for 1000 token context). Model loading takes 12gb free ram. (by arrmansa)

Basic-UI-for-GPT-J-6B-with-low-vram Alternatives

Similar projects and alternatives to Basic-UI-for-GPT-J-6B-with-low-vram

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a better Basic-UI-for-GPT-J-6B-with-low-vram alternative or higher similarity.

Basic-UI-for-GPT-J-6B-with-low-vram reviews and mentions

Posts with mentions or reviews of Basic-UI-for-GPT-J-6B-with-low-vram. We have used some of these posts to build our list of alternatives and similar projects.
  • How to run this service with a local GPU?
    1 project | /r/PygmalionAI | 27 Jan 2023
    You need a lot of VRAM to run the AI models, scaling somewhat with the amount of parameters a model uses. The most advanced model Pygmalion has is 6 billion parameters, which requires a minimum of 16GB of VRAM to run locally at decent speeds. There are methods of running 6b locally on low VRAM machines as listed here: https://github.com/arrmansa/Basic-UI-for-GPT-J-6B-with-low-vram but even then, the generations would be excruciatingly slow, and the lowest VRAM card used with this method has 6GB of VRAM.
  • Tesla M40 and GPT-J-6B
    1 project | /r/KoboldAI | 8 Aug 2021
    While waiting however I came across https://github.com/arrmansa/Basic-UI-for-GPT-J-6B-with-low-vram which allows you to use some of system memory to run the model. I was able to get a version working with 2.7B on my 2060 6GB with KoboldAI. The github above has an error that prevents it from working (https://github.com/arrmansa/Basic-UI-for-GPT-J-6B-with-low-vram/issues/1), but other than that it works.
  • How is any of this even possible?
    1 project | /r/GPT3 | 21 Jul 2021
    Just to add to this, there is a low VRAM version of GPT-J here (suggest 16GB RAM + 8GB GPU).
  • GPT-J 6B locally on my computer
    1 project | /r/KoboldAI | 25 Jun 2021
    I found this yesterday, is it somehow possible to use this with KoboldAI to run GPT-J on weaker graphics cards?
  • A note from our sponsor - SaaSHub
    www.saashub.com | 25 Apr 2024
    SaaSHub helps you find the best software and product alternatives Learn more →

Stats

Basic Basic-UI-for-GPT-J-6B-with-low-vram repo stats
4
113
0.0
over 2 years ago

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com