sui VS GLM-130B

And the start page looks like a modified version of this

is something like Flame https://github.com/pawelmalak/flame Heimdall https://github.com/linuxserver/Heimdall Sui https://github.com/jeroenpardon/sui You are looking for?

I have tried Heimdall and Homer. Currently I'm using Sui, but I want to try Flame soon.

I use a version of SUI (https://github.com/jeroenpardon/sui) that i have modified myself (https://github.com/Kussie/startpage)

jeroenpardon/sui - very elegant startpage. The thing is that if you want to add a new app or bookmark you have manually edit json files.

Check out SUI/dashboard/startpage (latter two are forks of SUI, but all projects seem active). I think it is much cleaner than dashmachine in my opinion. When they can get a config page or something similar to the way dashmachine does it, it will be really great.

It's a deployable dashboard that you can find here: https://github.com/jeroenpardon/sui

The https://github.com/THUDM/GLM-130B model is trained on The Pile and can run on 4x3090 when quantized to INT4. I'm wondering if anyone knows if this model could (or has) been quantized using GPTQ, which gives some impressive performance gains over traditional quantization, and I'm also wondering if anyone has tried a 3-bit or 2-bit quantization of such a massive model (using GPTQ). Are there any inherent limitations in this? Is there anything about this model that prevents it from being run on text-generation-webui?

> GLM-130B, a model comparable with GPT-3, has 130 billion parameters in FP16 precision, a total of 260G of GPU memory is required to store model weights. The DGX-A100 server has 8 A100s and provides an amount of 320G of GPU memory (640G for 80G A100 version) so it suits GLM-130B well.

https://github.com/THUDM/GLM-130B/blob/main/docs/low-resourc...

GLM-130B[1] (a 130 billion parameter model vs GPT-3's 175 billion parameter model) is able to run optimally on consumer level high-end hardware, 4xRTX 3090 in particular. That's < $4k at current prices, and as hardware prices go one can only imagine what it'll be in a year or two. It also enables running with degraded performance on lesser systems.

It's a whole lot cheaper to run neural net style systems than to train them. "Somebody on Twitter"[2] got it setup, and broke down the costs, demonstrated some prompts, and what not. Cliff notes being a fraction of a penny per query, with each taking about 16s to generate. The output's pretty terrible, but it's unclear to me whether that's inherent or a result of priority. I expect OpenAI spent a lot of manpower on supervised training, whereas this system probably had minimal, especially in English (it's from a Chinese university).

[1] - https://github.com/THUDM/GLM-130B

[2] - https://twitter.com/alexjc/status/1617152800571416577

GLM-130B in 4 bit mode is better than GPT3 and can run on 4 RTX-3090s. Still expensive but it’s getting closer. https://github.com/THUDM/GLM-130B

https://github.com/THUDM/GLM-130B might be a useful place to look