1-Bit LLMs Could Solve AI's Energy Demands

1. unilm

   1 44 21,071 8.2 Python

   Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

   

2. Judoscale

   judoscale.com featured

   Save 47% on cloud hosting with autoscaling that just works. Judoscale integrates with Django, FastAPI, Celery, and RQ to make autoscaling easy and reliable. Save big, and say goodbye to request timeouts and backed-up task queues.

   

3. BitNet-1.58-Instruct

   2 2 22 6.4 Python

   Implementation of BitNet-1.58 instruct tuning

   

4. 1.58BitNet

   3 1 61 6.8 Python

   Experimental BitNet Implementation

   

5. picoagent-rnd

   4 1 18 3.5 Python

   Web & CLI capable LLM agent (research prototype, no framework dependencies).

   

   While llama3-8b might be slightly more brittle under quantization, llama3-70b really surprised myself and others[1] in how well it performs even in the 2..3 bits per parameter regime. It requires one of the most advanced quantization methods (IQ2_XS specifically) but the reward is a SoTA LLM that fits on one 4090 GPU and allows for advanced usecases such as powering the agent engine I'm working on: https://github.com/kir-gadjello/picoagent-rnd

   For me it completely replaced strong models such as Mixtral-8x7B and DeepSeek-Coder-Instruct-33B.

   1. https://www.reddit.com/r/LocalLLaMA/comments/1cst400/result_...

6. quip-sharp

   5 2 531 7.4 Python

   

7. hn-search

   6 2,013 555 2.9 TypeScript

   Hacker News Search

   

   Yes but please try to avoid repetition on HN. The GP's response was rude and broke the site guidelines but https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que... does look excessive to me.

Suggest a related project