blog VS WizardLM

[4] https://github.com/huggingface/blog/blob/main/starcoder.md

If you just want to do inference/mess around with the model and have a 16GB GPU, then this[0] is enough to paste into a notebook. You need to have access to the HF models though.

0. https://github.com/huggingface/blog/blob/main/llama2.md#usin...

The hands-on 👉https://github.com/huggingface/blog/blob/main/notebooks/101_train-decision-transformers.ipynb

I'm also running the optimized script but it doesn't run with 512x512 on my RTX3050 Ti mobile. On this website, they recommend to switch to fp16 for GPUs with less than 10gb of vram.

Here's a notebook illustrating how to train a language model from scratch: https://github.com/huggingface/blog/blob/master/notebooks/01_how_to_train.ipynb

This is interesting work, and a good contribution, but there is no need to mislead people.

[1] https://github.com/nlpxucan/WizardLM

If you pair this with the latest WizardCoder models, which have a fairly better performance than the standard Salesforce Codegen2 and Codegen2.5, you have a pretty solid alternative to GitHub Copilot that runs completely locally.

(We will update the demo links in our github.)

In terms of speed, we're talking about 140t/s for 7B models, and 40t/s for 33B models on a 3090/4090 now.[1] (1 token ~= 0.75 word) It's quite zippy. llama.cpp performs close on Nvidia GPUs now (but they don't have a handy chart) and you can get decent performance on 13B models on M1/M2 Macs.

You can take a look at a list of evals here: https://llm-tracker.info/books/evals/page/list-of-evals - for general usage, I think home-rolled evals like llm-jeopardy [2] and local-llm-comparison [3] by hobbyists are more useful than most of the benchmark rankings.

That being said, personally I mostly use GPT-4 for code assistance to that's what I'm most interested in, and the latest code assistants are scoring quite well: https://github.com/abacaj/code-eval - a recent replit-3b fine tune the human-eval results for open models (as a point of reference, GPT-3.5 gets 60.4 on pass@1 and 68.9 on pass@10 [4]) - I've only just started playing around with it since replit model tooling is not as good as llamas (doc here: https://llm-tracker.info/books/howto-guides/page/replit-mode...).

I'm interested in potentially applying reflexion or some of the other techniques that have been tried to even further increase coding abilities. (InterCode in particular has caught my eye https://intercode-benchmark.github.io/)

[1] https://github.com/turboderp/exllama#results-so-far

[2] https://github.com/aigoopy/llm-jeopardy

[3] https://github.com/Troyanovsky/Local-LLM-comparison/tree/mai...

[4] https://github.com/nlpxucan/WizardLM/tree/main/WizardCoder

You talking about this? https://github.com/nlpxucan/WizardLM

The smallest model that is close to competent at code is WizardCoder 15B.. https://github.com/nlpxucan/WizardLM/

WizardCoder: Empowering Code Large Language Models with Evol-Instruct (https://github.com/nlpxucan/WizardLM/tree/main/WizardCoder)