blog VS llm

[4] https://github.com/huggingface/blog/blob/main/starcoder.md

If you just want to do inference/mess around with the model and have a 16GB GPU, then this[0] is enough to paste into a notebook. You need to have access to the HF models though.

0. https://github.com/huggingface/blog/blob/main/llama2.md#usin...

The hands-on 👉https://github.com/huggingface/blog/blob/main/notebooks/101_train-decision-transformers.ipynb

I'm also running the optimized script but it doesn't run with 512x512 on my RTX3050 Ti mobile. On this website, they recommend to switch to fp16 for GPUs with less than 10gb of vram.

Here's a notebook illustrating how to train a language model from scratch: https://github.com/huggingface/blog/blob/master/notebooks/01_how_to_train.ipynb

That's a great use case, you might be able to do this if you've got a copy and paste on the command line with

https://github.com/simonw/llm

In between. An alias like pdfwtf translating to "paste | llm command | copy"

I added support for this model to my LLM CLI tool via a new plugin: https://github.com/simonw/llm-command-r

So now you can do this:

    pipx install llm

If you're willing to use the CLI, Simon Willison's llm library[0] should do the trick.

[0] https://github.com/simonw/llm

I'm not thrilled about https://github.com/GoogleCloudPlatform/localllm/blob/main/ll... calling their Python package "llm" and installing "llm" as a CLI command, when my similar https://llm.datasette.io/ project has that namespace reserved on PyPI already: https://pypi.org/project/llm/

I've been gleefully exploring the intersection of LLMs and CLI utilities for a few months now - they are such a great fit for each other! The unix philosophy of piping things together is a perfect fit for how LLMs work.

I've mostly been exploring this with my https://llm.datasette.io/ CLI tool, but I have a few other one-off tools as well: https://github.com/simonw/blip-caption and https://github.com/simonw/ospeak

I'm puzzled that more people aren't loudly exploring this space (LLM+CLI) - it's really fun.

Seems nice if you're using c# or java. It also supports python, but for that Simon's llm library is nice because he designed it as both a library and a command line tool: https://github.com/simonw/llm