llm VS llama-gpt

That's a great use case, you might be able to do this if you've got a copy and paste on the command line with

https://github.com/simonw/llm

In between. An alias like pdfwtf translating to "paste | llm command | copy"

I added support for this model to my LLM CLI tool via a new plugin: https://github.com/simonw/llm-command-r

So now you can do this:

    pipx install llm

If you're willing to use the CLI, Simon Willison's llm library[0] should do the trick.

[0] https://github.com/simonw/llm

I'm not thrilled about https://github.com/GoogleCloudPlatform/localllm/blob/main/ll... calling their Python package "llm" and installing "llm" as a CLI command, when my similar https://llm.datasette.io/ project has that namespace reserved on PyPI already: https://pypi.org/project/llm/

I've been gleefully exploring the intersection of LLMs and CLI utilities for a few months now - they are such a great fit for each other! The unix philosophy of piping things together is a perfect fit for how LLMs work.

I've mostly been exploring this with my https://llm.datasette.io/ CLI tool, but I have a few other one-off tools as well: https://github.com/simonw/blip-caption and https://github.com/simonw/ospeak

I'm puzzled that more people aren't loudly exploring this space (LLM+CLI) - it's really fun.

Seems nice if you're using c# or java. It also supports python, but for that Simon's llm library is nice because he designed it as both a library and a command line tool: https://github.com/simonw/llm

wodner if you can pair with https://github.com/getumbrel/llama-gpt

I put up a draft PR to demo how to run it on a GPU: https://github.com/getumbrel/llama-gpt/pull/11

It breaks other things like model downloading, but once I got it to a working state for myself, I figured why not put it up there in case its useful. If I have time, I'll try to rework it a little bit with more parameters and less dockerfile repetition to fit the main project better.