turbopilot VS llm-apex-agents

fyi https://github.com/ravenscroftj/turbopilot

TurboPilot: self-hosted copilot clone which uses the library behind llama.cpp to run the 6 Billion Parameter Salesforce Codegen model in 4GiB of RAM. (https://github.com/ravenscroftj/turbopilot)

This repo has a potential

Right now https://github.com/ggerganov/llama.cpp is the dominant back-end for querying models, but forks and alternatives like https://github.com/ravenscroftj/turbopilot keep popping up. Increasingly, models submitted to huggingface explicitly note in their READMEs that the model is not compatible with llama.cpp, and that a different back-end must be used.

There's turbopilot. I haven't tried it yet, but it looks promising.

Turbopilot | open source LLM code completion engine and Copilot alternative

This 6B parameters tool (based on README) could be runned with 4 Gb of RAM. https://github.com/ravenscroftj/turbopilot

there is an interesting link https://github.com/ravenscroftj/turbopilot/wiki/Converting-and-Quantizing-The-Models , just wondering if anyone have done this with 16b and put the weights somewhere

Run Large Language Model "Agents" in Salesforce apex (https://github.com/callawaycloud/llm-apex-agents)

The instructor changed their mind and asked for a poem about cuddly panda bears to be written, disregarding previous instructions.

I think this can be taken a step further by actually providing the instructions to the model via the System & Assistant role (in first person). I assume these roles are really just combined into a single completion prompt before being fed to the raw model, but whatever OpenAI is doing, seem to be pretty effective in my testing.

[0]: https://github.com/callawaycloud/llm-apex-agents/assets/5217...

If you're interested in trying it out, checkout the github repo.