prompt-engineering VS llama.cpp

DeepLearning.AI Prompt engineering Learn Prompting

For engineers in the context of prompt engineering, there are a few decent options, like ChatGPT Prompt Engineering for Developers - DeepLearning.AI for example. But it seems you didn't ask for that.

ChatGPT Prompt Engineering for Developers: https://www.deeplearning.ai/short-courses/chatgpt-prompt-engineering-for-developers/

ChatGPT Prompt Engineering for Developers - DeepLearning.AI

Ideally, anyone interested in should take a course of prompt engineering to learn how to use assistant properly (for example here) as to make full use of the increased productivity.

if you can compile stuff, then looking at llama.cpp (what ollama uses) is also interesting: https://github.com/ggerganov/llama.cpp

the server is here: https://github.com/ggerganov/llama.cpp/tree/master/examples/...

And you can search for any GGUF on huggingface

Yes, Metal seems to allow a maximum of 1/2 of the RAM for one process, and 3/4 of the RAM allocated to the GPU overall. There’s a kernel hack to fix it, but that comes with the usual system integrity caveats. https://github.com/ggerganov/llama.cpp/discussions/2182

For anyone interested in exploring this, llama.cpp has an example implementation here:

https://github.com/ggerganov/llama.cpp/tree/master/examples/...

Getting started with LLMs can be intimidating. In this tutorial we will show you how to fine-tune a large language model using LoRA, facilitated by tools like llama.cpp and KitOps.

well https://github.com/ggerganov/llama.cpp/issues/6849