airoboros VS prompt-engineering

Just wanted to drop a note that I overhauled the airoboros tool not the models to have most of the prompts I've been using to build the datasets, plus a couple extras.

airoboros: using large language models to fine-tune large language models (https://github.com/jondurbin/airoboros)

This is a fine-tuned LlaMa model, using completely synthetic training data created by https://github.com/jondurbin/airoboros

I used airoboros, a python tool I wrote, to generate the synthetic instruction response pairs, and included a jailbreak prompt to attempt to bypass OpenAI censorship. This is the only dataset used to fine-tune the model.

This is a 7b parameter, fine-tuned on 100k synthetic instruction/response pairs generated by gpt-3.5-turbo using my version of self-instruct airoboros

GitHub Repo

An agent is a style of prompt that lets LLMs act as reasoning engines. It's also known as the ReAct pattern (which engineers are avoiding using for namespace collision reasions).

You can read a good intro example here: https://github.com/brexhq/prompt-engineering#react

Well, if developers get replaced by AI, then who are the managers going to manage :). I personally don't think AI is just going to replace us. The way we work will continue to change as new AI tools come out. I'm taking time to tinker with new tools and seeing how others do as well (e.g., I found Brex's tips and tricks for working with LLMs very insightful: https://github.com/brexhq/prompt-engineering).

I recognize there's plenty of catnip here when it comes to calling this "engineering" or not, however, whatever you want to call it (prompt fiddling?), the techniques are crucial if you want to achieve reasonably consistent output from current-state LLMs. As models improve concerns about context window limitations will be reduced and it will be easier to discern user intent.

These are good straight-to-the-point guides:

- Prompt Engineering by BrexHQ: https://github.com/brexhq/prompt-engineering

- OpenAI guidance: https://help.openai.com/en/articles/6654000-best-practices-f...

- https://devblogs.microsoft.com/dotnet/gpt-prompt-engineering...

- (great examples): https://www.deeplearning.ai/short-courses/chatgpt-prompt-eng...

tl;dr:

Brex's Prompt Engineering Guide (https://github.com/brexhq/prompt-engineering)