qlora VS Voyager

Marx 3B V3 is StableLM 3B 4E1T instruction tuned on EverythingLM Data V3(ShareGPT Format) for 2 epochs using QLoRA.

https://github.com/artidoro/qlora

The tools and mechanisms to get a model to do what you want is ever so changing, ever so quickly. Build and understand a notebook yourself, and reduce dependencies. You will need to switch them.

My own project right now is still in raw generated form, and this now makes me think about trying qlora's scripts since this gives me some confidence I should be able to get it to turn out now that someone else has carved a path and charted the map. I was going to target llamatune which was mentioned here the other day.

Most papers I did read showed at least a thousand, even 10000 at several cases, so I assumed that to be the trend in the case of Low rank adapter(PEFT) training.(source: [2305.14314] QLoRA: Efficient Finetuning of Quantized LLMs (arxiv.org) , Stanford CRFM (Alpaca) and the minimum being openchat/openchat · Hugging Face ; There are a lot more examples)

4-bit quantization training mainly refers to qlora. Simply put, qlora quantizes the weights of the LLM into 4-bit for storage, while dequantizing them into 16-bit during the training process to ensure training precision. This method significantly reduces GPU memory overhead during training (the training speed should not vary much). This approach is highly suitable to be combined with parameter-efficient methods. However, the original paper was designed for single-modal LLMs and the code has already been wrapped in HuggingFace's library. Therefore, we extracted the core code from HuggingFace's library and migrated it into LaVIN's code. The main principle is to replace all linear layers in LLM with 4-bit quantized layers. Those interested can refer to our implementation in quantization.py and mm_adaptation.py, which is roughly a dozen lines of code.

There have been some new development like QLora which help fine-tune LLMs without updating all the weights.

This might be helpful: https://github.com/artidoro/qlora/issues/10

curious, since qlora paper only reports lora/qlora comparison for full fine-tuning for small 7B models.for 13B/33B/65B, it does not do so (table 4 in paper)it would be helpful if anyone can please provide links where I can read more on efficacy of lora or disadvantages of lora?

This might not be appropriate answer but did you take a look at this repository? https://github.com/artidoro/qlora With artidoro's repository it's pretty easy to train qlora. You just prepare your own dataset and run the following command: python qlora.py --model_name_or_path --dataset="path/to/your/dataset" --dataset_format="self-instruct" This is only available for several dataset formats. But every dataset format has to have input-output pairs. So the dataset json format has to be like this [ { “input”: “something ”, “output”:“something ” }, { “input”: “something ”, “output”:“something ” } ]

Source: Conversation with Bing, 12/10/2023 (1) Wes Roth - YouTube. https://www.youtube.com/@WesRoth. (2) I've set most of my videos to Public again - Community. https://community.openai.com/t/ive-set-most-of-my-videos-to-public-again/24535. (3) AI Updates: Meta Develops Mind-Reading AI System, OpenAI’s Q* Is Here .... https://www.windermeresun.com/2023/11/20/ai-updates-meta-develops-mind-reading-ai-system-openais-q-is-here-how-economy-will-work-after-agi/. (4) David Shapiro. https://www.daveshap.io/. (5) undefined. https://natural20.com/. (6) undefined. https://arxiv.org/abs/2305.16291. (7) undefined. https://twitter.com/DrJimFan/status/1. (8) undefined. https://voyager.minedojo.org/. (9) undefined. https://minedojo.org/. (10) undefined. https://www.youtube.com/@DavidShapiroAutomator/videos.

> AI cannot sustain itself trained on AI work.

This isn’t true. You can train LLMs entirely on synthetic data and get strong results. [0]

> If new languages, engines etc pop up it cannot synthesize new forms of coding without that code having existed in the first place.

You can describe the semantics to a LLM, have it generate code, tell it what went wrong (i.e. with compiler feedback), and then train on that. For an example of this workflow in a different context, see [1].

> And most importantly, it cannot fundamentally rationalize about what code does or how it functions.

Most competent LLMs can trivially describe what some code does and speculate on the reasoning behind it.

I don’t disagree that they’re flawed and imperfect, but I also do not think this is an unassailable state of affairs. They’re only going to get better from here.

[0]: https://arxiv.org/abs/2309.05463

[1]: https://voyager.minedojo.org/

In a way it is the same thing, agents are mostly an abstraction that make it easier to know what’s going on.

I think of agents more or less as python classes with a mixture of natural language and code functions. You design them to do something with information they produce, and to interface with other agents or “tools” in some way.

But all the agents can be the same language model under the hood, they are frames used to build different kinds of contexts.

And yes I think the idea is that emergent behaviour can be useful. This comes to mind

https://github.com/MineDojo/Voyager

But I think we are still a small ways off from being really smart about agents. My opinion is that we haven’t quite figured out what we are doing yet.

This k8s application deploys an instance of Voyager along with a Fabric Minecraft server with required fabric mods. It assumes you have a local deployment of a Large Language Model (LLM) with 4K-8K token context length with a compatible OpenAI API, including embeddings support.

In my opinion the most interesting Agents: Auto-GPT Github: https://github.com/Significant-Gravitas/Auto-GPT BabyAGI Github: https://github.com/yoheinakajima/babyagi Voyager Github: https://github.com/MineDojo/Voyager / Paper: https://arxiv.org/abs/2305.16291 I would also add: ChemCrow: Augmenting large-language models with chemistry tools Github: https://github.com/ur-whitelab/chemcrow-public/ Paper: https://arxiv.org/abs/2304.05376

Does this beat the voyager? I read about it and wondered what if we add a skill library to langchain/llamaindex agents. It could be the same vector store for storing static data but after each task is performed, the agent will evaluate and archive the recipe of steps to perform a new task. Next time when the agent is asked to perform a task, it can just look at the library to retrieve a recipe. Unlike traditional fine tuning, you dont update the model parameters, these recipes are much more interpretable and can be manually edited/inserted by humans. There may also be an automatic way to convert wikihow articles or youtube tutorials into recipes.

Source. P.S. If you love geeking over AI updates, I have this free newsletter you might want to check out. Thank you!

Source.