tree-of-thought-llm VS Voyager

[2305.10601] Tree of Thoughts: Deliberate Problem Solving with Large Language Models (arxiv.org)

Language models are increasingly being deployed for general problem solving across a wide range of tasks, but are still confined to token-level, left-to-right decision-making processes during inference. This means they can fall short in tasks that require exploration, strategic lookahead, or where initial decisions play a pivotal role. To surmount these challenges, we introduce a new framework for language model inference, “Tree of Thoughts” (ToT), which generalizes over the popular “Chain of Thought” approach to prompting language models, and enables exploration over coherent units of text (“thoughts”) that serve as intermediate steps toward problem solving. ToT allows LMs to perform deliberate decision making by considering multiple different reasoning paths and self-evaluating choices to decide the next course of action, as well as looking ahead or backtracking when necessary to make global choices. Our experiments show that ToT significantly enhances language models’ problem-solving abilities on three novel tasks requiring non-trivial planning or search: Game of 24, Creative Writing, and Mini Crosswords. For instance, in Game of 24, while GPT-4 with chain-of-thought prompting only solved 4% of tasks, our method achieved a success rate of 74%. Code repo with all prompts: https://github.com/princeton-nlp/tree-of-thought-llm.

GPU utilization should be down when using this technique. I’m hoping this could allow for more efficient batch inference on GPUs. If you can predict 10 tokens for the price of 1 it should allow you to do tree of thought much more efficiently.

https://github.com/princeton-nlp/tree-of-thought-llm

For those familiar with Daniel Kahneman's Thinking Fast and Slow, the current LLMs (such as GPT-4 via ChatGPT) seem to resemble System 1 thinking (near-instantaneous, automatic, intuitive processes like next-word prediction). However, they lack System 2 thinking (slow, effortful, logical, planning, reasoning). What I learned today is that Google's Gemini (an LLM in training now) not only has more modalities (I think all Youtube Video and audio??), more compute, and almost twice the training data, but they're building in AlphaGo-type learning, which resembles tree of thoughts and looks a LOT like the missing puzzle piece of System 2 thinking. Will it be AGI? Maybe, and it's coming this winter.

Tree of thoughts: https://arxiv.org/abs/2305.10601

Good video on "Tree of thoughts" which also reviews / puts it in the context of other methods: https://www.youtube.com/watch?v=ut5kp56wW_4

Completion vs conversational interface is something you can read about in the OpenAI API documentation.

For the remaining things I don't have single specific pointer at hand.

What do you think about the combination of Tree of Thoughts: Deliberate Problem Solving with Large Language Models LongNet: Scaling Transformers to 1,000,000,000 Tokens Textbooks Are All You Need Attention Is All You Need Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation

You are right. Tree of Thoughts: Deliberate Problem Solving with Large Language Models proposes to solve this via MCTS-style generation (similar to how AlphaGo worked, and a lot of planning & control problems are executed).

The transformer was developed in 2017 and it powers all modern LLMs. If you're familiar with Daniel Kahneman's work from Thinking Fast and Slow, you could easily summarize LLMs as excellent System 1 thinking: our fast, automatic, unconscious responses (e.g. autocomplete). I'd argue that we're one development (similar to the transformer) away from creating System 2 thinking: deliberate and strategic thinking. In fact, with merely GPT-4 and some clever architectures, researchers have developed chain-of-thought prompting and, more recently, tree-of-thoughts reasoning. While external to the LLM architecture, embedding these concepts into a LLM could very likely solve the creation of System 2 thinking and produce the first real AGI. Adding more modalities (e.g. audio, images, video, topography, etc.) will simply add more nuance in the weights and biases of a complete system.

There are a bunch of other methods to improve quality and performance like tree-of-thought-llm, connecting a LLM to a database or have it review its own output.

Source: Conversation with Bing, 12/10/2023 (1) Wes Roth - YouTube. https://www.youtube.com/@WesRoth. (2) I've set most of my videos to Public again - Community. https://community.openai.com/t/ive-set-most-of-my-videos-to-public-again/24535. (3) AI Updates: Meta Develops Mind-Reading AI System, OpenAI’s Q* Is Here .... https://www.windermeresun.com/2023/11/20/ai-updates-meta-develops-mind-reading-ai-system-openais-q-is-here-how-economy-will-work-after-agi/. (4) David Shapiro. https://www.daveshap.io/. (5) undefined. https://natural20.com/. (6) undefined. https://arxiv.org/abs/2305.16291. (7) undefined. https://twitter.com/DrJimFan/status/1. (8) undefined. https://voyager.minedojo.org/. (9) undefined. https://minedojo.org/. (10) undefined. https://www.youtube.com/@DavidShapiroAutomator/videos.

> AI cannot sustain itself trained on AI work.

This isn’t true. You can train LLMs entirely on synthetic data and get strong results. [0]

> If new languages, engines etc pop up it cannot synthesize new forms of coding without that code having existed in the first place.

You can describe the semantics to a LLM, have it generate code, tell it what went wrong (i.e. with compiler feedback), and then train on that. For an example of this workflow in a different context, see [1].

> And most importantly, it cannot fundamentally rationalize about what code does or how it functions.

Most competent LLMs can trivially describe what some code does and speculate on the reasoning behind it.

I don’t disagree that they’re flawed and imperfect, but I also do not think this is an unassailable state of affairs. They’re only going to get better from here.

[0]: https://arxiv.org/abs/2309.05463

[1]: https://voyager.minedojo.org/

In a way it is the same thing, agents are mostly an abstraction that make it easier to know what’s going on.

I think of agents more or less as python classes with a mixture of natural language and code functions. You design them to do something with information they produce, and to interface with other agents or “tools” in some way.

But all the agents can be the same language model under the hood, they are frames used to build different kinds of contexts.

And yes I think the idea is that emergent behaviour can be useful. This comes to mind

https://github.com/MineDojo/Voyager

But I think we are still a small ways off from being really smart about agents. My opinion is that we haven’t quite figured out what we are doing yet.

This k8s application deploys an instance of Voyager along with a Fabric Minecraft server with required fabric mods. It assumes you have a local deployment of a Large Language Model (LLM) with 4K-8K token context length with a compatible OpenAI API, including embeddings support.

In my opinion the most interesting Agents: Auto-GPT Github: https://github.com/Significant-Gravitas/Auto-GPT BabyAGI Github: https://github.com/yoheinakajima/babyagi Voyager Github: https://github.com/MineDojo/Voyager / Paper: https://arxiv.org/abs/2305.16291 I would also add: ChemCrow: Augmenting large-language models with chemistry tools Github: https://github.com/ur-whitelab/chemcrow-public/ Paper: https://arxiv.org/abs/2304.05376

Does this beat the voyager? I read about it and wondered what if we add a skill library to langchain/llamaindex agents. It could be the same vector store for storing static data but after each task is performed, the agent will evaluate and archive the recipe of steps to perform a new task. Next time when the agent is asked to perform a task, it can just look at the library to retrieve a recipe. Unlike traditional fine tuning, you dont update the model parameters, these recipes are much more interpretable and can be manually edited/inserted by humans. There may also be an automatic way to convert wikihow articles or youtube tutorials into recipes.

Source. P.S. If you love geeking over AI updates, I have this free newsletter you might want to check out. Thank you!

Source.