Interactive-LLM-Powered-NPCs
GPT-HTN-Planner
| Interactive-LLM-Powered-NPCs | GPT-HTN-Planner | |
|---|---|---|
| 3 | 1 | |
| 449 | 34 | |
| - | - | |
| 6.9 | 4.9 | |
| 3 months ago | 3 months ago | |
| Python | Python | |
| MIT License | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Interactive-LLM-Powered-NPCs
-
AI: Startup vs Incumbent Value
It is different this time, though. Take a look at this open source project.[1]
This is a system which lets you talk to NPCs in video games. It's a collection of off the shelf components held together by some Python code. The components do this:
- Listen to the user talking and convert speech to text.
- Watch the user's facial expressions via webcam.
- Watch the game, and use face recognition on the game images to determine what character is being addressed.
- Run the user's text through a LLM preloaded with about 30 lines of info about the NPC to generate a reply.
- Generate voice output in a voice generated to match the character's persona.
- Modify the image of the character on screen to animate their facial expressions to match the voice output. This is done on the output image, not by animating the 3D character.
Five years ago, that was science fiction. A year ago, half that stuff wouldn't work right. Now it's someone's hobby project.
[1] https://github.com/AkshitIreddy/Interactive-LLM-Powered-NPCs
- LLM Powered NPCs: Enabling Dynamic Dialogue with LLM-Powered NPCs in Any Game
- Talk to GPT-4 Powered NPCs in your Favorite Games!
GPT-HTN-Planner
-
Can LLMs Reason and Plan?
"Plan" means a lot of things.
There has been some research in applying GPT-4 to Hierarchical Task Networks (HTN), one means of doing computerized semi-automated/automated planning of a complex task as a tree of less and less complex tasks [1].
There are other types of planning. Automated planning works better as there are more defined the tasks in a plan, less ambiguity in dependencies, more separate between the tasks. The OP article touches on that, noting LLMs are good at extracting planning knowledge but not good in their experience at creating executable plans. This is why I think the hybrid approach is best, using an LLM to inform and tweak other planning tools in order to create an executable plan.
[1] https://github.com/DaemonIB/GPT-HTN-Planner
What are some alternatives?
codeinterpreter-api - 👾 Open source implementation of the ChatGPT Code Interpreter
awesome-ai-agents - A list of AI autonomous agents
agents - An Open-source Framework for Autonomous Language Agents
AgentPilot - A framework to create, manage, and chat with AI agents + Multi agent chat, branching chat and multiple API providers
voxelgpt - AI assistant that can query visual datasets, search the FiftyOne docs, and answer general computer vision questions
Adala - Adala: Autonomous DAta (Labeling) Agent framework
E2B - Secure cloud runtime for AI apps & AI agents. Fully open-source.
Auto-GPT - An experimental open-source attempt to make GPT-4 fully autonomous. [Moved to: https://github.com/Significant-Gravitas/AutoGPT]
ACE_Model_Implementation - A python implementation of Dave Shap's ACE Model
PyMera - an application that track your face
ToRA - ToRA is a series of Tool-integrated Reasoning LLM Agents designed to solve challenging mathematical reasoning problems by interacting with tools [ICLR'24].
doppel-bot - Train a language model to answer Slack messages as you.