Adala VS GPT-HTN-Planner

We recently open sourced an agent framework [1] for automating data processing and labeling where the prompt is refined trough iterations (i.e. automatic prompt tuning). We tested it on the Math reasoning dataset GSM8k and where able to improve the baseline accuracy (GPT4) by 45% -> 74% using 25 labeled examples (I'll put the notebook and blog post linked below [2][3]). Results are definitively very interesting, if not surprising with some skills, and we see more and more of our open source users and customers showing interested in the framework for automating labeling / having it as a data processing / labeling copilot.

[1] https://github.com/HumanSignal/Adala

[2] https://github.com/HumanSignal/Adala/blob/master/examples/gs...

[3] https://labelstud.io/blog/mastering-math-reasoning-with-adal...

We have just open sourced Adala - a robust framework for implementing agents that specialize in advanced data processing tasks, starting with data labeling and generation.

"Plan" means a lot of things.

There has been some research in applying GPT-4 to Hierarchical Task Networks (HTN), one means of doing computerized semi-automated/automated planning of a complex task as a tree of less and less complex tasks [1].

There are other types of planning. Automated planning works better as there are more defined the tasks in a plan, less ambiguity in dependencies, more separate between the tasks. The OP article touches on that, noting LLMs are good at extracting planning knowledge but not good in their experience at creating executable plans. This is why I think the hybrid approach is best, using an LLM to inform and tweak other planning tools in order to create an executable plan.

[1] https://github.com/DaemonIB/GPT-HTN-Planner