memprompt VS pal

Quick Read: https://www.marktechpost.com/2022/12/18/allen-institute-for-artificial-intelligence-introduces-memprompt-a-new-method-to-fix-gpt-3-after-deployment-with-user-interaction/ Paper: https://arxiv.org/abs/2201.06009 Code: https://github.com/madaan/memprompt

It's already possible to get some of this effect with codex. The trick is to keep appending the interaction in the prompt (to maintain a memory of sorts).

For examples, you can replicate all the prompts here: https://twitter.com/yoavgo/status/1599200756631887872 with prompt + memory.

The notebook at https://github.com/madaan/memprompt/blob/main/YoavsPythonPro... shows a demo of this.

Some of these ideas were earlier discussed in our work on memory-assisted prompting [1].

[1] https://arxiv.org/pdf/2201.06009.pdf.

Code for https://arxiv.org/abs/2201.06009 found: https://github.com/madaan/memprompt

Using the terminology that I'm working with this is an example of a second-order analytic augmentation!

Here's another approach of second-order analytic augmentation, PAL: https://reasonwithpal.com

And third-order, Toolformer: https://arxiv.org/abs/2302.04761

The difference isn't in what is going on but rather with framing the approach within the analytic-synthetic distinction developed by Kant and the analytic philosophers who were influenced by his work. There's a dash of functional programming thrown in for good measure!

I have scribbled on a print-out of the article on my desk:

  Nth Order

GitHub: https://github.com/reasoning-machines/pal

tl;dr -- LLMs are bad at basic arithmetic and logic (as their opening examples with math word problems show), but they do much better if instead of asking them for the answer, you ask for code to compute the answer. Then evaluate or run the code to get the answer.