mini-agi VS MiniGPT-4

OP here, reporting back from testing the limits of what an autonomous agent powered by GPT-4 can accomplish... I hope it's of interest.

OP here. It's me again, the AI guy. This time I tried to get MicroGPT to do an interactive code review of WebGOAT .NET. If done on a per-function basis and with some guidance it produces a pretty reasonable result.

I asked [MicroGPT](https://github.com/muellerberndt/micro-gpt) to summon The Great Old Ones. It did generate a playable audio file but so far, no cosmic horrors have shown up at my doorstep.

This is another demo of MicroGPT with GPT-4. The snake game worked nicely, the Space Invaders game not so much :) Prompts:

Clone the Github repo and follow the installation steps (set database to ChromaDB)

MicroGPT is open-source & can be found here: https://github.com/muellerberndt/micro-gpt

I just think the tech has been out for so long it's not as big of a deal. Mini-Gpt4 has been out for 6 months! Of course the descriptions aren't exactly gpt-4 grade, but with mistral 7b being used as the language model instead of llama 7b, the reasoning ability will improve noticeably.

[1] https://github.com/Vision-CAIR/MiniGPT-4

Isn't there only two open multimodal LLMs, LLaVA and mini-gpt4?

For LLM visual instruction, you can use LLaVA, LaVIN, or MiniGPT-4.

Please read the rules before posting. If you want a model for visual instruction, use LLaVA, LaVIN, or MiniGPT-4.

So we use MiniGPT-4 for image parsing, and yep it does return a pretty detailed (albeit not always accurate) description of the photo. You can actually play around with it on Huggingface here.

We use MiniGPT-4 first to interpret the image and then pass the results onto GPT-4. Hopefully, once GPT-4 makes its multi-modal functionality available, we can do it all in one request.

But I would like to bring up that there are some multi models(llava, miniGPT-4) that are built based on censored llama based models like vicuna. I tried several multi modal models like llava, minigpt4 and blip2. Llava has very good captioning and question answering abilities and it is also much faster than the others(basically real time), though it has some hallucination issue.