SoM VS AiFilter

At https://openadapt.ai/ we are using LLMs to automate repetitive tasks in GUI interfaces. Think robotic process automation, but via learning from demonstration rather than no-code scripting.

The stack is mostly python running locally, and calling the OpenAI API (although we have plans to support offline models).

For better visual understanding, we use a custom fork of Set-of-Mark prompting (https://github.com/microsoft/SoM) deployed to EC2 (see https://github.com/OpenAdaptAI/SoM/pull/3).

A Twitter filter to take back control of your social media feed from recommendation engines. Put in natural language instructions like "Only show tweets about machine learning, artificial intelligence, and large language models. Hide everything else" and it will filter out all the tweets that you tell it to.

Runs on a local LLM, because even using GPT3 costs would have added up quickly.

Currently requires CUDA and uses a 10.7B model but if anyone wants to try a smaller one and report results let me know on github and I can give some help.

https://github.com/thomasj02/AiFilter

While exploring new applications for local LLMs, I built a Chrome extension that filters your Twitter feed based on natural language instructions.

For instance, you can instruct it to "Hide all tweets, except for tweets about machine learning (ML), artificial intelligence (AI) and large language models (LLMs)."

I've tested it and got good results with a 10B parameter model, but I suspect a high-quality small model like Phi-2 might work almost as well.

It's open source and available at https://github.com/thomasj02/AiFilter

Video demo: https://www.youtube.com/watch?v=CligVVTC5io