skyvern VS self-operating-computer

https://github.com/Skyvern-AI/skyvern

This is pretty much what we're building at Skyvern. The only problem is that inference cost is still a little bit too high for scraping, but we expect that to change in the next year

This is a great point. This is something already on our roadmap. We call it "prompt caching", but I realize writing this that it's a terrible name. Will update! (https://github.com/Skyvern-AI/Skyvern?tab=readme-ov-file#fea...)

Thank you for this feedback

We're also working in the space and just open sourced Skyvern

https://github.com/Skyvern-AI/Skyvern

This is quite different than https://github.com/OthersideAI/self-operating-computer

Self-operating-computer uses pixel mapping to control your computer. This is a very good approach, but it's extremely unreliable. GPT-4V frequently hallucinates pixel outputs, causing it to miss interactions, or enter fail-loops

>The approach by AI Jason

AI Jason is using image-only methods to interact with the browser. This is a great first step, but this approach tends to be rife with hallucinations or errors. We do dom parsing in addition to image anaylsis to help GPT-4V correlate information in the image to the interactable elements within the DOM. This dramatically boosts its ability to perform the same task over and over again reliably (which proved impossible with the image-only approach)

Something close to this is being worked on. https://github.com/OthersideAI/self-operating-computer

It would be fantastic if you could control the smartphone with gpt4 vision through screenshots something similar to this: https://github.com/OthersideAI/self-operating-computer but for Android.