Segment-Everything-Everywhere-All-At-Once VS datasaurus

Yes, you can. The model that I was talking about LLaVA only output text but other models such as SEEM (https://github.com/UX-Decoder/Segment-Everything-Everywhere-...) outputs a segmentation map. You could prompt the model "Where is the pickleball in the image?" and get a segmentation map that you could then use to compute its center. Please let me know if you would be interested to have SEEM available in Datasaurus

I think people are just seeing the rate of progress and rightfully think that this stuff will be possible at some point. For the rotoscoping for example, here's an example of progress being made on that.

Yeah, it's definitely possible especially with all the recent advances. With segment anything systems (like SAM) and segmentation on NeRF reconstructions already being a thing the feasibility of this is more a time investment thing. Naive "scene understanding" is already possible in a few AR headsets at real-time, but the new papers in the past few weeks have made this much more trivial and faster to implement.

Play with the demo on GitHub! https://github.com/UX-Decoder/Segment-Everything-Everywhere-All-At-Once

And let’s talk about development speed. By using text prompts to interact with your images, you can whip up a computer vision prototype in seconds. It’s fast, it’s efficient, and it’s changing the game.

So, what do you all think? Are we moving towards a future where foundational models take the lead in computer vision, or is there still a place for training models from scratch?

P.S. Shameless plug: I’ve been working on this open-source platform called Datasaurus https://github.com/datasaurus-ai/datasaurus) that taps into the power of vision-language models. It’s all about helping engineers get the insights they need from images, fast. Just wanted to share some thoughts and start a conversation. Let’s talk about the future of computer vision!