R1 Computer Use

1. r1-computer-use

   1 2 206 7.2 Python

   Applying the ideas of Deepseek R1 to computer use

   

2. CodeRabbit

   coderabbit.ai featured

   CodeRabbit: AI Code Reviews for Developers. Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.

   

3. AutoGPT

   2 184 173,433 9.9 Python

   AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.

   

   Check out AutoGPT you don't have to wait, it's already built.

   https://github.com/Significant-Gravitas/AutoGPT

4. clickclickclick

   3 10 386 7.5 Python

   A framework to enable autonomous android and computer use using any LLM (local or remote)

   

   Training a base model just for computer use seems like an overkill as normal reasoning model like o3 for planning + a vision model like gemini-flash is good enough[1] without being trained specifically for computer use.

   But if you still want to try out this path, Google has made the screenQA dataset(rico) available[2] along with bounding boxes.

   1. A framework to use local/hosted models for android use/control - https://github.com/BandarLabs/clickclickclick

   2. https://github.com/google-research-datasets/screen_qa

5. screen_qa

   4 1 106 2.7 Python

   ScreenQA dataset was introduced in the "ScreenQA: Large-Scale Question-Answer Pairs over Mobile App Screenshots" paper. It contains ~86K question-answer pairs collected by human annotators for ~35K screenshots from Rico. It should be used to train and evaluate models capable of screen content understanding via question answering.

   

   Training a base model just for computer use seems like an overkill as normal reasoning model like o3 for planning + a vision model like gemini-flash is good enough[1] without being trained specifically for computer use.

   But if you still want to try out this path, Google has made the screenQA dataset(rico) available[2] along with bounding boxes.

   1. A framework to use local/hosted models for android use/control - https://github.com/BandarLabs/clickclickclick

   2. https://github.com/google-research-datasets/screen_qa

Suggest a related project