deep-swe
arena-ai-leaderboards
| deep-swe | arena-ai-leaderboards | |
|---|---|---|
| 11 | 4 | |
| 101 | 14 | |
| 0.0% | - | |
| - | 7.8 | |
| 22 days ago | 4 days ago | |
| Shell | Python | |
| - | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
deep-swe
-
AWS Bedrock to require sharing data with Anthropic for Mythos and future models
That remains to be seen.
It's notable that Anthropic are still using SWEBench as a coding benchmark rather that the newer more difficult DeepSWE which shows them well behind GPT 5.5
https://deepswe.datacurve.ai/
Bear in mind that all the marketing efforts such as solving Erdos problem are the result of concerted RL training to impart those narrow capabilities, and how much of any benchmark results, or paid shill vibe reports, reflect improved performance for more general real-world use cases remains to be seen.
-
DeepSeek V4 Pro beats GPT-5.5 Pro on precision
This benchmark draws a very different picture having GPT5.5 on the very top with 70% and DeepSeek at 8%
https://deepswe.datacurve.ai
- DeepSWE results are unreliable – 3/3 DSv4 "failed" tasks solved with same model
- DeepSWE: Measuring frontier coding agents on original, long-horizon SWE tasks
- DeepSWE Audit: DeepSeek-v4-pro results are unreliable
-
DeepSWE: More and cheaper intelligence from maxed GPT 5.5 than maxed Opus 4.8
Source: https://deepswe.datacurve.ai
Just select the two models from the drop down.
-
Claude Opus 4.8
Where did you get that idea? It uses mini-swe-agent, same as SWE-Bench.
https://github.com/datacurve-ai/deep-swe
- DeepSWE: Measuring coding agents on original, long-horizon engineering tasks
- DeepSWE Measuring frontier coding agents
arena-ai-leaderboards
-
Claude Opus 4.8
Depends what you want it for. Probably Qwen
https://arena.ai/leaderboard
-
The Best LLMs for Agentic Coding in 2026 (Real-World, Not Just Benchmarks)
Claude Opus 4.7 currently holds the #1 spot on the LMSYS Arena leaderboard in thinking mode (arena.ai/leaderboard) and scores 87.6% on SWE-bench Verified (Anthropic launch post, Vellum breakdown) - the top vendor-reported result among all models as of May 2026. It's the strongest coding agent model available in a closed API today. The 4.7 release addressed community complaints about 4.6's habit of over-scoping - it now stays more focused, edits fewer files than asked, and explains its reasoning more clearly mid-run.
-
I Built an Auto-Updating Archive of Every AI Arena Leaderboard
So I built arena-ai-leaderboards — a GitHub repo that auto-fetches all 10 Arena AI leaderboards daily into structured JSON.
What are some alternatives?
anthropic-sdk-python
Pixelle-Video - 🚀 AI 全自动çŸè§†é¢‘引擎 | AI Fully Automated Short Video Engine
claude-code-system-prompts - All parts of Claude Code's system prompt, 27 builtin tool descriptions, sub agent prompts (Plan/Explore/Task), utility prompts (CLAUDE.md, compact, statusline, magic docs, WebFetch, Bash cmd, security review, agent creation). Updated for each Claude Code version.
tau2-bench - Ï„-Bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains