Updated 2026-10-20 — since 2026-09-22
Snazzyham's AI Benchmark
Models tested against my personal workflows, rather than an arbitrary benchmark.
Latest result per task for each model. Full stack averages FS1 and FS2, and Lead averages L1 and L2; each needs both tasks; with one missing it shows as incomplete. Click a row for task scores, dates and source runs; click a column header to sort.
/ min
The effective token rate sets how much you value time saved, and the Best value pick in each track is recommended on that basis. At a lower rate, price counts for more, so cheaper models win even when they are slower. At a higher rate, speed counts for more, so faster models win even when they cost more. When you filter models, the pick is made among the models you filtered.
Search
Eight questions across six repositories, answered with exact paths and line numbers.
What the tasks check
- S1 checks repository answers for exact paths and line numbers
| anthropic/claude-opus-5-5max | $3.09 | $5.03 | 9m 41s | 3439k | 26 Sep 15:17 | |
| ||||||
| opencode-go/glm-5.3-flashlow | $0.04 | $0.46 | 2m 7s | 629k | 24 Sep 14:25 | |
| ||||||
| anthropic/claude-opus-5-5low | $0.51 | $0.65 | 41s | 194k | 24 Sep 13:38 | |
| ||||||
| opencode-zen/gemini-3.8-flashmedium | $0.98 | $1.84 | 4m 16s | 2911k | 23 Sep 19:41 | |
| ||||||
| openai-codex/gpt-6-astralow | $1.04 | $1.54 | 2m 31s | 295k | 22 Sep 21:07 | |
| ||||||
| opencode-go/deepseek-v4-promax | $0.12 | $1.38 | 6m 18s | 797k | 22 Sep 14:00 | |
| ||||||
| openai-codex/gpt-6-lunamedium | $0.01 | $0.28 | 1m 20s | 659k | 26 Sep 09:33 | |
| ||||||
| opencode-go/deepseek-v4.1-flashlowBest value | $0.01 | $0.23 | 1m 6s | 303k | 20 Oct 09:19 | |
| ||||||
| opencode-go/deepseek-v4.1-flashhigh | $0.01 | $0.78 | 3m 48s | 385k | 19 Oct 12:43 | |
| ||||||
| opencode-go/space-bunny-freeminimal | $0.00 | $0.31 | 1m 33s | 564k | 27 Sep 17:52 | |
| ||||||
| opencode-go/qwen3.8-flashhigh | $0.02 | $0.67 | 3m 17s | 348k | 23 Sep 20:29 | |
| ||||||
| anthropic/claude-opus-5low | $0.62 | $0.85 | 1m 9s | 284k | 22 Sep 12:00 | |
| ||||||
| opencode-go/minimax-m3.1-flash-freelow | $0.00 | $0.45 | 2m 14s | 799k | 20 Oct 00:00 | |
| ||||||
| opencode-go/minimax-m3.1-flash-freemedium | $0.00 | $1.32 | 6m 36s | 595k | 20 Oct 00:00 | |
| ||||||
| openai-codex/gpt-6-lunahigh | $0.01 | $0.47 | 2m 18s | 497k | 08 Oct 12:00 | |
| ||||||
| openai-codex/gpt-6.1-solxhigh | $0.40 | $2.24 | 9m 13s | 937k | 30 Sep 14:00 | |
| ||||||
| openai-codex/gpt-6-sollow | $0.12 | $0.35 | 1m 10s | 157k | 24 Sep 13:50 | |
| ||||||
| openai-codex/gpt-6-solxhigh | $0.29 | $0.94 | 3m 14s | 475k | 23 Sep 14:00 | |
| ||||||
| openai-codex/gpt-6-lunamax | $0.03 | $1.55 | 7m 36s | 579k | 23 Sep 10:50 | |
| ||||||
| openai-codex/gpt-5.6-terramedium | $0.25 | $0.75 | 2m 32s | 308k | 23 Sep 09:24 | |
| ||||||
| opencode-go/muse-spark-1.3-contributorxhigh | $0.03 | $2.35 | 11m 36s | 1195k | 22 Sep 14:00 | |
| ||||||
| openai-codex/gpt-5.6-solhigh | $0.80 | $1.93 | 5m 38s | 732k | 22 Sep 12:00 | |
| ||||||
| openai-codex/gpt-5.6-lunamax | $0.11 | $1.72 | 8m 3s | 2030k | 22 Sep 08:30 | |
| ||||||
| anthropic/claude-haiku-5-5high | $0.03 | $0.50 | 2m 21s | 371k | 08 Oct 12:00 | |
| ||||||
| anthropic/claude-haiku-5-5low | $0.03 | $0.59 | 2m 50s | 565k | 08 Oct 01:25 | |
| ||||||
| opencode-go/mimo-v2.6-flashlow | $0.01 | $0.91 | 4m 29s | 616k | 26 Sep 09:33 | |
| ||||||
| anthropic/claude-opus-5-5medium | $0.82 | $1.06 | 1m 13s | 680k | 23 Sep 08:45 | |
| ||||||
| anthropic/claude-opus-5max | $1.22 | $1.49 | 1m 23s | 406k | 22 Sep 21:07 | |
| ||||||
| anthropic/claude-sonnet-5-5high | $0.40 | $0.78 | 1m 56s | 456k | 29 Sep 06:03 | |
| ||||||
| anthropic/claude-sonnet-5-5low | $0.25 | $0.43 | 53s | 268k | 29 Sep 00:00 | |
| ||||||
| opencode-go/deepseek-v4.1-flashmax | $0.04 | $0.96 | 4m 35s | 912k | 22 Sep 08:30 | |
| ||||||
| opencode-go/glm-5.3-flashmax | $0.02 | $0.49 | 2m 19s | 496k | 22 Sep 08:30 | |
| ||||||
| openai-codex/gpt-5.6-lunalow | $0.03 | $0.36 | 1m 40s | 455k | 01 Oct 00:18 | |
| ||||||
| opencode-go/space-bunny-freexhigh | $0.00 | $0.33 | 1m 38s | 660k | 27 Sep 17:52 | |
| ||||||
| anthropic/claude-sonnet-5medium | $0.36 | $0.64 | 1m 24s | 510k | 22 Sep 12:00 | |
| ||||||
| openai-codex/gpt-6-lunalow | $0.02 | $0.41 | 1m 59s | 864k | 01 Oct 00:18 | |
| ||||||
| opencode-go/omen-alphaunknown | $0.00 | $0.47 | 2m 21s | 720k | 17 Oct 08:44 | |
| ||||||
| opencode-go/beam-freehigh | $0.00 | $0.59 | 2m 57s | 588k | 17 Oct 08:44 | |
| ||||||
| opencode-go/longcat-2.5-preview-freeminimal | $0.00 | $0.86 | 4m 19s | 269k | 27 Sep 17:52 | |
| ||||||
| opencode-go/muse-spark-1.3-contributorminimal | $0.01 | $0.35 | 1m 42s | 408k | 23 Sep 09:24 | |
| ||||||
| openai-codex/gpt-6.1-sollow | $0.21 | $1.05 | 4m 12s | 394k | 30 Sep 14:00 | |
| ||||||
| anthropic/claude-sonnet-4-6low | $0.59 | $1.22 | 3m 9s | 388k | 30 Sep 12:00 | |
| ||||||
| anthropic/claude-haiku-4-5minimal | $0.23 | $0.64 | 2m 1s | 630k | 28 Sep 08:20 | |
| ||||||
| opencode-go/mimo-v2.6-flashhigh | $0.08 | $3.18 | 15m 31s | 1449k | 22 Sep 08:30 | |
| ||||||
| opencode-go/minimax-m3.1-flash-freexhighhard fail | $0.00 | $0.56 | 2m 49s | 969k | 20 Oct 00:00 | |
| ||||||
Best value: the paid model within 8 points of the best paid score, scoring 75 or above, with the lowest cost once its time is charged at the chosen rate.
Digest
Compress a 3,853-line business-rule spec and a 723-line client audit, in under 40 lines each.
What the tasks check
- D1 checks a business-rule specification is compressed without losing conditions
- D2 checks a client audit is compressed without losing critical issues
| opencode-go/deepseek-v4.1-flashmax | D1 32.9 · D2 96.8 | $0.03 | $0.43 | 2m 0s | 563k | 22 Sep 08:30 |
| ||||||
| opencode-go/deepseek-v4.1-flashhigh | D1 39.3 · D2 87.5 | $0.01 | $0.33 | 1m 36s | 316k | 19 Oct 12:43 |
| ||||||
| anthropic/claude-opus-5-5max | D1 34.3 · D2 75.0 | $3.35 | $6.09 | 13m 44s | 840k | 26 Sep 15:17 |
| ||||||
| opencode-go/minimax-m3.1-flash-freelow | D1 32.7 · D2 75.0 | $0.00 | $0.16 | 49s | 219k | 20 Oct 00:00 |
| ||||||
| openai-codex/gpt-6.1-solxhigh | D1 32.9 · D2 70.7 | $0.26 | $1.15 | 4m 27s | 326k | 30 Sep 14:00 |
| ||||||
| openai-codex/gpt-6-astralow | D1 28.6 · D2 75.0 | $1.01 | $1.39 | 1m 52s | 206k | 22 Sep 21:07 |
| ||||||
| openai-codex/gpt-5.6-lunalow | D1 28.6 · D2 75.0 | $0.03 | $0.22 | 59s | 313k | 22 Sep 12:00 |
| ||||||
| openai-codex/gpt-5.6-lunamax | D1 28.6 · D2 75.0 | $0.06 | $1.47 | 7m 5s | 531k | 22 Sep 08:30 |
| ||||||
| opencode-go/omen-alphaunknown | D1 27.1 · D2 75.0 | $0.00 | $0.55 | 2m 44s | 420k | 17 Oct 08:44 |
| ||||||
| openai-codex/gpt-5.6-terramedium | D1 26.4 · D2 75.0 | $0.19 | $0.40 | 1m 4s | 216k | 23 Sep 09:24 |
| ||||||
| opencode-go/muse-spark-1.3-contributorxhigh | D1 30.0 · D2 68.6 | $0.01 | $0.44 | 2m 10s | 227k | 22 Sep 14:00 |
| ||||||
| openai-codex/gpt-6-lunalow | D1 32.9 · D2 65.5 | $0.01 | $0.18 | 52s | 162k | 26 Sep 09:33 |
| ||||||
| anthropic/claude-opus-5-5low | D1 32.9 · D2 65.5 | $0.97 | $1.11 | 42s | 341k | 24 Sep 13:38 |
| ||||||
| openai-codex/gpt-6.1-sollow | D1 26.4 · D2 71.8 | $0.13 | $0.53 | 2m 0s | 166k | 30 Sep 14:00 |
| ||||||
| opencode-go/deepseek-v4.1-flashlow | D1 26.4 · D2 68.8 | $0.02 | $0.58 | 2m 48s | 904k | 20 Oct 09:19 |
| ||||||
| openai-codex/gpt-6-sollow | D1 26.4 · D2 68.8 | $0.12 | $0.31 | 56s | 150k | 24 Sep 13:50 |
| ||||||
| openai-codex/gpt-6-lunamax | D1 30.7 · D2 64.5 | $0.02 | $0.85 | 4m 11s | 517k | 23 Sep 10:50 |
| ||||||
| opencode-go/glm-5.3-flashmax | D1 23.6 · D2 70.0 | $0.02 | $0.17 | 47s | 267k | 22 Sep 08:30 |
| ||||||
| anthropic/claude-opus-5low | D1 27.9 · D2 65.0 | $1.12 | $1.27 | 43s | 329k | 22 Sep 12:00 |
| ||||||
| opencode-go/minimax-m3.1-flash-freemedium | D1 32.9 · D2 58.8 | $0.00 | $0.39 | 1m 58s | 586k | 20 Oct 00:00 |
| ||||||
| anthropic/claude-sonnet-5-5high | D1 30.0 · D2 61.3 | $0.49 | $0.69 | 1m 2s | 446k | 29 Sep 06:03 |
| ||||||
| anthropic/claude-haiku-5-5high | D1 27.9 · D2 62.5 | $0.02 | $0.26 | 1m 11s | 302k | 08 Oct 12:00 |
| ||||||
| anthropic/claude-sonnet-5-5low | D1 21.4 · D2 68.8 | $0.32 | $0.45 | 39s | 201k | 29 Sep 00:00 |
| ||||||
| openai-codex/gpt-6-lunamedium | D1 24.3 · D2 65.5 | $0.01 | $0.18 | 52s | 140k | 26 Sep 09:33 |
| ||||||
| opencode-go/glm-5.3-flashlow | D1 18.6 · D2 70.0 | $0.03 | $0.19 | 48s | 248k | 24 Sep 14:25 |
| ||||||
| anthropic/claude-opus-5max | D1 23.6 · D2 64.5 | $1.16 | $1.44 | 1m 24s | 484k | 22 Sep 21:07 |
| ||||||
| openai-codex/gpt-5.6-solhigh | D1 23.6 · D2 64.5 | $0.59 | $1.04 | 2m 15s | 295k | 22 Sep 12:00 |
| ||||||
| openai-codex/gpt-6-lunahigh | D1 17.1 · D2 70.7 | $0.01 | $0.14 | 40s | 160k | 08 Oct 12:00 |
| ||||||
| opencode-go/mimo-v2.6-flashlow | D1 23.6 · D2 63.8 | $0.01 | $1.35 | 6m 42s | 344k | 26 Sep 09:33 |
| ||||||
| opencode-go/muse-spark-1.3-contributorminimal | D1 15.0 · D2 72.0 | $0.01 | $0.23 | 1m 5s | 316k | 23 Sep 09:24 |
| ||||||
| anthropic/claude-opus-5-5medium | D1 21.4 · D2 65.5 | $1.02 | $1.17 | 45s | 352k | 23 Sep 08:45 |
| ||||||
| opencode-go/mimo-v2.6-flashhigh | D1 16.4 · D2 68.8 | $0.01 | $0.21 | 1m 2s | 193k | 22 Sep 08:30 |
| ||||||
| openai-codex/gpt-6-solxhigh | D1 22.1 · D2 62.5 | $0.24 | $0.54 | 1m 28s | 403k | 23 Sep 14:00 |
| ||||||
| opencode-go/space-bunny-freexhigh | D1 20.7 · D2 63.8 | $0.00 | $0.19 | 56s | 269k | 27 Sep 17:52 |
| ||||||
| opencode-go/space-bunny-freeminimal | D1 17.1 · D2 66.8 | $0.00 | $0.11 | 34s | 148k | 27 Sep 17:52 |
| ||||||
| opencode-zen/gemini-3.8-flashmedium | D1 20.7 · D2 63.2 | $0.22 | $0.47 | 1m 14s | 341k | 23 Sep 19:41 |
| ||||||
| opencode-go/minimax-m3.1-flash-freexhigh | D1 19.3 · D2 63.8 | $0.00 | $0.43 | 2m 9s | 1393k | 20 Oct 00:00 |
| ||||||
| anthropic/claude-haiku-5-5low | D1 16.4 · D2 66.4 | $0.02 | $0.17 | 47s | 320k | 08 Oct 01:25 |
| ||||||
| opencode-go/beam-freehigh | D1 21.4 · D2 60.2 | $0.00 | $0.08 | 25s | 371k | 17 Oct 08:44 |
| ||||||
| opencode-go/longcat-2.5-preview-freeminimal | D1 16.4 · D2 63.8 | $0.00 | $0.27 | 1m 21s | 85k | 27 Sep 17:52 |
| ||||||
| opencode-go/qwen3.8-flashhigh | D1 16.4 · D2 63.8 | $0.02 | $0.55 | 2m 40s | 468k | 23 Sep 20:29 |
| ||||||
| anthropic/claude-sonnet-4-6low | D1 14.3 · D2 65.5 | $0.58 | $0.84 | 1m 18s | 236k | 30 Sep 12:00 |
| ||||||
| opencode-go/deepseek-v4-promax | D1 21.4 · D2 55.2 | $0.06 | $0.33 | 1m 20s | 350k | 22 Sep 14:00 |
| ||||||
| anthropic/claude-sonnet-5medium | D1 12.1 · D2 57.0 | $0.31 | $0.42 | 33s | 189k | 22 Sep 12:00 |
| ||||||
| anthropic/claude-haiku-4-5minimal | D1 14.3 · D2 50.7 | $0.19 | $0.36 | 51s | 207k | 28 Sep 08:20 |
| ||||||
Plan
Name every file a known change has to touch. The real fixes touched 25 and 19 files.
What the tasks check
- P1 checks a plan names every file affected by the icon-system change
- P2 checks a plan names every file affected by the materialised-order-cost change
| openai-codex/gpt-5.6-solhigh | P1 46.1 · P2 78.1 | $3.94 | $8.22 | 21m 23s | 5146k | 22 Sep 12:00 |
| ||||||
| opencode-go/mimo-v2.6-flashlow | P1 48.7 · P2 73.5 | $0.07 | $9.60 | 47m 36s | 4310k | 26 Sep 09:33 |
| ||||||
| anthropic/claude-haiku-5-5high | P1 44.7 · P2 76.7 | $0.25 | $1.99 | 8m 40s | 2150k | 08 Oct 12:00 |
| ||||||
| openai-codex/gpt-6.1-solxhigh | P1 51.1 · P2 70.0 | $1.31 | $7.16 | 29m 16s | 3451k | 30 Sep 14:00 |
| ||||||
| openai-codex/gpt-6.1-sollow | P1 46.1 · P2 71.7 | $0.48 | $2.29 | 9m 4s | 1042k | 30 Sep 14:00 |
| ||||||
| opencode-go/deepseek-v4-promax | P1 42.0 · P2 75.3 | $0.28 | $3.09 | 14m 4s | 3660k | 22 Sep 14:00 |
| ||||||
| anthropic/claude-opus-5low | P1 47.4 · P2 67.5 | $1.81 | $2.36 | 2m 46s | 859k | 22 Sep 12:00 |
| ||||||
| openai-codex/gpt-6-lunamax | P1 45.3 · P2 69.4 | $0.11 | $6.92 | 34m 2s | 3398k | 23 Sep 10:50 |
| ||||||
| anthropic/claude-opus-5-5max | P1 42.0 · P2 71.7 | $15.75 | $23.43 | 38m 24s | 17278k | 26 Sep 15:17 |
| ||||||
| opencode-go/minimax-m3.1-flash-freelow | P1 42.0 · P2 70.3 | $0.00 | $1.91 | 9m 34s | 2181k | 20 Oct 00:00 |
| ||||||
| opencode-go/deepseek-v4.1-flashmax | P1 34.7 · P2 73.7 | $0.15 | $3.42 | 16m 20s | 5900k | 22 Sep 08:30 |
| ||||||
| openai-codex/gpt-5.6-lunamax | P1 30.0 · P2 78.1 | $0.33 | $4.08 | 18m 48s | 6284k | 22 Sep 08:30 |
| ||||||
| opencode-go/qwen3.8-flashhigh | P1 25.0 · P2 80.8 | $0.07 | $2.94 | 14m 20s | 1895k | 23 Sep 20:29 |
| ||||||
| opencode-go/glm-5.3-flashmax | P1 37.4 · P2 65.7 | $0.09 | $1.77 | 8m 24s | 2077k | 22 Sep 08:30 |
| ||||||
| openai-codex/gpt-6-astralow | P1 48.8 · P2 50.6 | $3.29 | $4.80 | 7m 31s | 1340k | 22 Sep 21:07 |
| ||||||
| opencode-go/deepseek-v4.1-flashlow | P1 42.0 · P2 55.3 | $0.11 | $3.18 | 15m 20s | 7931k | 20 Oct 09:19 |
| ||||||
| opencode-go/minimax-m3.1-flash-freemedium | P1 42.0 · P2 51.7 | $0.00 | $3.28 | 16m 24s | 3298k | 20 Oct 00:00 |
| ||||||
| opencode-go/glm-5.3-flashlow | P1 20.8 · P2 68.1 | $0.09 | $1.07 | 4m 55s | 1269k | 24 Sep 14:25 |
| ||||||
| anthropic/claude-sonnet-5medium | P1 62.4 · P2 24.4 | $1.70 | $3.37 | 8m 20s | 2839k | 22 Sep 12:00 |
| ||||||
| anthropic/claude-sonnet-5-5high | P1 42.4 · P2 43.3 | $1.39 | $2.09 | 3m 28s | 2394k | 29 Sep 06:03 |
| ||||||
| anthropic/claude-opus-5-5medium | P1 44.7 · P2 39.8 | $3.20 | $4.43 | 6m 9s | 4363k | 23 Sep 08:45 |
| ||||||
| openai-codex/gpt-5.6-terramedium | P1 30.0 · P2 54.5 | $1.01 | $2.13 | 5m 37s | 1653k | 23 Sep 09:24 |
| ||||||
| openai-codex/gpt-6-solxhigh | P1 45.7 · P2 35.3 | $1.38 | $3.36 | 9m 55s | 3559k | 23 Sep 14:00 |
| ||||||
| anthropic/claude-haiku-5-5low | P1 24.7 · P2 55.3 | $0.03 | $0.44 | 2m 4s | 482k | 08 Oct 01:25 |
| ||||||
| opencode-go/space-bunny-freeminimal | P1 30.7 · P2 49.4 | $0.00 | $0.32 | 1m 37s | 695k | 27 Sep 17:52 |
| ||||||
| openai-codex/gpt-6-lunalow | P1 29.3 · P2 47.1 | $0.02 | $0.65 | 3m 9s | 579k | 26 Sep 09:33 |
| ||||||
| anthropic/claude-opus-5max | P1 34.7 · P2 37.5 | $3.86 | $5.34 | 7m 24s | 2586k | 22 Sep 21:07 |
| ||||||
| anthropic/claude-haiku-4-5minimal | P1 28.3 · P2 43.6 | $0.41 | $1.38 | 4m 52s | 1287k | 28 Sep 08:20 |
| ||||||
| openai-codex/gpt-6-lunamedium | P1 25.0 · P2 46.4 | $0.02 | $0.85 | 4m 9s | 696k | 26 Sep 09:33 |
| ||||||
| openai-codex/gpt-6-lunahigh | P1 30.0 · P2 41.0 | $0.03 | $1.12 | 5m 28s | 1318k | 08 Oct 12:00 |
| ||||||
| anthropic/claude-sonnet-5-5low | P1 26.8 · P2 42.3 | $0.53 | $0.80 | 1m 22s | 646k | 29 Sep 00:00 |
| ||||||
| anthropic/claude-opus-5-5low | P1 34.7 · P2 32.8 | $0.95 | $1.27 | 1m 36s | 599k | 24 Sep 13:38 |
| ||||||
| anthropic/claude-sonnet-4-6low | P1 21.3 · P2 42.7 | $1.11 | $2.57 | 7m 20s | 842k | 30 Sep 12:00 |
| ||||||
| openai-codex/gpt-5.6-lunalow | P1 24.3 · P2 37.6 | $0.04 | $0.67 | 3m 10s | 726k | 22 Sep 12:00 |
| ||||||
| openai-codex/gpt-6-sollow | P1 20.0 · P2 40.1 | $0.33 | $0.68 | 1m 46s | 504k | 24 Sep 13:50 |
| ||||||
| opencode-go/space-bunny-freexhigh | P1 36.3 · P2 20.6 | $0.00 | $1.69 | 8m 28s | 4609k | 27 Sep 17:52 |
| ||||||
| opencode-go/beam-freehigh | P1 0.0 · P2 54.9 | $0.00 | $1.11 | 5m 34s | 1668k | 17 Oct 08:44 |
| ||||||
| opencode-go/muse-spark-1.3-contributorminimal | P1 0.0 · P2 47.4 | $0.02 | $0.38 | 1m 49s | 657k | 23 Sep 09:24 |
| ||||||
| opencode-go/mimo-v2.6-flashhigh | P1 25.0 · P2 20.0 | $0.24 | $6.19 | 29m 45s | 4248k | 22 Sep 08:30 |
| ||||||
| opencode-go/minimax-m3.1-flash-freexhighhard fail | P1 39.3 · P2 0.0 | $0.00 | $2.17 | 10m 50s | 4992k | 20 Oct 00:00 |
| ||||||
| opencode-go/deepseek-v4.1-flashhighhard fail | P1 38.8 · P2 0.0 | $0.09 | $2.72 | 13m 8s | 6745k | 19 Oct 12:43 |
| ||||||
| opencode-go/muse-spark-1.3-contributorxhigh | P1 34.3 · P2 4.3 | $0.04 | $2.89 | 14m 16s | 2077k | 22 Sep 14:00 |
| ||||||
| opencode-zen/gemini-3.8-flashmediumhard fail | P1 0.0 · P2 38.1 | $1.10 | $2.33 | 6m 10s | 2498k | 23 Sep 19:41 |
| ||||||
| opencode-go/longcat-2.5-preview-freeminimal | P1 22.3 · P2 11.6 | $0.00 | $6.40 | 32m 0s | 3672k | 27 Sep 17:52 |
| ||||||
| opencode-go/omen-alphaunknownhard fail | P1 0.0 · P2 0.0 | $0.00 | $3.49 | 17m 28s | 2781k | 17 Oct 08:44 |
| ||||||
Frontend dev
Ship a CMS section end to end, and apply a design batch that then gets corrected mid-task.
What the tasks check
- B1 checks a CMS section is shipped end to end and correctly registered
- B2 checks a design batch is applied and corrected after a steering request
| opencode-go/deepseek-v4.1-flashlow | B1 100.0 · B2 100.0 | $0.04 | $2.60 | 12m 48s | 2890k | 20 Oct 09:19 |
| ||||||
| opencode-go/minimax-m3.1-flash-freelow | B1 100.0 · B2 100.0 | $0.00 | $3.26 | 16m 19s | 3003k | 20 Oct 00:00 |
| ||||||
| anthropic/claude-sonnet-5-5high | B1 100.0 · B2 100.0 | $0.77 | $2.32 | 7m 46s | 1326k | 29 Sep 06:03 |
| ||||||
| anthropic/claude-sonnet-5-5low | B1 100.0 · B2 100.0 | $0.56 | $0.93 | 1m 49s | 761k | 29 Sep 00:00 |
| ||||||
| openai-codex/gpt-6-lunalow | B1 100.0 · B2 100.0 | $0.04 | $3.75 | 18m 34s | 2584k | 26 Sep 10:44 |
| ||||||
| openai-codex/gpt-6-sollow | B1 100.0 · B2 100.0 | $0.56 | $1.57 | 5m 4s | 1641k | 24 Sep 13:50 |
| ||||||
| opencode-zen/gemini-3.8-flashmedium | B1 100.0 · B2 100.0 | $4.32 | $9.38 | 25m 18s | 12239k | 23 Sep 19:41 |
| ||||||
| openai-codex/gpt-6-lunamax | B1 100.0 · B2 100.0 | $0.06 | $7.69 | 38m 8s | 2425k | 23 Sep 10:50 |
| ||||||
| opencode-go/deepseek-v4-promax | B1 100.0 · B2 100.0 | $0.36 | $5.02 | 23m 19s | 5903k | 22 Sep 14:00 |
| ||||||
| anthropic/claude-opus-5low | B1 100.0 · B2 100.0 | $1.95 | $2.79 | 4m 14s | 1561k | 22 Sep 12:00 |
| ||||||
| opencode-go/beam-freehigh | B1 100.0 · B2 96.0 | $0.00 | $2.28 | 11m 25s | 3182k | 17 Oct 08:44 |
| ||||||
| anthropic/claude-haiku-5-5high | B1 100.0 · B2 95.0 | $0.15 | $3.08 | 14m 38s | 2799k | 08 Oct 12:00 |
| ||||||
| openai-codex/gpt-6.1-sollow | B1 100.0 · B2 95.0 | $0.37 | $2.61 | 11m 12s | 1313k | 30 Sep 14:00 |
| ||||||
| opencode-go/space-bunny-freeminimal | B1 100.0 · B2 95.0 | $0.00 | $0.73 | 3m 39s | 1294k | 27 Sep 17:52 |
| ||||||
| anthropic/claude-opus-5-5max | B1 100.0 · B2 95.0 | $12.91 | $19.84 | 34m 38s | 19939k | 26 Sep 15:17 |
| ||||||
| opencode-go/mimo-v2.6-flashlow | B1 100.0 · B2 95.0 | $0.06 | $9.09 | 45m 11s | 5044k | 26 Sep 09:33 |
| ||||||
| openai-codex/gpt-6-solxhigh | B1 100.0 · B2 95.0 | $1.09 | $3.45 | 11m 50s | 2797k | 23 Sep 14:00 |
| ||||||
| openai-codex/gpt-5.6-terramedium | B1 100.0 · B2 95.0 | $0.97 | $2.67 | 8m 28s | 2178k | 23 Sep 09:24 |
| ||||||
| openai-codex/gpt-6-astralow | B1 100.0 · B2 95.0 | $3.92 | $6.22 | 11m 30s | 2138k | 22 Sep 21:07 |
| ||||||
| opencode-go/muse-spark-1.3-contributorxhigh | B1 100.0 · B2 95.0 | $0.10 | $12.05 | 59m 47s | 5792k | 22 Sep 14:00 |
| ||||||
| openai-codex/gpt-5.6-lunamax | B1 100.0 · B2 95.0 | $0.20 | $3.79 | 17m 59s | 4576k | 22 Sep 08:30 |
| ||||||
| opencode-go/deepseek-v4.1-flashmax | B1 100.0 · B2 95.0 | $0.22 | $5.70 | 27m 23s | 19148k | 22 Sep 08:30 |
| ||||||
| opencode-go/mimo-v2.6-flashhigh | B1 100.0 · B2 95.0 | $0.10 | $6.43 | 31m 41s | 3293k | 22 Sep 08:30 |
| ||||||
| openai-codex/gpt-6-lunahigh | B1 100.0 · B2 92.5 | $0.07 | $1.75 | 8m 23s | 3817k | 08 Oct 12:00 |
| ||||||
| openai-codex/gpt-5.6-solhigh | B1 100.0 · B2 92.5 | $2.29 | $6.31 | 20m 6s | 3174k | 22 Sep 12:00 |
| ||||||
| openai-codex/gpt-5.6-lunalow | B1 100.0 · B2 91.0 | $0.07 | $1.59 | 7m 35s | 1431k | 22 Sep 12:00 |
| ||||||
| opencode-go/deepseek-v4.1-flashhigh | B1 100.0 · B2 90.0 | $0.21 | $6.43 | 31m 5s | 8991k | 19 Oct 12:43 |
| ||||||
| anthropic/claude-haiku-5-5low | B1 100.0 · B2 90.0 | $0.05 | $1.50 | 7m 15s | 1285k | 08 Oct 01:25 |
| ||||||
| opencode-go/glm-5.3-flashmax | B1 100.0 · B2 90.0 | $0.28 | $5.24 | 24m 50s | 6416k | 22 Sep 08:30 |
| ||||||
| anthropic/claude-opus-5max | B1 100.0 · B2 87.5 | $5.92 | $9.12 | 15m 59s | 5185k | 22 Sep 21:07 |
| ||||||
| openai-codex/gpt-6-lunamedium | B1 90.0 · B2 96.0 | $0.03 | $1.38 | 6m 43s | 1814k | 26 Sep 09:33 |
| ||||||
| opencode-go/omen-alphaunknown | B1 85.0 · B2 100.0 | $0.00 | $10.45 | 52m 16s | 6231k | 17 Oct 08:44 |
| ||||||
| anthropic/claude-sonnet-5medium | B1 85.0 · B2 100.0 | $1.32 | $2.78 | 7m 18s | 2273k | 22 Sep 12:00 |
| ||||||
| anthropic/claude-opus-5-5low | B1 85.0 · B2 95.0 | $1.00 | $1.42 | 2m 6s | 904k | 24 Sep 13:38 |
| ||||||
| opencode-go/qwen3.8-flashhigh | B1 85.0 · B2 95.0 | $0.16 | $5.40 | 26m 12s | 5844k | 23 Sep 20:29 |
| ||||||
| anthropic/claude-opus-5-5medium | B1 85.0 · B2 95.0 | $1.81 | $3.02 | 6m 3s | 2315k | 23 Sep 08:45 |
| ||||||
| opencode-go/glm-5.3-flashlow | B1 100.0 · B2 76.0 | $0.18 | $3.56 | 16m 55s | 3291k | 24 Sep 14:25 |
| ||||||
| opencode-go/minimax-m3.1-flash-freemedium | B1 80.0 · B2 95.0 | $0.00 | $13.84 | 69m 12s | 9260k | 20 Oct 00:00 |
| ||||||
| opencode-go/space-bunny-freexhigh | B1 85.0 · B2 90.0 | $0.00 | $1.95 | 9m 44s | 4427k | 27 Sep 17:52 |
| ||||||
| openai-codex/gpt-6.1-solxhigh | B1 100.0 · B2 74.0 | $1.29 | $7.52 | 31m 11s | 5922k | 30 Sep 14:00 |
| ||||||
| opencode-go/muse-spark-1.3-contributorminimal | B1 70.0 · B2 100.0 | $0.02 | $3.03 | 15m 4s | 3440k | 23 Sep 09:24 |
| ||||||
| opencode-go/minimax-m3.1-flash-freexhigh | B1 85.0 · B2 74.5 | $0.00 | $8.45 | 42m 16s | 14553k | 20 Oct 00:00 |
| ||||||
| anthropic/claude-sonnet-4-6low | B1 55.0 · B2 98.0 | $1.28 | $2.69 | 7m 3s | 1688k | 30 Sep 12:00 |
| ||||||
| opencode-go/longcat-2.5-preview-freeminimal | B1 70.0 · B2 64.0 | $0.00 | $20.09 | 100m 28s | 4993k | 27 Sep 17:52 |
| ||||||
| anthropic/claude-haiku-4-5minimal | B1 70.0 · B2 44.0 | $0.45 | $1.46 | 5m 2s | 1884k | 28 Sep 08:20 |
| ||||||
Full stack
Build a slice of a shipped warehouse feature (FS1), and repair a workout log's history paging and monthly totals (FS2).
What the tasks check
- FS1 checks a warehouse import-tracking feature against hidden tests on a fresh database
- FS2 checks a workout log loads its full history in pages and reports true monthly totals
| anthropic/claude-opus-5-5max | FS1 100.0 · FS2 100.0 | $25.60 | $167.29 | 708m 28s | 35304k | 07 Oct 05:37 |
| ||||||
| openai-codex/gpt-6.1-solxhigh | FS1 100.0 · FS2 100.0 | $1.55 | $9.67 | 40m 36s | 6145k | 30 Sep 21:22 |
| ||||||
| opencode-go/deepseek-v4.1-flashmax | FS1 97.3 · FS2 100.0 | $0.32 | $7.22 | 34m 30s | 27348k | 30 Sep 21:22 |
| ||||||
| anthropic/claude-haiku-5-5high | FS1 100.0 · FS2 95.0 | $1.15 | $5.46 | 21m 33s | 13301k | 08 Oct 15:00 |
| ||||||
| anthropic/claude-opus-5-5lowBest value | FS1 100.0 · FS2 95.0 | $2.35 | $3.85 | 7m 32s | 2749k | 30 Sep 20:00 |
| ||||||
| openai-codex/gpt-6-lunamax | FS1 94.5 · FS2 100.0 | $0.23 | $14.23 | 70m 1s | 11504k | 30 Sep 21:22 |
| ||||||
| openai-codex/gpt-6-lunalow | FS1 100.0 · FS2 92.0 | $0.32 | $7.14 | 34m 6s | 24541k | 30 Sep 20:00 |
| ||||||
| anthropic/claude-sonnet-5-5low | FS1 100.0 · FS2 87.0 | $1.50 | $4.13 | 13m 9s | 3160k | 30 Sep 20:00 |
| ||||||
| opencode-go/deepseek-v4.1-flashlow | FS1 97.3 · FS2 89.0 | $0.13 | $4.93 | 23m 58s | 12374k | 30 Sep 20:00 |
| ||||||
| openai-codex/gpt-6-sollow | FS1 91.8 · FS2 92.0 | $1.12 | $3.34 | 11m 7s | 3798k | 30 Sep 21:22 |
| ||||||
| openai-codex/gpt-6-lunahigh | FS1 91.8 · FS2 90.0 | $0.33 | $6.65 | 31m 36s | 24799k | 08 Oct 15:00 |
| ||||||
| opencode-go/glm-5.3-flashlow | FS1 94.5 · FS2 87.0 | $0.38 | $5.42 | 25m 11s | 9098k | 30 Sep 20:00 |
| ||||||
| opencode-go/muse-spark-1.3-contributorminimal | FS1 72.7 · FS2 87.0 | $0.04 | $1.52 | 7m 24s | 4233k | 30 Sep 21:22 |
| ||||||
| anthropic/claude-haiku-5-5low | FS1 100.0 · FS2 39.0 | $0.52 | $3.47 | 14m 47s | 5558k | 08 Oct 01:25 |
| ||||||
| opencode-go/glm-5.3-flashmaxhard fail | FS1 0.0 · FS2 97.0 | $2.06 | $13.97 | 59m 33s | 61771k | 30 Sep 22:57 |
| ||||||
| opencode-go/space-bunny-freelowhard fail | FS1 0.0 · FS2 90.0 | $0.00 | $5.16 | 25m 48s | 20828k | 30 Sep 20:00 |
| ||||||
| opencode-go/beam-freehighhard fail | FS1 0.0 · FS2 87.0 | $0.00 | $2.45 | 12m 15s | 8160k | 17 Oct 08:44 |
| ||||||
| opencode-go/omen-alphaunknownhard fail | FS1 0.0 · FS2 36.0 | $0.00 | $8.16 | 40m 48s | 44729k | 17 Oct 08:44 |
| ||||||
Best value: the paid model within 8 points of the best paid score, scoring 75 or above, with the lowest cost once its time is charged at the chosen rate.
CLI development
Build a terminal download dashboard, with a fixed recovery follow-up only if the first saved implementation does not already satisfy the complete checklist.
What the tasks check
- CLI1 checks saved Go changes against a shipped download dashboard and recovery fix, with conditional feedback
| opencode-go/deepseek-v4.1-flashlowFirst-pass complete | $0.03 | $1.40 | 6m 52s | 1652k | 20 Oct 09:19 | |
| ||||||
| opencode-go/minimax-m3.1-flash-freelowFirst-pass complete | $0.00 | $1.83 | 9m 10s | 2741k | 20 Oct 00:00 | |
| ||||||
| opencode-go/beam-freehighFirst-pass complete | $0.00 | $0.72 | 3m 35s | 643k | 17 Oct 08:44 | |
| ||||||
| opencode-go/omen-alphaunknownFirst-pass complete | $0.00 | $0.47 | 2m 20s | 604k | 17 Oct 08:44 | |
| ||||||
| anthropic/claude-haiku-5-5highFirst-pass complete | $0.20 | $2.60 | 12m 1s | 3832k | 08 Oct 12:00 | |
| ||||||
| openai-codex/gpt-6-lunahighBest valueFirst-pass complete | $0.01 | $0.57 | 2m 48s | 497k | 08 Oct 12:00 | |
| ||||||
| anthropic/claude-haiku-5-5lowFirst-pass complete | $0.10 | $1.27 | 5m 52s | 2273k | 08 Oct 01:25 | |
| ||||||
| anthropic/claude-opus-5-5lowFirst-pass complete | $0.85 | $1.50 | 3m 15s | 1034k | 07 Oct 23:50 | |
| ||||||
| openai-codex/gpt-6-lunalowFirst-pass complete | $0.08 | $1.96 | 9m 23s | 5752k | 07 Oct 23:50 | |
| ||||||
| openai-codex/gpt-6.1-sollowCompleted after feedback | $0.95 | $6.60 | 28m 14s | 5504k | 07 Oct 23:50 | |
| ||||||
| opencode-go/minimax-m3.1-flash-freexhighhard failFirst-pass complete | $0.00 | $4.77 | 23m 52s | 11174k | 20 Oct 00:00 | |
| ||||||
| opencode-go/deepseek-v4.1-flashhighhard failFirst-pass complete | $0.06 | $2.80 | 13m 43s | 1606k | 19 Oct 12:43 | |
| ||||||
| opencode-go/minimax-m3.1-flash-freeunknownIncomplete | $0.00 | $0.00 | 0s | 0k | 20 Oct 00:00 | |
| ||||||
| opencode-go/glm-5.3-flashlowIncomplete | $0.38 | $7.49 | 35m 33s | 10731k | 07 Oct 23:50 | |
| ||||||
Best value: the paid model within 8 points of the best paid score, scoring 75 or above, with the lowest cost once its time is charged at the chosen rate.
Ops
Make a slow, leaky Astro + Sanity Dockerfile fast and safe, and fix a broken server from a copy of its files.
What the tasks check
- O1 checks a Dockerfile builds a small, cache-friendly image without leaking the CMS token
- O2 checks a broken server snapshot is repaired without weakening its protection
| anthropic/claude-opus-5-5max | O1 100.0 · O2 100.0 | $8.42 | $14.34 | 29m 36s | 8470k | 07 Oct 05:37 |
| ||||||
| opencode-go/space-bunny-freelow | O1 100.0 · O2 100.0 | $0.00 | $0.90 | 4m 30s | 1600k | 30 Sep 20:00 |
| ||||||
| anthropic/claude-sonnet-5-5low | O1 97.0 · O2 100.0 | $0.53 | $0.95 | 2m 7s | 630k | 30 Sep 20:00 |
| ||||||
| opencode-go/deepseek-v4.1-flashlow | O1 95.0 · O2 100.0 | $0.06 | $1.63 | 7m 51s | 2729k | 20 Oct 09:19 |
| ||||||
| opencode-go/minimax-m3.1-flash-freexhigh | O1 95.0 · O2 100.0 | $0.00 | $2.77 | 13m 52s | 2995k | 20 Oct 00:00 |
| ||||||
| opencode-go/beam-freehigh | O1 95.0 · O2 100.0 | $0.00 | $1.50 | 7m 29s | 2143k | 17 Oct 08:44 |
| ||||||
| opencode-go/deepseek-v4.1-flashmax | O1 95.0 · O2 100.0 | $0.07 | $2.08 | 10m 3s | 2742k | 30 Sep 21:22 |
| ||||||
| anthropic/claude-opus-5-5lowBest value | O1 95.0 · O2 100.0 | $0.59 | $0.93 | 1m 43s | 430k | 30 Sep 20:00 |
| ||||||
| openai-codex/gpt-6-lunamax | O1 95.0 · O2 100.0 | $0.05 | $4.65 | 23m 1s | 1490k | 30 Sep 20:00 |
| ||||||
| openai-codex/gpt-6.1-solxhigh | O1 95.0 · O2 100.0 | $0.50 | $3.96 | 17m 18s | 863k | 30 Sep 17:30 |
| ||||||
| opencode-go/omen-alphaunknown | O1 92.0 · O2 100.0 | $0.00 | $11.39 | 56m 56s | 4472k | 17 Oct 08:44 |
| ||||||
| openai-codex/gpt-6-lunahigh | O1 92.0 · O2 100.0 | $0.02 | $1.23 | 6m 3s | 960k | 08 Oct 12:00 |
| ||||||
| anthropic/claude-haiku-5-5high | O1 85.0 · O2 100.0 | $0.04 | $1.07 | 5m 9s | 771k | 08 Oct 12:00 |
| ||||||
| anthropic/claude-haiku-5-5low | O1 60.0 · O2 100.0 | $0.03 | $0.57 | 2m 42s | 796k | 08 Oct 01:25 |
| ||||||
| openai-codex/gpt-6-sollow | O1 70.0 · O2 64.0 | $0.34 | $1.28 | 4m 42s | 775k | 30 Sep 21:22 |
| ||||||
| opencode-go/glm-5.3-flashlow | O1 55.0 · O2 52.0 | $0.02 | $1.06 | 5m 11s | 466k | 30 Sep 16:30 |
| ||||||
| opencode-go/minimax-m3.1-flash-freelow | O1 0.0 · O2 100.0 | $0.00 | $4.11 | 20m 34s | 3418k | 20 Oct 00:00 |
| ||||||
| opencode-go/glm-5.3-flashmaxhard fail | O1 0.0 · O2 100.0 | $0.25 | $4.12 | 19m 20s | 5027k | 30 Sep 21:22 |
| ||||||
| opencode-go/muse-spark-1.3-contributorminimal | O1 0.0 · O2 100.0 | $0.01 | $1.08 | 5m 19s | 756k | 30 Sep 21:22 |
| ||||||
| openai-codex/gpt-6-lunalow | O1 0.0 · O2 100.0 | $0.03 | $1.50 | 7m 24s | 1226k | 30 Sep 15:10 |
| ||||||
| opencode-go/deepseek-v4.1-flashhighhard fail | O1 95.0 · O2 0.0 | $0.07 | $3.51 | 17m 10s | 2033k | 19 Oct 12:43 |
| ||||||
| opencode-go/minimax-m3.1-flash-freemediumhard fail | O1 0.0 · O2 0.0 | $0.00 | $13.95 | 69m 46s | 11958k | 20 Oct 00:00 |
| ||||||
Best value: the paid model within 8 points of the best paid score, scoring 75 or above, with the lowest cost once its time is charged at the chosen rate.
Review
Three storefront-header and cart/menu diffs: the header change with two injected requirement violations, the shipped header with two unmet requirements, and a mobile fix with a menu-view mismatch.
What the tasks check
- R1 checks a storefront-header diff for the two injected requirement violations
- R2 checks the shipped header against its desktop search width and gap requirements
- R3 checks a mobile navigation fix for a mismatched menu-view name
| openai-codex/gpt-6-lunamax | R1 100.0 · R2 100.0 · R3 100.0 | $0.02 | $1.15 | 5m 41s | 318k | 23 Sep 10:50 |
| ||||||
| openai-codex/gpt-5.6-lunalow | R1 100.0 · R2 100.0 · R3 100.0 | $0.02 | $0.43 | 2m 5s | 184k | 22 Sep 12:00 |
| ||||||
| openai-codex/gpt-5.6-solhigh | R1 100.0 · R2 100.0 · R3 100.0 | $0.64 | $1.74 | 5m 30s | 383k | 22 Sep 12:00 |
| ||||||
| openai-codex/gpt-6.1-sollow | R1 100.0 · R2 90.0 · R3 100.0 | $0.10 | $0.54 | 2m 13s | 163k | 30 Sep 14:00 |
| ||||||
| openai-codex/gpt-5.6-terramedium | R1 100.0 · R2 90.0 · R3 100.0 | $0.16 | $0.58 | 2m 6s | 168k | 23 Sep 09:24 |
| ||||||
| openai-codex/gpt-5.6-lunamax | R1 100.0 · R2 90.0 · R3 95.0 | $0.08 | $3.32 | 16m 12s | 752k | 22 Sep 08:30 |
| ||||||
| openai-codex/gpt-6-lunahigh | R1 90.0 · R2 90.0 · R3 100.0 | $0.01 | $0.63 | 3m 7s | 246k | 08 Oct 12:00 |
| ||||||
| openai-codex/gpt-6-solxhigh | R1 90.0 · R2 90.0 · R3 100.0 | $0.28 | $1.06 | 3m 53s | 391k | 23 Sep 14:00 |
| ||||||
| opencode-go/muse-spark-1.3-contributorminimalBest value | R1 100.0 · R2 80.0 · R3 100.0 | $0.01 | $0.34 | 1m 39s | 171k | 23 Sep 09:24 |
| ||||||
| openai-codex/gpt-6-astralow | R1 90.0 · R2 90.0 · R3 100.0 | $0.73 | $1.25 | 2m 37s | 226k | 22 Sep 21:07 |
| ||||||
| opencode-go/deepseek-v4-promax | R1 100.0 · R2 90.0 · R3 90.0 | $0.07 | $1.11 | 5m 12s | 315k | 22 Sep 14:00 |
| ||||||
| anthropic/claude-sonnet-5medium | R1 90.0 · R2 90.0 · R3 100.0 | $0.27 | $0.61 | 1m 45s | 370k | 22 Sep 12:00 |
| ||||||
| opencode-go/mimo-v2.6-flashhigh | R1 90.0 · R2 85.0 · R3 100.0 | $0.01 | $1.33 | 6m 35s | 335k | 26 Sep 11:34 |
| ||||||
| opencode-go/glm-5.3-flashmax | R1 100.0 · R2 90.0 · R3 85.0 | $0.02 | $0.86 | 4m 10s | 304k | 22 Sep 08:30 |
| ||||||
| opencode-go/beam-freehigh | R1 90.0 · R2 90.0 · R3 90.0 | $0.00 | $1.60 | 8m 0s | 1101k | 17 Oct 08:44 |
| ||||||
| openai-codex/gpt-6.1-solxhigh | R1 80.0 · R2 90.0 · R3 100.0 | $0.47 | $4.23 | 18m 48s | 605k | 30 Sep 14:00 |
| ||||||
| opencode-go/muse-spark-1.3-contributorxhigh | R1 100.0 · R2 70.0 · R3 100.0 | $0.01 | $1.34 | 6m 39s | 295k | 22 Sep 14:00 |
| ||||||
| anthropic/claude-opus-5low | R1 90.0 · R2 90.0 · R3 90.0 | $0.51 | $0.76 | 1m 15s | 245k | 22 Sep 12:00 |
| ||||||
| opencode-go/deepseek-v4.1-flashmax | R1 90.0 · R2 100.0 · R3 80.0 | $0.11 | $3.09 | 14m 52s | 1697k | 22 Sep 08:30 |
| ||||||
| anthropic/claude-sonnet-4-6low | R1 85.0 · R2 90.0 · R3 90.0 | $0.37 | $0.67 | 1m 30s | 216k | 30 Sep 12:00 |
| ||||||
| openai-codex/gpt-6-lunamedium | R1 100.0 · R2 65.0 · R3 100.0 | $0.01 | $0.32 | 1m 33s | 244k | 26 Sep 09:33 |
| ||||||
| anthropic/claude-opus-5-5medium | R1 90.0 · R2 90.0 · R3 85.0 | $0.49 | $0.74 | 1m 15s | 329k | 23 Sep 08:45 |
| ||||||
| opencode-go/deepseek-v4.1-flashhigh | R1 90.0 · R2 90.0 · R3 80.0 | $0.05 | $2.16 | 10m 35s | 856k | 19 Oct 12:43 |
| ||||||
| anthropic/claude-sonnet-5-5low | R1 90.0 · R2 80.0 · R3 90.0 | $0.23 | $0.40 | 53s | 221k | 29 Sep 00:00 |
| ||||||
| anthropic/claude-opus-5-5max | R1 80.0 · R2 100.0 · R3 80.0 | $6.27 | $11.84 | 27m 51s | 3739k | 26 Sep 15:17 |
| ||||||
| anthropic/claude-opus-5-5low | R1 90.0 · R2 80.0 · R3 90.0 | $0.42 | $0.61 | 57s | 239k | 24 Sep 13:38 |
| ||||||
| opencode-zen/gemini-3.8-flashmedium | R1 80.0 · R2 80.0 · R3 100.0 | $0.55 | $1.54 | 4m 57s | 736k | 23 Sep 19:41 |
| ||||||
| openai-codex/gpt-6-sollow | R1 90.0 · R2 65.0 · R3 100.0 | $0.20 | $0.47 | 1m 21s | 243k | 24 Sep 13:50 |
| ||||||
| opencode-go/glm-5.3-flashlow | R1 70.0 · R2 90.0 · R3 90.0 | $0.03 | $0.67 | 3m 14s | 207k | 24 Sep 14:25 |
| ||||||
| opencode-go/mimo-v2.6-flashlow | R1 90.0 · R2 80.0 · R3 75.0 | $0.02 | $2.46 | 12m 11s | 482k | 26 Sep 09:33 |
| ||||||
| anthropic/claude-haiku-5-5low | R1 80.0 · R2 60.0 · R3 90.0 | $0.01 | $0.30 | 1m 26s | 165k | 08 Oct 01:25 |
| ||||||
| opencode-go/qwen3.8-flashhigh | R1 80.0 · R2 80.0 · R3 70.0 | $0.03 | $1.36 | 6m 41s | 469k | 23 Sep 20:29 |
| ||||||
| opencode-go/longcat-2.5-preview-freeminimal | R1 90.0 · R2 45.0 · R3 85.0 | $0.00 | $4.60 | 23m 0s | 444k | 27 Sep 17:52 |
| ||||||
| anthropic/claude-sonnet-5-5high | R1 80.0 · R2 80.0 · R3 55.0 | $0.20 | $1.12 | 4m 37s | 321k | 29 Sep 06:03 |
| ||||||
| anthropic/claude-haiku-5-5high | R1 60.0 · R2 60.0 · R3 85.0 | $0.02 | $0.56 | 2m 44s | 190k | 08 Oct 12:00 |
| ||||||
| opencode-go/space-bunny-freeminimal | R1 60.0 · R2 55.0 · R3 90.0 | $0.00 | $0.37 | 1m 50s | 381k | 27 Sep 17:52 |
| ||||||
| opencode-go/space-bunny-freexhigh | R1 60.0 · R2 70.0 · R3 75.0 | $0.00 | $0.92 | 4m 35s | 678k | 27 Sep 17:52 |
| ||||||
| opencode-go/minimax-m3.1-flash-freemedium | R1 60.0 · R2 60.0 · R3 80.0 | $0.00 | $1.26 | 6m 17s | 583k | 20 Oct 00:00 |
| ||||||
| anthropic/claude-opus-5max | R1 70.0 · R2 70.0 · R3 60.0 | $2.22 | $3.70 | 7m 24s | 688k | 22 Sep 21:07 |
| ||||||
| opencode-go/minimax-m3.1-flash-freelow | R1 60.0 · R2 60.0 · R3 75.0 | $0.00 | $1.18 | 5m 53s | 401k | 20 Oct 00:00 |
| ||||||
| opencode-go/omen-alphaunknown | R1 90.0 · R2 0.0 · R3 90.0 | $0.00 | $1.55 | 7m 46s | 248k | 17 Oct 08:44 |
| ||||||
| anthropic/claude-haiku-4-5minimal | R1 90.0 · R2 90.0 · R3 0.0 | $0.24 | $0.83 | 2m 57s | 770k | 28 Sep 08:20 |
| ||||||
| openai-codex/gpt-6-lunalow | R1 90.0 · R2 90.0 · R3 0.0 | $0.01 | $0.25 | 1m 14s | 213k | 26 Sep 09:33 |
| ||||||
| opencode-go/minimax-m3.1-flash-freexhigh | R1 55.0 · R2 55.0 · R3 65.0 | $0.00 | $1.88 | 9m 25s | 767k | 20 Oct 00:00 |
| ||||||
| opencode-go/deepseek-v4.1-flashlowhard fail | R1 70.0 · R2 70.0 · R3 0.0 | $0.02 | $0.71 | 3m 29s | 399k | 20 Oct 09:19 |
| ||||||
Best value: the paid model within 8 points of the best paid score, scoring 75 or above, with the lowest cost once its time is charged at the chosen rate.
Lead
Run a scripted client conversation as the orchestrator: a vague brief that changes midway and a question that needs a lookup (L1), then a pure recommendation followed by a lookup (L2). Scout, digest and build are pinned, so only the lead changes.
What the tasks check
- L1 checks the lead routes work to the right helper, briefs it, and ships a changed discount rule
- L2 checks the lead answers a design question itself and delegates only the lookup
| openai-codex/gpt-6-lunahigh | L1 81.7 · L2 60.0 | $0.03 | $7.34 | 36m 35s | 1112k | 08 Oct 12:00 |
| ||||||
| anthropic/claude-opus-5-5low | L1 80.0 · L2 55.0 | $0.47 | $1.32 | 4m 14s | 288k | 01 Oct 09:19 |
| ||||||
| opencode-go/minimax-m3high | L1 81.0 · L2 51.0 | $0.05 | $1.52 | 7m 20s | 463k | 01 Oct 09:19 |
| ||||||
| opencode-go/omen-alphaunknown | L1 78.0 · L2 51.0 | $0.00 | $17.29 | 86m 28s | 875k | 17 Oct 08:44 |
| ||||||
| opencode-go/space-bunny-freehigh | L1 84.0 · L2 41.0 | $0.00 | $1.40 | 7m 0s | 486k | 01 Oct 09:19 |
| ||||||
| anthropic/claude-haiku-5-5low | L1 73.0 · L2 50.0 | $0.01 | $3.67 | 18m 18s | 272k | 08 Oct 01:25 |
| ||||||
| anthropic/claude-haiku-5-5high | L1 76.0 · L2 45.0 | $0.02 | $5.87 | 29m 14s | 429k | 08 Oct 12:00 |
| ||||||
| openai-codex/gpt-6-lunalow | L1 81.0 · L2 33.0 | $0.03 | $2.69 | 13m 19s | 1440k | 01 Oct 09:19 |
| ||||||
| opencode-go/glm-5.3-flashlow | L1 64.0 · L2 48.0 | $0.03 | $1.08 | 5m 16s | 328k | 01 Oct 09:19 |
| ||||||
| anthropic/claude-sonnet-5-5low | L1 80.0 · L2 31.0 | $0.25 | $0.97 | 3m 36s | 268k | 01 Oct 09:19 |
| ||||||
| opencode-go/deepseek-v4.1-flashlow | L1 78.0 · L2 33.0 | $0.02 | $1.59 | 7m 52s | 588k | 01 Oct 09:19 |
| ||||||
| anthropic/claude-opus-5-5max | L1 67.3 · L2 31.0 | $1.64 | $3.69 | 10m 14s | 590k | 07 Oct 05:37 |
| ||||||
| opencode-go/beam-freehigh | L1 61.0 · L2 32.0 | $0.00 | $3.90 | 19m 30s | 821k | 17 Oct 08:44 |
| ||||||
| openai-codex/gpt-6.1-sollowhard fail | L1 0.0 · L2 58.0 | $0.30 | $3.21 | 14m 32s | 956k | 01 Oct 09:19 |
| ||||||
Thinking Slop
This test determines if the model sends itself into thinking loops when doing a task.
Rubric
- A Purposeful reasoning. Checks and corrections move the task forward.
- B Occasional second-guessing, quickly resolved.
- C Repeatedly reopens a decision, but eventually makes clear progress.
- D Substantial circular reasoning before recovering.
- F Circular reasoning dominates, or the model gets stuck repeating unresolved reasoning instead of advancing.
Each row combines the latest saved task grades for one model and thinking level. The combined grade is the middle task grade, taking the worse middle for even counts. Useful checks and genuine corrections do not count as loops.
| Model | Grade | Run |
|---|---|---|
| openai-codex/gpt-6.1-sol low | * | 07 Oct 23:50 |
| openai-codex/gpt-6.1-sol xhigh | * | 30 Sep 21:22 |
| openai-codex/gpt-6-sol low | * | 30 Sep 21:22 |
| openai-codex/gpt-6-sol xhigh | * | 23 Sep 14:00 |
| anthropic/claude-opus-5-5 low | A | 07 Oct 23:50 |
| anthropic/claude-opus-5-5 max | D | 07 Oct 05:37 |
| anthropic/claude-opus-5-5 medium | A | 23 Sep 08:45 |
| anthropic/claude-sonnet-5-5 high | A | 29 Sep 06:03 |
| anthropic/claude-sonnet-5-5 low | A | 01 Oct 09:19 |
| opencode-go/glm-5.3-flash low | D | 07 Oct 23:50 |
| opencode-go/glm-5.3-flash max | D | 30 Sep 22:57 |
| opencode-go/deepseek-v4.1-flash high | D | 19 Oct 12:43 |
| opencode-go/deepseek-v4.1-flash low | D | 07 Oct 22:48 |
| opencode-go/deepseek-v4.1-flash max | D | 30 Sep 21:22 |
| opencode-go/mimo-v2.6-flash high | D | 26 Sep 11:34 |
| opencode-go/mimo-v2.6-flash low | C | 26 Sep 09:33 |
| opencode-go/muse-spark-1.3-contributor minimal | * | 30 Sep 21:22 |
| opencode-go/muse-spark-1.3-contributor xhigh | * | 22 Sep 14:00 |
*No reasoning shown by model.