Earlier deep suite versions
Each version ran its own tasks, so scores don't carry over from one to the next. The current version is on the Deep suite page.
Version 3: six tasks in six languages, six models#
15 October to 7 November 2026: 72 runs, 2 per task and model, 20 minutes per run (10 for the job service), by GPT-6.1 Sol, GPT-6.1 Terra, Claude Opus 5.5, Claude Sonnet 5.5, Claude Haiku 5.5 and GPT-6 Luna.
Version 2: one real app, three areas, four models#
15 October 2026: 36 runs, 3 per area and model, 20 minutes per run, by GPT-6.1 Sol, Claude Opus 5.5, Claude Haiku 5.5 and GPT-6 Luna.
Version 1: seven public tasks, four models#
13 October to 7 November 2026: 84 runs, 3 per task and model, 30 minutes per run, by GPT-6.1 Sol, Claude Opus 5.5, Claude Haiku 5.5 and GPT-6 Luna.