UK's AI Safety Institute Evaluated Claude Mythos on Cyberattacks — It Solved a 32-Step Corporate Network Breach 30% of the Time
The UK AI Security Institute published its evaluation of Claude Mythos Preview's offensive cyber capabilities. The model achieved 73% on expert-level capture-the-flag challenges and completed a full 32-step corporate network attack simulation in 3 out of 10 attempts — a first for any AI model.
Original sourceThe UK's AI Security Institute has published a detailed evaluation of Claude Mythos Preview's offensive cybersecurity capabilities, and the findings deserve attention even if the conclusions are hedged.
On capture-the-flag challenges at expert difficulty, Mythos Preview succeeded 73% of the time — a number that no previous model had reached. On "The Last Ones," a 32-step corporate network attack simulation, it became the first model to complete the scenario end-to-end, doing so in 3 out of 10 attempts and averaging 22 of 32 steps. Previous frontier models topped out around 16 steps.
The AISI researchers were careful to contextualize these results. Their test environments lacked active defenders, security tooling, and hardened configurations — all the factors that make real enterprise networks harder to compromise than a simulation. The model also struggled with operational technology scenarios, a category that covers industrial control systems and critical infrastructure.
The evaluation was conducted under controlled conditions as part of Anthropic's voluntary safety testing program. Anthropic restricted Mythos from providing direct cyberattack assistance for public deployment, but the evaluation shows how capable the underlying model is when those restrictions are not in place.
The AISI's recommendation is blunt: organizations should treat this as a signal to invest in cyber defense fundamentals now, before the next model generation makes the attack surface significantly larger. Patch your systems. Implement proper access controls. Log everything.
Panel Takes
The Builder
Developer Perspective
“73% on expert CTF challenges is a legitimate benchmark milestone. The 32-step network simulation result is more meaningful to me than most AI capability claims — that's not autocomplete, that's strategic multi-step reasoning under adversarial conditions. Security engineers need to be building their threat models around this right now.”
The Skeptic
Reality Check
“The caveats here are doing a lot of work. 'No active defenders, no security tooling, no hardened configurations' — that's not a corporate network, that's a parking lot. These benchmarks measure what models can do against cardboard targets. The gap between 73% on CTFs and 'can actually breach a defended enterprise' is enormous. Don't panic, but don't celebrate either.”
The Futurist
Big Picture
“The trajectory is what matters, not the absolute number. Every generation of frontier models moves this benchmark meaningfully. The AISI is right that the investment window for defensive cybersecurity infrastructure is closing. In three years, the attack capability of freely available models will exceed what only nation-state actors can deploy today.”