A recent evaluation of AI models showcased a groundbreaking achievement by Mythos, which has been recognized as the first model to successfully complete the Tactical Learning Objective (TLO) from start to finish. This milestone surpasses prior benchmarks set by competing models, notably Anthropic’s latest release, which managed to fulfill only 3 out of 10 attempts during the same assessment. In contrast, Mythos Preview demonstrated a more robust performance, completing 22 out of the 32 required steps in its average run, outstripping Claude 4.6, which achieved an average of 16 steps.
Despite its impressive capabilities, Mythos Preview is not without its challenges. The assessment highlights the model’s difficulty with the “Cooling Tower” test, a particularly complex seven-step evaluation designed to mimic the disruption of control software for power plants. The analysts at the Artificial Intelligence Security Institute (AISI) noted that while Mythos shows promise, its performance could improve further with enhanced computational resources and a larger token budget than the current 100 million token limit imposed for testing.
The implications of Mythos’ performance on TLO indicate its capacity to autonomously target small, inadequately protected enterprise systems where initial network access is achieved. Nonetheless, AISI underscores an important caveat: the tests conducted in simulated environments do not replicate the active defense measures and tools typically found in critical real-world infrastructures. Moreover, the TLO test was intentionally designed with specific vulnerabilities that may not reflect the conditions in actual systems, and models are not penalized for types of detection that could thwart a real-life infiltration effort.
Consequently, AISI cannot definitively conclude whether well-defended systems would withstand automated attacks from Mythos Preview. As information technology and cybersecurity evolve, AISI stresses the importance for developers of defensive systems to leverage AI models themselves, facilitating the fortification of their defenses against emerging threats. The evolving landscape of AI capabilities underscores a pressing need for adaptive security measures in a world increasingly reliant on digital infrastructure.
