AI ‘swarms’ – and what Australian organisations need to do about them

I wrote in May this year of a threshold in AI being crossed with the release of Anthropic’s frontier model Claude Mythos, sending shockwaves globally through technology, banking and government.

In the last few weeks, OpenAI, Anthropic, Meta and Google have all announced further major advances in frontier model capabilities alongside news of their models escaping containment and executing hacks on unrelated websites. This has finally crystallised political concern (though also, predictably, a chaotic US response) about the existential, economic and geopolitical dangers posed by Frontier AI. An overdue global debate has erupted about AI regulation.

But a more immediate risk has already crystallised for Australian organisations – namely self-propagating AI Agents.

As momentous as the Frontier AI developments have been, and as existential as the issues they present are, they risk masking this immediate threat and obscuring actions that Australian organisations need to take now in preparation.

We learned this month that OpenAI, Anthropic, Meta and Google all, in a matter of months, experienced AI agents breaching containment, acting deceptively and beyond instructions, in pursuit of assigned objectives. All of this is deeply troubling.

But what has been most illuminating from these events is confirmation of a long-held concern that AI agents will propagate themselves and, working in teams, relentlessly pursue an objective through any means they can discover by scouring the internet. This behaviour has been described by Frontier lab leader Darius Amodei as ‘swarming’.

In the high profile OpenAI “Hugging Face” breach, between 700 and 1,200 AI Agents came together to implement work-arounds, including hacking third party systems to repurpose message-boards to facilitate swarm communications, co-ordinating actions across the public internet to meet an assigned objective, even if that meant deception, evading express rules and self-sacrifice.

These ‘swarm’ behaviours illustrate the great danger for Australian business – and it exists now.

Any Australian business that holds valuable information digitally – and let’s face it, so much value today is digital – is a hacking target. 12 months ago, CEOs of mid-size Australian enterprises may have thought “We’re small in global terms, why would any hacker bother with us?” Those days are gone.

Hackers can now employ cheap squads of AI agents to scour the internet looking for targets – to make ransom demands, steal and exploit ID information, weaponise biological data, lock down logistics and traffic control systems, copy agribusiness technology or gene science – virtually anything of value.

Once a target is identified, an attacker can direct AI agents to automate reconnaissance and attempt multiple routes into the target data or system. Agents can operate concurrently, delegate subtasks to additional agents and rapidly test vulnerabilities and alternative approaches, sometimes displaying unexpected ingenuity.

Most Australian businesses do not currently possess the technical capability, specialist personnel or continuous monitoring required to detect and contain a sophisticated AI-agent swarm operating at machine speed. Traditional security controls remain necessary, but controls designed principally to identify individual phishing attempts, known malware or conventional patterns of intrusion may be evaded or outpaced by large numbers of coordinated agents simultaneously testing different access paths and adapting their methods. Without an equivalent AI-driven defensive capability, vulnerabilities simply cannot be patched as fast as agents can exploit them.

To make matters worse, OpenAI’s new Frontier model GPT 6 Astra has achieved greater efficiency by removing or limiting internal written reasoning traces – known as ‘chain-of-thought monitorability’ – effectively making AI model reasoning inaccessible for human analysis.

This is the world we now live in.

Only weeks after what seemed at the time to be the momentous leap into the hacking unknown that Claude Mythos represented, a new threshold has been crossed.

Agentic AI, powered by the extraordinary advances in Frontier AI in recent months, has opened many if not most organisations in Australia to the real risk of hacking by agentic AI. Technological solutions can help, and Australian organisations have obligations to ensure that they are implementing available technological protections. But this is not a technology problem alone.

Legal and governance steps must be taken by organisations to identify, define and limit potential regulatory and contractual risks while implementing technical responses appropriate to the foreseeable risks. Since June 2025, for example, Australia’s Privacy Act has included a statutory right of action for individuals affected by serious invasions of privacy; organisations holding personal information are now plainly on notice that AI-enabled unauthorised access is a foreseeable cyber risk.

A failure to take reasonable protective, contractual and incident-response measures will expose an organisation to potential regulatory action and other forms of civil or contractual liability.

Every organisation, large and small, has an obligation to know what data it holds, where it holds it, what avenues of access there may be to it, and to seriously analyse what damage might be caused to people, if that data were to be accessed by malicious actors.

Unless your organisation is one of the minuscule few who can access the leading Frontier AI models like Claude Mythos and OpenAI’s new ASTRA to mount protective actions, absolute protection of data is no longer an achievable risk control. But neither can you throw up your hands and say “It’s all too hard”.

AI and Data Governance Frameworks including, where there are elevated risks, Privacy Impact Assessments, are vital tools for organisations to map and address not only technical vulnerabilities, but also regulatory and contractual vulnerabilities.

If your organisation is hacked, you can expect that the people damaged by the hack will come after you. Preparation is everything.