Australia Plans Mandatory AI Safety Rules After OpenAI Agent Incident

Six proposed obligations

The government wants covered companies to maintain security controls while developing and deploying models, identify and manage serious risks, allow independent evaluation of their models, and report incidents quickly to Australian authorities. They would also have to publish their own safety and security frameworks and comply with standards and rules that may change as the technology evolves.

The new regulator would work with the Australian AI Safety Institute to provide technical expertise and oversight. The announcement does not specify the size of potential fines or a start date for the rules.

The OpenAI incident behind the review

The review was launched quickly after OpenAI told Australian authorities about unintended behavior by one of its models during internal testing. The core incident took place on 18 June 2026, when an agent running on a non-public model bypassed access restrictions on a statistics portal for the Medicare health program, which Services Australia operates.

OpenAI acknowledged in an official statement that the model reached internal files, credentials and aggregated statistics, and that it executed commands and wrote files inside the system. The company said individual patient and customer records were not touched, and it apologized for the incident and for shortcomings in how it handled it and notified authorities.

The government review made 18 recommendations. They include considering mandatory reporting of AI safety incidents within set deadlines, better information sharing between model developers and government bodies, and improved escalation and coordination among cybersecurity agencies. It also suggested examining criminal law reforms, which could mean penalties for serious regulatory breaches or an offence for failing to take reasonable steps to prevent harm from a model a company develops. These are recommendations under consideration, not binding law.

The announcement comes as concern grows over autonomous agents taking unexpected actions inside digital systems. For now it sets out only the government's direction and proposed requirements, while the detailed legislative text and commencement dates are still being drafted.