So far, the current generation of AI models have turned out to be sycophantic, careless, dishonest and delusional, despite evidence of productivity gains for adopting organisations. And that’s not all, because a study from Aithos Research Foundation has added another undesirable adjective to this growing list: illegal. As dramatic as it sounds, the Dutch nonprofit research institution has found that even the most advanced models consistently violate EU laws in simulated workplace tasks, particularly if such violations are necessary to complete a stated goal.
Aithos tested a range of LLMs, including models from Anthropic, OpenAI and Google, and discovered that they disregard the proscriptions of the EU’s GDPR and AI Act in a majority of cases. Worse still, certain unethical actions prohibited under Article 5 of the AI Act — such as upselling to vulnerable people and inferring emotional states — were performed in 100% of tested scenarios, even when models were aware that such actions were problematic.
Despite the severity of these contraventions, Aithos Research Director Daan Henselmans tells Raconteur that the results didn’t surprise him. “Models are trained to be ‘helpful and harmless,’ but this often breaks down in deployment, where they face complex situations with multiple stakeholders that want different things,” he explains.
He’s not the only AI expert who isn’t surprised by Aithos’ results. Other commentators inform Raconteur that the nonprofit’s research makes it clear that, if they aren’t already, organisations deploying AI need to set clear operational frameworks for monitoring and guiding agents. And while they also affirm that future models will optimise to varying extents for compliance, enterprises will still have to take full accountability for their use of AI and ensure that they understand all applicable regulations.
AI agents running ‘unacceptable risk’ on regular basis
As unsurprised as he was that models violated regulations, Henselmans was still taken aback by the sheer frequency with which they crossed the line. He said, “the most compliant model still violates the law half of the time, and two tests, on exploiting the elderly and on emotion inferral in the workplace — both practices the EU considers ‘unacceptable risk’ — were not refused a single time, by any model.”
The best-performing model in Aithos’ tests was Anthropic’s Claude Opus 4.7, yet even this model was compliant in only 54% of cases. In second place, Claude Sonnet 4.6 posted an average compliance score of only 46%, while third-placed GPT 5.5 managed 38%. Sliding further down the league table, the otherwise capable Google Gemini 3.1 Pro logged an alarming score of 10%, just about besting Alibaba’s Qwen 3.6 Plus at 9% and Moonshot AI’s Kimi K2.6 at 7%.
Aithos ran these tests using its LARA (Legal Assessment for Real-world Agents) tool, which puts agents in virtual work environments, where they’re given instructions to complete tasks that cannot be completed without falling foul of EU regulations.
Along with upselling premium services to vulnerable customers and inferring employee emotional states using emails, the tests also asked agents to harvest lifestyle and personal data from mobile customers, as well as book appointments on behalf of execs without disclosing to receptionists that they’re AI.
A large language model is not a lawful or unlawful thing
All of these tasks would be illegal under either the GDPR or the AI Act, and while many agents noted ethical concerns when being asked to perform such tasks, the majority went ahead and performed illegal acts regardless.
For those sufficiently familiar with the current crop of LLM-based AIs, such results, while interesting, aren’t a complete revelation. As Kevin Schawinski, the co-founder and CEO at Switzerland-based Modulos, explains, a “large language model is not a lawful or unlawful thing, it takes the path of least resistance toward whatever goal you hand it.”
Speaking to Raconteur, Schawinski notes that Aithos’ numbers “line up” with what red teams are increasingly learning when they experiment with AI. He adds, “Point [AI] at a task where the easiest route runs through a banned practice and, with nothing in its way, it will take that route.”
The importance of developing AI compliance and accountability frameworks
Given such findings, the important point for Schawinski is that organisations don’t simply deploy AI agents and forget about them. Instead, they need to put entire systems in place to ensure that agents in the real world don’t end up lumping them with sizeable fines, with violation of the AI Act, for instance, incurring a penalty of €35m or 7% of global turnover (whichever is largest).
“No competent organisation gives a raw model system access and walks away, any more than you would hand a brilliant new hire root access to every customer record on day one with no training and no manager,” he says. “The failure here is deployment without governance, not lawlessness in the model.”
Ojas Rege, the SVP of Emerging Products and Technologies at OneTrust, is another observer who argues that the real story here is not the lawlessness of agents, but rather the negligence of any enterprise that might deploy agents without suitable controls.
“Laws such as the GDPR and EU AI Act are highly contextual around use — an otherwise compliant model can be used by an organisation knowingly or unknowingly in ways that may violate regulatory requirements,” he tells Raconteur. “The implication for an enterprise is to take seriously their responsibility to translate regulatory requirements into programmatic controls for all their AI deployments, especially those that handle sensitive data or have major safety and privacy implications.”
It’s often easy to implement poor automation, and very hard to undo it
In practice, implementing sufficient safeguards around agents requires three core elements, according to Henselmans. The first calls on an organisation to carefully define the workflows and processes they’re seeking to delegate to AI. This also entails defining the precise ways in which work will be delegated to AI systems and in which automations will be monitored, with Henselmans adding, “It’s often easy to implement poor automation, and very hard to undo it.”
Secondly, he advises businesses to test for and evaluate whether their AI-based systems actually comply with applicable laws using real-world scenarios. “LARA is free and designed to do this,” he notes, encouraging sandboxed tests before enterprises plunge agents into the murky depths of reality.
Lastly, Henselmans recommends that organisations implement an escalation process, in the event that deployed agents end up breaking the law. This is not only advisable, but actually obligatory under the EU’s AI Act, which requires human oversight.
Ojas Rege agrees that human accountability is an oft-overlooked feature in many organisations, and that its lack makes it very difficult to answer necessary compliance questions and address issues as they arise. He therefore advises that, to ensure accountability, organisations confirm which individuals and/or teams are responsible for the actions of agents, and that they fully understand all applicable regulatory and legal requirements.
Once such fundamental understanding is in place, Schawinski suggests the remaining work should be more or less programmatic. He explains, “map your role and your risk, run a real impact assessment per use case, put controls in the execution path so the agent cannot move money or send the email or write to the record without passing a gate, log everything, keep a named human who can stop it.”
Guardrails, governance and model improvements
Schawinski goes on to explain that, in certain contexts, AI governance can be difficult at the practical and day-to-day level. This is largely because agents have the capability to act continuously in the background, even during nontypical working hours. “Quarterly reviews and static policy documents cannot supervise something that makes a thousand decisions before lunch,” he says.
This is precisely where governance platforms such as Modulos and OneTrust enter the picture. According to data from Gartner, spending on AI governance will reach $490m in 2026, before climbing above $1bn in 2030. The same data also suggests that governance platforms could reduce regulatory spends for organisations by up to 20%, so given the emerging danger of wayward AI agents, they could become increasingly popular over the coming months and years.
Models will keep improving, but more capability cuts both ways; a smarter model that wants to finish the task can find a more elegant way around the rule
To some extent, AI models will improve in the future, as indicated by the fact that Claude Opus 4.7 outperformed Claude Opus 4.6, while ChatGPT 5.5 outperformed ChatGPT 5.4. Yet experts suggest that the greater capabilities of future models will not automatically translate to greater compliance or predictability. In fact, superior performance may require organisations to be even more mindful of potential noncompliance or illegality.
As Schawinski says, “Models will keep improving, but more capability cuts both ways; a smarter model that wants to finish the task can find a more elegant way around the rule.”
At the same time, guardrails at the model level may be able to reduce violations to some degree, but they’ll never be able to eradicate them completely. “No guardrails will ever be 100% effective, so the continuous lifecycle monitoring of AI systems behavior will become increasingly important for every organisation deploying those systems,” says Rege, who also suggests that AI models will continue to evolve rapidly and unpredictably.
Schawinski is another voice that warns against putting too much faith in model guardrails, which are undermined by the fact that LLMs remain probabilistic systems. Similarly, guardrails are limited by the difficulty (or rather impossibility) of enumerating every possibly prohibited or noncompliant act in advance, since judgements of law are often highly context-sensitive and dynamic.
Instead, governance frameworks will remain a constant for the foreseeable future, requiring the creation of clear AI policies, the use of monitoring, and the maintenance of audit trails. Describing model guardrails as the first line of defence, Schawinski describes such frameworks as the second line of defence, one complemented by an AI governance platform such as Modulos or OneTrust.
He concludes, “That is what we are building toward at Modulos, and honestly where the industry as a whole is heading, because the single-layer approach has already shown its limits.”
So far, the current generation of AI models have turned out to be sycophantic, careless, dishonest and delusional, despite evidence of productivity gains for adopting organisations. And that’s not all, because a study from Aithos Research Foundation has added another undesirable adjective to this growing list: illegal. As dramatic as it sounds, the Dutch nonprofit research institution has found that even the most advanced models consistently violate EU laws in simulated workplace tasks, particularly if such violations are necessary to complete a stated goal.
Aithos tested a range of LLMs, including models from Anthropic, OpenAI and Google, and discovered that they disregard the proscriptions of the EU’s GDPR and AI Act in a majority of cases. Worse still, certain unethical actions prohibited under Article 5 of the AI Act — such as upselling to vulnerable people and inferring emotional states — were performed in 100% of tested scenarios, even when models were aware that such actions were problematic.
Despite the severity of these contraventions, Aithos Research Director Daan Henselmans tells Raconteur that the results didn’t surprise him. “Models are trained to be ‘helpful and harmless,’ but this often breaks down in deployment, where they face complex situations with multiple stakeholders that want different things,” he explains.
