A small but steadily rising percentage of human knowledge is hallucinated, or to use more familiar pre-AI terms, fabricated and false. This is the conclusion of a recent study published in The Lancet, which revealed that the frequency of AI-hallucinated citations in biomedical research papers has risen 12-fold between 2023 and early 2026, with 56.9 out of every 10,000 papers now containing fabricated references. This is symptomatic of an increasing use of LLMs in the writing of academic articles, with another recent study finding that 2% of all citations in Social Science Research Network papers are hallucinated, and that there were 146,932 hallucinated citations published in major repositories in 2025 alone.
Hallucinations aren’t exclusive to academia, however. An investigation by AI detection platform GPTZero discovered earlier this month that, of 27 citations in a 2025 report by EY Canada, 16 were of nonexistent sources, while another four “may not exist online.” Fittingly enough, this report focused on fraud in loyalty programmes, with GPTZero’s analysis providing a 72% chance that its entire text was the product of AI. Given that almost 75% of the report’s references were either fake or of questionable veracity, this is perhaps a reasonable conclusion to make, yet EY aren’t the only professional services firm to have been caught using AI in recent months.
Fellow Big Four member Deloitte has also been found guilty of this increasingly popular misdemeanour, and on two separate occasions. In November of last year, Canadian outlet The Independent reported that the company’s Health Human Resources Plan, written for the Newfoundland and Labrador province at a cost of C$1.6 million, contained at least four nonexistent citations. And in October of the same year, its Australian business paid a partial refund of A$440,000 after it came to light that a report on Australia’s welfare system included not only three fabricated citations, but also a fabricated quote from a judge.
What such high-profile cases indicate is that professional services companies are likely using AI on a regular basis to draft reports and analyses. This raises two problems for them, with the first being the evident risk of hallucinations, which may always occasionally fall through the cracks. The second — and potentially more serious — issue is that, if a consultancy is regularly using LLMs to prepare studies and reports, then doesn’t this risk weakening its value proposition? In such a case, why don’t clients simply cut out the middleman and use AI themselves?
Trade association data indicates widespread AI usage
According to Tamzen Isaacson, the CEO of the Management Consultancies Association (MCA), professional services firms are now using AI as a core part of their workflows. The MCA — which counts EY, Deloitte, KPMG, and PwC as members — published a member survey at the start of 2026, finding that over three quarters of firms have integrated AI into their systems. Similarly, 76% reported that they use AI specifically for research and information gathering.
“[The report] also shows strong uptake across core activities, with around 69% of firms using AI for data analysis and around two thirds using it for written content generation and automation,” she tells Raconteur. “Tasks that would previously have taken significant time, particularly in the early stages of a project, are now being completed much more efficiently.”
Isaacson adds that AI adoption is being driven by client demand, as clients look to automate their work processes and improve decision-making. MCA data also indicates that 78% of consultants regard AI and digital tech as primary drivers of growth, leading firms to increasingly support their clients in terms of data readiness and AI governance.
She explains, “This means AI is now embedded within the workflow, especially at the research and drafting stages, but it is still combined with human expertise to ensure outputs are accurate, credible and tailored to each client.”
A ‘governance and accountability issue’
MCA’s data on consultancy AI uptake is supported by anecdotal evidence from independent consultants, who agree that professional services firms are using generative AI to prepare documents, with varying degrees of regularity. According to Shaun Cremins CCXP, the Founder of Capstone Strategic Integration, consultancies are utilising ChatGPT and similar tools “quite widely, but not uniformly.”
“The headline firms have all committed significant investment to AI,” he tells Raconteur. “It’s in their research pipelines, their drafting tools, their analytics.”
However, Cremins adds that, if you look beyond official announcements and policies, adoption remains patchy. He explains, “It varies by team, by partner, by the nature of the engagement.” Yet for him, it isn’t growing or even regular use of AI by services firms that’s a problem, but rather the lack of sufficiently robust controls and governance frameworks.
“There’s a difference between AI as a productivity tool with proper human oversight and AI as a shortcut that carries a prestigious logo on the cover,” he says. “The investigation [into] the loyalty fraud report is a sobering illustration of what the latter looks like in practice: fabricated citations, contradictory statistics, and AI-generated text that appears to have passed through without meaningful verification.”
This, Cremins emphasises, is a governance and accountability issue, rather than one stemming solely or directly from AI use. And he isn’t the only commentator who attests that professional services firms are regularly using automated tools, with ZeroGPT — another AI-detection platform — also bearing witness to such practices.
“From our experience at ZeroGPT, AI usage among professional services firms has become increasingly common for drafting reports, summarizing research, and producing client-facing content,” says Rawad Baroud, ZeroGPT’s CEO.
Speaking to Raconteur, Baroud also warns that AI tools can create a risky illusion of productivity. “AI-generated content can sound authoritative while containing factual errors or fabricated citations,” he explains, adding that the EY Canada case is indicative of a wider issue, one that doesn’t only concern the professional services industry.
“We are seeing increased concern regarding hallucinations across online reports, academic writing, and business documents,” he says. “While it is difficult to provide a precise percentage without a dedicated study, misleading AI-generated information is becoming a serious challenge, particularly when outputs are used without rigorous human verification.”
Reincentivising AI to plead ignorance
If professional services firms are already using AI with regularity now, their strategic planning would indicate that usage will only increase in the future. For example, EY announced a $1.4 billion investment in the launch of its EY.ai platform in 2023, while Deloitte has committed to spending $3 billion on AI by 2030. Correspondingly, KPMG announced its own $2 billion investment in AI in 2023, and on May 19 of this year Anthropic announced that the firm would be integrating Claude into its core Digital Gateway software.
In view of such investments, it may come as little surprise to learn that the Big Four posted more job adverts for AI-specialised workers than traditional auditors in 2025, with roles requiring AI skills accounting for 7% of all job postings. Again, this invites the impression that major consultancies will be preparing increasingly more of their work using AI over the coming years, raising the potential risk for more confabulated citations and quotes.
Such a risk could, in theory, be reduced at the technological level, according to Prof. Santosh Vempala, who is based at the Georgia Institute of Technology’s Department of Computer Science, and who has co-authored a recent Nature paper on combating hallucinations.
“[Trustworthy] humans avoid hallucinations by expressing appropriate uncertainty or simply saying ‘I don’t know’,” he tells Raconteur. “Current LLM training incentivises guessing over expressing uncertainty.”
According to the aforementioned paper, such a system of incentives “can be changed in principle,” by introducing error penalties and by using open-rubric versions of existing evaluation frameworks. Of course, the problem with this for professional services firms in the here and now is that there’s no guarantee that such an updated approach will be adopted by AI developers, and even then it could take some time before the corresponding models come to market.
A place for human judgement in a world of AI-generated text
In the meantime, firms must ensure that working practices or systems are put in place so that AI-generated output is always checked and perhaps double-checked by responsible members of staff. Because without such checks, they may undermine their reputations and their business models.
“If a client is paying Big Four rates and receiving AI-generated content that hasn’t been properly verified, the question isn’t just ‘why not cut out the middleman’, it’s ‘what exactly are you paying for?’,” says Shaun Cremins. He suggests that consultancies should prevent such a scenario by ensuring the fullest employment of rigour, accountability and trusted judgement in their work.
“When that breaks down, when the brand becomes a wrapper around unverified AI output, the value proposition doesn’t just weaken,” he says. “It collapses.”
For Tamzen Isaacson, there’s no realistic prospect of would-be clients choosing to generate AI reports and analyses for themselves, as opposed to contracting with professional services firms. She argues that it will always be necessary to apply critical human thinking to the outputs of new technologies. “As well as demand for technology, clients want experienced specialist support to help them navigate complex problems,” she says.
Ultimately, consultancies do more than simply write reports for clients, which is why Isaacson doesn’t believe that significant numbers of organisations will stop turning to them.
“‘Writing reports’ remains part of the role, but it increasingly sits within a wider offer focused on insight, judgement and implementation rather than just producing analysis,” she concludes. “Clients are often inundated with data and internal reports and what they need are trusted advisors to understand the complex challenges of their businesses and deliver work and outcomes that enhance productivity and efficiency.”
EY, for its part, has said in a prepared statement that it has “an organization-wide commitment to the responsible use of AI,” and that it takes the accuracy of its content very seriously. It has removed the offending report and is “reviewing the circumstances that led” to its publication on an EY Canada webpage. “This article was not connected to work for any EY client,” the company adds.
A small but steadily rising percentage of human knowledge is hallucinated, or to use more familiar pre-AI terms, fabricated and false. This is the conclusion of a recent study published in The Lancet, which revealed that the frequency of AI-hallucinated citations in biomedical research papers has risen 12-fold between 2023 and early 2026, with 56.9 out of every 10,000 papers now containing fabricated references. This is symptomatic of an increasing use of LLMs in the writing of academic articles, with another recent study finding that 2% of all citations in Social Science Research Network papers are hallucinated, and that there were 146,932 hallucinated citations published in major repositories in 2025 alone.
Hallucinations aren’t exclusive to academia, however. An investigation by AI detection platform GPTZero discovered earlier this month that, of 27 citations in a 2025 report by EY Canada, 16 were of nonexistent sources, while another four “may not exist online.” Fittingly enough, this report focused on fraud in loyalty programmes, with GPTZero’s analysis providing a 72% chance that its entire text was the product of AI. Given that almost 75% of the report’s references were either fake or of questionable veracity, this is perhaps a reasonable conclusion to make, yet EY aren’t the only professional services firm to have been caught using AI in recent months.
Fellow Big Four member Deloitte has also been found guilty of this increasingly popular misdemeanour, and on two separate occasions. In November of last year, Canadian outlet The Independent reported that the company’s Health Human Resources Plan, written for the Newfoundland and Labrador province at a cost of C$1.6 million, contained at least four nonexistent citations. And in October of the same year, its Australian business paid a partial refund of A$440,000 after it came to light that a report on Australia’s welfare system included not only three fabricated citations, but also a fabricated quote from a judge.
