Anyone who’s been using LLMs for an appreciable amount of time must be familiar with hallucinations by now. When an AI model is a next-token predictor that operates probabilistically, there will always be scope for confabulations and untruths, no matter the size of the model. This seems to be an accepted risk among adopters of ChatGPT, Claude and Gemini, despite data indicating that hallucinations cost businesses $67.4bn worldwide as early as 2024. However, one area where hallucinations haven’t been fully appreciated, at least not yet, is brand management.
According to a report published at the end of June Gartner, this is becoming an increasingly serious area of concern, particularly as consumers come to rely more on Google’s AI Overviews (and other AI services) for information about brands and their products. Gartner reports that companies are facing an expanding range of AI-caused brand-management problems, from hallucinated defamations to the inaccurate recommendation of competitors. Such issues are tearing up the previously “well-established” corporate reputation management playbook, and finding many companies glaringly unprepared.
In the face of such challenges, the advisory firm recommends that companies adopt systematic Trust Operations, a suite of measures aimed at quickly neutralising false AI-generated information and distributing more positive counter-information. That way, they can clamp down on falsehoods before they spread more widely, ensuring that wayward AI answers don’t damage the brands they’ve spent years building.
80% of enterprises experienced disinformation in past four years
Hallucinations are one thing, but recent months have also revealed AI’s susceptibility to picking up deliberately injected misinformation and regurgitating it as fact. Tech reporter Thomas Germain entertainingly highlighted this issue in February, when he published an article on his personal blog titled, “The best tech journalists at eating hot dogs.” Within 24 hours, ChatGPT and Google’s Gemini (and AI Overviews) were repackaging info from his planted article, telling him that he was indeed the best hot dog-eating tech journalist in the world. Needless to say, pretty much everything in the article was false.
While Germain’s fake-news injection was a bit of fun intended to highlight a serious point, Gartner’s report draws attention to how this issue is already negatively impacting brands. Authored by Andrew Frank, a VP Distinguished Analyst at the firm, it describes how ChatGPT and Google’s AI Overviews are producing false information about brands in roughly 2% of cases, a percentage which seems less negligible in view of the fact that both tools handle billions of prompts every day.
According to Frank, this threat became visible three years ago, around the time brand reputation company Terakeet published a blog on the subject. “It started coming up on my radar about two years ago with the arrival of AI Overviews and the adoption of ChatGPT as a search substitute,” he says. “We increasingly hear concerns about this from brands, often in the context of AEO/GEO strategy more generally.”
We increasingly hear concerns from brands, often in the context of AEO and GEO strategy more generally
Aside from brand defamation, which can involve hallucinated claims about a company’s products, ethics, financial status, or legal situation, Gartner also highlights how automated media campaigns can make incidents worse via amplification. Because a negative AI-generated response might generate a spike in online traffic, automated bidding systems may therefore bid on related ad space. This creates a vicious circle where a brand is effectively funding and perpetuating its “own character assassination,” as the report warns.
Frank affirms that defamatory AI hallucinations have become relatively common, even if most brands would “prefer to deal with them quietly,” making it harder to assess just how widespread they now are. He adds, “While it’s hard to gauge the prevalence of AI answer engine content that’s both negative and false, Gartner’s survey data suggests almost 80% of enterprises worldwide have experienced disinformation incidents in the past four years.”
If this percentage is accurate, then it would suggest that every company would benefit from having formal policies and procedures in place to preempt AI defamation. For Frank, the importance of such measures is heightened by the difficulty of introducing failsafe solutions at the model level, given the probabilistic nature of LLMs.
“I think models will continue to improve their citation accuracy and safeguards but the skills of deliberate detractors are also improving,” he says. “I foresee an escalating challenge for brands to assert control over the narratives that AI spreads about them and their practices more generally.”
Implementing TrustOps to combat AI-hallucinated brand defamation
With the onus on organisations themselves to guard against reputational damage, Gartner concludes its report by recommending the adoption of AI-ready Trust Operations (TrustOps). This is a suite of measures covering three complementary areas, with the first concerning the ability to deactivate programmatic ad campaigns, while also switching campaigns to ‘exact match’ settings in order to stop AI platforms from generating copy based on hallucinated misinformation.
Secondly, Gartner recommends the deployment of cross-platform content, meaning that brands should consider “aggressively” publishing structured data (with schema markup), as well as comparison charts and buyer guides. In other words, they should increase the production of positive AEO-friendly content. This, according to the report, should help to favourably alter the citation layer.
Thirdly, the report advises businesses and trade associations to implement frameworks for escalating complaints with AI developers. This could incorporate the use of agentic AI, harnessing the latter to automate safety filters, reviews and complaints.
Taken together, these three pillars should enable brands to disseminate true, positive content, while debunking false content in faster and more comprehensive ways, according to Frank. “Debunking false content means detecting it, investigating it, responding to it in a variety of ways, and constantly learning and improving the process,” he says. “It’s about reframing trust cultivation as an ongoing operational priority rather than an episodic, reactive escalation.”
Anyone who’s been using LLMs for an appreciable amount of time must be familiar with hallucinations by now. When an AI model is a next-token predictor that operates probabilistically, there will always be scope for confabulations and untruths, no matter the size of the model. This seems to be an accepted risk among adopters of ChatGPT, Claude and Gemini, despite data indicating that hallucinations cost businesses $67.4bn worldwide as early as 2024. However, one area where hallucinations haven’t been fully appreciated, at least not yet, is brand management.
According to a report published at the end of June Gartner, this is becoming an increasingly serious area of concern, particularly as consumers come to rely more on Google’s AI Overviews (and other AI services) for information about brands and their products. Gartner reports that companies are facing an expanding range of AI-caused brand-management problems, from hallucinated defamations to the inaccurate recommendation of competitors. Such issues are tearing up the previously “well-established” corporate reputation management playbook, and finding many companies glaringly unprepared.
In the face of such challenges, the advisory firm recommends that companies adopt systematic Trust Operations, a suite of measures aimed at quickly neutralising false AI-generated information and distributing more positive counter-information. That way, they can clamp down on falsehoods before they spread more widely, ensuring that wayward AI answers don’t damage the brands they’ve spent years building.
