Information Integrity

Definition

Information integrity refers to the condition in which information circulating within a society is accurate, attributable, and resistant to undetected manipulation. It encompasses both the factual correctness of individual claims and the structural reliability of the systems through which information is produced, distributed, and verified. Information integrity is undermined when false or misleading content is indistinguishable from authentic material, when provenance and attribution are absent or falsified, and when the institutions responsible for verification lack the capacity or credibility to perform that function. AI technologies affect information integrity in dual directions — enabling both novel attacks on integrity and new tools for detection and verification.

How It Relates to AI Threats

Information integrity is a foundational concern spanning the Information Integrity and Agentic and Autonomous AI Threats domains. Within the information integrity domain, generative AI enables the creation of deepfakes, synthetic text, and fabricated evidence that directly undermine the reliability of public information. Under the misinformation and hallucinated content sub-category, large language models produce confident but false statements that enter public discourse without adequate labelling. Within the agentic domain, the cascading hallucinations sub-category describes how AI agents operating in multi-step pipelines can propagate and compound false information without human verification checkpoints, degrading information integrity at machine speed.

Why It Occurs

• Generative AI enables production of synthetic media indistinguishable from authentic content at negligible marginal cost
• Large language models produce hallucinated outputs with the same confident tone as factually grounded responses
• Content provenance infrastructure remains immature, making it difficult to verify the origin and authenticity of digital media
• Verification institutions — journalism, fact-checking, academic peer review — operate at human speed against machine-scale content production
• Economic incentives in digital media reward engagement and novelty over accuracy, providing distribution advantages to false content

Real-World Context

Multiple incidents in the TopAIThreats taxonomy demonstrate information integrity threats in practice. INC-24-0001, the Hong Kong deepfake CFO fraud, illustrates how synthetic media can override institutional verification procedures. INC-23-0007, the Slovakia election deepfake, shows information integrity attacks targeting democratic processes. INC-24-0003, the Pikesville High School deepfake principal incident, demonstrates integrity threats at the community level. International responses include UNESCO’s guidelines on information integrity, the EU AI Act’s transparency requirements for AI-generated content, and industry efforts such as the Coalition for Content Provenance and Authenticity developing technical standards for content authentication.