Large Language Model

Definition

A large language model (LLM) is a neural network trained on massive text datasets to generate, summarise, translate, and reason about natural language. LLMs form the foundation of systems such as ChatGPT, Claude, and Gemini. They operate by predicting the most likely next tokens in a sequence, which enables fluent text generation but also produces confident-sounding outputs that may be factually incorrect (hallucinations).

How It Relates to AI Threats

LLMs intersect with threats across multiple domains. Within Information Integrity, they enable the production of misinformation and hallucinated content at scale. Within Human-AI Control, they create risks of overreliance and automation bias as users treat LLM outputs as authoritative. LLMs also underpin agentic AI systems, where autonomous action introduces additional risk vectors.

Why It Occurs

• Scale of training data includes both accurate and inaccurate information
• The prediction mechanism optimises for plausibility rather than factual accuracy
• Users frequently lack the ability to verify LLM outputs
• Commercial deployment incentivises broad capability over narrow reliability
• Rapid adoption has outpaced the development of appropriate governance frameworks

Real-World Context

LLM-related incidents include Samsung engineers leaking proprietary code via ChatGPT (INC-23-0002), Italy’s temporary GDPR-based ban on ChatGPT (INC-23-0003), a lawyer citing hallucinated case law in federal court (INC-23-0005), and AI-generated phishing attacks leveraging LLM fluency (INC-23-0006).