AI Threats Affecting Developers & AI Builders

How AI Threats Appear

For developers and AI builders, AI-enabled threats most commonly surface through:

• Model theft and extraction — Adversaries using systematic queries or side-channel attacks to replicate proprietary models
• Prompt injection and jailbreaking — Attacks that bypass safety controls, extract system prompts, or cause AI systems to behave in unintended ways
• Data poisoning — Manipulation of training data to introduce backdoors, biases, or vulnerabilities into models
• Supply chain attacks — Compromised dependencies, malicious model weights, or poisoned datasets introduced through the AI development pipeline
• Downstream liability — Harms caused by AI systems that generate legal, financial, or reputational consequences for the developers who built them

Developers and AI builders are affected both as direct victims (when their systems are attacked) and as parties bearing responsibility when their systems cause downstream harm.

---

Relevant AI Threat Domains

• Security & Cyber — Model extraction, adversarial attacks, and supply chain compromises
• Agentic Systems — Autonomous AI agent failures, tool misuse, and privilege escalation
• Human-AI Control — Loss of control over deployed AI behavior
• Systemic Risk — Recursive self-improvement risks and strategic misalignment

---

What to Watch For

Indicators of AI-related developer and builder risk:

• Model APIs accessible without rate limiting, query logging, or anomaly detection for extraction attempts
• Training pipelines that ingest uncurated web data without poisoning detection
• Deployed systems with safety controls that can be bypassed through prompt engineering
• Development tool dependencies (MCP servers, IDE integrations, package managers) with insufficient security vetting
• Autonomous AI agents with tool access that lacks adequate sandboxing or permission boundaries

---

Regulatory Context

• EU AI Act — Imposes obligations on AI providers (developers) for high-risk systems, including conformity assessments, post-market monitoring, and incident reporting
• ISO/IEC 42001 — Provides management system requirements for organizations developing AI
• NIST AI RMF — Addresses AI risk management throughout the development lifecycle
• Open-source AI development carries unique liability and governance questions across jurisdictions

For classification rules and evidence standards, refer to the Methodology.