Blog · Chapter 09 · Security & guardrails
Safe, compliant and in control
Prompt injection, excessive agency, data leakage, layered guardrails, GDPR and the EU AI Act.
AI applications inherit every classic security risk and add new ones. Chapter 9 builds a threat model for AI systems and a layered defence.
The AI-specific threat model
- Prompt injection: instructions hidden in user input or, more dangerously, in content the agent reads, such as emails, web pages and documents (indirect injection).
- Excessive agency: an agent with more tools or permissions than its task needs, which an attacker can then use.
- Data leakage: personal or confidential data exposed through answers, logs, traces or third-party providers.
- Insecure output handling: model output passed unchecked to a browser, database or shell.
Agents act as the user, never more
An agent acts on behalf of a user and must never have more permissions than that user. Enforce this in code: in tool implementations and in retrieval filters, not in the prompt. A prompt can ask the model not to show salary data; only an access filter can guarantee it.
Minimise and mask sensitive data
Send the model only what the task needs. Detect and mask personal data (names, IBANs, phone numbers) before it leaves your boundary where possible. Choose providers and regions with appropriate agreements.
Layer the guardrails
| Layer | Examples |
|---|---|
| Identity and access | Authentication, per-user authorization, tool permissions |
| Sensitive-data masking | PII detection and redaction, secrets scanning |
| Deterministic guardrails | Input limits, schemas, allow-lists, tool policies, approval for consequential actions |
| LLM guardrails | Injection detection, safety classification, groundedness checks |
| Governance and audit | Use-case register, risk classification, audit logs, human oversight |
Deterministic layers are cheap and reliable; LLM guardrails catch what rules cannot. Use both.
GDPR and the EU AI Act
Design for GDPR from day one, especially the right to erasure. Personal data now lives in vector stores, memories, traces and caches as well as databases. Classify each use case under the EU AI Act: high-risk systems need risk management, documentation, human oversight and logging. Decisions with legal effect on people need a human in the loop.
Key takeaways
- Threat-model prompt injection, excessive agency and leakage.
- Agents never get more permissions than their user.
- Layer deterministic and LLM guardrails; design for GDPR from day one.