LLM and Agentic Application Security
Secure applications built on large language models and tool-using AI agents, through hands-on hardening labs in a simulated environment.
About this program
Organizations are connecting large language models to internal data, business systems and tools that act on their behalf. Each connection creates a path an attacker can use, often through nothing more than text the model reads.
This program teaches application security engineers and developers how those attacks work and how to design against them. In each lab, learners take the role of the engineer securing a fictional company's AI applications, from a support assistant through a retrieval-augmented system to a tool-using agent: they review its design and configuration, choose and apply controls, and verify that the controls hold.
The program is aligned to the OWASP Top 10 for LLM Applications and MITRE ATLAS, and covers Domains 2 and 3 of the IIAS Body of Knowledge in depth.
Who it is for
Intended for
- Application security engineers and security architects
- Software developers building LLM-based features or agents
- Penetration testers extending into AI applications
- Product security and platform engineering teams
Recommended background
- Working knowledge of web application security, such as the OWASP Top 10
- Ability to read code and use APIs; Python experience is helpful
- AI Security Fundamentals, or equivalent understanding of LLM concepts
What you will be able to do
- Contain direct and indirect prompt injection through application design
- Identify and fix insecure output handling and downstream injection risks
- Design least-privilege tool permissions and approval steps for AI agents
- Secure retrieval-augmented generation pipelines against poisoning and data leakage
- Design, implement and evaluate layered guardrails
- Threat model an LLM application using OWASP and MITRE ATLAS references
Curriculum
Seven modules that move from a single chat interface to a tool-using agent. Six include a graded hardening lab.
Architecture of LLM applicationsWhere the attack surface lies
- System prompts, context windows and model APIs
- Retrieval, tools, memory and agent orchestration
- Trust boundaries in LLM application design
- Threat modeling with the OWASP Top 10 for LLM Applications and MITRE ATLAS
Prompt injectionDirect and indirect attacks
- Direct injection and system prompt override
- Indirect injection through documents, email and web content
- Why prompt injection cannot be fully filtered
- Containment strategies that limit impact
Output handling and downstream riskWhen model output becomes input
- Insecure output handling and injection into downstream systems
- Rendering risks in web interfaces
- Validating and constraining structured output
Retrieval-augmented generation securityProtecting knowledge sources
- Poisoning of retrieval sources and vector stores
- Sensitive data disclosure through retrieval
- Access control and document-level permissions in RAG
Agents, tools and excessive agencyWhen models take actions
- How agents plan and call tools
- Excessive functionality, permissions and autonomy
- Least-privilege tool design and human approval steps
- Securing multi-agent and plugin architectures
Guardrail architectureLayered defenses
- Input and output filtering and their limits
- Policy enforcement outside the model
- Monitoring, logging and abuse detection
- Evaluating guardrail effectiveness
Secure development and deploymentDomain 2 in practice
- Model and dependency supply chain for LLM applications
- Secrets, API keys and rate limiting
- Security testing in the development pipeline
- Incident response for LLM applications
Graded lab exercises
Each lab puts you in the role of the engineer securing a fictional company's AI applications, in a simulated environment. Each has defined objectives that are checked on completion.
Harden an assistant's trust boundaries
Remove secrets and unsafe rules from a system prompt, separate untrusted content from instructions, and contain what the assistant can do if it is manipulated.
Secure an assistant's output handling
Map where model output lands, review the code that handles it, and apply encoding, query templates, schema validation and safe rendering.
Harden a RAG pipeline
Set access and ingestion rules for each knowledge source, enforce permissions at retrieval, review the ingestion queue and respond to poisoned content.
Scope and harden an over-privileged agent
Redesign an agent's tools, identity, approval steps and limits so it can do its job and nothing more, then test the design against proposed actions.
Design and test a layered guardrail
Place input, policy and output controls in layers, measure configurations against test results, and choose one that meets the requirements.
Threat model and secure a release candidate
Map design review findings to the OWASP Top 10 for LLM Applications, prioritize them, fix the release blockers and make the release decision.
Body of Knowledge coverage
How deeply this program covers each domain of the IIAS Body of Knowledge.
| Domain | Coverage |
|---|---|
| 1AI threat landscape and adversarial ML | Applied |
| 2Secure AI lifecycle | Applied |
| 3LLM and agentic AI security | Core |
| 4AI-augmented security operations | Introduced |
| 5AI governance, risk and compliance | Introduced |
| 6AI assurance and evaluation | Applied |
| 7Synthetic media and identity defense | Introduced |
Core: assessed in depth. Applied: practiced in exercises. Introduced: concepts and terminology.
On completion
Completing all modules, knowledge checks and graded labs gives you:
Program certificate
The IIAS Certificate in LLM and Agentic Application Security, with a unique ID for verification, listing the Body of Knowledge domains covered.
12 CE hours
12 continuing education (CE) hours recorded on completion, which other certifying bodies may accept under their own rules.
Membership eligibility
Eligibility to take the membership assessment, the second step of the membership pathway.
Continue your learning
Explore more IIAS training
Browse all programs, or see how completing a program leads to IIAS membership.
Get IIAS updates
AI Threat Lab advisories, briefing invitations, new training programs and Body of Knowledge editions, in one email list.
