Most threat modeling frameworks were written for systems where the developer controls every branch of logic. STRIDE, attack trees, and data-flow diagrams still work for AI applications, but they need one addition before they produce useful results: a step where the team assumes the model itself is saying whatever an attacker wants it to say, and then asks what happens next.
Skipping that step is the most common mistake in AI threat modeling. Teams diagram the API, the database, and the network boundary correctly, treat the model as a trusted internal component, and miss the scenario that actually matters: the model is not a trusted component, it is an interpreter of whatever content reaches it.
Start by drawing trust boundaries, not data flows
A standard data-flow diagram shows how information moves. For an AI system, draw trust boundaries first: which content is fully trusted, which is partially trusted, and which is fully untrusted. A system prompt written by your own team is trusted. A message from an authenticated user is partially trusted. A retrieved web page, an email from an external sender, or the output of a tool call is untrusted, even if it arrives through an authenticated, encrypted, well-behaved connection.
The point of this exercise is to notice where untrusted content crosses into the model's context without a checkpoint in between. If a retrieved document goes straight into the prompt with no filtering, that is a boundary crossing with no control on it. If a tool's response is fed back to the model unvalidated, that is another one. Most AI threat models find their first real finding at this step, before any attack technique has even been discussed.
Run the "assume the model is compromised" exercise
Once boundaries are drawn, ask a specific question for each one: if an attacker fully controlled what the model says at this point in the flow, what could they cause the system to do? Not what could they make the model say, what could they make the system do.
This reframes the exercise around consequences instead of model behavior. A model that can only produce text that a human reads has a bounded answer: worst case, it says something false or offensive. A model that can trigger a tool call has a different answer: worst case, it triggers whatever that tool allows, up to the limit of the tool's own permissions. Walking through this question for every tool, every downstream consumer of model output, and every memory write path produces a concrete list of worst-case outcomes, which is far more actionable than a general sense that "prompt injection is a risk."
Map the tools like you would map an API
Every tool exposed to a model deserves the same scrutiny a new API endpoint would get in a conventional threat model. What does it do. What does it require as input. What checks does it perform on that input beyond schema validation. What identity does it act as. Whether it can be called repeatedly without limit. Whether its effect is reversible.
A tool that looks narrow on paper, like "look up an order," can still be risky if it accepts an arbitrary order ID with no ownership check, letting a manipulated model retrieve any customer's order rather than only the current user's. The threat model should catch this the same way it would catch a missing authorization check on a REST endpoint, because it is the same missing authorization check.
Account for memory and multi-turn state
If the system retains any memory across turns or sessions, add a specific question to the threat model: can something written into memory in one turn or by one user influence behavior in a later turn, possibly for a different user. If the answer is yes, memory is a persistence mechanism for whatever gets written into it, including attacker-controlled content, and it needs the same scrutiny as any other durable store that an attacker might be able to write to indirectly.
This matters most in systems that summarize or store user-provided content for later use. A single successful injection that would otherwise be contained to one session can become a standing instruction that resurfaces repeatedly until someone notices and clears it.
Consider multi-agent and delegation paths
If one agent's output feeds another agent, treat that hand-off as another trust boundary. The receiving agent should not automatically treat the sending agent's output as more trustworthy than any other untrusted input, even though it came from "your own system." A compromised or manipulated upstream agent can pass along instructions that a downstream agent executes, and the threat model should trace that path the same way it traces a user-to-service call.
Prioritize by reachability and impact, not novelty
Once the exercise produces a list of scenarios, prioritize using two questions: how easily can an attacker reach this boundary, and how bad is the worst outcome if they do. A publicly reachable chatbot with a document-search tool that has no ownership check deserves more urgent attention than an internal tool with no external content path, even if the internal tool sounds more technically interesting.
This is the same triage logic used in conventional threat modeling. The difference in AI systems is that "reachability" now includes indirect paths: a document that might get retrieved, an email that might get summarized, a tool response that might get fed back into context. Those paths are less visible than a public endpoint, so they need to be actively hunted for rather than assumed away.
Where TELEON fits
TELEON is useful in the mitigation step that follows a threat model, not the modeling step itself. Where a threat model identifies a tool call as a high-impact boundary, TELEON's policy enforcement can be configured to require approval or apply limits at that specific point, and its audit trail gives the team a record to check against the scenarios the threat model predicted.
The threat modeling work itself, drawing boundaries, mapping tools, and tracing memory and delegation paths, has to happen first. A policy layer can only enforce boundaries that have already been identified and defined.
A practical threat-modeling checklist
- Draw trust boundaries around every source of content the model receives.
- For each boundary, assume the model is fully compromised and ask what the
system could be made to do.
- Map every tool like an API: input, checks, identity, reversibility.
- Trace memory writes and reads across sessions and users.
- Trace hand-offs between agents as trust boundaries, not internal calls.
- Prioritize findings by reachability and worst-case impact.
Threat modeling an AI application is mostly the same discipline teams already use for conventional systems, applied to a wider set of components. The addition that matters most is treating the model as an interpreter of untrusted content rather than a trusted piece of your own code, because that single assumption changes which boundaries actually need a checkpoint.