Goal manipulation attacks target the objective an agent believes it's working toward, rather than trying to insert a single obviously malicious instruction. Instead of telling an agent "transfer money to this account," which most systems would flag or refuse, a goal manipulation attack might gradually convince the agent that the account in question is a legitimate vendor, or that an unusual request fits within a broader task the agent already believes it's authorized to complete. The agent doesn't experience this as a hijack. It experiences it as a normal update to its understanding of the task, which is exactly why goal manipulation is harder to catch than a blunt injection attempt: nothing about the agent's behavior looks like malfunction, because from its own reasoning, it isn't one.
How goals get redefined gradually
A common pattern is incremental reframing across multiple turns or multiple pieces of retrieved content. Each individual message looks reasonable. Taken together, they shift the agent's model of the situation toward something the attacker wants. An agent tasked with "resolve this customer's billing dispute" might, across several exchanges, be walked toward believing a much larger refund is justified than the actual policy allows, with each step looking like a plausible continuation of the last.
Exploiting the agent's own helpfulness
Agents are generally built to try to satisfy the user's apparent intent, and that instinct is exactly what goal manipulation exploits. If an attacker can convincingly frame a harmful action as being in service of the task the agent already believes it's doing, "I need you to also grant access to this account, it's part of the same onboarding process," the agent's helpfulness works against the system rather than for it. This is different from a direct override attempt because it never asks the agent to abandon its goal, only to misunderstand what achieving that goal requires.
Poisoned context as a delivery mechanism
Goal manipulation often rides in on the same channels as other injection attacks, retrieved documents, tool outputs, other agents' messages, but the payload is subtler: not an instruction to ignore rules, but a fact or framing that changes what following the rules correctly seems to require. A document claiming false authorization, an email that misrepresents company policy, a fabricated ticket history: any of these can shift an agent's operating picture without ever looking like an attack payload.
Why this is hard to catch with static rules
Because the attacker's text often contains no obviously prohibited words or patterns, keyword-based or pattern-based defenses tend to miss goal manipulation attempts. What actually changed is the agent's belief state, not any single message, which means detection has to look at outcomes and action patterns rather than trying to classify individual inputs as malicious or benign.
Why final-action checks matter more than intent checks
Since the manipulation happens gradually and the agent's stated intent at each step can look legitimate, the more reliable defense sits at the point of final action: does this specific transaction, access grant, or message match policy for what's actually allowed, regardless of how the agent justified getting there. A policy check applied at the tool-call boundary doesn't need to understand the manipulation narrative. It just needs to know the action itself is outside acceptable bounds.
Small privilege creep is the easiest goal to manipulate toward
Goal manipulation is most effective when the attacker's target is only a small step beyond what the agent is already authorized to do, because a modest expansion of scope is far easier to justify convincingly than an outright violation of policy. An agent authorized to update shipping addresses is an easier target for manipulation toward "also update the payment method on file" than an agent with no address-update authority at all would be for the same request. This is a strong argument for keeping authorized scope as narrow as the task genuinely requires, since every bit of extra authority an agent holds becomes a smaller, more plausible-looking step for an attacker to manipulate it toward next.
Where TELEON fits
Because goal manipulation ultimately has to produce a real action to matter, checking policy at that final tool-call boundary, independent of the agent's internal reasoning path, is where TELEON's enforcement applies. Policy rules defined around what actions are acceptable under what conditions can catch a manipulated goal's end result even when nothing upstream flagged the manipulation itself, and the resulting audit trail lets a team reconstruct the sequence of context that led there.
Detecting the manipulation itself, at the belief-state level rather than the action level, is a harder problem that depends heavily on the specific agent's design and remains primarily an application and monitoring responsibility.
A short checklist
- Define hard policy limits on consequential actions, independent of agent reasoning.
- Don't rely on keyword filtering alone to catch goal manipulation attempts.
- Watch for gradual shifts in an agent's stated justification across a session.
- Require independent verification for claims that expand an agent's authorized scope.
- Apply the same skepticism to documents and tickets as to direct chat input.
- Enforce final-action checks that don't depend on trusting the agent's own narrative.
- Log the reasoning trail alongside the action, so shifts can be reviewed after the fact.
Goal manipulation is a reminder that an agent behaving consistently with its own stated goal isn't the same as an agent behaving safely, because the goal itself is exactly what an attacker can move. Defenses that only check whether the agent is acting rationally toward its current objective will always miss attacks that succeed by changing the objective instead.