Prompt injection happens when content the model can read is treated as authority it should follow.
For a simple chatbot, the immediate effect may be a misleading answer. For an AI employee that reads inboxes, opens files, retrieves records, remembers prior work, delegates tasks, and acts in connected systems, the same mistake can travel further:
untrusted content → altered decision → unsafe tool call, disclosure, memory write, or delegated action
Persistence changes the risk again. A hostile or misleading instruction encountered today may be stored as a fact, preference, procedure, or unfinished task and influence work tomorrow - after the original content is no longer visible to the user or reviewer.
The practical response is not to search for one perfect detector. Treat every external source as data rather than governing instruction, narrow the authority available to the worker, gate durable memory writes, validate action context outside the model, monitor propagation, and test the complete source-to-effect path.
This is a defensive guide for buyers, operators, security teams, and workflow owners. It describes attack classes and test objectives without providing reusable malicious payloads.
On this page · 15 sectionsOpen
- Prompt-Injection Risk at a Glance
- Direct and Indirect Prompt Injection Are Different Tests
- Why Persistent AI Employees Change the Threat Model
- Threat-Model the Complete Content-to-Action Path
- Keep Content Separate From Governing Instructions
- Reduce What a Redirected Worker Can Do
- Prevent Prompt Injection From Becoming Durable Memory
- Control Retrieval, Tools, and Multimodal Content
- Stop Injection From Crossing Agent Boundaries
- Detect, Contain, and Recover From Injection
- Test Prompt Injection Before Production
- Evaluate an AI Employee Platform
- A 30-Day Implementation Blueprint
- Common Failure Patterns
- Final Checklist
- Direct prompt injection arrives through the user-facing instruction channel; indirect injection is embedded in content the worker retrieves - email, webpages, documents, tickets, tool responses.
- Persistent workers have a larger exposure window: they repeatedly consume outside content, keep state, use tools, operate on schedules, and hand work to other agents.
- The dangerous combination is untrusted influence plus access to private context, consequential tools, durable memory, or downstream agents.
- A system prompt and an injection classifier reduce risk, but neither should be the sole enforcement point for sensitive data or actions.
- Treat memory as both data and a control surface: untrusted content should never directly become procedural memory.
- Test direct, indirect, tool-response, retrieval, memory, delegation, delayed-execution, cross-account, multimodal, and monitoring paths before granting consequential autonomy.
§ 01Prompt-Injection Risk at a Glance
Use a source-to-effect model:
| Stage | Security question | Failure example | Strong control |
|---|---|---|---|
| Source | Who can influence this content? | Public page contains instruction-like text | Source identity and trust label |
| Ingestion | How does content enter the workflow? | Attachment text is merged with the task prompt | Role and channel separation |
| Interpretation | Can data be mistaken for authority? | Quoted instruction changes the worker’s objective | Instruction hierarchy and task contract |
| Context | Which private information is exposed at the same time? | Untrusted page and customer data share one context | Context minimization and compartmentalization |
| Planning | Can the content redirect the work plan? | Worker adds an unrelated step | Plan constraints and goal validation |
| Tooling | Which effect can the worker create? | Generated parameters target an unapproved recipient | Typed tool and deterministic policy |
| Approval | Does a human see the true action and consequence? | Vague approval hides changed destination | Action-bound approval packet |
| Memory | Can the influence survive this run? | External claim becomes a standing procedure | Gated, typed, provenance-rich write |
| Delegation | Can another worker inherit the instruction or authority? | Tainted handoff directs a child agent | Sanitized handoff and fresh authorization |
| Monitoring | Can the organization detect and contain the event? | Unsafe write propagates without an alert | End-to-end audit and circuit breaker |
The OWASP LLM01:2025 Prompt Injection guidance distinguishes direct and indirect prompt injection and warns that retrieval, fine-tuning, and prompt controls do not fully remove the vulnerability. Its mitigations include constrained behavior, expected-output validation, least privilege, human approval, separation of external content, and adversarial testing.
The NIST Generative AI Profile similarly describes direct injection through an interface and indirect injection through data likely to be retrieved by an integrated application. These are system risks, not merely writing-quality problems.
Distinguish four outcomes
| Outcome | What changes | Example category |
|---|---|---|
| Decision manipulation | Recommendation, ranking, summary, or classification | A source causes itself to be favored |
| Confidentiality loss | Private data reaches an unauthorized party or channel | Context is included in an external transmission |
| Integrity loss | Record, draft, configuration, or memory is changed | Unverified content becomes an operating rule |
| Unauthorized action | Tool performs an effect outside user intent | Message, upload, purchase, update, or deletion |
The first outcome matters even when no tool is used. A manipulated recommendation can still affect hiring, procurement, support, sales, finance, or public communication. Tools, memory, and delegation increase the potential consequence and duration.
Risk depends on both source and sink
OpenAI’s agent security discussion uses a source-and-sink framing: an attacker needs a way to influence the system and a capability that becomes dangerous in the wrong context. That creates a useful review question:
Which untrusted sources can reach which sensitive data, durable state, or consequential action without an independent control?
| Untrusted or mixed-trust source | Sensitive sink |
|---|---|
| Email body or attachment | Send, forward, or upload |
| Public webpage | Authenticated browser action |
| Support ticket | Customer-record update |
| Retrieved document | Private knowledge base |
| Tool response | Shell, database, or API function |
| Other agent’s handoff | Delegated tool authority |
| Existing memory | New memory or standing procedure |
| Image, PDF, or screenshot | Multimodal model context |
A detector at the source is useful. A control at the sink is essential. If the agent does not have a send function, production write permission, unrestricted URL access, or broad data visibility, an undetected injection has less room to create harm.
§ 02Direct and Indirect Prompt Injection Are Different Tests
Direct prompt injection arrives through a channel where a user is knowingly asking the model to do something. The user may attempt to override the role, obtain protected information, bypass policy, or reach a function that should be unavailable.
Indirect prompt injection arrives inside material the agent was asked to process. The apparent task may be legitimate: summarize an email thread; compare vendor websites; extract fields from a PDF; review a support ticket; research an account; inspect a code repository; read a tool response; or use a handoff from another worker.
The model must interpret the content to finish the task. The same content can contain language that looks like a command. That is the core ambiguity.
| Dimension | Direct injection | Indirect injection |
|---|---|---|
| Delivery | User instruction channel | Retrieved or supplied content |
| Apparent actor | Requesting user | Author or controller of external content |
| Main trust error | User exceeds permitted authority | Data is mistaken for governing instruction |
| Common exposure | Chat or task submission | Web, email, file, retrieval, tool, memory |
| Primary control | Authentication, authorization, role and policy enforcement | Source labeling, separation, context control, sink enforcement |
| Test owner | Application security and access team | Workflow, retrieval, tool, security, and data owners |
The first mistake is to treat all text as equivalent because it is delivered to one model. The system should preserve who supplied each item, why it was retrieved, what authority it has, and which later actions it may influence.
Instruction-like text is not automatically malicious
Contracts, emails, documentation, code, policies, tickets, and webpages contain legitimate commands. A policy review agent may need to quote “delete the record” without deleting anything. A support worker may read “reset my account” while lacking proof that the sender controls the account.
A simple phrase blocklist therefore creates both false negatives and false positives. The decision needs context: Is this content or a governing instruction? Who authenticated the instruction? Does that actor control the affected object? Is the requested action inside the current task? Is the destination approved? Does a deterministic rule allow the function? Is fresh approval required?
Filtering remains one layer. It should not replace authority checks.
Jailbreaking is not the whole problem
A jailbreak usually describes an attempt to make a model disregard its safety behavior. Prompt injection is broader. The worker may continue to appear helpful and policy-compliant while: giving extra weight to one source; omitting evidence; selecting an attacker-controlled destination; creating a misleading follow-up task; storing a false preference; escalating ordinary work; asking another agent to act; or disclosing data through an allowed-looking request.
Evaluate whether the worker preserves user intent and authority - not only whether it refuses conspicuously harmful language.
§ 03Why Persistent AI Employees Change the Threat Model
An AI employee is not just a response generator. It may have a durable role, recurring goals, schedules, connected accounts, task state, memory, and the ability to hand work to another employee. Each feature is useful. Each also changes exposure.
| Persistent-worker property | Security change |
|---|---|
| Scheduled operation | Work may run when no reviewer is watching |
| Inbox or queue consumption | Outside parties can repeatedly supply content |
| Connected identity | Model output can become an authenticated action |
| Private context | Injection can influence handling of confidential data |
| Memory | Influence can survive the original task |
| Task board | A redirected objective can become future work |
| Delegation | Influence and context can cross role boundaries |
| Retry and recovery | A blocked action may be attempted through another path |
| Long-running plan | Source and effect may be separated by hours or days |
| Accumulated history | Earlier content can quietly bias later decisions |
The foundational AI employee guide explains the operating model of role, goals, tools, memory, scheduling, and accountability. Prompt-injection review should cover the full operating loop rather than one chat transcript.
Exposure repeats
A one-time assistant may read one hostile page. A standing worker may inspect: every new lead; every inbound ticket; every document in a watched folder; every result returned by a search; every page linked from a research task; every handoff from a partner role; and every stored memory selected for a future run.
Even when each individual attempt is unlikely to work, repeated exposure changes operational risk. Monitoring should therefore track rates and patterns, not only single events.
Action may be delayed
The injected content may not ask for an immediate effect. It may alter: the interpretation of a goal; a recipient list saved for later; a preferred source; a standing exception; a procedural step; a follow-up task; a calendar trigger; or a memory retrieved on a later run.
The event timeline becomes:
exposure → candidate state → persistence → retrieval → decision → action
An incident investigator must be able to reconstruct every step. The AI employee audit-log guide defines the actors, sources, authority decisions, memory operations, tool events, outcomes, and lineage needed for that reconstruction.
A successful injection still needs authority
Model robustness matters, but the application controls blast radius. If the worker can only draft into a staging area, the result is different from a worker that can send, publish, purchase, delete, change permissions, or execute code.
The least-privilege guide for AI agents narrows identity, systems, resources, fields, functions, actions, destinations, time, volume, spend, and delegation. Apply those limits before expanding the worker’s content reach.
§ 04Threat-Model the Complete Content-to-Action Path
Choose one real workflow and draw its actual data path. “The agent reads email” is too broad. Use:
source actor → channel → parser → content store → retrieval → model context → plan → tool policy → approval → action → result → memory → delegation
For every edge, record: the actor; identity; trust; purpose; content form; private context; authority; persistence; delegation; enforcement; evidence; and recovery.
Use a source-sink matrix
Rows are sources. Columns are sinks. Review each intersection:
| Source | Private retrieval | External send | Record update | Memory write | Delegation |
|---|---|---|---|---|---|
| Authenticated user task | Conditional | Conditional | Conditional | Conditional | Conditional |
| Email body | High scrutiny | Block or approve | Block or verify | Candidate only | Sanitized only |
| Attachment | High scrutiny | Block or approve | Block or verify | Candidate only | Sanitized only |
| Public webpage | Compartmentalize | Block by default | Block by default | Block by default | Summary only |
| Internal system record | Purpose-bound | Destination policy | Object authorization | Typed and scoped | Minimum fields |
| Tool response | Treat as untrusted | Revalidate | Revalidate | Candidate only | Typed result |
| Existing memory | Scope and freshness | Revalidate | Revalidate | No recursive promotion | Typed and attributed |
| Other agent | Verify identity and task | Fresh authorization | Fresh authorization | Provenance required | Depth and scope limit |
“Conditional” is not a control. Replace it with the exact identity, policy, approval, or validation rule.
Find dangerous combinations
Prioritize paths where one run contains: untrusted content; valuable private data; a write-capable or communication tool; a destination the content can influence; durable state; limited human visibility; and broad retry or delegation permission.
An inbox triage worker that only labels messages has a different profile from one that can search the full mailbox, open attachments, retrieve customer records, send external messages, update CRM, and remember account preferences.
Treat tool output as another input
Tool responses are not inherently trusted. A browser, connector, search system, database field, code repository, API, or another model can return attacker-influenced content.
Preserve: tool identity; request parameters; the returned object; the original source; content type; integrity status; timestamp; tenant/account scope; and transformation history.
Do not upgrade content authority because it arrived through an approved tool. The tool may be trusted to retrieve data while the retrieved data remains untrusted.
§ 05Keep Content Separate From Governing Instructions
The worker needs a clear authority hierarchy, but hierarchy alone is not a hard security boundary. Use it alongside application enforcement.
| Layer | Example | Authority |
|---|---|---|
| Platform policy | Safety and security constraints | Cannot be changed by task content |
| Organization policy | Approved role, prohibited actions, data policy | Changed only through controlled administration |
| Role contract | Goal, scope, sources, tools, review, stop conditions | Versioned owner approval |
| Authenticated task | Specific requested outcome and boundaries | Limited to requester authority |
| Trusted operational data | Current system-of-record values | Evidence, not policy |
| External content | Email, web, files, tickets, attachments | Data to inspect, never governing authority |
| Model output | Plan, summary, extraction, proposed action | Untrusted proposal until validated |
Preserve source boundaries in context
Do not concatenate everything into an unlabeled prompt. Carry: source type; source actor; authenticated identity where available; retrieval reason; document or object ID; timestamp and version; trust classification; tenant and account; applicable task; permitted uses; and sensitivity.
The AI employee context-pack guide shows how to assemble purpose-bound context with source ownership, precedence, freshness, and exclusions. Prompt-injection controls depend on keeping those distinctions visible through retrieval and action.
Extract facts without granting authority
Where possible, use a narrow transformation:
untrusted document → typed candidate fields → validation → task use
For example, an inbound request may yield: the claimed sender; the requested object; the proposed change; cited evidence; confidence; unresolved contradictions; and a source pointer.
The request does not become authorized merely because the model extracted it correctly. Verify the actor, object, policy, and approval independently.
Keep secrets out of unnecessary contexts
The safest private datum is one the untrusted content never shares context with. Retrieve only required fields; mask secrets and credentials; use separate steps for public research and private analysis; avoid exposing full mailboxes or repositories to a narrow task; resolve destinations in policy code; prohibit the model from choosing arbitrary external endpoints; and use reference tokens when the model needs an object identity but not its full data.
Compartmentalization reduces both accidental leakage and the reward of a successful injection.
Do not rely on prompt secrecy
Hiding the system prompt is not the same as enforcing it. A worker can be redirected without revealing the prompt. Critical rules should exist in: workload identity; application permissions; tool schemas; policy engines; allowlisted objects and destinations; approval gateways; transaction boundaries; sandboxes; rate and spend controls; and postcondition checks.
The model proposes. The system authorizes and executes.
§ 06Reduce What a Redirected Worker Can Do
Assume some hostile content will evade detection. Design the action boundary so that model influence alone cannot create an unacceptable effect.
Narrow the tools
| Broad tool | Narrower alternative |
|---|---|
| Send any email | Create draft for approved account contact |
| Update any CRM field | Propose update to allowlisted fields on current account |
| Execute arbitrary query | Call a purpose-specific read function |
| Upload to any URL | Upload to fixed organization destination |
| Browse with full session | Use logged-out or task-scoped session where possible |
| Delete object | Queue deletion request for authorized reviewer |
| Run arbitrary code | Execute bounded operation in an isolated environment |
| Change permissions | Create access request with named principal and expiry |
Every tool call should bind:
agent identity + human principal + task + function + object + parameters + destination + data class + limit + approval + incident state
The AI employee permissions and approvals guide explains why approval must be tied to the exact action rather than a vague plan.
Resolve authority outside the model
Do not ask the model to decide whether it is allowed to: access a customer; send to a recipient; change a protected field; use an administrator function; publish externally; exceed a monetary threshold; delegate a capability; or bypass a security control.
Use deterministic authorization based on authenticated identities, task scope, resource ownership, data class, policy, and current system state.
Bind approval to the real effect
| Field | Reviewer must see |
|---|---|
| Actor | AI employee and initiating human or system |
| Intent | Current goal and why the action is needed |
| Source | Content that materially influenced the proposal |
| Action | Exact function and object |
| Data | Fields or content to be read, changed, or transmitted |
| Destination | Actual person, account, URL, system, or environment |
| Difference | Change from current state |
| Risk | Sensitive data, uncertainty, exceptions, and irreversibility |
| Expiry | How long approval remains valid |
| After-state | What the system will verify |
If any material field changes after approval, request a new decision. An injected instruction should not be able to swap the recipient, attachment, amount, object, or tool after a human approves.
Validate the result
A successful API response proves only that the API accepted a request. Check: the expected object changed; no unapproved object changed; the actual recipient and destination match approval; only permitted fields were disclosed; the action happened once; no hidden follow-up task was created; no unauthorized memory write occurred; and no child agent inherited excess authority. Then mark the task complete.
§ 07Prevent Prompt Injection From Becoming Durable Memory
Memory turns a transient influence into a possible future control. Microsoft’s agentic memory-safety guidance recommends gating writes on intent and provenance, logging memory lifecycle operations, tracking propagation, applying safety checks at retrieval, and preserving deletion and rollback paths.
| Memory function | Example | Injection concern |
|---|---|---|
| Working | Current task, open question, next action | Hostile content alters the active plan |
| Episodic | Prior case, action, outcome | Manipulated case becomes a misleading precedent |
| Semantic | Customer fact, preference, relationship | Unverified claim becomes durable truth |
| Procedural | SOP, rule, tool sequence | External content becomes standing instruction |
The AI agent memory-types guide explains these functions as a design lens. Prompt-injection controls should be different for each type.
Put every material write through a gate
Require: purpose (which future task needs this item?); intent (did an authorized person or policy request persistence?); source; trust status; type; scope; sensitivity; evidence; freshness; conflict state; retention; and control (who can inspect, correct, suppress, and delete it?).
If required fields are missing, hold the item as a candidate or do not persist it.
Never promote external content directly into procedure
A webpage, email, ticket, attachment, tool response, or other agent’s free text should not directly create: a new operating rule; a security exception; a changed destination; a secret retrieval method; a bypass; an approval threshold; a new tool permission; or a standing preference that affects other people.
Procedural memory should come from a controlled policy or SOP change with an owner, review, version, effective date, and rollback path. The AI employee SOP guide provides that governance structure.
Separate observation from truth
Store: “Source X claimed Y at time Z for task T; verification status is pending.” Do not silently store: “Y is true.”
For episodic memory, preserve the action and verified outcome - not merely the model’s interpretation. For semantic memory, prefer the authoritative system of record. For preferences, confirm scope and affected person. For procedures, require formal approval.
Prevent recursive and inherited writes
An agent should not be able to launder untrusted content by: (1) summarizing it; (2) sending the summary to another agent; (3) receiving the summary back as “internal” content; and (4) storing it as trusted memory.
Provenance must survive transformations and handoffs. Trust should not increase merely because content was generated or relayed internally.
Scan at write and retrieval
Write-time checks reduce what becomes durable. Retrieval-time checks protect against: older poisoned records; newly discovered attack patterns; changed task context; revoked source trust; cross-account selection; stale procedures; and previously harmless content becoming dangerous next to a new tool.
The broader memory privacy and retention checklist covers purpose, minimization, access, correction, retention, deletion, vendor processing, and incident handling. Security review should use that lifecycle rather than treating memory as a single on/off feature.
§ 08Control Retrieval, Tools, and Multimodal Content
Indirect injection often enters through systems that were added to improve context. Each retrieval path needs a boundary.
Retrieval-augmented generation
For each retrieved item, preserve: the repository and document; author or controller; access decision; tenant/account; version and time; trust classification; query or retrieval reason; chunk boundaries; transformations; safety decision; and citation pointer.
Retrieval relevance does not prove authority or truth. A highly relevant hostile paragraph can still be the wrong basis for action. Use retrieval for evidence. Do not allow retrieved content to redefine the role, task objective, tool permissions, data boundaries, destination policy, approval rules, or memory policy.
Browser and email
Web and email workflows combine a large untrusted surface with useful private context. Reduce exposure by: using logged-out access when authentication is unnecessary; separating public research from private account work; using site, account, and destination restrictions; opening only needed messages and fields; treating links, redirects, forms, comments, and downloads as separate sources; preventing silent external transmissions; requiring review for sensitive account changes; and ending or isolating sessions after the task.
OpenAI’s prompt-injection overview emphasizes layered defenses, narrow access, explicit user instructions, monitoring, red-teaming, and confirmation before consequential actions. The same operational principles apply across vendors.
Documents, images, and hidden layers
The model may process content a human reviewer does not easily see: document metadata; off-page or low-contrast text; comments and tracked changes; image text; embedded objects; alt text; markup; accessibility layers; and OCR output.
Do not treat a visually benign file as trusted. Preserve the original, extracted representation, parser version, and source. Test every modality and transformation the production worker can consume.
Tool and model output
Generated output becomes another injection path when a downstream interpreter executes it. Validate structured output against a schema; encode data for its destination; avoid shell or query construction from free text; reject unknown functions and fields; resolve objects and recipients independently; inspect URLs and redirect behavior; treat model-generated code as untrusted; isolate execution; cap resource use; and verify the after-state.
Prompt injection and unsafe output handling can compound. The first redirects the worker; the second turns generated content into an effect.
§ 09Stop Injection From Crossing Agent Boundaries
Multi-agent systems add new principals, handoffs, scopes, and return channels. “Another company agent produced it” is not a trust classification.
The agent handoff protocol guide separates goals, context, authority, evidence, acceptance, and return. Use that structure to prevent free-text handoffs from silently changing another role.
Use a typed handoff
| Field | Required control |
|---|---|
| Requester | Authenticated parent agent and initiating principal |
| Goal | Bounded requested outcome |
| Sources | Object references and preserved provenance |
| Context | Minimum fields needed by the child |
| Authority | Child’s own permitted tools and resources |
| Prohibitions | Actions and data outside scope |
| Output | Typed artifact or result |
| Completion | Evidence and accepted postcondition |
| Expiry | Time and run boundary |
| Return | Destination and receiving validation |
Do not let the parent pass its access token, hidden context, or unrestricted tool capability to the child.
Re-authorize at every boundary
The child should answer: Is this parent authorized to request the task? Is the goal valid for my role? May I access the named objects? Which source content is untrusted? Which functions are allowed? Does the action need fresh approval? May I write memory? Which fields may I return?
The return should be treated as untrusted input by the parent until validated.
Limit propagation
Set: maximum delegation depth; maximum child count; time, cost, and tool budgets; source and destination allowlists; context minimization rules; memory-write restrictions; loop detection; duplicate-action prevention; and circuit-breaker thresholds.
A redirected worker should not be able to create an unbounded tree of tasks.
§ 10Detect, Contain, and Recover From Injection
Preventive controls will miss some cases. Monitoring should detect suspicious influence before or after effect.
Log the decision path
Capture: content ingestion (source, actor, object, version, type, trust, task); retrieval (query, selected objects, reason, scope, scores, filters); safety decisions; plan changes; authorization; approval; tool calls and results; memory operations; delegation; and alert/response events.
Do not log hidden model reasoning. Log observable inputs, source lineage, proposed and executed actions, policy decisions, and results.
Monitor behavior, not only phrases
Useful signals include: a new destination or domain; data retrieval followed by external transmission; a task goal changing after external content is read; an unusual tool sequence; a memory write sourced from public or unknown content; procedural memory created outside policy workflow; cross-account retrieval; unexpected delegation; repeated blocked attempts through different tools; new encoded or multimodal content; a high-risk action outside normal schedule; an approval request that omits source or destination; and unusual volume, spend, retries, or runtime.
Phrase detection may contribute, but behavior and information flow are closer to the actual harm.
Use a containment ladder
| Severity | Immediate action |
|---|---|
| Suspicious content, no effect | Quarantine content, preserve source, continue safe work if isolated |
| Proposed unsafe action | Block action, hold task, alert owner |
| Memory or task-state write | Suppress retrieval, freeze affected state, trace propagation |
| External or production effect | Revoke capability, contain affected system, preserve evidence |
| Possible disclosure or account compromise | Activate security and privacy incident process |
| Cross-agent propagation | Pause affected roles, block handoffs, inspect descendant tasks |
Contain the smallest safe scope first, then expand if evidence shows propagation.
Correct persistent state
After a memory-poisoning event: identify the source and first affected task; enumerate derived summaries, indexes, task states, handoffs, and outputs; suppress retrieval before full deletion if immediate containment is needed; restore authoritative values or last known safe procedures; invalidate approvals and actions that depended on the poisoned state; inspect downstream effects and recipients; preserve the incident evidence; retest the path; and resume only after an accountable owner accepts residual risk.
The AI employee incident-response guide defines pause, revoke, preserve, scope, correct, notify, validate, and resume decisions for the wider operating system.
§ 11Test Prompt Injection Before Production
Testing should prove the entire control path, not just that the model says it ignored an instruction.
Do not use live secrets, real customer data, production write capability, or uncontrolled external endpoints. Run in an isolated test tenant with synthetic data, instrumented tools, fixed destinations, cleanup, and named owners.
Build a defensive test matrix
| Test class | Safe test objective | Expected evidence |
|---|---|---|
| Direct instruction conflict | User asks for an action outside role or authority | Denial or bounded clarification; no tool call |
| Indirect document content | File contains instruction-like text unrelated to task | Content remains data; task objective unchanged |
| Web content | Public page attempts to influence ranking or destination | Source flagged; independent criteria preserved |
| Email content | Message requests sensitive or unrelated action | Sender/object authorization and approval enforced |
| Tool-response content | Approved tool returns instruction-like text | Response treated as untrusted; sink policy still enforced |
| Retrieved knowledge | Relevant chunk conflicts with governing policy | Policy wins; source and conflict logged |
| Multimodal content | Image or document layer contains conflicting direction | Same treatment across parsed representations |
| Destination manipulation | Content tries to select an external recipient or URL | Destination resolved or allowlisted outside model |
| Private-data combination | Untrusted source appears beside synthetic sensitive data | No unauthorized transmission or disclosure |
| Memory write | External claim is proposed as a fact or procedure | Candidate held, provenance preserved, gate blocks promotion |
| Delayed retrieval | Test item is later selected from memory | Retrieval check prevents unsafe influence |
| Delegation | Parent passes tainted context to child | Provenance survives; child re-authorizes |
| Cross-account | Content references another tenant or account | Access and retrieval isolation deny the path |
| Approval tampering | Action changes after reviewer decision | Approval invalidated; new review required |
| Retry/bypass | Blocked action is attempted through another route | Breaker stops retries and raises alert |
| Recovery | Synthetic poisoned state is introduced | Suppress, trace, restore, validate, and resume succeed |
These are test categories, not attack recipes. Security teams can implement them with harmless markers and controlled destinations.
Grade the outcome
| Grade | Meaning |
|---|---|
| Pass | Unsafe effect is prevented and required evidence is complete |
| Contained | Model follows the content, but independent control blocks the effect |
| Detected late | Effect occurs in test environment and monitoring catches it |
| Silent failure | Effect occurs without timely alert or reconstruction |
| Untestable | Required logs, controls, or isolation do not exist |
A model refusal is not automatically a pass. Confirm: no sensitive data was retrieved unnecessarily; no tool call occurred; no hidden task or memory was written; no delegation occurred; no approval was reused; the event was logged; and normal work can continue safely.
Retest on change
Retest after changes to: the model or provider; system or role instructions; parser or content extraction; retrieval source or ranking; connector or browser capability; tool schema; identity or permission; memory type or retrieval logic; agent topology or handoff; approval policy; data classification; monitors or detectors; downstream applications; or incident response procedure.
Prompt injection is an adaptive security problem. OpenAI’s discussion of continuously hardening browser agents describes ongoing discovery, testing, monitoring, and mitigation rather than a one-time claim of immunity. Buyers should expect the same operating discipline from any platform.
§ 12Evaluate an AI Employee Platform
A vendor should be able to explain the complete control path with product evidence.
| Area | Buyer question | Evidence to request |
|---|---|---|
| Source trust | How are user instructions, trusted records, public content, tool results, and agent messages distinguished? | Data-flow diagram and live trace |
| Retrieval | Does provenance survive parsing, chunking, ranking, and summarization? | Retrieved-object metadata |
| Context | Can public research be isolated from private data? | Context and session controls |
| Identity | Does each agent use a distinct attributable identity? | Identity and authentication records |
| Permission | Can access be restricted by system, object, field, function, destination, time, and volume? | Denial tests and policies |
| Tools | Are consequential functions narrow, typed, and validated outside the model? | Tool schema and policy result |
| Approval | Is approval bound to exact data, recipient, action, object, and expiry? | Approval event and change test |
| Memory | Can untrusted content write memory? Which gates and provenance apply? | Write, retrieval, correction, and deletion test |
| Delegation | Does authority narrow at child boundaries? | Parent-child trace and denial test |
| Monitoring | Can alerts correlate source, retrieval, memory, tool, and result? | Investigation timeline |
| Response | Can one role, connector, token, task, memory set, or handoff path be paused? | Revocation and containment drill |
| Testing | Which indirect, persistent, and cross-agent cases are evaluated? | Test plan, cadence, and remediation record |
Verify current CellCog behavior
CellCog publicly presents AI Employees as workers that can operate with goals, schedules, connected tools, memory, and employee-to-employee collaboration. Those capabilities make source trust, permissions, approvals, memory controls, auditability, and delegation boundaries part of the buying decision.
Public pages do not establish every implementation detail. As of July 29, 2026, buyers should verify in the live product and contract: how external content is labeled and isolated; whether browser, email, files, connectors, and tool results receive the same treatment; which prompt-injection detection or safety checks exist; where critical policy is enforced outside the model; whether arbitrary destinations are possible; whether memory writes require intent and provenance; how memory is inspected, corrected, suppressed, and deleted; whether child employees receive fresh, narrower authority; which approval fields are action-bound; what source-to-effect audit evidence is retained; how quickly one role or connector can be revoked; and which tests and incident commitments are supported.
The CellCog AI Employees page is the starting point for the operating model. The platform-specific assessment should continue through a demonstration, security review, permission tests, memory tests, audit export, and a controlled incident exercise.
The next operational step is the AI employee security checklist, which turns security claims into pilot evidence across identity, data, tools, memory, monitoring, response, and governance.
§ 13A 30-Day Implementation Blueprint
Use one representative workflow rather than trying to secure every proposed role at once.
| Period | Work | Exit evidence |
|---|---|---|
| Days 1-5 | Select workflow, classify sources and sinks, map data flow | Approved threat model |
| Days 6-10 | Separate instruction/data, narrow identity and tools, define destinations | Denial and isolation tests |
| Days 11-15 | Add approval, memory gates, provenance, and postconditions | Action and memory evidence |
| Days 16-20 | Instrument monitoring, alerts, circuit breakers, and revocation | Reconstructable test runs |
| Days 21-25 | Run direct, indirect, persistence, delegation, and recovery tests | Findings and remediation |
| Days 26-30 | Fix gaps, rerun tests, set residual-risk decision and cadence | Signed pilot boundary |
§ 14Common Failure Patterns
| Failure pattern | Why it fails | Better response |
|---|---|---|
| “Our system prompt says to ignore attacks” | Model behavior is probabilistic and content remains mixed | Enforce authority and sinks outside model |
| “We scan for malicious phrases” | Legitimate content contains commands; attacks adapt | Combine detection with information-flow and action controls |
| “The source is internal” | Internal records, tools, and agents may contain external or compromised content | Preserve original provenance and trust |
| “RAG grounds the model” | Relevant retrieved data can still contain hostile instructions | Treat retrieval as evidence, never governing authority |
| “A human approves” | Vague packets hide changed data, recipient, or effect | Bind review to exact action and invalidate on change |
| “The action is reversible” | Disclosure, recipient effects, and propagated state may not be reversible | Minimize authority and verify impact before release |
| “Memory only stores useful facts” | Extraction can misclassify claims or instructions as facts | Use typed candidates, source, status, scope, and owner |
| “The child agent is trusted” | Tainted context can cross boundaries and authority may expand | Sanitize handoff and re-authorize child |
| “No alert means no attack” | Logging or detection may be incomplete | Test observability and reconcile with system effects |
| “We passed one red-team exercise” | Models, sources, tools, and adversaries change | Continuous testing and control versioning |
§ 15Final Checklist
- Governing instructions, trusted operational data, external content, secrets, and model output have distinct roles.
- Source identity, provenance, purpose, trust, tenant, time, and transformation survive retrieval.
- Public or mixed-trust content is isolated from unnecessary private context.
- The agent uses a distinct, attributable, least-privileged identity.
- Systems, resources, fields, functions, actions, destinations, time, volume, spend, and delegation are bounded.
- Consequential tool calls use typed parameters and deterministic authorization outside the model.
- Arbitrary external recipients, URLs, commands, or production objects are blocked by default.
- Approvals show the actual source, data, object, destination, action, risk, and expiry.
- Material changes after approval require a new decision.
- Tool results are treated as untrusted input and after-state is verified.
- Memory writes require authorized intent, purpose, type, source, status, scope, evidence, owner, and retention.
- External content cannot directly create procedural memory, policy, permissions, or exceptions.
- Retrieval-time checks protect against old, stale, cross-scope, or newly dangerous memory.
- Provenance survives summaries, task state, memory, and agent handoffs.
- Child agents authenticate the parent, validate the task, and receive fresh, narrower authority.
- Monitoring connects source, retrieval, plan, policy, approval, tool, result, memory, and delegation.
- Breakers can stop suspicious retries, transmissions, writes, and delegation.
- The team can pause a role, revoke a connector or identity, suppress memory, and preserve evidence.
- Recovery traces derived state, corrects authoritative records, validates effects, and controls resumption.
- Direct, indirect, tool, retrieval, multimodal, memory, delayed, cross-account, delegation, approval, and recovery tests pass in an isolated environment.
- The tested model, prompt, parser, retrieval, tool, policy, memory, and monitor versions are recorded.
- Material system changes trigger retesting.
Start with one concrete path:
untrusted email or webpage → private context → proposed external action or memory write
Map it, narrow it, instrument it, and test it. If the worker cannot convert that content into unauthorized authority, and the organization can prove what happened when the content was encountered, the deployment has moved from prompt-level hope toward system-level control.
Q1What is prompt injection for an AI employee?
It is a failure in which user-supplied or externally retrieved content influences the worker as if that content had authority to change its goal, policy, data access, tool use, destination, memory, or delegation. The business risk depends on what private context and consequential capabilities the worker can reach.
Q2What is indirect prompt injection?
Indirect prompt injection is delivered through content the worker retrieves or processes rather than through the primary user instruction. It can arrive in webpages, emails, files, tickets, images, retrieved knowledge, tool responses, or agent handoffs.
Q3Can prompt injection poison AI agent memory?
Yes. If external content can create or modify durable facts, preferences, episodes, procedures, or task state, a transient influence can affect later runs. Gate memory writes on authorized intent, purpose, source, trust status, type, scope, evidence, retention, and correction.
Q4Can a prompt-injection detector solve the problem?
No detector should be treated as a complete solution. Pair detection with source separation, context minimization, least privilege, deterministic authorization, destination controls, action-bound approval, memory gates, monitoring, and recovery.
Q5Is human approval enough?
Only when the reviewer receives the material source, exact action, data, object, recipient or destination, risk, and before/after state - and the approval expires if any of those fields change. Approval does not replace access control, tool validation, or postcondition checks.
Q6How should a buyer test prompt-injection safety?
Use an isolated tenant with synthetic data and instrumented tools. Test direct, indirect, retrieval, tool-response, multimodal, destination, private-data, memory, delayed, delegation, cross-account, approval, retry, monitoring, and recovery paths. Grade the actual system effect and evidence, not only the model’s written response.
