Prompt injection
An attempt to make an AI system follow instructions embedded in untrusted material such as email, documents or websites instead of the authorized user’s task.
- Trust boundary
- Retrieved content is information, not authority
Personal agents encounter third-party content through browsers and connected apps. A malicious message can try to redirect tool use or extract data. Anthropic acknowledges that no prompt-injection defense is perfect.
Limit connected data and action permissions, review consequential approvals and verify source records. A source citation helps trace a claim; it does not itself make the source trustworthy.
Connected
- Claude DuetAnthropic’s proactive personal agent: it works in connected apps on a cloud computer and returns briefs, files and approval requests. Launched as an invitation-only US Pro and Max research preview in October 2026.
- Agent evaluationRepeatable testing of an AI agent's complete task, tool use, permissions and final system state against explicit acceptance criteria. A successful demo is not a production reliability estimate.
- Model Context Protocol (MCP)An open protocol, introduced by Anthropic in November 2024, that standardises how AI applications connect to external tools and data sources.
More concepts & methods
- machine learningscientific study of algorithms and statistical models that computer systems use to perform tasks without explicit instructions
- deep learningspecialized form of machine learning that uses artificial neural networks to identify patterns in data
- genetic algorithmcompetitive algorithm for searching a problem space
- Altitude, depth and rangeFelix Rieseberg’s framework for ambition with AI models: speed repeats work faster, altitude targets outcomes, depth uses expert instruments and range combines disciplines.
- Outcome-based promptingAsking an AI agent to pursue a specified change for a user or system, with evidence and boundaries, instead of merely producing a named deliverable.
- AI outside judgeAn acceptance check outside an agent’s self-assessment, such as a deterministic test, held-out evaluation, measurement or separately briefed reviewer.