3. Permissions by design: Bind instruments to duties, to not fashions
A standard anti-pattern is to provide the mannequin a long-lived credential and hope prompts hold it well mannered. SAIF and NIST argue the other: credentials and scopes must be certain to instruments and duties, rotated recurrently, and auditable. Brokers then request narrowly scoped capabilities by way of these instruments.
In observe, that appears like: “finance-ops-agent might learn, however not write, sure ledgers with out CFO approval.”
The CEO query: Can we revoke a particular functionality from an agent with out re-architecting the entire system?
Management information and conduct
These steps gate inputs, outputs, and constrain conduct.
4. Inputs, reminiscence, and RAG: Deal with exterior content material as hostile till confirmed in any other case
Most agent incidents begin with sneaky information: a poisoned internet web page, PDF, e-mail, or repository that smuggles adversarial directions into the system. OWASP’s prompt-injection cheat sheet and OpenAI’s personal steering each insist on strict separation of system directions from consumer content material and on treating unvetted retrieval sources as untrusted.
Operationally, gate earlier than something enters retrieval or long-term reminiscence: new sources are reviewed, tagged, and onboarded; persistent reminiscence is disabled when untrusted context is current; provenance is hooked up to every chunk.
The CEO query: Can we enumerate each exterior content material supply our brokers be taught from, and who accredited them?
5. Output dealing with and rendering: Nothing executes “simply because the mannequin stated so”
Within the Anthropic case, AI-generated exploit code and credential dumps flowed straight into motion. Any output that may trigger a aspect impact wants a validator between the agent and the actual world. OWASP’s insecure output dealing with class is express on this level, as are browser safety greatest practices round origin boundaries.
