Python: Samples: deterministic action-boundary validation middleware (#5366)#6528
Open
eeee2345 wants to merge 1 commit into
Open
Python: Samples: deterministic action-boundary validation middleware (#5366)#6528eeee2345 wants to merge 1 commit into
eeee2345 wants to merge 1 commit into
Conversation
…ary validation, microsoft#5366) Adds python/samples/02-agents/middleware/atr_validation_middleware.py: a FunctionMiddleware that validates tool arguments at the execution boundary and raises MiddlewareTermination before call_next() when they match an attack pattern, so the tool never runs. This is the deterministic, single-enforcement- point pattern named in microsoft#5366 and answers its open follow-up about a recommended validation-at-execution-boundary sample. The check is a small self-contained deny-list mirroring Agent Threat Rules (ATR) intent (prompt injection, exfiltration, credential access in tool args); a docstring notes how to swap in the full open ruleset via pyatr. No external dependency, so the sample stays import-clean. Updates the middleware README Files table. Signed-off-by: Adam Lin <adam@agentthreatrule.org>
github-actions
Bot
changed the title
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds a new middleware sample demonstrating deterministic validation of tool arguments at the tool-execution boundary (ATR-style deny-list), and documents it in the middleware samples index.
Changes:
- Introduces
ATRValidationMiddlewaresample that blocks suspicious tool calls viaMiddlewareTerminationbefore tool execution. - Adds a small regex-based deny-list to detect common prompt-injection/exfil/credential-access patterns in tool arguments.
- Updates the middleware README to include the new sample.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 6 comments.
| File | Description |
|---|---|
| python/samples/02-agents/middleware/atr_validation_middleware.py | New sample middleware + demo agent that blocks tool calls when arguments match ATR-like patterns |
| python/samples/02-agents/middleware/README.md | Adds an entry describing the new ATR validation middleware sample |
Comment on lines
+18
to
+22
| from dotenv import load_dotenv | ||
| from pydantic import Field | ||
|
|
||
| # Load environment variables from .env file | ||
| load_dotenv() |
Comment on lines
+49
to
+50
| r"\b(?:ignore|disregard|forget|override)\b.{0,40}" | ||
| r"\b(?:previous|prior|above|earlier)\b.{0,40}\binstructions?\b", |
|
|
||
| def _matches_attack_pattern(arguments: dict[str, object]) -> str | None: | ||
| """Return the first matched pattern string, or None when the arguments look benign.""" | ||
| text = " ".join(str(value) for value in arguments.values()) |
Comment on lines
+105
to
+108
| print( | ||
| f"[ATRValidationMiddleware] Blocked tool '{context.function.name}': " | ||
| f"arguments matched an ATR-style attack pattern." | ||
| ) |
| # Raise BEFORE call_next() so the tool is never executed. | ||
| raise MiddlewareTermination(f"ATR validation blocked tool '{context.function.name}'") | ||
|
|
||
| print(f"[ATRValidationMiddleware] Tool '{context.function.name}' passed ATR validation.") |
| f"arguments matched an ATR-style attack pattern." | ||
| ) | ||
| # Raise BEFORE call_next() so the tool is never executed. | ||
| raise MiddlewareTermination(f"ATR validation blocked tool '{context.function.name}'") |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds a function-middleware sample that answers #5366: a single deterministic enforcement point that validates a tool call right before it executes.
ATRValidationMiddleware subclasses FunctionMiddleware, inspects the validated arguments in FunctionInvocationContext.arguments, and raises MiddlewareTermination before calling call_next() when the arguments match a known attack pattern, so the tool never runs. The decision is deterministic (no model call, no network), which is what #5366 asks for at the execution boundary.
The check is a small, self-contained deny-list illustrating the pattern. To enforce a full, maintained ruleset instead of the illustrative subset, install the open-source engine (pip install pyatr) and swap the matcher; the sample documents this inline.
Files
Local checks
Disclosure: the deny-list patterns mirror the open-source Agent Threat Rules (ATR) project, which I maintain. The sample has no ATR dependency; the reference is for users who want the full ruleset.