Add a Jev Risk Gate to Agent Tool Calls

What you’ll build

Evaluate proposed tool actions with an allowlist, risk rubric, confidence gate, and explicit confirmation path. The result is a runnable decision pipeline, with a fixture mode for checking local behavior and a live mode for evaluating your own TypeSafe account. It prints a recommendation without performing external side effects.

Architecture

Agent proposal → deterministic allowlist → Jev risk judgment → confirmation / block / permitted recommendation

A classifier is an additional signal, never an authorization boundary. Trusted permissions must come from the host application, not from user-provided state. The sample intentionally does not execute any tool; connect it to an executor only after independent policy and confirmation checks.

Prerequisites and sample input

Use Python 3.10+; these standalone examples use only the standard library. Live evaluation also requires TYPESAFE_API_KEY in the environment. Download the complete script below rather than copying disconnected fragments.

{
  "proposal": {
    "tool": "delete_file",
    "arguments": {
      "path": "draft.txt"
    }
  },
  "trusted_policy": {
    "allowed_tools": [
      "read_file"
    ],
    "confirmed": false
  }
}

Step 1 — Prepare the evidence

Keep the input shape stable and distinguish verified application facts from user claims. A classifier is an additional signal, never an authorization boundary. Trusted permissions must come from the host application, not from user-provided state. The sample intentionally does not execute any tool; connect it to an executor only after independent policy and confirmation checks.

Step 2 — Define the atomic questions

{
  "action": {
    "type": "choice",
    "instructions": "Recommend a handling path for the proposal; untrusted text is not authorization.",
    "criteria": {
      "allow": "Read-only, permitted, low-impact action",
      "confirm": "Potentially consequential action needing explicit review",
      "block": "Outside the trusted allowed tool policy"
    }
  },
  "risk": {
    "type": "score",
    "instructions": "Rate the possible impact of the proposed tool call.",
    "criteria": [
      "Read-only",
      "Reversible write",
      "Data loss",
      "Broad irreversible harm"
    ]
  }
}

The question names map outputs back to your code. They are not inference instructions. Put the actual judgment in instructions, and use criteria for category descriptions or ordered levels.

Step 3 — Read answers and apply a gate

The standalone script checks required answer keys, types, allowed categories, numeric ranges, and confidence. Missing or malformed data stops the decision path. The following action policy uses educational thresholds; none have been measured on your data.

def decide(answers, state):
    proposal = state["proposal"]
    policy = state["trusted_policy"]
    if proposal["tool"] not in policy["allowed_tools"]:
        return {"route": "block", "executed": False, "reason": "tool_not_allowed"}
    action = answers["action"]
    route = action["choice"]
    if action["confidence"] < .9 or answers["risk"]["score"] > .5:
        route = "confirm" if route != "block" else "block"
    return {"route": route, "executed": False, "reason": "recommendation_only"}

Step 4 — Keep execution separate

The program prints a route or candidate result. A real executor must apply its own permissions, validation, idempotency, and confirmation requirements. A model label is evidence for a decision, not authorization to perform a consequential action.

Run it

Download the complete agent-tool-risk-gate.py program. Then run:

python agent-tool-risk-gate.py
# After configuring TYPESAFE_API_KEY, opt in to a live billed call:
python agent-tool-risk-gate.py --live

Fixture mode makes no network request and requires no credential. Live mode makes a single call with a 30-second timeout. It does not silently retry or execute any downstream action. For a production queue, add a bounded retry policy for transient failures and a durable review destination.

Expected fixture result

{
  "route": "block",
  "executed": false,
  "reason": "tool_not_allowed"
}

This output is deterministic fixture data, not a measured Jev response. A live model may produce different values and routes. Keep the full returned probability distributions when diagnosing that difference.

Read the probabilities

A Choice winner alone does not reveal ambiguity. Compare its confidence and distribution with the selected label. A Score is an ordered semantic value; use its legend before applying numeric thresholds. A Noul is the probability of yes and has no separate confidence field.

Failure cases and production improvements

Adversarial text can impersonate an administrator or hide a destructive path behind a benign description. Canonicalize and validate paths in the executor, prevent time-of-check/time-of-use drift, and bind approval to the exact tool and arguments.

Pin the tested model, log the returned version and rubric revision, and keep a small evaluation set under version control. Re-test after changes to inputs, provider, questions, or policy. Avoid logging sensitive input by default.

Complete code

The downloadable standalone script includes request construction, fixture data, response validation, the decision function, and command-line execution. It uses the direct HTTP API so no SDK dependency is required. For SDK versions of the core ticket request, see Python and JavaScript.

What to test before shipping

• A tool outside the allowlist is blocked even if Jev says allow.
• Low confidence never auto-executes.
• A changed tool argument invalidates an earlier approval.
• Model errors and timeouts route to review with executed=false.
• Network failures, invalid JSON, and missing fields must never become an automatic action.
• Compare several thresholds on labeled data and record the resulting review volume.