C4 · Jev: a model that decides · Interactive, in your browser

Gating an agent’s action

Checking an agent’s proposed action before it runs, and why a refusal is not the same as asking a person.

A gate that scores risk still needs a decision about what happens to the calls it holds back.

Hold a call when its risk is at least50%

What happens to a held call

Cost of a risky call that ran12

Cost of a fine call that was refused4

Every call, its risk percentage and whether it was in fact risky are invented. Costs are invented units, not money. When a person looks, the person is assumed to be right every time.

Run: 8
Held back: 6
Risky, but ran: 1
Fine, but refused: 1
Total cost: 16
If nothing were gated: 72

On this set, the cheapest threshold at these costs is 21% (total 12).

Proposed calls, riskiest first

Export the customer list to a fileRisk 92%
Refused. In fact it was risky.
Delete a customer’s recordRisk 88%
Refused. In fact it was risky.
Email every customerRisk 78%
Refused. In fact it was risky.
Pay a supplier to a new account numberRisk 71%
Refused. In fact it was risky.
Cancel a booking that appears twiceRisk 66%
Refused. In fact it was fine.A fine call was refused.
Refund an order over the usual limitRisk 63%
Refused. In fact it was risky.
Move one visit to another dayRisk 48%
Runs. In fact it was fine.
Refund a small orderRisk 42%
Runs. In fact it was fine.
Email an invoice to an address typed in todayRisk 31%
Runs. In fact it was risky.A risky call ran.
Send one appointment reminderRisk 20%
Runs. In fact it was fine.
Add a note to a jobRisk 12%
Runs. In fact it was fine.
Draft a reply without sending itRisk 8%
Runs. In fact it was fine.
Read tomorrow’s visit listRisk 5%
Runs. In fact it was fine.
Look up a job’s statusRisk 3%
Runs. In fact it was fine.

My viewA gate checks what an agent proposes before it runs. The figure has fourteen invented actions an agent for an invented repair firm might propose, six of them in fact risky, each with an invented probability that it is risky. Drag the threshold: a call at or above it is held back and one below it runs. Then choose what happens to a held call: refuse it, or ask a person.

LangChain’s documentation for the TypeSafe integration describes a tool-gating middleware. It asks for the probability that a tool call is risky or insufficiently authorised, and it classifies only the tools you list. A call it judges risky comes back to the agent as an error message, and the tool does not run.

The same page says the middleware does not ask a person to approve a held call, and points to LangChain’s human-in-the-loop middleware for that. So as documented, the gate refuses, and asking is a second piece you add. The page I read gives no cutoff probability, and like the routing middleware this one is experimental, with an interface that may change without notice.

The risk question is a yes or no, which in TypeSafe’s terms is a Noul. TypeSafe’s docs say a yes or no answer has no confidence value, so the gate has the probability alone to work with, and the choice of cutoff is yours.

TypeSafe’s docs describe what to do with an answer it is unsure about, for its Choice and Score answers: route it to a person, ask for clarification, or fall back to another system, and use higher cutoffs for actions whose mistakes cost more. They give an example in which a transfer needs more confidence than showing a balance. The docs say the right cutoffs depend on your domain and the model’s performance on your use case, and recommend starting with conservative thresholds, testing with your own data and adjusting. That is advice about confidence, not about this middleware.

The integration docs also warn against putting secrets in tool arguments or conversation state unless sending them to TypeSafe is acceptable, which implies that is what the gate sends.

TechCrunch reports an engineer at a software company saying his company replaced a language model it used as a classifier to review commands for safety with Jev and got better results; the article gives no benchmark or sample size. That is the nearest reported use to this chapter. Separately, it reports TypeSafe’s founder saying he sees users deploying Jev to track agent traces and prevent jailbreaks, and names no customer or result.

My viewMy reading of the figure: scoring a call is half of a gate, and the other half is a decision about the calls it holds back. Refusing stops a risky call, but every fine call it holds is work that did not happen. Asking puts a person on every held call, which is a cost of its own. At the starting costs and the 50% start, refusing totals 16 against 18 for asking, and at each policy’s cheapest threshold asking wins, 9 against 12; with nothing gated the risky calls cost 72. Change the three costs and the order changes. All of it is invented.

My viewThe figure assumes the person is always right and always has time, which a real person may not be. A gate is also one layer: the integration docs say it classifies only the tools you list, and chapter 8 comes back to what a decision node should be allowed to decide.

My viewI would not take a cutoff, or a policy for the held calls, from a vendor’s page, and this chapter does not show that Jev gates well. It shows how the decision is shaped. Putting a cost on your own mistakes and testing the probabilities on your own calls is chapter 6.

Exercise

Decide what happens to a held call.

Raise the cost of a person looking at a held call to its highest. Where is the cheapest threshold for asking now, and does refusing beat it? Then lower it and the cost of a blocked fine call and compare again.
Pick one call in the list that you would never let run whatever its probability. Is that a threshold, or a list of tools the agent should not have?
Write down, for one of your own tools, who would see a held call, how soon, and what the agent does while it waits.

Sources

LangChain docs TypeSafe integration. Integration documentation; read 2026-11-01.
The LangChain classifier, the experimental model-routing and tool-gating middleware, what the gating middleware classifies (only the tools you list) and does with a call it judges risky (an error message instead of running the tool, no approval step, a pointer to human-in-the-loop middleware for that), what is sent to TypeSafe and the warning about secrets. The provider package (langchain-typesafe) may be written by TypeSafe, so I read the page as the integration’s documentation, not as an independent view.
- The tool-gating middleware: the probability a tool call is risky or insufficiently authorised; only listed tools classified; a risky call returns an error message instead of running
- The gating middleware does not ask a human for approval; pair it with human-in-the-loop middleware; no cutoff stated; experimental, APIs may change without notice
- The risk can be redefined with criteria for a yes or no question
- What is sent to TypeSafe: tool call arguments and conversation state; do not put secrets in them unless sending is acceptable
- The gating middleware refuses a risky call and does not ask a person
TypeSafe docs Confidence. The vendor’s documentation; read 2026-11-01.
Confidence as a statistic of the probabilities, no confidence for a yes/no answer, bands of action, what to do at low confidence, thresholds that scale with risk, testing with your own data.
- Noul answers carry no confidence value
- Low confidence: route to a human, ask for clarification or fall back; higher cutoffs for higher-stakes actions; the transfer and balance example
- Routing an unsure answer to a person; cutoffs that scale with risk
TechCrunch A new kind of AI model from a ChatGPT inventor is thrilling developers. Press article, 18 September 2026; read 2026-11-01.
Developers’ early uses (a command-safety review and an email test, both one person’s account; for tracking agent traces and preventing jailbreaks, the founder’s expectation and no reported deployment), one developer’s caution, anecdotal testing, and that outside observers suspect an open-weight base which the article does not confirm.
- One engineer’s account of a command-safety review, anecdotal, no benchmark; the founder’s expectation about tracking agent traces and preventing jailbreaks, no named deployment or results

Each fact is paraphrased from the page named, read on the date shown. TypeSafe’s own pages are marked as the vendor’s claims: they say what the company reports, and I have not tested any of it. Where a sentence is my own reading, it is marked “My view”.