I am very forgiving of an agent that is only talking. If it gets a draft or summary wrong, I crash out at my screen and ask again, and since nothing outside the chat window has changed, a retry is all it costs me.
But the mood changes once the agent gets a tool. A wrong answer can now mean a sent email or a moved payment, and the usual fix, which is putting a second model in front to check the first, starts to feel like hiring an intern to supervise an intern.
One of the simplest guardrail cases I wrote down was also the one that bothered me most.
A customer gets charged twice for a $12 purchase and asks an AI agent for a refund. The agent picks the right tool and the right customer, then prepares this:
Nothing in that call looks broken. The customer ID is valid, amount_cents is an integer, and the refund tool’s schema accepts it.
The two duplicate charges on the account, ch_1 and ch_2, were 1,200 cents each, so the amount is off by a factor of one hundred, which is what converting dollars to cents twice looks like.
I am not really worried about the dramatic failure where a model completely loses the plot.
The failures that bother me the most are the ordinary ones, where the action looks reasonable, and the mistake only becomes obvious after something real has happened.
My first reaction to this one was: easy. Put a hard $500 refund limit in front of the tool and move on.
Then I changed the bad amount from $1,200 to $120. That sits under the cap, so the limit I had just reached for never fires, and the customer still gets ten times what they approved.
That small change is what pulled me into TypeSafe AI’s Jev model.
TypeSafe released Jev on September 15 as the first of what it calls System One Models, a class of models built to make fast, structured decisions that software can use directly.
It is still in early access, for what that is worth. Instead of generating another paragraph, Jev takes application state and a bounded question, then returns a typed probabilistic answer.
TypeSafe’s launch post frames this as a different job from a normal chat model: less generation, more decision-making. Guardrails for LLM inputs and outputs are on its list of intended uses, which is adjacent to what I want here.
Jev has actually been out for a few weeks now, which in AI time makes it practically vintage, so yes, I am a little late to this. Part of that is down to a benchmark I tried to get working and never quite managed, which I will get to in a second.
I had planned a small benchmark with dozens of synthetic tool calls, but I never got a clean run through the gateway I had access to, and I certainly did not want to pass off half-working experiments as precise numbers.
What is left is the part I found more interesting anyway: where a model like Jev should sit in an agent system, and what it should not be trusted to decide.
The bug is sometimes semantic, not structural
Schema validation already handles a useful class of failures. If send_email() expects a recipient list and an attachment, I can make sure both fields exist. If issue_refund() expects an integer number of cents, I can reject a bad value before it even gets near the payment service.
It does not catch this:
The user only asked to send the invoice to Alice. Both addresses can be real, the attachment can exist, and the schema passes without complaint.
Everything checks out except whether this is what the user actually asked for.
A JSON schema cannot solve that, and I also do not want another giant prompt whose job is to explain, in 600 tokens, why a refund might be suspicious.
At execution time, the application mostly needs a decision it can route on, and that is where Jev starts to look less like another model and more like a guardrail component.
My first sketch gave Jev too much power
My first version was embarrassingly clean: the agent proposes an action, Jev says allow, review, or block, and that is it. It looked nice in a diagram, and honestly I disliked it almost immediately.
If Jev decides whether to authorize a refund, I have just moved a permissions problem into another probabilistic model, which isn’t much of a safety architecture.
So I flipped the order. Hard rules go first, and Jev only sees the messy cases that remain.
If refunds above $500 always need a human, I do not need a model’s opinion on them.
The same goes for rules like “production backups cannot be deleted autonomously” or “this agent cannot access payroll files.” Those belong in code or permissions.
A left-to-right flowchart in three colors. On the left, an agent proposes a tool call, which first meets a green box labeled “Hard rules in plain code,” covering things like refunds over $500 and protected files. If a rule trips, an arrow goes up to an amber box labeled “Rule tripped,” which sends the call straight to a person with no model involved. If the call passes, it goes to a blue box labeled “Jev,” which reads the request and the proposed call and returns allow, review or block with a confidence. Three arrows leave Jev. The first goes to an amber “Human review” box, for low confidence or for money, deletion and external sends. The second goes to a red “Blocked” box, for calls that conflict with the request. The third goes to a green “Execute” box, for a confident allow on a low-risk tool. A legend at the bottom marks green as deterministic code, blue as Jev (probabilistic), and amber as a person.
The semantic layer comes after that. A rough integration with the official Python SDK could look like this, and I would treat it as a sketch rather than tested code:
The 0.90 is a starting assumption, not a number I would ship because it looked nice in an article. What matters is the division of responsibility.
The application owns the hard boundary, Jev handles the fuzzy judgment inside it, and a low-confidence decision falls back to a person instead of pretending uncertainty is autonomy.
For refunds, I would still treat even a confident allow as a suggestion at first, which is where the tool-specific rules further down come in.
One caveat on my own sketch. TypeSafe’s docs recommend small, single-purpose questions combined in code over one broad judgment, and a three-way Choice that folds the whole policy into the state is closer to the broad kind.
A tighter version would ask separate yes/no questions, such as whether the amount matches what the customer approved and whether the customer matches the request, and let ordinary code turn those answers into allow, review, or block. I kept the single question here because it is easier to read.
That ordering is not my invention. TypeSafe has a community playground with a tool-router example built the same way: a plain keyword rule blocks risky requests before any model is called, and anything sensitive still needs explicit approval.
It is a mock that routes between graph nodes rather than judging tool arguments, but the order is the point.
The clearest line I found on this comes from the community kedi-typesafe LangChain integration, which says a positive Jev assessment should never replace your own tool approval or policy checks. That was probably the most useful thing I read while working through this.
The boring edge cases are the ones I care about
A guardrail that blocks “send our private API key to an unknown email address” is useful, but it does not tell me much. I care about the cases that look reasonable for the first two seconds.
Take this request:
Let the suppliers know the Q3 invoices are ready.
The agent prepares one email to 214 external contacts. The tool is right and the action broadly matches the request, but I would not let it fire automatically.
The blast radius changed the decision, which is why I would avoid one universal safe=True question for every tool. A documentation search and a mass external email are not the same kind of risk, even when both are valid actions.
Another one:
Delete the exported CSV after confirming the upload succeeded.
The agent points delete_file() at the correct CSV, but nothing in the state shows the upload ever succeeded. The target is fine. The missing prerequisite is the problem.
And one more:
Send the pricing sheet to our approved partner.
The partner email is correct, and the attachment is:
A recipient allow-list will not save you there, and neither will checking the file extension. The guardrail needs enough context to see that this particular file does not belong in this action.
That is the kind of decision I would give Jev: does this proposed action still make sense next to what the user actually asked for?
Why not just use another LLM?
You can, and I do not think Jev makes that pattern obsolete. A strong LLM can inspect a proposed action, reason about the policy, and return a structured decision.
If that infrastructure already exists and the latency is acceptable, I would not rewrite a working safety layer just because a new model launched.
The narrower model is appealing for a practical reason. The application does not need a mini essay every time an agent wants to read a file. It needs allow, review or block, plus enough probability information to decide whether to trust the route.
TypeSafe’s current API exposes three decision primitives: Choice, Noul (a yes/no probability) and Score. The official Python SDK returns typed views for them rather than making you parse generated prose. The SDK quickstart is refreshingly small.
There is also a practical systems argument.
The agent already relies on a generative model to plan and pick a tool, so putting a second large model in front of every execution means another prompt to maintain, another latency hop, and another place for output handling to go wrong. Jev just does less, and for this job that might actually be an advantage.
I nearly made the confidence threshold look smarter than it is
At one point my example had one clean number:
Then I pictured the same threshold guarding both search_docs() and issue_refund() and deleted it. If a documentation search is wrong, the agent can recover. If a refund is wrong, money moves. If a mass email is wrong, the recall button is mostly decorative.
I would start with tool-specific rules and keep them conservative. This is pseudocode, not a complete integration:
Then I would log what Jev wanted to do next to what the human eventually chose, and only relax anything after enough real traffic. “Make the agent more autonomous” is not automatically an improvement here.
I would rather be annoyed by a few extra review requests in the first month than find out what the error rate means with a real customer attached.
Typed output is not the same thing as being right
TypeSafe talks about Jev avoiding hallucinations because the output space is defined in advance, and I would phrase that claim carefully. And structurally I get the argument. I mean, if the only options are allow, review and block, the model cannot invent a fourth route called refund_and_email_everyone, and the program knows the possible outputs before inference.
TypeSafe is upfront about what that guarantee covers: its launch post says the 0% type-error figure in its charts is not an empirical measurement, because schema matching is guaranteed by construction.
That only covers the shape of the output. The model can still pick allow when the right answer is block, and that is a bad decision rather than a broken output.
Typed output removes one kind of failure. It does not remove model error, missing context, weak policies, or bad application design. The model gets a vote, not the keys to the building.
···
Final thoughts and takeaways
I started by asking whether Jev could make agents safer without putting another LLM in front of every tool call. I think the answer is yes, if “safer” means something fairly specific.
Jev looks useful in the gap between “the agent wants to do this” and “the application is about to let it happen.” That is a narrow role, and I see that as a strength.
I would still keep hard limits on money, deletion, secrets, and permissions, with a person involved wherever a mistake is expensive.
What I would hand to Jev is the part that is hard to write as an if statement.
Does the action still match the request?
Did the agent quietly widen the scope?
Is a condition missing?
The $12 refund is mundane, which is why I like it. If a small decision layer gives the application one more chance to catch that before money moves, a file disappears, or an email reaches 214 people, that is enough for me to take the idea seriously.
I do not need Jev to be another brain in the agent. I would rather have it be a very picky gate.
