Jev: The New
Judge in Town

A model built for decisions

On September 15, 2026, TypeSafe founder Diogo Almeida introduced Jev as the company’s first System One model. The official announcement describes an interface built for software: “unstructured state in, typed probabilistic decisions out.”

The developer supplies the available answers. A question might ask which team should handle a ticket, how strongly a passage supports a claim, or whether a message requests a refund. Jev evaluates the input and returns a choice, a score, or a probability, depending on the question type. It accepts text and structured text data; it does not generate explanations or inspect screenshots directly. TypeSafe’s documentation explains these primitives.

The attraction is architectural. Software already knows how to branch on a value. A model that can make a useful judgment at that boundary gives us a small, explicit place to introduce intelligence. We can inspect its inputs, measure its errors, and decide how much authority to give its answer.

That is the part I find brilliant: a focused capability that fits into code we can understand.

People are finding the decision points

Other developers are exploring quite different uses for the same interface.

Browser Use’s Jev Ultrafast turns a page into indexed controls. Jev chooses an operation and a target; a separate generative model supplies text when typing is needed. Its flight-search demonstration shows the appeal of rapid decisions in an interactive loop. The project also documents the limits of its measurements and requires independent verification after a DONE choice.

JevTicketRouter applies the idea to support requests in Persian and English. It asks about category, destination team, priority, sensitive data, and human review. Application rules then determine the handling. This is a useful example of the division of responsibility: the model supplies judgments; explicit rules decide how those judgments affect a ticket.

A community Discord moderation bot uses Jev to assess messages and their context. OpenRouter’s marketplace-moderation tutorial develops a related pattern, showing how typed questions and thresholds become publish, hold, or reject decisions.

These are projects and worked examples, with different maturity levels. I haven’t independently benchmarked them. What interests me is the shared pattern: find a bounded judgment inside a workflow, give it a clear interface, and keep the surrounding policy visible.

Why OpenSidebar gives it a judge’s chair

A browser agent can perform every apparent step and still miss the objective. A form can contain the right values before saving. A confirmation can refer to the wrong record. An executor’s final summary can sound certain while the page says otherwise.

The completion judge has a focused job: assess the claimed outcome against explicit criteria and supplied evidence. OpenSidebar’s Jev adapter asks for one of three judgments for each criterion.

“Unsupported” does essential work here. It preserves the distinction between an outcome shown to be wrong and an outcome we cannot yet establish. That difference should influence recovery: inspect the existing result before repeating an action that might already have succeeded.

This is why Jev’s interface feels appropriate. The question has a defined scope, a small answer space, and evidence against which we can evaluate the result.

From a claim to a checked outcome

Consider a request to add a specific internal note to a customer record. The executor enters the note and clicks Save. Its summary says the task is finished.

Our rubric needs to describe the actual outcome: the intended record contains the requested text, the note has internal visibility, and the change was committed. Evidence might include the observed save confirmation and a readback of the resulting record. A screenshot or page view showing text in the editor alone leaves persistence unresolved.

The implementation sends the claim, evidence, and relevant facts through OpenRouter’s Decisions API. Each criterion becomes a typed question. The instructions require concrete evidence, distinguish editable values from committed changes, and tell the judge to prioritize page observations when they conflict with the executor’s summary.

We validate the returned model identity, required answers, labels, and probability values. We also preserve the distinction between classifier probabilities and a chat model’s self-reported confidence. Jev supplies no prose rationale, so the adapter does not invent one.

In the current adapter, a criterion passes only when Jev selects entailed with at least 0.90 probability. Every required criterion must pass. That is a declared operating threshold, still requiring calibration for our workload. A failed judgment can send the workflow back to inspect missing evidence.

The scope matters: Jev is integrated as an explicitly selectable judge and is used in our evaluation work. It is not the general default judge. The current transport requires a direct OpenRouter credential; cloud Decisions transport is not configured. This is an account of what we have built and tested so far.

A promising small experiment

Our October 5 pilot compared three judges on the same twelve synthetic cases. Here is the recorded result.

We have left comparative latency out of this table. The GPT-OSS calls used six unpinned providers, none of them Groq or Cerebras, and reasoning effort was not explicitly set. That run cannot tell us how Jev compares with GPT-OSS on a fast, deliberately selected endpoint. Groq and Cerebras document high-speed GPT-OSS inference. Their output-throughput figures are a different metric from our complete-request timings; a fair comparison needs measured end-to-end latency on pinned endpoints.

The accuracy columns also reflect the pilot’s original adapter. GPT-OSS’s recorded false rejection involved a criterion-ID mismatch: the raw answer supported the outcome, but its renamed criterion did not satisfy the adapter. It should not be read as a purely semantic judgment error.

Jev’s median complete-request time in this pilot was 271 ms, including local bookkeeping. That was encouraging. A judge sits on the path between doing the work and reporting the result; its delay is something the user experiences. A fast judgment could make additional verification affordable in more places.

But this was a smoke test: twelve simple synthetic cases, only two positive cases, and six held-out cases using the same template in a different workflow. The 0.90 threshold was declared in advance, not calibrated for production. Zero false accepts in this sample does not establish a dependable false-accept rate. These results justified more investigation; they did not justify removing safeguards or changing defaults.

There is useful outside evidence too. An independent preprint evaluating Jev across 37 datasets reports strong results on many classification tasks, while also finding difficulty with some rubric-based judgments and sensitivity to probability thresholds. That is relevant context for our work. It does not measure OpenSidebar’s completion decisions.

The judge needs good evidence

One of our most useful findings came from a CRM note task. An earlier evaluator accepted a marker meaning a note had been added. It did not establish that the note contained the requested text or had the correct internal visibility.

That made the disagreement easy to misread. A stricter judge could look obstructive beside a green evaluator result, even though the evaluator had not checked the whole objective. The repair was to improve the evaluator and the preserved evidence. Lowering the bar would have hidden the problem.

In the repaired workflow, the check covered the saved text and visibility. A separate generic project-update case also verified persistence after a reload. Both were development cases, so their success still leaves broader coverage to establish.

The definition of success is part of the system you have to test.

The same discipline applies to Jev. Typed output gives the software a predictable answer shape. Whether the selected answer is correct remains an empirical question. TypeSafe’s own documentation makes this clear: calibration is measured across groups of predictions and does not guarantee an individual answer.

We need to measure false acceptance, unnecessary rejection, ambiguous evidence, and unavailable judgments. We also need to distinguish a model’s verdict from a transport failure. The quality of the entire decision path matters to the person waiting for a trustworthy answer.

Intelligence that fits into software

I am excited about Jev because it gives a familiar engineering idea a useful new capability: break a system into responsibilities, give each responsibility a clear contract, and measure how well it performs.

For OpenSidebar, the next questions are practical. How does the judge behave across more real browser workflows? Where should thresholds sit when false acceptance is especially costly? How much evidence is enough, and which observations are we still failing to preserve?

That is how I want to earn confidence in this approach. The idea is brilliant. Its place in the product should grow with the evidence.

A fast judge makes verification more practical.
Good evidence makes its verdict worth trusting.