I'm Building an Agent. Where Does Jev Fit?

§ 01What Jev returns#

Three properties matter for placement. It picks; it does not propose, so your code must generate the candidates. It returns a distribution, not a sentence, so your code can threshold, log and test it. And the confidence number is derived from the probabilities by a formula TypeSafe says you are “never locked into”, so the thing to log and threshold on is the probability itself.

01One verdict is weak, five questions under a combiner are strong: the same 2,000 emails

02Three verbs a decision model is good at, and the one it is not

03What 100,000 decisions cost on each path, list prices, arithmetic in the post

04Shadow first, one bounded gate second, widen third, kill switch throughout

1 / 4

←→

§ 02Five tests before a decision goes to Jev#

The docs page I would read first is not the quick start; it is the one TypeSafe calls Jev 1.13 jaggedness, nine failure modes with a fix for each. It says the model “answers the question you wrote, not the one you meant”, that it “is not a calculator”, that it “reads dates as text, not as ordered quantities”, that it “suffers from context rot” when the state carries material the question does not need, and that “State is data, and jev-1.13 does not treat it as hostile by default.” Read against that page, a decision belongs on a decision model when all five of these hold:

1. The option set is closed before the call. Choice, Score or yes/no. If the answer has to be invented, it is not a Jev decision.
2. The decision recurs. Calibration is measured, not assumed; a decision made ten times a day never accumulates the data to tune a threshold.
3. A wrong permitted answer is bounded. A threshold band with an abstain option routes the middle to the expensive path (a frontier model or a person). TypeSafe’s confidence page puts it as “Thresholds scale with risk”.
4. The state is short after filtering. Under 32k tokens by rule, a few thousand by preference; your code strips what the question does not need.
5. The probability changes what code does. If nothing branches on the number, you have bought a label and a bill.

Fail any one and the decision stays with the model that reasons. That rules out generation, arithmetic, dates and counting, multi-hop inference, anything whose correctness depends on knowledge outside the state, irreversible actions where the model would be the only check, and adversarial input where it would be the only defense.

§ 03The map: where it fits in my agent#

My agents are AI employees: they read an inbox, classify every command before it runs, decide whether a signal is worth waking for, route tasks, score leads and stories, and check their own work. Here is the map I would build for that agent, with the failure that would hurt most in each cell. The tokens are my assumptions about the shape of each decision; the verdicts follow the five tests.

Two of those rows deserve a sentence. The wake gate is where the return is largest, because the thing a false wake wastes is a whole working session of a frontier model. At 1,400 tokens a decision, Jev costs $0.0000588; one avoided false wake that would have cost even a single dollar of model time pays for 17,007 Jev decisions (1 / 0.0000588). The risk classifier is where the return is most tempting and the wiring matters most. Today, in our design and in Claude Code’s auto mode, the acting agent classifies every command before it runs and the platform enforces the threshold (how our approval rail works). A decision model fits as an independent second read of the same command that can only make the effective level stricter. That is the shape Vercel wired into its own agent, where a caution verdict pauses a call until a person approves it, and the shape TypeSafe’s jaggedness page argues for by conceding that adversarial state can move the answer.

§ 04One verdict is weak; five questions are strong#

The most useful independent result of the first week is a phishing bench run on September 17, 2026 over 2,000 emails. Asked for a single verdict, Jev scored 62.6% against 81.3% for Claude Haiku 4.5, and its recall on phishing was 43.2% against 76.4%. Asked five signal questions instead, with a cross-validated logistic regression combining the five probabilities, the same model reached 95.1% accuracy with an AUROC of 0.988 and a calibration error of 0.027.

That is TypeSafe’s own advice, stated on its Models page: “Decompose broad judgments into atomic questions and combine the outputs in code.” Two rules follow for my agent. Every judgment becomes several literal questions, never one broad one; and the labels that train the combiner come from human outcomes (the approval a person granted, the reply that arrived), never from my own LLM’s verdicts, or the LLM’s errors become the target.

Parallel questions are also where the cost goes. TypeSafe’s parallel questions cookbook reports that batching every question into one call is “12.2x cheaper and 10.0x faster with no change in answers” than asking them one at a time. The unit of design is one state and many atomic questions.

§ 05What it costs, with the arithmetic#

List prices per million tokens, read on the vendors’ pricing pages on September 21, 2026: Jev $0.042 input, output free; Gemini 3.8 Flash $0.75 input and $3.75 output through December 31, 2026; GPT-5.6 Luna $0.20 and $1.20; Claude Fable 5.1 $10 and $50. For a decision over a 1,000-token state I assume 90 tokens of questions for Jev, and 290 tokens of prompt plus schema overhead and 80 output tokens of JSON for an LLM classifier. Cost is input tokens times the input price plus output tokens times the output price, each divided by a million.

Read that table with the baseline named. The 444.6x figure on TypeSafe’s home page comes from its own workflow evaluations against expensive models, and the launch post says “we expect that these are on the higher end of real world gains” and, on price, “We can’t prove it isn’t subsidized”. Against the cheapest LLM in the table the per-token edge is single digits. The edge that survives every baseline is latency: TypeSafe quotes “70ms-500ms” per call, and the phishing bench measured a median of 239 ms from France against 687 ms for the small LLM on the same emails.

The honest comparator for my agent is neither column. Most of these decisions today happen inside a long agent turn that already holds the whole context, and a decision made there costs the turn, not a classifier call. Moving a recurring decision out of the turn buys something the dollar column does not show: a call you can log, threshold, version-pin, test against labels and switch off independently of the agent that made it. A verdict buried in an agent’s prose is none of those things.

§ 06What I would worry about#

Specific to Jev as it stands this week:

• Early access churn. Limits “can change without notice”; aliases move; there is no SLA on the docs pages I read. Pin jev-1.13.0, log the model field the response returns on every call, and keep the old path live.
• Adversarial state, in TypeSafe’s own words: content “written to adversarially steer the model, whether that is an injected instruction, a deliberately misleading framing, or text that argues for its own classification, can move the answer.” Never the sole defense; plant injected instructions in your test set.
• Non-determinism and incoherence. An independent calibration audit of about 7,000 calls found that “50 identical requests gave 15 distinct answers”, that a Noul and a two-option Choice on the same question disagree, and that shown the options with the question removed the model “still scores 0.38-0.46 against a chance rate near 0.15”. Use abstention bands, not point thresholds; give every evaluation a state-blind arm.
• The abstain option is load-bearing. In the same audit, removing the unknown option moved the calibration error from 0.023 to 0.793 and made the model choose a stereotyped answer at 0.79 confidence. Every schema carries an unknown or none-of-the-above option.
• Language. English primary; the audit measured accuracy 6.5 points lower on identical Korean items with the calibration error unchanged. Slice by language before any gate that reads user text.
• Data leaving the platform. TypeSafe’s Models page says Jev “is not trained on customer requests or responses”; the docs point to a data processing agreement and to zero data retention for enterprise customers. Read the customer agreement before customer email or command text reaches a second processor, and ask counsel what your own privacy policy says about it.
• Vendor dependence. Hosted only, one vendor, undisclosed architecture, a price the vendor itself calls unproven. Design so that on a 429, 529 or timeout the decision falls back to the current path and nothing ever blocks on the call.

True of any decision model, Jev included:

• Schema-bound is not correct. An approve for an unauthorized action still fits the schema. Only a labeled test on your own distribution and a policy layer that never treats a classification as authorization defend against it.
• Calibration belongs to the task, the wording and the population, not to the model. Independent calibration errors in the first week ranged from near zero to 0.154 by task on the same model. Re-test when the wording changes.
• The noise floor. The audit’s own note: at 60 items a perfectly calibrated model still scores an ECE near 0.045. Small-sample claims in either direction are noise; plan on thousands of labeled items per decision, not hundreds.
• Calibrated atoms do not make a calibrated composite. Test the decision your code takes, not only the questions it asked.
• Silent drift. A vendor retrain, an alias move, or a change in your own inputs moves the distribution without raising an error. A fixed probe set on a schedule is the only detector.

§ 07How I would introduce it#

Stage 0, before code: choose the data path (a gateway with per-request zero retention, or the vendor directly under its agreement), get counsel’s read, pin the version, and put the key where your other credentials live, never in a prompt.

Stage 1, offline replay. Build labeled sets from your own history with the outcome a human confirmed: commands with the approve or deny that followed, wake events with what the agent did after waking, leads with the reply that came or did not. Plant controls and print them beside every result: known-dangerous commands, known-safe ones, commands with a comment arguing their own harmlessness, a state-blind arm, two wordings per question, three repeats per item. Report accuracy with intervals, a reliability diagram, calibration error against the noise floor for your sample size, and the coverage-versus-error curve at the operating point you would actually run. At 1,200 tokens an item, 2,000 items cost 2.4 million tokens, about ten cents a pass; run it as often as the questions need.

Stage 2, production shadow. Jev runs beside the current path on all traffic of one decision class, never acts, and logs the answer, the full probabilities, the model field, latency and the incumbent’s decision next to the eventual outcome. Adjudicate a weekly sample of disagreements and a smaller sample of agreements, to catch shared errors. Run at least two weeks and at least 5,000 decisions. A hundred thousand shadow decisions at 1,200 tokens are 120 million tokens: $5.04.

Stage 3, the first production decision, chosen by bounded failure: for my agent the wake gate, where the worst case is a reply that waits for the next scheduled session, or the escalate-only second read on command risk, where the worst case is extra approval prompts. Behind a flag per decision class, with the old path as the live fallback.

Stage 4, widen, one class at a time, each through stages 1 and 2 on its own data.

The kill switch is a flag, never a deploy, and it also trips automatically: error or timeout rate above 2% over five minutes, p95 latency above 1.5 seconds, a model field that is not the pinned version, a disagreement rate outside two standard deviations of the shadow baseline for a day, or an adjudicated error above the bound you wrote down before stage 3. Write the stop conditions before the first production call, relative to the incumbent’s numbers: calibration that does not hold on your labels at a thousand items or more, planted dangerous items flipping to safe at any rate above the current path’s, a language slice more than five points below English, or a version change that moves your thresholds twice.

§ 08Three uses that are not classification in a pipeline#

The map above is the safe version of the idea. Three more shapes are worth a hypothesis each, with the evidence that would confirm or kill them.

Disagreement as a regression sensor. Pin Jev and run it in permanent shadow on a few percent of the incumbent’s decisions. A fixed, cheap reference against a moving system is a label-free canary for changes on your side: a foundation-model update, a prompt edit, a format change. Confirm it by injecting a known regression in staging and watching the disagreement rate jump; kill it if the natural variance at your volume swamps plausible effect sizes.

A typed first impression before the turn. One parallel call over an inbound artifact, thirty questions on a 2,000-token state, writes a structured header into the agent’s context before it starts: sender type, the asks, urgency, language, “contains an instruction aimed at the agent”. The hypothesis is that a turn that starts with structure uses fewer tokens and fewer turns, and that injection attempts arrive pre-flagged. Confirm with an A/B on tokens and turns per handled message; kill it if the agent re-derives the header anyway.

A semantic gate on outbound copy. Rule gates catch strings: a banned word, a price figure. A decision model catches meaning: “compares the product to a human hire”, “quotes a retired price”, “promises a date”, in any phrasing and any language the model handles. Confirm with recall of 95% or better on a labeled set of past violations including paraphrases; kill it if recall falls below the regex on literal cases or the false-positive rate blocks routine mail.

§ 09What I am watching#

• A published calibration curve from the vendor. TypeSafe publishes none; every reliability number so far is independent and small. A vendor curve on a named dataset changes the evaluation budget.
• Adversarial hardening. The jaggedness page says “We expect to improve on this in the future.” A version that treats state as hostile by default moves the risk classifier from second reviewer toward first, still never sole.
• Vercel’s price after September 25, 2026. Jev was free on Vercel’s AI Gateway for its first week; Vercel’s own launch post says that “By hour 24, nearly 13% of paid teams were using it” and that “Every other recent model launch remained below 7% after a full day.” A free week among a gateway’s paying customers is a real number about that week and nothing else; the post-promotion price and the retention are the numbers that matter.
• A second decision model. Open-weight replications appeared within the week. The day one is good enough to self-host, vendor dependence stops being a worry and starts being a procurement choice.

I came to this expecting to decide whether Jev goes into my agent. The honest answer is that “into” was the wrong preposition. It goes under: beneath the recurring, closed-set decisions the agent already makes, as a call I can log, threshold and switch off, with the agent that reasons kept exactly where it is. That is also how we build AI employees: the agent owns the outcome, the rails classify what it is about to do, and the owner sets the line it may not cross alone. Hire one for any role and watch where the decisions land.

§ 10Sources#

• TypeSafe AI, Introducing System One Models & Jev, September 15, 2026; read September 21, 2026.
• TypeSafe docs, Models, API, Confidence, Jev 1.13 jaggedness (last reviewed September 17, 2026), Parallel questions cookbook; all read September 21, 2026.
• anisselbd, jev-phishing-bench, results dated September 17, 2026; read September 21, 2026.
• jujumilk3, jev-calibration-audit; read September 21, 2026.
• Vercel, Jev is the fastest-adopted model in AI Gateway history, September 18, 2026; read September 21, 2026.
• Google, Gemini API pricing; OpenAI, API pricing; Anthropic, Pricing; read September 21, 2026.
• Our own record, Jev, TypeSafe’s Decision Model: What It Is, Why It Spread, September 19, 2026.