Yuval Avidani
Author
A decision model is an AI model that takes in text and a question with a closed set of answers, and returns one choice with a probability, in a single pass, without writing a single word. Since TypeSafe launched Jev on September 15th, this category has exploded: an open-source library called Laya hit 33 thousand stars on GitHub in under a month, Cloudflare released Clef with open weights, and Ollaya lets us run all of this on our own machine, Ollama-style. We already covered the Jev vs. OpenAI race in our DevDay coverage. This time we're rolling up our sleeves and building.
The exercise that'll carry us through this whole piece: a small online store with an inbox that gets a few hundred messages a day. We want every message automatically routed into one of four buckets: billing, shipping, technical, or spam, plus we want to know if it's urgent. We'll build that exact same router three ways: through TypeSafe's API, with Laya on our own machine, and with Ollaya as a local server. At the end, we'll compare speed, price, and what happens when a message lands in Hebrew.
What even is a decision model, and why now
Let's break it down. A regular language model, an LLM, writes an answer token by token (a token is a chunk of a word, the unit the model reads and writes). To classify a message, it's essentially "typing out" the word billing, letter by letter, and we need to parse the text that comes out, hope it didn't tack on a polite explanation, and handle the cases where it returns something we never asked for.
A decision model works differently. At TypeSafe they call it System One, named after "System 1" from Daniel Kahneman's "Thinking, Fast and Slow": the fast, intuitive thinking, as opposed to the slow, deliberate "System 2." The model reads the entire input once, and immediately returns a probability distribution over the answers we defined in advance, so it simply cannot return an answer outside the list. Think of it like the difference between asking an employee to write you an email with their recommendation, versus handing them a form with four checkboxes.
There are three types of questions, and it turns out every tool in this article uses the same names for them:
- A Choice question - picking from a list, for example which bucket a message belongs to.
- A Score question - a rating on a graded scale, for example how angry the customer is.
- A Noul question - the probability of "yes," from 0 to 1, for example whether the message is urgent.
And why should this matter to us? Because a huge chunk of what AI actually does in a real product is exactly this: routing, classification, spam filtering, and guardrails (checks that stop dangerous input or output before it moves forward). According to TypeSafe, Jev takes between 70 and 500 milliseconds end-to-end, and the price is $0.042 per million input tokens, with output completely free.
The obstacle: why not just ask an LLM
The fair question is why not just keep doing what already works. A good language model will classify our messages just fine, and with structured output you can even force it to return valid JSON. That's a perfectly legitimate approach, and in plenty of cases it's good enough.
But in a router, three problems pile up. The first is time: every message becomes a call to a model generating text, and in an app routing every single request, that's noticeable. The second is money: we're paying for output tokens, which are usually a lot more expensive than input tokens. And the third, the one I care about most, is confidence. A language model tells us "billing" in exactly the same tone whether it's completely certain or just guessing. We don't have a single number we can point to and say: "below this, hand it to a human."
A decision model gives us exactly that number. TypeSafe says the model is trained so its probabilities are calibrated: if it says 80% across a thousand messages, roughly 800 of them should actually be correct. They themselves warn that this is measured across large groups, and doesn't guarantee that any single answer is right.
Way one: Jev through the API
The fastest way to get started is TypeSafe's official Python SDK. Install it with pip install typesafe-sdk, drop an API key into the TYPESAFE_API_KEY environment variable, and send the message (called "state") along with your questions to the POST /v1/systemone endpoint. Here's our store's router, based on the example from the official docs:
Pay attention to the confidence line. That's the heart of the whole thing: if the model isn't sure which bucket a message belongs to, we don't guess, we hand it to a human. And notice both questions, the bucket and the urgency, get asked in the same single call, on the same message. Worth knowing: TypeSafe's docs explicitly state that English is the primary training language, and that the model handles other languages "but not at the same level." Meaning, before we trust it with Hebrew inquiries, we absolutely have to test it on our own data.
Way two: Laya on your machine, for free
Now let's pull the router out of the cloud. Laya is an open-source decision model, under an Apache 2.0 license, built by researcher Nandakishor M from ConvAI Innovations. The repo opened on September 18th, three days after Jev, and today it has over 33 thousand stars. Under the hood it's a small encoder: not a model that writes text, but one that only reads and understands, similar to the familiar BERT. The English version weighs in at 421 million parameters, and the multilingual version at 322 million, supporting over 100 languages.
Installation is one line, python -m pip install laya, and it's the exact same router, this time with no key and no account:
The cool part is the Router: it detects the language of the message on its own and sends an English message to the English model, and a Hebrew message to the multilingual one. According to the project's README, a single question takes around 33 milliseconds on a T4 card (an old, widely-available cloud GPU). And there's another perk: the laya-serve command spins up a server that speaks the exact same format as Jev's API, so code we wrote for Jev works against it almost without changes.
Way three: Ollaya, and what's hiding under the hood
If Laya is a library, Ollaya is the tool turning this whole category into an Ollama-like experience. Ollaya is an open-source local server that downloads decision models by name and runs them on our machine, for developers who want routing and classification without sending data out. It supports Laya, Cloudflare's Clef, and over twenty more open models, and speaks TypeSafe's format. The practical upshot: you can point the code from way one at your own machine by changing a single environment variable:
The model the project recommends is winnow:e4b, a fine-tune of Gemma 4 at around 4 billion parameters. According to the README, it hits 0.722 accuracy on a typed-decisions benchmark (Jev scores 0.738 there), and answers five questions in 89 milliseconds on an RTX 4090. And they say it plainly: without an NVIDIA card, start with laya instead, which answers in a fraction of a second even on a regular CPU.
And what's actually happening under the hood? A post called "Jev in 25 Lines of Python" made the rounds in September, and its author, Duarte O. Carmo, calls it a parody himself. But it demonstrates something real: take a small language model (Qwen3-0.6B) through llama.cpp, present it with the options as A, B, and C, and instead of letting it write, read the logits, the raw scores the model assigns to every possible token, just for those three letters. Convert them to probabilities, and that's it. On an email asking for a password on a fake login page, it scored 0.885 for phishing. The difference between this trick and Jev or Laya isn't the idea, it's the training: decision models are specifically trained so their probabilities come out calibrated.
The comparison: speed, price, and Hebrew
Here's the part we built the router three times for. Cloudflare published, in the blog post launching Clef, one measurement comparing everyone on the same test. Median response time (the time that half of all requests are faster than):
In Cloudflare's blog, Jev clocked in at 524 milliseconds, the big Clef at 209, Kev-9B at 51, Clef-flash at 39, and Laya at just 6 milliseconds. They themselves flag the caveat too: Laya is very fast, but it trades away accuracy on benchmarks. And keep in mind this is a competitor's measurement, and that the numbers depend on hardware, network, and the number of questions. In Laya's own README, for instance, they cite independent measurements where Jev answered in 236 to 276 milliseconds.
On price, the order flips a bit. Let's run the numbers for our store: in TypeSafe's official example, a message with three questions consumed 428 input tokens. Round that to 400, assume 10,000 messages a month, that's 4 million tokens. On Jev that comes out to about 17 cents a month. On Clef-flash, priced at $0.038 per million tokens on Workers AI, about 15 cents. On the big Clef, priced at $0.24, about 96 cents. And on Laya or Ollaya on our own machine, zero dollars, aside from electricity and hardware. At the scale of a small store, price barely matters: what actually decides things is speed, privacy, and whether the model understands Hebrew.
And we need to be honest about Hebrew. Laya published a measurement across 51 languages, on an intent-detection task with 20 options, where random guessing gets 5%. The multilingual model hit 37% accuracy in Hebrew. That's seven times better than chance, but very far from something you'd put into production without testing. And our router only has four buckets, so the task is easier, but the rule doesn't change: measure on your own real messages before you trust it.
My take: when yes, and when no
In my view, decision models are one of the most useful tools that came out this year for product builders, precisely because they're modest. They don't replace the language model. They sit in front of it, deciding fast and cheap where each request should go: to regular code, to an expensive language model, or to a human. That's exactly the pattern TypeSafe themselves recommend in their docs, confidence-based routing.
So which way should you pick? If you want to start today without managing servers, Jev through the API. If the data is sensitive, or you want zero marginal cost, Laya or Ollaya on your own machine. And if you're already inside Cloudflare Workers, Clef is a single call away. The nice part is they all speak the same format, so switching between them is basically just changing an address.
And it's important to say the limitations out loud. According to TypeSafe's docs, Jev only accepts text, and in our DevDay coverage we already noted it struggles with arithmetic, counting, and comparing dates, and that Check Point researchers showed you can inject malicious instructions into its input, exactly like with a language model. Laya is also openly upfront that it's weak once you go past 20 options: on a banking benchmark with 77 labels, Jev hit 0.870 and Laya only 0.425. And the promise that "the model can't hallucinate" just means it won't return an answer outside the list, not that it's always right. A decision model can be confidently wrong, which is exactly why our confidence threshold has to be tested on our own data, not copy-pasted from an example.
So, which single decision in your product is currently going through a full language model, and could instead be a 40-millisecond checkbox?
Want to take this further?
The pattern of a fast router that decides when to call the big model, and how to bring it into a real project alongside a coding agent, I break down in depth in the Claude Code Mastery course. And if you want to build automations, agents, and MCP connections with Claude, in Hebrew and without writing much code, you'll find that in the Practical AI with Claude course. And if the idea of running everything on your own machine, with no data going out, speaks to you, there's also the End-to-End Local Models course.
I post these demos, including the router we built here, to YouTube and LinkedIn, and short updates on X, Instagram, and TikTok. My code lives on GitHub.