Pranav Srivastava

11 lessons

0/11 done
Lesson 3 of 11·14 min·Intermediate
214 min

Does it really save money?

What you will learn
  • Explain the two sources of saving: cheaper input and no paid output
  • Use cost per correct decision, not cost per token
  • Run the cascade calculator and read its trade-offs
  • List the hidden costs that eat the saving

The short answer

Yes, it can cut the cost of an LLM system, but only in a specific way, and not automatically. It saves LLM tokens you would otherwise have spent on decisions a cheap, fast model can make just as well. It does not save anything on the hard cases, and it adds costs of its own.

Let us see why, from the ground up.

Refresher: what a token is, and who pays for which

A token is a chunk of text, roughly three-quarters of a word. Model providers charge per million tokens, and they charge differently for two things:

  • Input tokens: what you send (the ticket, your instructions)
  • Output tokens: what the model writes back. These usually cost several times more than input tokens, because generating text is the slow part.

Now the key move. An LLM has to write its decision. Jev's decision is a value, and at list price output is free. So one source of the saving is simply that you stop paying for the paragraph.

Where the money goes in one decision

Slide the numbers. With the defaults, the written answer alone is half the LLM bill.

Where the money goes in one decision

LLM decision: $0.003600

■ input $0.001800 · ■ written answer $0.001800 (50% of the cost)

Typed decision: $0.000038

■ input only. Output tokens: free at list price.

The typed decision costs about 95x less per call with these numbers. Two things produce that: a much lower input price, and no paid output. Notice how much of the LLM bill is the written answer alone.

That is the first source of saving: no paid output. The second is a much lower input price. Put together, one decision can be around a hundred times cheaper per call. That is the number that makes headlines.

Why the headline number is not your saving

The vendor says "193.6x faster, 444.6x cheaper". One independent reviewer noted (as did Layer3 Labs) that the comparison model is not named and that, averaged across all eight of TypeSafe's own workflow setups, the figures come to 97.8x faster and 149.2x cheaper. Against Claude Haiku 4.5 on a phishing task, the same reviewer measured 2.9x faster and 12x cheaper. Against a local open model it was slower, and against one DeepSeek setup it cost more.

So the multiple depends on what you compare to. Cheap per call is not the same as cheap per problem solved. One author put it well: the useful metric is cost per solved task, not cost per token.

Try it: the cascade

The way most systems would actually use this is a cascade. Jev looks at every item. The ones it handles confidently are done. The rest go on to a bigger model (or a person). The LLM only sees the hard cases.

A cascade
  every ticket ──► typed decision ──┬── confident ──► act on it            (cheap)
                                    │
                                    └── unsure ────► LLM, or a human       (expensive)

Play with the calculator. Watch two outputs, not one: the monthly bill and the number of wrong decisions.

Do the sum: LLM on everything, or Jev first and the LLM for the rest?

LLM on everything

$3,600 / month

40,000 wrong · $0.0038 per correct decision

Jev first, LLM for the rest

$1,118 / month

54,000 wrong · $0.0012 per correct decision

You save $2,482 a month (69%). It costs you 14,000 more wrong decisions. Is each one worth less than $0.1773?

Defaults are placeholders from public reporting (Jev list price from TypeSafe, LLM rates as quoted in press on 16 Sept 2026). Prices change and TypeSafe says it cannot yet prove its price is sustainable. Use measured numbers from your own traffic.

With the defaults (a million decisions a month, Jev handling 70% alone at 94% accuracy, the LLM at 96%), you should see roughly:

Monthly costWrong decisionsCost per correct one
LLM on everythingabout $3,600about 40,000about $0.0038
Jev first, LLM for the restabout $1,118about 54,000about $0.0012

The cascade saves about two-thirds of the bill. It also makes about 14,000 more wrong decisions. Both facts are true at once. Whether that is a good trade is a business question: is each extra wrong decision worth less than the roughly $0.18 each one saved you? For a low-stakes tag, yes. For a refund decision at a bank, probably not, and you would push the "handled alone" share down until it is.

When it does not save money

Try these in the calculator:

  • Set "share Jev handles alone" to 20%. You still pay Jev to look at everything, and the LLM still sees 80%. The saving nearly vanishes.
  • Drop Jev's accuracy to 80%. The bill falls; the wrong-decision count balloons.
  • Make the LLM prices tiny. If your LLM decision already costs a fraction of a cent, there is little left to save, and the engineering effort dominates.

And some cases the calculator cannot show:

  • Every decision needs a real explanation anyway. You will call the LLM regardless, so Jev is just extra cost.
  • A regex would do. If the decision is "does the text contain one of these ten strings", the cheapest baseline is free, exact and explainable.
  • Volume is small. At Jev's list price a million 900-token decisions costs about $38 a month. One review worked out that 40,000 decisions a month costs about $1.34. If your monthly LLM bill for that decision is $20, you are optimising pennies. The reason to adopt it at low volume is speed or a cleaner output, not money.

The hidden costs

CostWhy it exists
Labelled dataYou need a few hundred examples to test it, and more to calibrate it
Shadow testingRunning beside your current process before trusting it (chapter 6)
Human reviewSomeone has to check the unsure and sampled cases
DecompositionAsking five small questions instead of one is more calls (still cheap, but not free)
EngineeringIntegrating, monitoring, a fallback for when the API is down
Vendor riskA ten-day-old service. A price rise, an outage, or a model version change lands on you
Re-testingSmall wording changes moved a third of answers in one test, so every prompt edit is a release

None of these are reasons not to use it. They are the reasons to do the sum on cost per correct decision, including these, before you claim a saving.

The Jevons twist

The model is named after the economist behind the Jevons paradox: when something gets cheaper, we use more of it. Watch for this in your own system. If a decision costs almost nothing, someone will add fifteen more questions to every ticket, and the total bill can rise even as the unit price falls. That is not a failure; it may be exactly the value you wanted. But budget for it. The Cost and Rate Governor applies here too.

A fair way to claim a saving

  1. Measure your current cost and accuracy per decision. Real traffic, real numbers.
  2. Run the typed-decision model in shadow on the same items.
  3. Measure its accuracy against labels, by confidence band.
  4. Pick the "handle alone" line where the error is acceptable for the stakes.
  5. Then, and only then, compute the saving, with the wrong-decision count next to it.
Chapter summary
  • Two sources of saving: much cheaper input, and no paid output
  • The headline "hundreds of times cheaper" depends on the comparison; independent tests saw 12x down to none
  • A cascade saves LLM tokens on the easy cases only, and can add wrong decisions
  • Judge it by cost per correct decision, including labels, testing, review and vendor risk
  • At low volume the dollar saving is tiny; the case then rests on speed and clean output
Check your understanding
  1. In the calculator, what happens to the saving when Jev handles only 20% alone? Why?
  2. Your current LLM decision costs 0.2 cents and you make 50,000 a month. Is cost the reason to adopt Jev? What might be?
  3. Why is "cost per correct decision" a better measure than "cost per token"?

Sources

All pages read on 25 September 2026. Figures were pulled through a summarising fetch tool, so check any number on the linked page before you quote it.

  • TypeSafe AI, launch post. List price, comparison methodology, and its own stated caveats.
  • The Register, 16 Sept 2026. Press-quoted LLM prices used as calculator placeholders.
  • xbill, independent tests. Speed and cost multiples against Haiku 4.5, local models and DeepSeek; averaged vendor figures.
  • Layer3 Labs, Jev benchmarks. Unnamed comparator; "terrible tradeoff" warning for costly errors.
  • Flowtivity, Laya vs Jev. The 40,000-decisions arithmetic.
  • Flavio Copes, A deep dive into Jev. Cost per solved task.
  • William Stanley Jevons, The Coal Question (1865). The paradox the product name refers to.

Finished this lesson?

Mark it done — your progress is saved automatically.