← Wednesday, September 30, 2026InferenceDeep dive

Jevstiller uses a disagreement bound to route confident labels to a local model

jevstiller.pages.dev 9 min read Editor score 8/10

Jevstiller trains a CPU-local model to match a vendor’s labels on a target share of requests, reporting about 15 ms latency versus 300 ms remotely. Its agreement contract and threshold selection matter for low-latency classification in agent loops.

What: Jevstiller routes classification requests to a local model when it is confident and the request is in-distribution; otherwise, it sends them to Jev.
How: A frozen sentence encoder feeds a logistic regression head; a k-nearest-neighbour OOD scorer and routing policy decide whether the head may answer.
Why it matters: It can answer locally in about 15 ms on CPU while targeting a chosen agreement rate with Jev, useful for latency-sensitive agent loops.

The encoder and OOD scorer inform a router that selects a local answer or Jev. Sketched from the article's own text.

Read on jevstiller.pages.dev → Hacker News · 14 comments

Discussion

Discussion is not switched on yet. Use Note above to keep your own thoughts on this story.

Headline, summary, brief and diagram are written by a model from the source's own text; the link, byline and numbers come from the source. Read the original before relying on a claim.