AnyJev open-sources training-free calibration for open LLM decisions
Nokia’s applied research team has released AnyJev, an Apache-2.0 Python library that converts an open large language model into a typed decision model without additional training. It is designed for tasks where a system selects one answer from a fixed set rather than generating prose. AnyJev is available through PyPI and provides Hugging Face Transformers and vLLM backends with shared-prefix scoring.
The library supports multiple-choice, yes-or-no and ordered-score questions. Its L0 mode presents answer choices in cyclic rotations, combining the results to reduce position bias, and applies a running prior correction after eight items. L1 adds temperature scaling based on 100 to 500 labeled examples. Neither mode generates or parses an answer; the decision probability comes from the model’s next-token distribution.
In a BANKING77 test using Qwen3-8B and 20 choices, the option-reversal flip rate fell from 0.230 with raw logits to 0.073 with L0. Accuracy increased from 0.747 to 0.803, while L1 reduced expected calibration error from 0.240 to 0.095. The share of cases automatically decidable at 5% error rose from 7.7% to 52.0% with L1. Nokia said it also observed promising results on an internal routing task.
Written by Sakhanda Wire AI from reporting by MarkTechPost. Facts, quotes and full context are in the original article below.