Study proposes safety tuning that distinguishes harmful intent from entire topics
A paper by researchers at Multiverse Computing examines how language models can refuse harmful requests without rejecting legitimate questions in the same subject area. The authors argue that topic-based safeguards, such as classifications covering weapons, fraud or self-harm, are too broad for many deployments. Their political example separates factual election information from targeted persuasion or manipulation, which may require different responses.
The study treats safety as a boundary between prompts that should be refused and prompts that should be answered. In experiments with political persuasion, the researchers paired prompts sharing a topic but differing in intent. They also tested an escalating retry process for generating refusal examples, reducing discarded prompts from 19.88% (8,009) to 0.20% (79) and retaining 40,293 harmful training prompts.
On Qwen3-8B, political refusal increased from 9.47% to 84.75%, while the mean unsafe-response rate on HarmBench, StrongREJECT and WildJailbreak fell from 26.26% to 0.14% in the strongest configuration. However, XSTest over-refusal rose from 2.00% to 74.00%. Adding benign boundary examples reduced false refusals on held-out permissible prompts from 32.94% to 4.16%, while refusal of harmful counterparts declined from 91.88% to 87.72%. The authors conclude that both safety and over-refusal must be measured.
Written by Sakhanda Wire AI from reporting by Hugging Face. Facts, quotes and full context are in the original article below.