Kyutai has released Voice of Reason, 2 open-weight speech-to-speech models that solve math problems out loud. Both start from GLM-4-Voice-9B and add supervised fine-tuning (SFT) and reinforcement learning (RL). There is no transcription step and no separate text LLM in the loop. On spoken GSM8K, accuracy climbs from 27.3% for the base model to 77.1%.
Is it deployable? Yes, for self-hosting. Kyutai ran both BF16 checkpoints on a single H100. You also need the GLM-4-Voice repo for its speech tokenizer and decoder. Weights inherit the GLM-4-Voice license, and no Hugging Face inference provider hosts them yet.
Why Speech Models Lag on Math
Cascaded pipelines (speech-to-text, text LLM, text-to-speech) still lead on reasoning. However, each stage adds latency, and the pipeline loses paralinguistic cues like tone. Speech-native models must emit audio at regular intervals to stay interactive. That limits how many hidden reasoning tokens they can afford.
Base GLM-4-Voice scores 27.3% on GSM8K. The earlier STITCH method raised that to 58.7% by adding reasoning chunks. The research team call their work the first application of RL to math reasoning in speech-native models.
How the Training Works
GLM-4-Voice interleaves its output: 13 text tokens, then 26 audio tokens, repeating.
- Stage-1 SFT: Training uses 150,616 problems from Orca-Math. Qwen3-235B rewrote each problem for speech. Kyutai’s DSM TTS then voiced them in many voices. SFT alone lifts GLM-4-Voice from 27.3% to 61.7%.
- Stage-2 RL: For each spoken question, the model samples 4 replies at temperature 0.9. A judge, Qwen3-235B-A22B-2507, scores the decoded text with a binary reward. The judge never sees the reference answer. On 100 hand-checked cases, it agreed with humans 88% of the time.
Rewards are centered within each group, forming a group-relative REINFORCE objective. It is related to GRPO but drops PPO clipping and KL regularization. Training ran on 16 H100 GPUs, with 1,500 RL updates.
2 design choices matter most:
- Temperature correction: Logits are divided by the sampling temperature before the log-softmax in the loss. Without it, GSM8K collapsed from 65.5% to 12.3%.
- Audio-token merging: At each audio position, all audio-vocabulary probabilities are summed into 1 abstract token. The loss asks only whether audio came next, not which audio token. The paper proves this estimator is unbiased and lower-variance under a value-invariance assumption.
Interactive Explainer
‘;d.appendChild(lab);d.appendChild(pr);});
q(‘#kvr-lkey’).style.background=fix?’#7DB7FF’:’#FF6B6B’;
q(‘#kvr-tv’).textContent=”Mismatch (total variation): “+(tv*100).toFixed(1)+’%’;
q(‘#kvr-tnote’).textContent=fix?’Logits are divided by T before the log-softmax in the RL loss, so the gradient is computed for the exact policy that produced the samples.’:’Samples are drawn from softmax(z / T) but scored under softmax(z). The gradient estimator no longer matches the sampler. In the paper this collapsed GSM8K from 65.5% to 12.3%.’;
resize();}
function drawAbl(){var rows=[[‘Default setup’,’merged audio, correction on, group 4′,65.5,’65.5 \u00b1 1.1′,’ours’],[‘Group size 8′,”,67.0,’67.0’,”],[‘Loss on all tokens’,”,64.4,’64.4′,”],[‘Loss on text tokens only’,”,63.8,’63.8′,”],[‘No temperature correction’,”,12.3,’12.3′,fix?”:’hot’]];paint(q(‘#kvr-abl’),rows);}
q(‘#kvr-T’).addEventListener(‘input’,temp);
seg(‘kvr-fix’,function(b){fix=b.getAttribute(‘data-f’)===’1′;temp();drawAbl();});
temp();
/* slide 4: results */
var dec=”paper”;
function drawRes(){var P=dec===’paper’;var rows=[
[‘PersonaPlex’,’8B, full-duplex’,3.2,’3.2′,”],
[‘GLM-4-Voice (base)’,’9B’,27.3,’27.3′,”],
[‘STITCH (Chiang et al.)’,’9B, reasoning chunks’,58.7,’58.7′,”],
[‘Voice of Reason’,’9B, direct answers’,P?65.5:70.3,P?’65.5 \u00b1 1.1′:’70.3′,’ours’],
[‘Voice of Reason (Stitch)’,’9B, silent reasoning’,P?74.8:77.1,P?’74.8 \u00b1 1.1′:’77.1′,’ours’],
‘Top lines, not matched on size or architecture’,
[‘Qwen2.5-Omni’,’7B, text output’,84.7,’84.7′,”],
[‘Qwen3-Omni’,’30B, text output’,94.6,’94.6′,”],
[‘Cascaded ASR, LLM, TTS, ASR’,’Gemma-4-31B-IT + Kokoro’,95.7,’95.7′,”]];paint(q(‘#kvr-res’),rows);}
function paint(el,rows){el.innerHTML=”;var fills=[];rows.forEach(function(r){if(typeof r===’string’){var s=document.createElement(‘div’);s.className=”kvr-sep”;s.textContent=r;el.appendChild(s);return;}
var n=document.createElement(‘div’);n.className=”kvr-rname”;n.innerHTML=r[0]+(r[1]?’‘+r[1]+’‘:”);
var tr=document.createElement(‘div’);tr.className=”kvr-track”;var f=document.createElement(‘div’);f.className=”kvr-fill “+r[4];tr.appendChild(f);
var v=document.createElement(‘div’);v.className=”kvr-val”;v.textContent=r[3]+’%’;var row=document.createElement(‘div’);row.className=”kvr-rrow”;row.appendChild(n);row.appendChild(v);row.appendChild(tr);el.appendChild(row);fills.push([f,r[2]]);});
requestAnimationFrame(function(){requestAnimationFrame(function(){fills.forEach(function(x){x[0].style.width=x[1]+’%’;});});});resize();}
seg(‘kvr-dec’,function(b){dec=b.getAttribute(‘data-d’);drawRes();});
})();
Credit: Source link
