Perplexity and NVIDIA cut post-training tokens up to 13x with dual-role JOLT

On October 21, Perplexity Research and NVIDIA introduced JOLT (Joint On-policy Learning and Teaching), an approach to model post-training that lets a single language model teach itself while curbing runaway token costs.

Standard post-training for reasoning models largely relies on reinforcement learning with sparse terminal rewards, such as GRPO (Group Relative Policy Optimization). Because the model only discovers if an entire chain of logic succeeded at the very end, training on complex, multi-step agent tasks requires a staggering volume of generated tokens. Traditional distillation solves the sparse feedback problem by having a stronger external teacher evaluate each token, but that requires maintaining two separate models, and an expert solver does not necessarily provide accessible explanations.

JOLT bypasses external teachers by running a single model in two simultaneous roles. The teacher receives privileged data—such as verified answers or execution logs—while the student sees only the raw prompt. To prevent the teacher from providing hints the student cannot possibly parse, the teacher is trained on its own rollouts with Kullback-Leibler regularization anchored to the student's current capabilities. The student then updates via standard reverse-KL distillation on the teacher's per-token evaluations.

According to the authors, combining this mechanism with the student's own centered terminal reward—a setup termed JOLT+—outperformed GRPO on benchmarks spanning from Qwen3-0.6B to Qwen3-30B-A3B. Reported gains included 4.2 percentage points on MATH and 9.9 points on the agent-focused AppWorld benchmark.

The efficiency differential represents the primary practical advantage: on MATH, matching a 63.4% accuracy threshold reportedly took 1.3 million processed tokens under JOLT compared to 8.4 million with GRPO—a 6.5-fold reduction, widening to roughly 13-fold when counting only generated tokens.

These figures come directly from the researchers, with testing limited to relatively small base models and no independent third-party replications yet available. Whether the token savings hold up at frontier parameter scales remains an open question.

Source: habr

Source: Perplexity Blog