← Reading list
June 26, 2026

ReNIO: Reweighting Negative Trajectory Importance for LLM On-Policy Distillation

Authors
Chen Lin, Kedi Chen, Wei Zhang
Venue
arXiv 2606.23104
Link
Open →
Tags
llmdistillationrlreasoning

Core idea

Incorrect traces preserve exploratory behavior near the model’s capability boundary, while correct traces tend to be shorter and less reflective. ReNIO uses the student/teacher prefix probability ratio to find pivotal tokens leading to wrong reasoning and weights those trajectories more — keeping OPD’s prefix-training advantage (no full rollouts needed).