https://sky-2002.github.io/tags/pass@k/Recent content in Pass@K on Aakash ThatteHugoen-usSun, 30 Aug 2026 00:00:00 +0000https://sky-2002.github.io/posts/2026-08-30-tiny-rlvr-01/Sun, 30 Aug 2026 00:00:00 +0000https://sky-2002.github.io/posts/2026-08-30-tiny-rlvr-01/<p>I have been following the discourse around RL, hearing a lot of terminology (like async RL, sample efficiency etc) and how people are using it to post-train models, and I did RL on smaller models like Qwen by following the docs of trl, unsloth. I could see the rewards going up, sometimes wiggling, sometimes 0, at times answer still being wrong at the end of training. I knew the math, the code, but I wanted to go deeper and had a lot of questions, like: <strong>what exactly changes in a model&rsquo;s behaviour after RL?</strong> <strong>does it depend on how the pretraining was done?</strong> <strong>how stale can the rollouts be?</strong> - and many more. So I did some experiments and now writing this post to present those.</p>