Posts tagged with "long-context"

Long-Sequence MoE RL Training: From First Principles to MI300X

A first-principles read of Yan Bai's long-sequence MoE RL optimizations — Path B recompute, linear cross-entropy, FSDP2, chunked expert-parallel overlap — and what each one means on AMD MI300X / MI355X.