Miles opens PyTorch-native RL training
RadixArk's Miles is an open-source, PyTorch-native stack for large-scale LLM RL post-training, pairing SGLang rollouts with Megatron-LM training over Ray.
RadixArk introduced Miles on the PyTorch blog on June 30: an open-source, PyTorch-native framework for reinforcement learning post-training of large language models at cluster scale. It combines SGLang for high-throughput rollout generation, Megatron-LM for distributed training, and Ray for orchestration, with PyTorch supplying the model and autograd layer underneath.
The feature list reads like a checklist of what made RL post-training painful. Rollout and training can run disaggregated or colocated, with a fully asynchronous mode. Weights synchronize over NCCL and RDMA channels. A MoE-aware routing replay keeps expert routing aligned across the rollout-training boundary, one of the classic silent-corruption bugs in mixture-of-experts RL. Precision support spans BF16, FP8, MXFP8, and INT4-QAT, LoRA works on both sides, and the core stays small: rewards, losses, sample filters, and training hooks are plain Python extension points, while placement, sync, and fault tolerance stay out of user code.
Supported models include DeepSeek-V4, GLM-5, and Qwen3.5 on NVIDIA Hopper and Blackwell hardware, which says clearly who this is for: teams tuning open models with RL rather than training from scratch.
The open-model flywheel needs exactly this layer. Models like Meituan’s LongCat-2.0 and the Ornith coding family arrive as raw weights; stacks like Miles are how they get shaped into specialists, and hosts like Together AI are where the results end up served.