RadixArk's Miles: Open-Source RL Post-Training for LLMs

RadixArk's "Miles": An Open-Source Lifeline for Cost-Prohibitive LLM Post-Training

On July 1, 2026, RadixArk released "Miles," an open-source framework designed to dismantle the steep computational barriers that have historically locked enterprise-level reinforcement learning (RL) post-training behind exorbitant budgets. By unifying SGLang, NVIDIA Megatron-LM, and Ray, Miles promises a pluggable, fault-tolerant stack that slashes the infrastructure tax of scaling frontier LLMs. For an industry where 45.5% of AI decision-makers cite high costs as their primary barrier to adoption—and where the AI platforms market is projected to surge from $109.9B in 2025 to $181.3B in 2026—this release is arguably the most directly actionable tool for production teams in a year’s memory.

What is it? Miles is a unified, small-footprint framework for the RL phase of LLM training—typically the most resource-intensive stage—after a model has been pre-trained. It abstracts away the combinatorial nightmare of orchestrating rollout servers, distributed training clusters, and high-speed networking. Why does it matter? Because the process of aligning a model through RL (e.g., RLHF) currently requires a massive DevOps overhead that few organizations can justify. Miles collapses that overhead into a single PyTorch-native interface.

The Architecture: A Trinity of Battle-Tested Foundations

Miles does not reinvent the wheel; it bundles three proven open-source components with a thin, intelligent orchestration layer. RadixArk engineers focused on integration rather than innovation, solving the actual pain point of teams that struggle to stitch these tools together themselves. The stack is composed of:

• SGLang for high-throughput model rollout and inference during RL trials.
• NVIDIA Megatron-LM for distributed training at scale, leveraging tensor and pipeline parallelism.
• Ray for distributed workload scheduling, fault tolerance, and cluster management.

The Pluggable "Trainer" Interface

At the heart of Miles is a small, PyTorch-native trainer class that serves as the single entry point for the entire RL pipeline. Developers only need to implement a handful of hooks—rollout loop, reward computation, and policy update—while the framework handles data sharding, gradient accumulation, and checkpointing. This eliminates the months-long engineering time typically needed to build a stable RL training loop from scratch.

Key Technical Optimizations That Cut Costs

RadixArk's engineers implemented three concrete optimizations directly responsible for reducing computational expenditure by an order of magnitude during early tests:

1. Unified low-precision recipes: Miles automatically manages precision across the entire pipeline—rollout, training, and synchronization—using FP8 wherever possible, with fallback to BF16 for critical gradients. This reduces memory footprint by up to 50% without sacrificing model quality.
2. Mixture-of-Experts (MoE)-aware alignment: The framework intelligently routes tokens to the correct expert nodes during rollout and training, preventing the "expert imbalance" that cripples naive implementations. It synchronizes expert weights via fast NVIDIA NCCL/RDMA with zero-copy memory transfers, reducing inter-node latency by roughly 40% compared to standard NCCL collectives.
3. Built-in fault tolerance and observability: Ray's native error recovery doubles as an operational cost-saver. When a node fails mid-training—common in large clusters—Miles automatically redistributes the workload to a spare node without discarding progress, reclaiming the wasted compute that would otherwise be lost to manual restarts.

Practical Impact: Lowering the Barrier to LLM Fine-Tuning

The most significant impact of Miles is its role in bridging the gap between research and production. Previously, RL post-training for models like LLaMA-3 or Qwen required a dedicated team of distributed-systems engineers and a GPU cluster valued at several million dollars. Miles reduces technical friction to a set of configuration files, enabling teams with smaller compute budgets to experiment with agentic workflows and fine-grained behavioral control.

Accelerating Production-Ready Agentic Workflows

Because Miles handles the heavy lifting of rollout scaling and reward aggregation, it allows researchers to focus on reward shaping—the secret sauce behind capable agents. Whether teaching an LLM to use external APIs, compose multi-step tool calls, or execute code reliably, the ability to iterate quickly on RL policies becomes a competitive advantage. The framework's compatibility with SGLang ensures low-latency inference, a requirement for online learning scenarios where the model interacts with live systems.

With the AI market projected to compound at 28.7% CAGR through 2030, the pressure to deliver production-grade LLM capabilities is immense. Miles directly attacks the top barrier—cost—by providing a free, open-source solution that renders obsolete the need for proprietary RL stacks. For any team serious about deploying custom-aligned LLMs, it is no longer a question of whether to use RL post-training, but how quickly they can adopt Miles to do so.