Скачать или смотреть Contextual + Ray: Boosting SFT, RL & Inference at Scale | Ray Summit 2025

At Ray Summit 2025, Fanhai Lu from Contextual AI shares how the company builds enterprise-grade AI agents and applications by leveraging Ray as the backbone for scalable training, reinforcement learning, and low-latency serving across multi-node clusters.

He begins by presenting Contextual AI’s end-to-end architecture—a platform designed to accelerate supervised fine-tuning (SFT), reinforcement learning (RL), and large-scale inference for real-world agentic workloads. The system is optimized for flexibility and performance, enabling rapid iteration on complex agent behaviors.

Fanhai then walks through key architectural components and lessons learned, including:

Asynchronous RL pipelines and large-scale multi-turn training

LoRA-based adaptation for fast specialization

Context, data, and tensor parallelism to scale training and inference efficiently

Autoscaling and cold-start mitigation strategies

Latency-aware routing for low-latency real-time agent serving

Disaggregated prefill and decode, improving throughput under dynamic traffic patterns

He also dives into the operational side of running enterprise AI agents at scale, covering:

Distributed observability—logging, metrics, tracing, and alerting

Multi-host deployment patterns for reliability and redundancy

Techniques for maintaining system resilience, consistency, and service quality in production

Attendees will gain a holistic understanding of how Contextual AI uses Ray to power the entire lifecycle of enterprise AI agents—from large-scale training pipelines to mission-critical, low-latency production serving.

Liked this video? Check out other Ray Summit breakout session recordings    • Ray Summit 2025 - Breakout Sessions  

Subscribe to our YouTube channel to stay up-to-date on the future of AI!    / anyscale  

🔗 Connect with us:
LinkedIn:   / joinanyscale