Скачать или смотреть Rob de Wit-Liezenga - Scaling Python to thousands of nodes with Ray - PyData Eindhoven 2025

Rob de Wit-Liezenga - Scaling Python to thousands of nodes with Ray - PyData Eindhoven 2025
https://pydata.org/eindhoven2025

Python is the language of choice for anything to do with AI and ML. While that has made it easy to write code for one machine, it's much more difficult to run workloads across clusters of thousands of nodes. Ray allows you to do just that. I'll demonstrate how to implement this open source tool with a few lines of code. As a demo project, I'll show how I built a RAG for the Wheel of Time series.

DESCRIPTION

I don’t read as much as I used to. As a result, ploughing through the Wheel of Time series by Robert Jordan has become a multi-year project for me. With 2787 named characters across 14 books, I often have to search on wikis for what happened six books ago. And who was that one minor character again…?

With AI we can do better. What if I fine-tuned an LLM with RAG to help me keep track of the entire saga? I could ask questions and it would give me spoiler-free answers… ✨

Whipping up a few lines of Python to train and tune models is easier than ever. But while executing scripts on a laptop or VM is trivial, it becomes much more difficult when you want to parallellize such a workload across an entire cluster. That’s where Ray comes in. With a few lines of code, we can modify our code to distribute our model training to tens or hundreds of machines. If you ever wondered how you companies like OpenAI train foundational models: this is how.

In this talk, I’ll cover Ray’s core concepts and show how to bring it in action. I’ll introduce Ray tasks and actors, and show how to set up large-scale pipelines with Ray Data. As a demo project, I’ll finetune an LLM to help me with my Wheel of Time read-through. And I’ll show how can scale your workloads from a single machine to thousands of nodes.

AGENDA

Introduction (2m)
Problem (3m)
Ray explained (10m)
Demo project (10m)
Conclusion and takeaways (5m)
KEY TAKEAWAYS

For AI workloads we need frameworks that are hardware-agnostic and work on both CPUs and GPUs
With Ray tasks and actors, we can achieve distributed computing with just a few decorators
Automatic up- and downscaling is a must if you want to keep your cloud bill in check
While a literary masterpiece, The Wheel of Time has too many characters and some pacing issues


www.pydata.org

PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for data management, processing, analytics, and visualization. PyData communities approach data science using many languages, including (but not limited to) Python, Julia, and R.

PyData conferences aim to be accessible and community-driven, with novice to advanced level presentations. PyData tutorials and talks bring attendees the latest project features along with cutting-edge use cases.

00:00 Welcome!
00:10 Help us add time stamps or captions to this video! See the description for details.

Want to help add timestamps to our YouTube videos to help with discoverability? Find out more here: https://github.com/numfocus/YouTubeVi...