Скачать или смотреть Exploiting Parallelism in Large Scale DL Model Training: From Chips to Systems to Algorithms

We live in a world where hyperscale systems for machine intelligence are increasingly being used to solve complex problems ranging from natural language processing to computer vision to molecular modeling, drug discovery, climate modeling and recommendation systems. Beyond images and text, graphs are fast emerging as a key datatype that AI models are being used to process. A convergence of breakthrough research in machine learning models and algorithms, increased accessibility to hardware systems at cloud scale for research and thriving software ecosystems are paving the way for an exponential increase in model sizes. There has also been a sharp uptick in the range of problems that AI is addressing and the rate of innovation in emerging model architectures (like GNNs). Effective parallel processing and model decomposition techniques and large clusters of accelerators will be required to train these models of the future economically. Our network disaggregated architecture uniquely positions us to build highly scalable systems (IPU-PODs) with thousands of accelerators aimed at exploiting various dimensions of parallelism.

Speaker: Saurabh Kulkarni, VP and GM, Graphcore (North America)
Webinar Date: April 4, 2022