2-D Parallelism using DistributedTensor and PyTorch DistributedTensor

Описание к видео 2-D Parallelism using DistributedTensor and PyTorch DistributedTensor

PyTorch 2.0 Q&A:
🗓️ March 1
⏰ 11am PT
✅ Register: https://community.linuxfoundation.org...

Introduction to 2-D Parallelism (FSDP + Tensor Parallel) to train large scale ViT models and Introduction to PyTorch DistributedTensor, a fundamental tensor level primitives that expresses tensor distribution & computation across devices/hosts.

Join Wanchao Liang & Junjie Wang
Host DA: Shashank Prasanna

Learn more in advance: https://docs.google.com/presentation/...

Комментарии

Информация по комментариям в разработке