Accelerate Transformer training with AWS Trainium

In this video, I show you how to accelerate Transformer training with AWS Trainium, a new custom chip designed by AWS.

First, I walk you through the setup of an Amazon EC2 trn1.32xlarge instance, equipped with 16 Trainium chips. Then, I run a natural language processing job where I adapt existing Transformer training code for Trainium, accelerating a BERT model to classify the Yelp review datatset. Finally, I run the job on 1, 8, and 32 Neuron cores.

⭐️⭐️⭐️ Don't forget to subscribe to be notified of future videos ⭐️⭐️⭐️
⭐️⭐️⭐️ Want to buy me a coffee? I can always use more :) https://www.buymeacoffee.com/julsimon ⭐️⭐️⭐️

AWS Trainium: https://aws.amazon.com/ec2/instance-t...
AWS Neuron SDK documentation: https://awsdocs-neuron.readthedocs-ho...
AWS Neuron SDK samples: https://github.com/aws-neuron/aws-neu...
Hugging Face tutorial: https://huggingface.co/docs/transform...
Setup steps and code: https://gitlab.com/juliensimon/huggin...

Interested in hardware acceleration? Check out my other videos :
Habana Gaudi:    • Accelerate Transformer training with ...  
Graphcore:    • Accelerate Transformer training with ...  
Trainium on SageMaker:    • Accelerate Transformers on Amazon Sag...