The GPU evolution: from simple graphics to AI brains

This video dives into the history of GPUs, from when they powered simple graphics to shader languages and custom rendering, and the recent history of using GPUs as accelerators. Cryptocurrencies and AI have both being using GPUs to perform their calculations (mining, and training and inference). But why is a GPU an effective tool in these very different domains?

Apparently, I had a lot to say about GPUs because this is my longest video yet. We discuss some early DOS games I made. Later we discuss my attempts to build a local GPU training cluster, the reasons that GPUs are so expensive, and NVidia's marketing strategies. We also touch on the only other prominent accelerator for AI, namely Google's TPUs.

If you find any part of this video too technical, I suggest fast forwarding to the next part. The three major parts cover different types of content.

Thanks for watching!

#gpu #ai #nvidia


0:00 Intro
0:16 Contents
0:23 Part 1: How GPUs evolved
0:45 VGA standard graphics
1:07 How monitors draw pixels from memory
1:52 Vertical sync to avoid tearing
2:26 Double buffering definition
3:30 Example: DOS games, star cluster
4:05 Fixed pipeline 3D rendering
4:57 Programmable pipeline (shaders)
5:44 OpenGL and DirectX
6:01 Textures drove larger GPU memory
6:48 Raytracing and modern effects
7:13 Part 2: Booming GPU demand
7:42 Example: GPUs for speech recognition
8:57 Segments of GPU market (GPUs in the cloud)
9:57 Unexpected twist: consumer GPUs top the charts
10:31 GPUs in the cloud are prohibitively expensive
11:34 Part 3: GPUs as accelerators
11:59 Why use a GPU to solve problems in other domains?
12:57 "if" statements in Cuda
13:28 CPU is still involved with GPU programs
13:45 GPU influence on CPUs: SIMD instructions
14:42 Why not make special purpose accelerators?
15:08 PCIe interface for GPUs (technical)
16:13 What do different applications care about?
16:30 Application: cryptocurrencies
17:02 Application: AI
17:52 When special purpose accelerators make sense
19:00 TPU AI accelerator from Google
19:33 Pixel phones have mini TPU
20:11 New accelerators for AI? (Graph-based)
20:53 Conclusion
21:14 Cuda for Nvidia
21:35 Accelerators for AI
22:21 Outro (with quantum computers)