Large Language Models Vulnerability From GPU Local Memory Leak. AI Security

Our guest, Tyler Sorensen, a security researcher at Trail of Bits and an Assistant Professor at UC Santa Cruz, covered for the ‪@BuzzRobot‬ community the vulnerability that he and his colleagues recently discovered.

They call the vulnerability LeftoverLocals which affected devices from companies like AMD, Apple, Qualcomm and Imagination. By exploiting GPU local memory leaks attackers can restore an LLM output with high precision.

In this talk Tyler gives a brief overview of GPU and LLM architectures to allow everyone better understand the root cause of the vulnerability and dives into how the vulnerability actually works.

Timestamps:
0:00 Introduction to the lecture and explanation about the discovered vulnerability.
2:20 The description of the GPU architecture to better grasp the root cause of the vulnerability.
14:33 The description of local memory leak type of vulnerability.
17:45 What LeftoverLocals vulnerability is.
19:13 List of GPU devices that were affected by the vulnerability.
21:32 Overview of Large Language Models architecture as one of the most popular applications to better understand potential vulnerability risks.
24:25 How an open-source LLM was tested for LeftoverLocals vulnerability and description of how researchers replicated the attack.
29:20 Reflecting on the impact of the attack.
31:55 The current state of which devices are still impacted by the vulnerability after it was discovered and reported.
32:52 Demo of how the vulnerability works.

Social Links:
Newsletter: https://buzzrobot.substack.com/
X: https://x.com/sopharicks
Slack: https://buzzrobot.slack.com/join/shar...