GenAI on the Edge Forum: Edge of Tomorrow: Unleashing the Power of Small LLMs for Generative AI...

Edge of Tomorrow: Unleashing the Power of Small LLMs for Generative AI at the Edge
Mallik P. MOTURI, VP Product and Business Development, Syntiant

This talk dives into the innovations crafting small, efficient Large Language Models (LLMs) for edge devices. By distilling the essence of LLMs, we achieve real-time processing, circumventing cloud latency and bandwidth issues. We’ll explore advancements in LLM architectures and the strategic optimizations that enable their miniaturization without sacrificing performance. The focus will be on how these developments not only reduce on-chip memory demands but also pave the way for bespoke hardware, tailored to enhance edge AI’s capabilities. Addressing the economic impact, we’ll discuss the direct correlation between memory footprint and chip cost, and how innovations in this realm are crucial for affordable AI deployment.