Mykola Boiko - Running Neural Networks and LLMs on Edge Devices

This talk will explore the limits of running neural networks and small-scale language models (LLMs) on edge devices such as laptops, smartphones, and microcontrollers. We’ll examine the challenges and limitations of deploying AI in resource-constrained environments, as well as the opportunities it presents. The session will include a discussion of potential applications, including the feasibility of building tools like a local voice-to-voice translator.

The demo will feature a local text-to-text translator, showcasing the feasibility of AI on edge hardware. We’ll also discuss the next step: upgrading this translator to include voice-to-voice capabilities, highlighting potential use cases and development paths.