Скачать или смотреть Itiel Shwartz @ AIDevTLV '25 | From Chaos to Quality – A Big Data Approach to Agentic System

AIDevTLV 2025
aidevtlv.com
Israel's largest conference for LLM App developers
Powered by EventHandler and AT&T

Building reliable agentic AI systems in production environments presents unique challenges when dealing with massive, noisy datasets. This talk shares hard-won lessons from developing Klaudia, Komodor's AI agent that processes millions of Kubernetes events daily to deliver autonomous troubleshooting with 95%+ accuracy.

The fundamental challenge isn't LLM capability—it's building systems that maintain reliability when 90% of your data is noise. We'll explore why most agentic AI fails in production: hallucinations masquerading as insights, inability to validate reasoning chains, and the brittle nature of RAG systems when dealing with complex, interconnected failure modes.

This session covers practical know-how learned through painful production iterations: how to build validation frameworks that catch LLM errors before they reach users, architectural patterns for constraining problem spaces without losing effectiveness, and techniques for creating evidence-based reasoning that can be audited and improved systematically.

You'll learn specific strategies for LLM validation in high-stakes environments, including confidence scoring systems, multi-agent verification patterns, and iterative investigation loops that prevent runaway reasoning. We'll cover the hard-earned lessons about what works and what spectacularly fails when building trustworthy AI agents that must deliver accurate results rather than plausible-sounding explanations.

Itiel is the CTO and co-founder of Komodor, a startup building the first Kubernetes-native troubleshooting platform. A big believer in dev empowerment and moving fast, he has worked at eBay, Forter and Rookout (as the founding engineer). Itiel is a backend and infra developer turned “DevOps”, an avid public speaker that loves talking about things such as cloud infrastructure, Kubernetes, Python, observability, and R&D culture.