Скачать или смотреть Cloud Native Testing and AI - Why Tests Provide the Context

In this episode, Ole Lensmar joins Richard Li, founder of Polar Sky, to explore how testing strategies must evolve for the age of AI. They discuss the journey of cloud-native tools like Telepresence and Richard's key insight for agentic coding: using automated tests as the "context" to guide AI behavior, rather than relying on complex prompts to describe system rules.

The conversation also digs into the mechanics of building AI-driven software, emphasizing why "evals" are critical for measuring success. Richard shares practical strategies for treating evaluations as data management problems and explains how his team uses simple AI agents to "babysit" CI pipelines and automatically resolve flaky tests.

Topics discussed:

The Evolution of Telepresence: Moving from local debugging to headless execution in CI pipelines.
Tests as Context: Why running a test suite is more effective than complex prompting for AI agents.
Agentic Coding Strategy: Shifting focus from unit tests to integration and behavioral verification.
AI Evals: Why evaluation is the most critical aspect of building reliable AI products.
Babysitting CI: Using simple AI agents to identify and retry flaky tests automatically.

#llm #qatesting #softwaretesting #aitesting