DHS 2024: Debugging & fixing Gen AI applications

Generative AI has revolutionized the world, by making complex AI / ML techniques very easy to use. It has enabled non-Data Science users like business and engineering folks to create AI solutions with ease. But as we know there are no free lunches, ease of use comes with challenges of handling non-trivial cases. This is where a new Gen AI user gets stuck and feels frustrated, as they have invested a good amount of time in it and now they struggle to find a way out. We will walk through some ways to debug a Gen AI solution and see how we can find the problem area. Then based on symptoms we can choose the appropriate remedial measures.

In this talk we will talk about various technologies like using RAG efficiently on private enterprise data. Then how to improve RAG performance by playing with some internal layers like embedding models, similarity metric and data segmentation to tune it to our needs. In case the context is spread in multiple places, or it is spread in a large section of text. We will see how we can use Knowledge Graphs and Ontologies to summarize information for improving performance. Given we have the right context and problem persists, then we will see how to break the problem into smaller subproblems to find the root cause and fix it. We will do a walk-through for some of these useful techniques in the talk.