Скачать или смотреть Tom Zehle - CAPO: Cost-Aware Prompt Optimization

LLMs have shown to be capable of solving a wide range of tasks, yet their performance is surprisingly sensitive to prompt formulation. Manually tuning prompts can be time-consuming, and we lack clear guidelines on what works well. This makes automated prompt optimization an important tool for improving performance in a systematic and efficient way. In this talk, we introduce CAPO (Cost-Aware Prompt Optimization), a method that finds prompts achieving high performance while keeping costs low. CAPO has been shown to outperform existing prompt optimization techniques across various benchmarks, achieving substantial improvements such as a 25%p accuracy boost on GSM8K using a Llama-3.3-70B. To make these advances accessible to the broader community, we have developed Promptolution, an open-source framework for implementing and experimenting with prompt optimization approaches.

Speaker
Tom Zehle is a Master’s student in Statistics & Data Science at LMU Munich and currently works as a Data Scientist at Airbus. He will begin his PhD this October at the ELLIS Institute in Tübingen, where his doctoral research will focus on specialization techniques for generalist foundation models, with a particular emphasis on prompt optimization, as well as finetuning and in-context learning. Together with Timo Heiß and Moritz Schlager, he co-developed CAPO and Promptolution, exploring practical and cost-efficient approaches to automated prompt optimization.

Paper:
https://arxiv.org/abs/2504.16005

About Munich NLP:

Munich🥨NLP is a community founded in May 2022 by LMU and TUM students focusing on NLP topics. Within the first year, the community has already grown to over 1000 members consisting not only of current students, but also including PhD students, professors, and industry practitioners. We host weekly workshops and/or paper-reading events, both to learn from guests and to gather inspiration for our own (research) projects, as well as to establish and keep going an active student NLP community in the Munich area. The goal is to promote NLP-related exchange between students, researchers, and practitioners inside and outside the university and to showcase paths and possibilities during and after university.

Join our Discord server:   / discord  

Homepage: https://munich-nlp.com
LinkedIn:   / munich-nlp  
Twitter / X:   / munichnlp  
Bluesky: https://bsky.app/profile/munich-nlp.b...

#AutoML #NLP #naturallanguageprocessing #AI #KI #Munich #Research #AGI #ASI #Statistics #Math #llm #ChatGPT #Qwen #deepseek #finetuning