Скачать или смотреть UEval: New Benchmark for Unified Generation

In this AI Research Roundup episode, Alex discusses the paper: 'UEval: A Benchmark for Unified Multimodal Generation' This paper introduces UEval, a new benchmark designed to evaluate unified models that can generate both images and text simultaneously. The benchmark consists of 1,000 expert-curated questions across eight real-world tasks that require complex reasoning for outputs. Unlike previous methods, UEval uses a fine-grained rubric-based scoring system with over 10,000 validated criteria to ensure accurate evaluation. The study finds that even top models like GPT-5-Thinking struggle with these tasks, highlighting a significant gap in current multimodal capabilities. Researchers also discovered that incorporating reasoning traces can significantly improve performance in these unified generation tasks. Paper URL: https://arxiv.org/pdf/2601.22155 #AI #MachineLearning #DeepLearning #Multimodal #Benchmark #ComputerVision #NLP #GenerativeModels