Скачать или смотреть FutureX: Live Benchmark for LLM Forecasting

In this AI Research Roundup episode, Alex discusses the paper:
'FutureX: An Advanced Live Benchmark for LLM Agents in Future Prediction'
FutureX introduces a live, contamination-free benchmark to evaluate LLM agents on forecasting real-world future events. It runs a daily pipeline that sources future-oriented questions from 195 sites, generates templated events, executes agent predictions at start time, and crawls for ground truth at resolution—avoiding temporal and retrieval leakage. The benchmark covers ~500 weekly events across 11 domains with four difficulty tiers based on volatility and required skills. It assesses 25 models spanning base LLMs and agentic systems using Think&Search and deep research capabilities.
Paper URL: https://arxiv.org/abs/2508.11987

#AI #MachineLearning #DeepLearning #LLMAgents #ForecastingBenchmark #FuturePrediction #Evaluation