Скачать или смотреть Evaluating Large Language Models in Scientific Discovery (Dec 2025)

Title: Evaluating Large Language Models in Scientific Discovery (Dec 2025)
Link: http://arxiv.org/abs/2512.15567v1
Date: December 2025

Summary:
This paper introduces the Scientific Discovery Evaluation (SDE) framework, a benchmark designed to assess Large Language Models (LLMs) on realistic, iterative scientific research tasks rather than static knowledge retrieval. Spanning biology, chemistry, materials, and physics, SDE evaluates models at two levels: question-level accuracy within specific research scenarios and project-level performance involving hypothesis generation and experimental simulation. The study reveals a significant gap between performance on general science exams and actual discovery capabilities, highlighting diminishing returns from current scaling strategies and identifying shared failure modes across top-tier models like GPT-5 and DeepSeek-R1.

Key Topics:
Scientific Discovery Evaluation (SDE)
LLM Benchmarking
Iterative Reasoning
Project-level Evaluation
Computational Biology
Materials Science
Hypothesis Generation

Chapters:
00:00 - Introducing SDE Framework
00:46 - Key Research Findings
01:39 - The Evaluation Gap
02:25 - Structuring Real-World Scenarios
03:05 - Testing Deep Technical Skills
04:04 - Performance Drop-Offs
05:25 - Diminishing Returns on Scaling
06:20 - Systematic Model Failures
07:28 - End-to-End Discovery Loops
08:35 - The Serendipity Paradox
09:40 - No Scientific Superintelligence
10:15 - Roadmap for AI Developers

Stock video credits:
Anete Lusina - https://www.pexels.com/@anete-lusina
Yaroslav Shuraev - https://www.pexels.com/@yaroslav-shuraev
Mikhail Nilov - https://www.pexels.com/@mikhail-nilov
Kindel Media - https://www.pexels.com/@kindelmedia
Colin Jones - https://www.pexels.com/@larchmedia
KATRIN BOLOVTSOVA - https://www.pexels.com/@ekaterina-bol...
Colors Motion Graphics - https://www.pexels.com/@colors-motion...
cottonbro studio - https://www.pexels.com/@cottonbro
Kelly - https://www.pexels.com/@kelly
Pressmaster - https://www.pexels.com/@pressmaster
StefWithAnF - https://www.pexels.com/@stefwithanf-1...
Engin Akyurt - https://www.pexels.com/@enginakyurt
Pavel Danilyuk - https://www.pexels.com/@pavel-danilyuk
Pachon in Motion - https://www.pexels.com/@pachon-in-mot...
Stas Knop - https://www.pexels.com/@stasknop
Silviu Din - https://www.pexels.com/@silviu-din-16...
Bedrijfsfilmspecialist.nl - https://www.pexels.com/@bedrijfsfilms...
Danil Shostak - https://www.pexels.com/@danil-shostak...
Pixabay - https://www.pexels.com/@pixabay
@svetjekolem - https://www.pexels.com/@svetjekolem
Trippy Lagoon - https://www.pexels.com/@trippy-lagoon...
Dan Cristian Pădureț - https://www.pexels.com/@paduret
tunnel motions - https://www.pexels.com/@tunnelmotions
Soumya - https://www.pexels.com/@soumya-1446957
Ron Lach - https://www.pexels.com/@ron-lach
Oleg Gamulinskii - https://www.pexels.com/@oleg-gamulins...
José Alfredo Munguía Lira - https://www.pexels.com/@rectorretro