Скачать или смотреть Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Models (Apr 2025)

Title: Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Models (Apr 2025)
Link: http://arxiv.org/abs/2504.04823v2
Date: April 2025

Summary:
This paper conducts a comprehensive empirical study on the impact of quantization—specifically weight, KV cache, and activation quantization—on reasoning-enhanced Large Language Models (LLMs) such as the DeepSeek-R1-Distill series and QwQ. The study reveals that while 8-bit quantization remains largely lossless, lower bit-widths (e.g., 3-bit or 4-bit) introduce significant accuracy risks, particularly for smaller models and harder reasoning tasks. The authors also investigate the relationship between quantization and chain-of-thought length, finding that quantized models do not necessarily extend reasoning steps to compensate for precision loss, and provide recommendations for optimal quantization configurations.

Key Topics:
Quantization
Reasoning Models
Large Language Models
Chain-of-Thought
KV Cache Quantization
Model Compression
DeepSeek-R1

Chapters:
00:00 - Introduction to Quantized Reasoning
01:00 - Six Core Study Insights
02:36 - Motivation and Study Setup
03:52 - Stress Testing Reasoning Benchmarks
04:30 - Identifying Safe Bit-Widths
05:43 - Weight and Cache Compromises
06:45 - Choosing Quantization Algorithms
07:44 - Architecture Specific Failures
08:43 - Impact of Task Difficulty
09:54 - How Model Training Matters
10:55 - Analyzing Output Length Changes
11:41 - The Model Scaling Payoff
12:48 - Calibration Data Sensitivity
14:00 - Summary and Future Challenges

Stock video credits:
cottonbro studio - https://www.pexels.com/@cottonbro
@svetjekolem - https://www.pexels.com/@svetjekolem
Engin Akyurt - https://www.pexels.com/@enginakyurt
Trippy Lagoon - https://www.pexels.com/@trippy-lagoon...
Yaroslav Shuraev - https://www.pexels.com/@yaroslav-shuraev
Pressmaster - https://www.pexels.com/@pressmaster
Colin Jones - https://www.pexels.com/@larchmedia
Pachon in Motion - https://www.pexels.com/@pachon-in-mot...
Bedrijfsfilmspecialist.nl - https://www.pexels.com/@bedrijfsfilms...
Pixabay - https://www.pexels.com/@pixabay
Kindel Media - https://www.pexels.com/@kindelmedia
Pavel Danilyuk - https://www.pexels.com/@pavel-danilyuk
StefWithAnF - https://www.pexels.com/@stefwithanf-1...
crazy motions - https://www.pexels.com/@crazy-motions...
Mikhail Nilov - https://www.pexels.com/@mikhail-nilov
Soumya - https://www.pexels.com/@soumya-1446957
José Alfredo Munguía Lira - https://www.pexels.com/@rectorretro
Oleg Gamulinskii - https://www.pexels.com/@oleg-gamulins...
KATRIN BOLOVTSOVA - https://www.pexels.com/@ekaterina-bol...
Kelly - https://www.pexels.com/@kelly
Ron Lach - https://www.pexels.com/@ron-lach
Stas Knop - https://www.pexels.com/@stasknop
Colors Motion Graphics - https://www.pexels.com/@colors-motion...
Anthony 🙂 - https://www.pexels.com/@inspiredimages
tunnel motions - https://www.pexels.com/@tunnelmotions
Danil Shostak - https://www.pexels.com/@danil-shostak...
Silviu Din - https://www.pexels.com/@silviu-din-16...
Anete Lusina - https://www.pexels.com/@anete-lusina