Скачать или смотреть Integer Quantization for Deep Learning Inference: Principles and Empirical Evaluation

This talk is a part of Neural Network Accelerator Study#3. To watch the others, please refer to here: https://github.com/ConstantPark/Neura...

This paper describes a broad summary of Int8 quantization. If you are new, this paper is a good start point for quantization. The paper is uploaded on Arxiv. The original talk was presented in GTC 2020.

OUTLINE:
0:00 Overview
3:46 Introduction
6:24 Related Work
11:37 Quantization Fundamentals
31:55 Post-Training Quantization
41:20 Techniques to Recover Accuracy (Partial Quantization and Quantization Aware Training)
55:17 Recommended Workflow
57:50 Conclusion


Original Paper: INTEGER QUANTIZATION FOR DEEP LEARNING INFERENCE: PRINCIPLES AND EMPIRICAL EVALUATION, https://arxiv.org/abs/2004.09602

Slide: https://www.slideshare.net/leejaymin/...

Abstract: Quantization techniques can reduce the size of Deep Neural Networks and improve inference latency and throughput by taking advantage of high throughput integer instructions. In this paper we review the mathematical aspects of quantization parameters and evaluate their choices on a wide range of neural network models for different application domains, including vision, speech, and language. We focus on quantization techniques that are amenable to acceleration by processors with high-throughput integer math pipelines. We also present a workflow for 8-bit quantization that is able to maintain accuracy within 1% of the floating-point baseline on all networks studied, including models that are more difficult to quantize, such as MobileNets and BERT-large.