Скачать или смотреть CVPR 2021 Tutorial on Normalization Techniques in Deep Learning-Part4 Applications

This is the Part 4 of CVPR 2021 Tutorial Normalization Techniques in Deep Learning: Methods, Analyses, and Applications.

Presenter: Lei Huang

Normalization for applications (50 min): We show that the normalization methods can be used to ‘edit’ the statistical properties of layer activations. These statistical properties, when designed well, can represent the style information for a particular image or the domain-specific information for a distribution of a set of images. This characteristic of normalization methods has been thoroughly exploited in CV tasks and potentially beyond them:
a. Normalization for domain adaptation. We illustrate how normalization can be used to align the distributions of activations for different domains in domain adaptation. We also illustrate how to exploit BN to learn the domain specific information, while other convolutional/linear parameters to learn domain invariant representation. These ideas can be further used to learn universal representation for multiple task learning.
b. Normalization for style transfer. We provide how the statistical information of an internal layer can represent a style of an image, and how normalization can be used to disentangle the style from the content. This idea can be further used for more general image translation tasks.
c. Normalization for training generative adversarial networks (GANs). We show the recent theoretical and empirical progresses of normalization in training GANs.
d. Normalization for efficient deep models. We show how to exploit normalization for network slimming and how to effectively quantize networks with BN.