How to Design a Neural Network | 2020 Edition

In this video, I covered some of the useful neural network design techniques that came out or popularized between 2018 and 2020. At the end of the video, I went through some of our recent papers and explained how my colleagues and I designed neural networks for constrained environments.

Earlier video:
How to Design a Convolutional Neural Network
   • How to Design a Convolutional Neural ...  

My work mentioned in the video:
Seeing Through the Clouds With DeepWaterMap
http://www.isikdogan.com/files/isikdo...

A Machine Learning Imaging Core using Separable FIR-IIR Filters
https://arxiv.org/pdf/2001.00630.pdf

Other papers referenced in the video:
ShuffleNetV2: Practical Guidelines for Efficient CNN Architecture Design
https://arxiv.org/pdf/1807.11164.pdf

MobileNetV2: Inverted Residuals and Linear Bottlenecks
https://arxiv.org/pdf/1801.04381.pdf

Learning Transferable Architectures for Scalable Image Recognition (NASNet)
https://arxiv.org/pdf/1707.07012.pdf

Attention Is All You Need (the Transformer)
https://arxiv.org/pdf/1706.03762.pdf

Squeeze-and-Excitation Networks
https://arxiv.org/pdf/1709.01507.pdf

Searching for MobileNetV3
https://arxiv.org/pdf/1905.02244.pdf