Скачать или смотреть AI 360: 01/03/2021. Unified Transformer, Sebastian Ruder, OpenAI's DALL-E, GLOM and StudioGAN

For the full experience, and links to everything referenced, visit our website: https://lamaai.io


FAIR proposes Unified Transformer
Recently, we have seen that Transformers have led to a paradigm shift in AI and NLP research (now even computer vision). Multi-modal research has recently employed Transformers in large Vision/Language Pretraining frameworks (such as VILBERT, VLP etc). Models such as these are usually only trained on one or two pre-training tasks. Facebook AI Research (FAIR) propose a multi-modal model they call the Unified Transformer (UniT), which is a Transformer based model jointly trained on 7 different tasks: object detection, VQA, SNLI-VE, MNLI, QNLI, QQP and SST-2. The architecture, which achieves comparable results to task specific Transformer based models with a signficantly reduced parameter set uses two Transformer encoders and one Transformer decoder. At a very high level, one Transformer encoder is responsible for encoding the image, and the other for encoding the text. The one Transformer decoder takes in an enumerated task as input, and the concatenation of the image and text encodings during the cross multi-head attention step.

Sebastian Ruder: Language Model Fine-tuning
Sebastian Ruder has a new blog post! He posts about the Recent Advances in Langguage Model Fine-tuning. Ruder discusses how the pre-training/fine-tuning paradigm has evolved and advanced since the introduction of large pre-trained langauge models. As is usually the case with his posts, we are given great intuitive descriptions of: adaptive fine-tuning, behavioural fine-tuning, parameter-efficient fine-tuning, text-to-text fine-tuning and how to mitigate fine-tuning instabilities.

OpenAI's DALL-E
DALL-E is a zero-shot text-to-image that was announced via a blog post by OpenAI recently. OpenAI have now sinced released their paper and part of their model - the encoder/decoder used by the discrete VAE. They don't plan to release the model in it's entirety, however they are currently considering whether to enable semi-open access through an API (similar to what is currently in place for GPT-3). However, the community has been quick to try and recreate their work! Check out an implementation by @advadnoun here.

Geoffrey Hinton proposes GLOM
The father of modern deep learning himself proposes the paper: How to Represent Part-Whole Hierarchies in a Neural Network... or more simply, GLOM. GLOM is a conceptual architecture which discusses a method for representing an image as a tree-like hierarchy using a (fixed) neural network. Hinton claims that, if GLOM can be made to work, representations created by Transformer vision/language systems can have significantly improved interpretability.

StudioGAN
Reddit user /u/Minkkowski introduces StudioGAN - an open source PyTorch library featuring SoTA GAN models. StudioGAN contains extensive GAN implementations, benchmarked on a wide variety of image datasets. The library also includes pre-trained models and support for multi-GPU systems alongside many other implementation tricks. Check out his Reddit post announcing the project here!