AI-Generated Game Graphics: One Step Closer With World-Consistent Vid2Vid | Game Futurology #19

This is episode #19 of the video series "Game Futurology" covering the paper "World-Consistent Video-to-Video Synthesis" by Arun Mallya, Ting-Chun Wang, Karan Sapra and Ming-Yu Liu.
PDF: https://arxiv.org/pdf/2007.08509.pdf
Authors' Project Page: https://nvlabs.github.io/wc-vid2vid/

Game Futurology:
This is a video series consisting of short 2-3 minute overview of research papers in the field of AI and Game Development. This series aims to ponder over what the future games might look like based on the latest academic research going on in the field today. Subscribe for more weekly videos!

Abstract:
Video-to-video synthesis (vid2vid) aims for converting high-level semantic inputs to photorealistic videos. While existing vid2vid methods can achieve short-term temporal consistency, they fail to ensure the long-term one. This is because they lack knowledge of the 3D world being rendered and generate each frame only based on the past few frames. To address the limitation, we introduce a novel vid2vid framework that efficiently and effectively utilizes all past generated frames during rendering. This is achieved by condensing the 3D world rendered so far into a physically-grounded estimate of the current frame, which we call the guidance image. We further propose a novel neural network architecture to take advantage of the information stored in the guidance images. Extensive experimental results on several challenging datasets verify the effectiveness of our approach in achieving world consistency - the output video is consistent within the entire rendered 3D world.

Music Credits: https://www.fesliyanstudios.com/


----------------------------------------------------------------
• YouTube -    / deepgaminga.  .
• Twitter -   / deepgamingai  
• Medium -   / chintan.t93  
• GitHub - https://github.com/ChintanTrivedi
--------------------------------------------------------------------

#DeepLearning #GameDesign #GenerativeModeling #GameDevelopment