Скачать или смотреть I made real-time optical flow, because we can.

We use optical flow in many things: frame generation, object detection, motion stabilization, and more.

I built a real-time motion estimation algorithm that runs on modest GPUs, and in this video, I'll show you how I did it. This project was a dream I've had since my days as a video editor in 2015, when GPU-based motion effects were a distant possibility. Now, it's a reality, and I'm excited to share the journey with you all.

CShade
• Website: https://papadanku.github.io/CShade/
• Source Code: https://github.com/papadanku/CShade
• Blog Post & Source Code: https://papadanku.github.io/source/bl...

Disclaimer
This video, from 2:33 to 5:32, recommends that the viewer have some familiarity with linear algebra. I added [MATH] to the math-heavy chapters in the video.

Key Takeaways
• We start with the Brightness Constancy Assumption, which states that a pixel's brightness doesn't change as it moves. We linearize this to create the Optical Flow Equation. (1:02 to 1:54)
• We quickly encounter the Aperture Problem, a fundamental limitation of the Optical Flow Equation, because it gives us multiple possible answers for a single pixel's motion. (1:54 to 3:20)
• The Lucas-Kanade method solves this by assuming Local Constancy, meaning all pixels in a small neighborhood move the same way. It then uses the Least Squares method to find a unique, single motion vector for that whole neighborhood. (2:33 to 5:32)
• Lucas-Kanade works great for small motions, but it fails with large motion. The Brightness Constancy Assumption and our linearization break down when pixels move too far between frames. (6:28 to 6:52)
• To overcome the large motion limitation, we use a coarse-to-fine approach. This involves analyzing the image at multiple scales, starting with a low-resolution version where large motions appear small, and then refining the results at higher resolutions. (6:52 to 8:25)

Chapters
0:00 · Optical Flow is Everywhere.
0:13 · What is Optical Flow?
0:28 · The Lore.
1:02 · The Brightness Constancy Assumption.
1:24 · The Optical Flow Equation.
1:54 · The Aperture Problem.
2:33 · [MATH] 1 Equation, 2 Unknowns.
3:20 · [MATH] Lucas-Kanade: Local Constancy.
3:37 · [MATH] Lucas-Kanade: Why Least Squares?
3:55 · [MATH] Lucas-Kanade: Solving Least Squares.
5:32 · The Johnny Zest Test.
6:28 · The Project Reality Test.
6:52 · Lucas-Kanade: Large Motion Gaps.
7:09 · Downscaling.
7:28 · Lucas-Kanade: Coarse to Fine.
8:13 · Lucas-Kanade: Coarse to Fine Comparison.
8:25 · Acknowledgements.

References
• Plyer, A., Le Besnerais, G., & Champagnat, F. (2016). Massively parallel Lucas Kanade optical flow for real-time video processing applications. Journal of Real-Time Image Processing, 11(4), 713-730.
• Rojas, R. (2010). Lucas-Kanade in a nutshell. Freie Universit at Berlinn, Dept. of Computer Science, Tech. Rep.
• Sastrasinh, P. (n.d.). Dense Realtime Optical Flow on the GPU. https://cs.brown.edu/courses/csci1290...

Audio Sources
• BGM_011 (Kikiyama, Yume Nikki): https://yume.wiki/File:Yume_Nikki_BGM...
• e40 (Mike, Yume 2ikki): https://yume.wiki/File:E40-slow.ogg
• 2_18 (Camellia, Yume 2ikki): https://yume.wiki/File:2_18(051E).ogg
• ...But Still We Go On (CCPGames, Eve Online):   / eve-online-but-still-we-go-on  

What I Used to Make This Video
• Amazon Basics USB Condenser Microphone: https://a.co/d/ajRty0H
• Amazon Basics 50-inch Tripod Stand: https://a.co/d/7eTKA8J
• Asap: https://fonts.google.com/specimen/Asap
• Audacity: https://www.audacityteam.org/
• DaVinci Resolve: https://www.blackmagicdesign.com/prod...
• FFmpeg: https://ffmpeg.org/
• GIMP: https://www.gimp.org/
• OBS Studio: https://obsproject.com/
• MOVEMENT: https://www.nmtype.com/movement/
• Sony α6000 with E PZ 16–50mm F3.5–5.6 OSS (SELP1650)

#graphics #shaders #reshade