Tobi Delbruck - Event Camera Tutorial 2020 - v4 1

Tutorial on event cameras prepared for the 2020 Telluride Neuromorphic workshop and ESSCIRC.
Please ask questions in comment thread. See expanded description below to go to individual slides. See https://drive.google.com/open?id=1w7e... for the PDF slides with links.
See also Davide Scaramuzza's video    • Event Cameras: Opportunities and the ...   "Event Cameras: Opportunities and the Road Ahead (CVPR 2020)"
- Tobi Delbruck, http://sensors.ini.uzh.ch

------- Version history
v4.1 Rerecorded introduction and added slide about inivation DVS token ring readout
v4.0 Added Mahowald&Mead retina and How to build event camera slides
v3.0 New slides added on pixel circuits, no silent gaps
v2.1 Updated from the first version with better timing and no silent gaps.
v2.0 original

---------------------------------------

00:00:00 1 Event Cameras
00:00:38 2 This tutorial has 3 parts

00:01:47 3 The Human Eye as a digital camera
00:04:20 4 Concise (Tobi biased) timeline of Electronics, Neuroscience, AI and NE
00:06:15 5 Historical development
00:06:42 6 Mahowald/Mead retina
00:11:22 7 Mead Physics of Computation Lab, Caltech (1985-96)

00:13:40 8 PART 2
00:13:48 9 Conventional cameras (Static vision sensors) output a stroboscopic sequence of frames
00:16:37 10 DVS (Dynamic Vision Sensor) Pixel
00:18:42 11 Compare event camera with frame camera
00:19:38 12 DVS pixel has wide dynamic range
00:21:08 13 And DVS has sparse, quick output
00:22:03 14 DAVIS (Dynamic and Active Pixel Vision Sensor) Pixel
00:22:58 15 An event camera chip (DAVIS346)
00:24:27 16 2019 example
00:25:49 17 DAVIS camera sample data
00:28:23 18 DVS sensor specifications
00:30:35 19 Key functional aspects of DVS pixel #1: Mismatch reduction
00:32:04 20 Key functional aspects of DVS pixel #2: Bandwidth enhancement
00:33:38 21 Effect of pixel bias currents
00:35:47 22 Event threshold matching measurement
00:37:33 23 Recent matching results
00:39:35 24 Latency and jitter
00:41:44 25 Pixel simulation shows what’s going on
00:44:33 26 A key part of pixel is the 7T-2C change detector
00:47:10 27 Together with log photoreceptor, it provides temperature independent temporal contrast threshold
00:48:49 28 Temperature independence
00:50:19 29 Reduction in pixel size
00:51:52 30 Example of stacked pixel: Finateu ISSCC 2020 paper 5.10
00:52:59 31 Advances in DVS pixel circuits
00:55:03 32 Increasing photoreceptor gain and reducing “leak events”
00:59:40 33 Advances in DVS readout speed
01:04:27 34 Example of token-ring group readout for DVS with low power inivation DVS
01:09:27 35 Word serial AER burst mode
01:09:27 36 Building a complete USB event camera
01:18:53 37 Using event cameras

01:20:29 38 PART 3
01:20:52 39 Representing and processing events
01:22:24 40 Tracking objects from DVS events using spatio-temporal coherence
01:24:41 41 Robo Goalie
01:25:20 42 Using DVS allows 2 ms reaction time at 4% processor load with USB interfaces, under Windows, using Java
01:26:46 43 Yann LeCunn, CNNs: MNIST, LeNet, DAVE (1988-2008)
01:27:57 44 Activity-driven sampling strategies for exposing DVS ‘frames’
01:30:43 45 How much information is in the DVS stream?
01:33:22 46 How much information is in the DVS stream?
01:34:00 47 Tools for simulating DVS from frame camera video
01:35:48 48 Synthetic slow motion interpolates the frames
01:36:20 49 See v2e paper for detailed explanation, including pixel non-idealities
01:36:35 50 Demo - RoShamBo
01:36:41 51 RoShamBo training images
01:36:41 52 Live demo of Rock Scissors Paper
01:37:54 53 Live demo of Rock Scissors Paper

01:41:01 54 Summary
01:42:29 55 Additional sources