Скачать или смотреть Anthony Fuller & Yousef Yassin - LookWhere? Efficient Visual Recognition by Learning Where to Look

Vision transformers are ever larger, more accurate, and more expensive to compute. The expense is even more extreme at high resolution as the number of tokens grows quadratically with the image size. We turn to adaptive computation to cope with this cost by learning to predict where to compute. Our LookWhere method divides the computation between a low-resolution selector and a high-resolution extractor without ever processing the full high-resolution input. We jointly pretrain the selector and extractor without task supervision by distillation from a self-supervised teacher, in effect, learning where and what to compute simultaneously. Unlike prior token reduction methods, which pay to save by pruning already-computed tokens, and prior token selection methods, which require complex and expensive per-task optimization, LookWhere economically and accurately selects and extracts transferrable representations of images. We show that LookWhere excels at sparse recognition on high-resolution inputs (Traffic Signs), maintaining accuracy while reducing FLOPs by up to 34x and time by 6x. It also excels at standard recognition tasks that are global (ImageNet classification) or local (ADE20K segmentation), improving accuracy while reducing time by 1.36x.

Anthony is a 3rd / final-year Ph.D. student at Carleton University / Vector Institute supervised by Jim Green (from Carleton) and Evan Shelhamer (from UBC / Vector). He is interested in deep learning, specifically, computer vision, self-supervised learning, transformers, and applications (e.g., remote sensing and biomedical). He has an M.A.Sc. in Systems and Computer Engineering (2023) and a B.Eng. in Aerospace Engineering (2015), both from Carleton.

Yousef is a second-year master’s student at Carleton University, supervised by Junfeng Wen. His work focuses on self-supervised and reinforcement learning, aiming to combine intuition and theory with simple, scalable algorithms. He holds a B.Eng in Computer Systems Engineering from Carleton, earned in 2024.

This session is brought to you by the Cohere Labs Open Science Community - a space where ML researchers, engineers, linguists, social scientists, and lifelong learners connect and collaborate with each other. We'd like to extend a special thank you to Ahmad Anis and Kanwal Mehreen, Lead of our Geo Regional Asia group for their dedication in organizing this event.

If you’re interested in sharing your work, we welcome you to join us! Simply fill out the form at https://forms.gle/ALND9i6KouEEpCnz6 to express your interest in becoming a speaker.

Join the Cohere Labs Open Science Community to see a full list of upcoming events (https://tinyurl.com/CohereLabsCommuni....