Optimization within Latent Spaces

Abstract: Large language models are good at learning semantic latent spaces, and the resulting contextual embeddings from these models serve as powerful representations of information. In this talk, I present two novel uses of semantic distances in these latent spaces. In the first part, I introduce BERTScore, an algorithm designed to measure the similarity between machine translation outputs and human gold standards. BERTScore approximates a form of transport distance to match tokens in the generated and human text. In the second part, I will focus on an information retrieval setting, where transformers are trained end-to-end in order to map search queries to corresponding documents. In this setting, I describe IncDSI, a method to add new documents to a trained retrieval system by solving a constrained convex optimization problem to obtain new document representations. Finally, I will also briefly discuss my ongoing work on controllable language generation using diffusion in a semantic latent space.

Bio: Varsha Kishore is a computer science PhD student at Cornell, where she is advised by Kilian Weinberger. She is broadly interested in evaluation metrics, retrieval based generation systems, and text diffusion. During her PhD, she has interned at Google, Microsoft Research and ASAPP. Before starting her PhD, she studied Math and Computer Science at Harvey Mudd College.