Colin Raffel: A call to build models like we build open-source software

This talk was held on March 3, 2022 as a part of the MLFL series, hosted by the Center for Data Science, UMass Amherst.

Abstract: Large pre-trained models have become a cornerstone of modern ML pipelines thanks to the fact that they facilitate improved performance with less labeled data on downstream tasks. However, these models are typically created by a resource-rich research group that unilaterally decides how a given model should be built, trained, and released, after which point it is left as-is until a better pre-trained model comes along to completely supplant it. In contrast, open-source development has proven that it is possible for a distributed community of contributors to work together to iteratively build complex and widely-used software. This kind of large-scale distributed collaboration is made possible through a mature set of tools including version control, continuous integration, merging, and more. In this talk, I will present a vision for building machine learning models in the way that open-source software is developed, including preliminary work from my lab on "merging" and "patching" models. I will also give some insight into the future work required to make this vision a reality.

Bio: Colin Raffel is an assistant professor at UNC Chapel Hill. He also spends one day a week as a faculty researcher at Hugging Face.

About Machine Learning and Friends Lunch: MLFL is a lively and interactive forum held weekly where friends of the UMass Amherst machine learning community can sit down, have lunch, and give or hear a 50-minute presentation on recent machine learning research. This semester of the UMass MLFL series has been graciously sponsored by our friends at Oracle Labs.

Please follow this link to know more about the past and upcoming talks: http://umass-mlfl.github.io