Yedid Hoshen - What Can Darwin’s Tree of Life Tell Us About Neural Networks?

Thousands of new neural network models are uploaded every day to online repositories such as Hugging Face, the number of available models should cross 1 million by the end of 2024. These numbers lead us to hypothesize that models are emerging as a major data modality. However, differently from images or audio, researchers do not know how to interpret neural network weights; in practice model collections are mostly left disorganized and unused.

Motivated by Darwin’s tree-of-life, we present the Model Graph for organizing models and representing their relations as a directed graph. The Model Graph contains multiple Model Trees, each rooted by a foundation model with directed edges connecting parent models to child models that were finetuned from it. We then introduce an automatic algorithm for recovering the Model Graph from a disorganized collection, much like in evolutionary analysis. In practice, our method succeeds in unmasking the hereditary structure of in-the-wild model collections such as the Llama and Stable Diffusion families.

To complete missing data on the Model Graph, we present a method for imputing the weights of a model given those of its children. Our method works on large-scale models e.g., Mistral, and may have serious implications for model safety.