The Future of Earth-System Data Infrastructure with Ryan Abernathey

Abstract: Responding to the climate crisis requires a coordinated mobilization of academic research, government agencies, and a rapidly growing suite of private companies (broadly described as “climate tech”) aimed at addressing climate change adaptation and mitigation through commercial products and services. At the heart of this work is data: exabytes of data about the earth system, originating from satellites, sensors, and simulations and passing through many stages of processing and refinement as they reach end-user applications.

The growth of AI is fueling an insatiable demand for data across the board while also producing new sources of data, as AI-driven forecasts begin to emerge. Earth system data has many more users and use cases than it did a decade ago.

These trends require us to rethink our approach to data infrastructure, which has traditionally emphasized a one-way exchange of data files from a few large data providers to data consumers. The talk will review exciting recent progress in moving towards a cloud-native earth-system-data ecosystem, incorporating lessons from my work on open-source software such as Xarray, Zarr, and Pangeo. Abernathey conclude with a vision for how a truly frictionless global data infrastructure can enable a radically more effective response to the climate crisis while also empowering those most impacted by climate change to play a greater role in solutions.

Bio: Ryan Abernathey is a scientist, startup founder, and open-source software developer. He is the CEO and co-founder of Earthmover PBC, an early stage startup on a mission to empower people to use scientific data to solve humanity's greatest challenges. He's also an Associate Professor of Earth And Environmental Science at Columbia University and Lamont Doherty Earth Observatory. As a physical oceanographer, he studies the large-scale ocean circulation and its relationship with Earth's climate using climate models and satellite data. He received his Ph.D. from MIT in 2012 and did a postdoc at Scripps Institution of Oceanography. He has received an Alfred P. Sloan Research Fellowship in Ocean Sciences, an NSF CAREER award, The Oceanography Society Early Career Award, and the AGU Falkenberg Award. He is a member of the NASA Surface Water and Ocean Topography (SWOT) science team and Director of Data and Computing for a new NSF Science and Technology Center called Learning the Earth with Artificial Intelligence and Physics (LEAP). Prof. Abernathey is an active participant in and advocate for open source software, open data, and reproducible science. In 2016, he helped found the Pangeo project, an open science community focused on big scientific data analytics.