Towards Using Neural Networks for Geoscientific Discovery
October 02, 2020
Benjamin Toms
Committee: Elizabeth Barnes (Adviser), James W. Hurrell, David W. J. Thompson, Imme Ebert-Uphoff (Electrical and Computer Engineering)
Abstract
How can we use computational methods to extract physically meaningful patterns from geoscientific data? This question has been asked in some form for decades within the geoscientific community, with many landmark discoveries resulting from the novel application of computational methods to a geoscientific dataset. Many methods exist to discover patterns within geoscientific data, although each is limited by its own set of assumptions. The most common assumption is that of linearity, which oftentimes conflicts with our understanding that the earth system can be both dynamically and statistically nonlinear. However, a recently popularized subset of methods within the computer science community known as neural networks can identify nonlinear patterns and are therefore potentially powerful tools for geoscientific discovery. Regardless of the application, a common limitation of neural networks has been the difficulty to understand how and why they make their decisions. Therefore, while they have been used in geoscience for more than two decades, they have mostly been applied when accuracy is valued more than understanding, such as for making forecasts.Within this presentation, we first propose a framework for how neural networks can be used for geoscientific discovery by applying recently invented methods from the computer science community. We focus on methods that explain which aspects of the input dataset are useful for the neural network when making connections to the output dataset. This framework enables physical interpretations of how and why neural networks make decisions, since the geoscientist that designs the neural network is likely familiar with the physical meaning of each input.
In the first study of this presentation, we outline the framework and apply it to two simple tasks to ensure the neural network interpretations abide by our current understanding of the earth system. The interpretable neural networks successfully identify the pattern of the El Ni\~no Southern Oscillation and oceanic patterns that lend seasonal predictability, which lends confidence that the framework is reliable. In the second study, we then further test the methods by applying them to a more spatially and temporally complex oscillation called the Madden-Julian Oscillation (MJO). The interpretable neural networks correctly identify the known spatial structures and seasonality of the MJO, and also suggest that the MJO is nonlinear and expresses its nonlinearity through the uniqueness of each event. The final study assesses whether the proposed framework can be used to identify predictable patterns of earth-system variability within climate models through its application to decadal predictability. We find that the interpretable neural networks identify known modes of oceanic decadal variability that contribute to predictability of continental surface temperatures. The interpretations can also be used to identify distinct regimes of predictability, wherein spatially and temporally unique oceanic modes contribute predictability for the same location at different times.
From a broader perspective, these studies indicate that neural networks are a viable tool for geoscientific discovery and are particularly useful given their ability to capture nonlinear, time-evolving patterns. It is likely that new neural network algorithms and methods for their interpretation will continue to be developed by the computer science community, and so this research offers a guideline for how such methods can be gainfully applied within the geosciences.