Insights from Machine Learning-Based Forecasts of Convective Hazards and Environments
December 06, 2024
Allie Mazurek
Committee: Russ Schumacher (Advisor); Aaron Hill; Susan van den Heever; Kristen Rasmussen; Haonan Chen (Electrical and Computer Engineering)
Abstract
Severe convective thunderstorms and their associated hazards are costly, damaging, and difficult to predict. Machine learning (ML) techniques are rapidly being developed and deployed in an effort to predict severe thunderstorms more quickly and with greater accuracy than traditional methods. With these developments, there is a need to understand how ML-based weather prediction systems rely on atmospheric data and generate their forecasts. This work probes a number of ML-based convective thunderstorm-related forecasts over the contiguous United States to 1) understand how they make their predictions, 2) diagnose where their strengths and deficiencies may lie, and 3) explore how well their predictions resemble physical characteristics of the atmosphere. The insights gleaned from this research aim to support operational use of ML-based forecast guidance.First, probabilistic ML-based forecasts of severe convective hazards (i.e., tornadoes, hail, and thunderstorm-driven winds) from the Colorado State University Machine Learning Probabilities (CSU-MLP) system are studied using an explainable machine learning technique known as Tree Interpreter (TI). TI provides context to the CSU-MLP forecasts by disaggregating its forecast probabilities into “contributions” by each of the environmental variables that are used to train the model. This technique allows one to see the extent to which each atmospheric “ingredient” contributes to the final predictions. Results of this work show that CSU-MLP uses environmental information to make its predictions in ways that resemble the climatology and environments of severe storms, and the values of the contributions generally scale with values of the environmental inputs, effectively enhancing the interpretability of the ML system.
Second, CSU-MLP forecast performance is examined across different synoptic regimes in an effort to understand which types of environmental conditions tend to lead to skillful versus less-skillful forecast performance. Self organizing maps (SOMs), which are a type of ML, are employed to statistically diagnose regimes across two years of reanalysis data. The skill of day-2 CSU-MLP probabilistic tornado, wind, and hail forecasts are examined across the SOM-identified regimes. This work shows that SOMs are successful at identifying distinct atmospheric patterns using only surface-based convective available potential energy (SBCAPE) and vertical wind shear as inputs. At times, the best- and worst-performing CSU-MLP forecasts occur under highly similar atmospheric conditions, though there is some evidence that suggests that the best-performing forecasts at times occur when greater synoptic forcing is present.
Third, forecast output from three deep learning weather prediction (DLWP) models, GraphCast, Pangu-Weather, and FourCastNetv2, is studied to investigate how well they model severe storm environments and capture convection-related parameters. This work explores both native and derived fields from 22 months of daily forecasts from these three models, all of which were initialized with input conditions from the Global Forecasting System (GFS). The output is compared to ERA-5 reanalysis and GFS forecasts, both broadly and for specific convective events. Overarching results from this study show that the DLWP model forecasts tend to be characterized by less moisture and greater instability compared to ERA-5. For specific events, the DLWP forecasts can reasonably capture convective environments at least a week in advance and are competitive with the GFS. However they tend to underforecast the vertical wind shear magnitude, and their limited vertical resolution can lead to overly smooth profiles that lack key details such as stable layers.