CSU-MLP GEFS Day-1 "First-Guess" Excessive Rainfall Forecasts: Aggregate Evaluation and Synoptic Regimes of Best- and Worst-Performing Forecasts

May 16, 2022

Jacob Escobedo

Committee: Russ Schumacher (Adviser), Susan van den Heever, Daniel Cooley (Statistics)

Abstract

Forecasting for excessive rainfall, particularly flash-flood producing rainfall, is an important problem that remains difficult due to the relatively short temporal and small spatial scales at which they occur. One important operational product that highlights areas for potential excessive rainfall and flash flood occurrences is the Excessive Rainfall Outlook (ERO) issued by the NOAA Weather Prediction Center (WPC), which provides outlooks for lead times of 1-3 days. To address a need for additional tools for WPC forecasters while forming a given ERO, the Colorado State University Machine Learning Probabilities (CSU-MLP) system, a probabilistic forecast system for excessive rainfall (and other convective hazards), was developed to produce forecasts to be used as a "first-guess" ERO. CSU-MLP employs the use of a random forest (RF) algorithm trained using NOAA’s Second-Generation Global Ensemble Forecast System Reforecast (GEFS/R) and precipitation observations, while using the operational GEFS with the trained model to produce real-time forecasts. Initially developed as a medium range guidance (2-3 day lead time), CSU-MLP has produced day-1 forecasts that have been evaluated favorably during the 4-week Flash Flood and Intense Rainfall Experiment (FFaIR) in the summer of 2020. However, CSU-MLP day-1 forecasts have been observed to have daily forecast skill that can vary widely between days. This work will include an aggregate evaluation of CSU-MLP day-1 forecasts over a longer period of study (3 March 2019 – 15 October 2020) and an identification of synoptic regimes that these forecasts tend to perform at their best and worst.

Results show that CSU-MLP day-1 forecasts are reliable, provide adequate discrimination of excessive rainfall events and non-events (AuROC =0.819), and have comparable performance, evaluated by use of the Brier skill score (BSS), to that of the ERO (CSU-MLP BSS = 0.083; ERO BSS = 0.085). However, CSU-MLP forecasts have a higher frequency of categorical probabilities (≥ 0.05) which results in larger variations of daily BSS. Synoptic regimes of best-performing daily forecasts reveal a tendency for these regimes to be characterized by moderate to strong large-scale forcing and relatively high low-level and column moisture. This would include warm-season regimes with moderate amplitude upper-level troughs, tropical cyclones, cut-off lows, and cool-season regimes where strong forcing is co-located near an abundant moisture source. Forecasts tend to perform worst when there is strong large-scale forcing and low-level and column moisture is relatively low, such as cool-season regimes with large amplitude troughs and surface cyclones but higher levels of atmospheric moisture are not present or not as widespread. This work has implications for WPC forecasters as they use the "first-guess" forecasts while developing the ERO for a given day, as well as implications for future CSU-MLP system model iterations and/or designs.