HIGH-DIMENSIONAL NONLINEAR DATA ASSIMILATION WITH NON-GAUSSIAN OBSERVATION ERRORS FOR THE GEOSCIENCES
July 26, 2023
Chih-Chi Hu
Committee: Peter Jan van Leeuwen (Adviser); Jeffrey Anderson (NCAR); Michael Bell; Christian Kummerow; Michael Kirby (Mathematics, Computer Science)
Abstract
Data assimilation (DA) plays an indispensable role in the modern weather forecasting. DA aims to provide better initial conditions for the model by combining the model forecast and the observations. This can be described elegantly by Bayes theorem. However, directly applying Bayes theorem for atmospheric DA problems is practically unfeasible, due to the extremely high computational cost. Therefore, modern DA methods for weather forecasting make assumptions, namely, Gaussian and linear assumptions, to seek efficient solutions from Bayes theorem. However, these assumptions can be inappropriate, e.g., for convective-scale problems, or for the assimilation of remotely-sensed observations. These inappropriate assumptions can compromise the effectiveness of DA, thereby limiting the utilization of the valuable information within the observations. Therefore, the goal of this dissertation is to seek solutions to tackle the issues arising from the linear and Gaussian assumptions in DA.In the first part, we explore a fully-nonlinear DA method, the particle flow filter (PFF) in high-dimension systems. We first show that the PFF can improve the DA over existing methods for non-Gaussian problems, even when the solution is a multi-modal probability density function, in a high-dimensional Lorenz model. Next, in order to implement PFF in atmospheric problems, we devise a parallelizable algorithm for the PFF in the Data Assimilation Research Testbed (DART), called PFF-DART. We then demonstrate that, for the first time, implementing the PFF in an atmospheric model is indeed possible. We also show that PFF-DART is able to improve the assimilation of non-linear and non-Gaussian observations in a year-long data-assimilation experiment with a simplified general circulation model.
In the second part, we shift our focus to the observation errors. The observation errors play a crucial role in DA as it essentially determines how much weight we put on the observations. Yet, the distribution of the observation errors is often not well known and is mostly assumed to be Gaussian. To better understand the observation errors, we develop a new method, the Deconvolution-based Observation Error Estimation (DOEE), that can estimate the full distribution of the observation error without Gaussian assumptions. We apply DOEE to show that the all-sky microwave radiances indeed have non-Gaussian observation errors, especially in a cloudy and humid environment. Next, in order to evaluate how the non-Gaussian observation errors can affect the forecast, we propose a new way, the evolving-Gaussian method, that can incorporate a general non-Gaussian error into the incremental 4D-Var, which is a commonly used DA method in operational centers. We apply the evolving-Gaussian method for all-sky microwave observations in the Integrated Forecasting System of the European Centre for Medium-Range Weather Forecasts (ECMWF). Preliminary results show improvement for the short-term forecast of lower-tropospheric humidity, cloud, and precipitation, especially in the tropics.
In all, this dissertation provides possible solutions for outstanding non-linear and non-Gaussian DA problems in high-dimension systems. While there are still important remaining issues, we hope this dissertation lays a foundation for future non-linear and non-Gaussian DA research and practice.