Publications

Preprints

1. A Multi-Model Ensemble Kalman Filter for Data Assimilation and Forecasting

   Bach, Eviatar, and Ghil, Michael

   AbsarXiv

   Data assimilation (DA) aims to optimally combine model forecasts and noisy observations. Multi-model DA generalizes the variational or Bayesian formulation of the Kalman filter, and we prove here that it is also the minimum variance linear unbiased estimator. However, previous implementations of this approach have not estimated the model error, and have therewith not been able to correctly weight the separate models and the observations. Here, we show how multiple models can be combined for both forecasting and DA by using an ensemble Kalman filter with adaptive model error estimation. This methodology is applied to the Lorenz96 model, and it results in significant error reductions compared to the best model and to an unweighted multi-model ensemble.
2. Deep Learning-Enhanced Ensemble-Based Data Assimilation for High-Dimensional Nonlinear Dynamical Systems

   Chattopadhyay, Ashesh, Nabizadeh, Ebrahim,  Bach, Eviatar, and Hassanzadeh, Pedram

   AbsarXiv HTML

   Data assimilation (DA) is a key component of many forecasting models in science and engineering. DA allows one to estimate better initial conditions using an imperfect dynamical model of the system and noisy/sparse observations available from the system. Ensemble Kalman filter (EnKF) is a DA algorithm that is widely used in applications involving high-dimensional nonlinear dynamical systems. However, EnKF requires evolving large ensembles of forecasts using the dynamical model of the system. This often becomes computationally intractable, especially when the number of states of the system is very large, e.g., for weather prediction. With small ensembles, the estimated background error covariance matrix in the EnKF algorithm suffers from sampling error, leading to an erroneous estimate of the analysis state (initial condition for the next forecast cycle). In this work, we propose hybrid ensemble Kalman filter (H-EnKF), which is applied to a two-layer quasi-geostrophic flow system as a test case. This framework utilizes a pre-trained deep learning-based data-driven surrogate that inexpensively generates and evolves a large data-driven ensemble of the states of the system to accurately compute the background error covariance matrix with less sampling error. The H-EnKF framework estimates a better initial condition without the need for any ad-hoc localization strategies. H-EnKF can be extended to any ensemble-based DA algorithm, e.g., particle filters, which are currently difficult to use for high dimensional systems.

2022

1. Towards physics-inspired data-driven weather forecasting: integrating data assimilation with a deep spatial-transformer-based U-NET in a case study with ERA5

   Chattopadhyay, Ashesh, Mustafa, Mustafa, Hassanzadeh, Pedram,  Bach, Eviatar, and Kashinath, Karthik

   Geoscientific Model Development Mar 2022

   Abs HTML

   There is growing interest in data-driven weather prediction (DDWP), e.g., using convolutional neural networks such as U-NET that are trained on data from models or reanalysis. Here, we propose three components, inspired by physics, to integrate with commonly used DDWP models in order to improve their forecast accuracy. These components are (1) a deep spatial transformer added to the latent space of U-NET to capture rotation and scaling transformation in the latent space for spatiotemporal data, (2) a data-assimilation (DA) algorithm to ingest noisy observations and improve the initial conditions for next forecasts, and (3) a multi-time-step algorithm, which combines forecasts from DDWP models with different time steps through DA, improving the accuracy of forecasts at short intervals. To show the benefit and feasibility of each component, we use geopotential height at 500 hPa (Z500) from ERA5 reanalysis and examine the short-term forecast accuracy of specific setups of the DDWP framework. Results show that the spatial-transformer-based U-NET (U-STN) clearly outperforms the U-NET, e.g., improving the forecast skill by 45 %. Using a sigma-point ensemble Kalman (SPEnKF) algorithm for DA and U-STN as the forward model, we show that stable, accurate DA cycles are achieved even with high observation noise. This DDWP+DA framework substantially benefits from large (O(1000)) ensembles that are inexpensively generated with the data-driven forward model in each DA cycle. The multi-time-step DDWP+DA framework also shows promise; for example, it reduces the average error by factors of 2–3. These results show the benefits and feasibility of these three components, which are flexible and can be used in a variety of DDWP setups. Furthermore, while here we focus on weather forecasting, the three components can be readily adopted for other parts of the Earth system, such as ocean and land, for which there is a rapid growth of data and need for forecast and assimilation.
2. Impacts of 319 wind farms on surface temperature and vegetation in the United States

   Qin, Yingzuo, Li, Yan, Xu, Ru, Hou, Chengcheng, Armstrong, Alona,  Bach, Eviatar, Wang, Yang, and Fu, Bojie

   Environmental Research Letters Feb 2022

   Abs HTML

   The development of wind energy is essential for decarbonizing energy production. However, the construction of wind farms changes land surface temperature (LST) and vegetation by modifying land surface properties and disturbing land–atmosphere interactions. In this study, we used moderate resolution imaging spectroradiometer satellite data to quantify the impacts on local climate and vegetation of 319 wind farms in the United States. Our results indicated insignificant impacts on LST during the daytime but significant warming of 0.10 \textdegree C of annual mean nighttime LST averaged over all wind farms, and 0.36 \textdegree C for those 61% wind farms with warming. The nighttime LST impacts exhibited seasonal variations, with stronger warming in winter and autumn, up to 0.18 \textdegree C, but weaker effects in summer and spring. We observed a decrease in peak normalized difference vegetation index (NDVI) for 59% of wind farms due to infrastructure construction, with an average reduction of 0.0067 compared to non-wind farm areas. The impacts of wind farms depended on wind farm size, with winter LST impacts for large and small wind farms ranging from 0.21 \textdegree C to 0.14 \textdegree C, and peak NDVI impacts ranging from -0.009 to -0.006. The LST impacts declined with the increasing distance from the wind farm, with detectable impacts up to 10 km. In contrast, the vegetation impacts on NDVI were only evident within the wind farm locations. Wind farms built in grassland and cropland showed larger warming effects but weaker vegetation impact than those built on forests. Furthermore, spatial correlation analyses with environmental factors suggest limited geographical controls on the heterogeneous wind farm impacts and highlight the important role of local factors. Our analyses based on a large sample offer new evidence for wind farm impacts with improved representativeness compared to previous studies. This knowledge is important to fully understand the climatic and environmental implications of energy system decarbonization.

2021

1. Ensemble Oscillation Correction (EnOC): Leveraging Oscillatory Modes to Improve Forecasts of Chaotic Systems

   Bach, Eviatar, Mote, Safa, Krishnamurthy, V., Sharma, A. Surjalal, Ghil, Michael, and Kalnay, Eugenia

   Journal of Climate Jul 2021

   Abs HTML

   Oscillatory modes of the climate system are among its most predictable features, especially at intraseasonal time scales. These oscillations can be predicted well with data-driven methods, often with better skill than dynamical models. However, since the oscillations only represent a portion of the total variance, a method for beneficially combining oscillation forecasts with dynamical forecasts of the full system was not previously known. We introduce Ensemble Oscillation Correction (EnOC), a general method to correct oscillatory modes in ensemble forecasts from dynamical models. We compute the ensemble mean—or the ensemble probability distribution—with only the best ensemble members, as determined by their discrepancy from a data-driven forecast of the oscillatory modes. We also present an alternate method that uses ensemble data assimilation to combine the oscillation forecasts with an ensemble of dynamical forecasts of the system (EnOC-DA). The oscillatory modes are extracted with a time series analysis method called multichannel singular spectrum analysis (M-SSA), and forecast using an analog method. We test these two methods using chaotic toy models with significant oscillatory components and show that they robustly reduce error compared to the uncorrected ensemble. We discuss the applications of this method to improve prediction of monsoons as well as other parts of the climate system. We also discuss possible extensions of the method to other data-driven forecasts, including machine learning.
2. parasweep: A template-based utility for generating, dispatching, and post-processing of parameter sweeps

   Bach, Eviatar

   SoftwareX Jan 2021

   Abs HTML

   We introduce parasweep, a free and open-source utility for facilitating parallel parameter sweeps with computational models. Instead of requiring parameters to be passed by command-line, which can be error-prone and time-consuming, parasweep leverages the model’s existing configuration files using a template system, requiring minimal code changes. parasweep supports a variety different sweep types, generating parameter sets accordingly and dispatching a parallel job for each set, with support for local execution as well as common high-performance computing (HPC) job schedulers. Post-processing is facilitated by providing a mapping between the parameter sets and the simulations. We demonstrate the usage of parasweep with an example.

2019

1. Local Atmosphere–Ocean Predictability: Dynamical Origins, Lead Times, and Seasonality

   Bach, Eviatar, Motesharrei, Safa, Kalnay, Eugenia, and Ruiz-Barradas, Alfredo

   Journal of Climate Oct 2019

   Abs HTML

   Due to the physical coupling between atmosphere and ocean, information about the ocean helps to better predict the future of the atmosphere, and in turn, information about the atmosphere helps to better predict the ocean. Here, we investigate the spatial and temporal nature of this predictability: where, for how long, and at what frequencies does the ocean significantly improve prediction of the atmosphere, and vice versa? We apply Granger causality, a statistical test to measure whether a variable improves prediction of another, to local time series of sea surface temperature (SST) and low-level atmospheric variables. We calculate the detailed spatial structure of the atmosphere-to-ocean and ocean-to-atmosphere predictability. We find that the atmosphere improves prediction of the ocean most in the extratropics, especially in regions of large SST gradients. This atmosphere-to-ocean predictability is weaker but longer-lived in the tropics, where it can last for several months in some regions. On the other hand, the ocean improves prediction of the atmosphere most significantly in the tropics, where this predictability lasts for months to over a year. However, we find a robust signature of the ocean on the atmosphere almost everywhere in the extratropics, an influence that has been difficult to demonstrate with model studies. We find that both the atmosphere-to-ocean and ocean-to-atmosphere predictability are maximal at low frequencies, and both are larger in the summer hemisphere. The patterns we observe generally agree with dynamical understanding and the results of the Kalnay dynamical rule, which diagnoses the direction of forcing between the atmosphere and ocean by considering the local phase relationship between simultaneous sea surface temperature and vorticity anomaly signals. We discuss applications to coupled data assimilation.

2018

1. How sensitive are mountain glaciers to climate change? Insights from a block model

   Bach, Eviatar, Radić, Valentina, and Schoof, Christian

   Journal of Glaciology Apr 2018

   Abs HTML

   Simple models of glacier volume evolution are important in understanding features of glacier response to climate change, due to the scarcity of data adequate for running more complex models on a global scale. Two quantities of interest in a glacier’s response to climate changes are its response time and its volume sensitivity to changes in the equilibrium line altitude (ELA). We derive a simplified, computationally inexpensive model of glacier volume evolution based on a block model with volume–area–length scaling. After analyzing its steady-state properties, we apply the model to each mountain glacier worldwide and estimate regionally differentiated response times and sensitivities to ELA changes. We use a statistical method from the family of global sensitivity analysis methods to determine the glacier quantities, geometric and climatic, that most influence the model output. The response time is dominated by the climatic setting reflected in the mass-balance gradient in the ablation zone, followed by the surface slope, while volume sensitivity is mainly affected by glacier size, followed by the surface slope.
2. Climate model shows large-scale wind and solar farms in the Sahara increase rain and vegetation

   Li, Yan, Kalnay, Eugenia, Motesharrei, Safa, Rivas, Jorge, Kucharski, Fred, Kirk-Davidoff, Daniel,  Bach, Eviatar, and Zeng, Ning

   Science Sep 2018

   Abs HTML

   Wind and solar farms offer a major pathway to clean, renewable energies. However, these farms would significantly change land surface properties, and, if sufficiently large, the farms may lead to unintended climate consequences. In this study, we used a climate model with dynamic vegetation to show that large-scale installations of wind and solar farms covering the Sahara lead to a local temperature increase and more than a twofold precipitation increase, especially in the Sahel, through increased surface friction and reduced albedo. The resulting increase in vegetation further enhances precipitation, creating a positive albedo–precipitation–vegetation feedback that contributes ~80% of the precipitation increase for wind farms. This local enhancement is scale dependent and is particular to the Sahara, with small impacts in other deserts.