GFDL - Geophysical Fluid Dynamics Laboratory

Climate-Weather Modeling Studies Using a Prototype Global Cloud-System Resolving Model

Clouds remain the largest source of uncertainty in our understanding of global climate change. Different aspects of the planetary cloud field can provide positive and negative feedbacks to the Earth’s energy balance, and clouds of course are directly implicated in any changes to the planetary distribution of precipitation. A fundamental problem in our current understanding of the role of clouds in the dynamics of climate are that current resolutions do not resolve the fundamental length scales associated with clouds. We expect our understanding of the role of clouds in climate to undergo a qualitative change as the resolutions of global models begin to encompass clouds. At these resolutions (which roughly scale with the tropopause height of 10km) non-hydrostatic dynamics become significant and deep convective processes are resolved. We are poised at the threshold of being able to run global scale simulations that include direct, non-parametrized, simulations of deep convective clouds. The goal of this research is to use the Argonne Leadership Computing Facility to to explore the frontier of weather prediction and climate modeling with the newly developed Geophysical Fluid Dynamics (GFDL) global cloud-resolving model. A single unified atmospheric modeling system with a cubed-sphere dynamical core and bulk cloud microphysics running at hydrostatic (12.5km) and non-hydrostatic (3.5km) resolutions will be run with the goal of capturing the climatology of clouds and severe storms in a warming world. The ability to reproduce historical tropical storm statistics will be used as a test of this ground-breaking model.

Benefits to Science:

It has long been hypothesized that global cloud-resolving models will provide fundamental new advances in understanding the role of clouds in climate. The simulations proposed here will attempt to test that hypothesis. Simulations attempting to reproduce known tropical storm climatologies will be run at a series of resolutions ranging from 12.5km (hydrostatic) to 3.5 km (non-hydrostatic), and eventually (in 2012 and beyond) to 1km. This will provide direct measures of the benefits due to resolution.

Benefits to HPC:

The model uses a hybrid distributed/threaded parallel programming model encoded in a high-level API in the Flexible Modeling System (FMS). This includes a high-performance layer for parallel I/O. While the principal expression of hybrid parallelism is currently based on MPI/OpenMP, it is widely believed that conventional programming models will begin to fail as we approach the exascale.  (ExaScale Computing Study: Technology Challenges in Achieving Exascale Systems, U.S. Defense Advanced Research Projects Agency, 2008) We believe that the challenges of producing a working model at 1km global resolution will begin to expose limits of traditional programming models on the IBM-BG/Q. We have proposed several projects that serve as a basis for both making incremental improvements to the current software infrastructure and well as developing alternate parallel programming models at the exascale. GFDL has been in extended discussions with IBM Watson Research about a research program on extended parallelism semantics. These projects include:

  • Improve single core performance of the code. The areas for study include the implementation of: prefetch, transactional memory, and vectorization in the code.
  • Exploit additional forms of higher-level parallelism in the codes including improvements to our OpenMP implementation.
  • Examine the trade-offs between process and thread based parallelism.
  • Improve the current implementation of the I/O scheme.

Science Summary:

The purpose of the experiments proposed is to validate the global cloud-resolving climate model via hurricane hindcasts. For this purpose, we propose to perform hurricane verification studies for the 2008 Atlantic Season. These 2008 storms lasted a total of 100-days and performing 5-day forecasts on each of the days. We propose to utilize the 12.5km global model for these forecasts. In addition to these hydrostatic, moderate resolution studies, we propose to perform high-resolution 3.5km non-hydrostatic model runs on five selected storms, yet to be determined. We propose to perform high-resolution long-term climate simulations with the 12.5km GFDL climate model for the year 2008. The year 2008 was chosen to correspond to Year of Tropical Convection research program which is an initiative jointly organized by the World Climate Research Programme and the World Weather Research Programme.

Application Summary:

The GFDL weather and climate models are built on the Flexible Modeling System (FMS). FMS uses a hybrid MPI/OpenMP model. Parallel I/O is run from only the MPI ranks. The parallel I/O layer allows single and multi-threaded I/O, as well as quilted I/O from a subset of MPI ranks. Output data uses the netCDF4 library, including its parallel I/O, chunking and deflation options.

Considerable progress has been made on improvements to the FMS infrastructure which is needed to perform experiments on the IBM-BG/P. These improvements to FMS have included:

  • Implementing a memory footprint that scales with increasing core counts, incorporating a high-level hybrid programming model and providing an I/O scheme that scales with increasing core counts.
  • Several strong-scaling studies for the atmospheric component model at 3.5km and 12.5km resolutions are encouraging as they show reasonable scaling characteristics on core counts typically available on the IBM-BG/P.

This development activity is intended to expand on the success of this work and enable the entire FMS software which would include the ocean, ice, … component models to run the experiments described below at ANL on the IBM-BG/P platform.

Research Team Members:

  • V. Balaji – Principal Investigator
  • Isaac Held – Senior Research Scientist
  • Christopher Kerr – Computational Scientist
  • Shian-Jiann Lin – Physical Scientist

Figure 1: The figure demonstrates the capabilities of the GFDL’s prototype global cloud resolving model at 12.5 km resolution. The research is a key part of an extensive study to evaluate the prototype cloud resolving model’s predictive capabilities with different spatial structures.