GFDL - Geophysical Fluid Dynamics Laboratory

Climate-Weather Modeling Studies Using a Prototype Global Cloud-System Resolving

Clouds remain the largest source of uncertainty in our understanding of global
climate change. Different aspects of the planetary cloud field can provide positive
and negative feedbacks to the Earth?s energy balance, and clouds of course are directly
implicated in any changes to the planetary distribution of precipitation. A fundamental
problem in our current understanding of the role of clouds in the dynamics of climate
are that current resolutions do not resolve the fundamental length scales associated
with clouds. We expect our understanding of the role of clouds in climate to undergo
a qualitative change as the resolutions of global models begin to encompass clouds.
At these resolutions (which roughly scale with the tropopause height of 10km) non-hydrostatic
dynamics become significant and deep convective processes are resolved. We are poised
at the threshold of being able to run global scale simulations that include direct,
non-parametrized, simulations of deep convective clouds. The goal of this research
is to use the Argonne Leadership Computing Facility to to explore the frontier of
weather prediction and climate modeling with the newly developed Geophysical Fluid
Dynamics (GFDL) global cloud-resolving model. A single unified atmospheric modeling
system with a cubed-sphere dynamical core and bulk cloud microphysics running at
hydrostatic (12.5km) and non-hydrostatic (3.5km) resolutions will be run with the
goal of capturing the climatology of clouds and severe storms in a warming world.
The ability to reproduce historical tropical storm statistics will be used as a
test of this ground-breaking model.

Benefits to Science:

It has long been hypothesized that global cloud-resolving models will provide
fundamental new advances in understanding the role of clouds in climate. The simulations
proposed here will attempt to test that hypothesis. Simulations attempting to reproduce
known tropical storm climatologies will be run at a series of resolutions ranging
from 12.5km (hydrostatic) to 3.5 km (non-hydrostatic), and eventually (in 2012 and
beyond) to 1km. This will provide direct measures of the benefits due to resolution.

Benefits to HPC:

The model uses a hybrid distributed/threaded parallel programming model encoded
in a high-level API in the Flexible Modeling System (FMS). This includes a high-performance
layer for parallel I/O. While the principal expression of hybrid parallelism is
currently based on MPI/OpenMP, it is widely believed that conventional programming
models will begin to fail as we approach the exascale.  (ExaScale Computing
Study: Technology Challenges in Achieving Exascale Systems, U.S. Defense Advanced
Research Projects Agency, 2008) We believe that the challenges of producing a working
model at 1km global resolution will begin to expose limits of traditional programming
models on the IBM-BG/Q. We have proposed several projects that serve as a basis for
both making incremental improvements to the current software infrastructure and
well as developing alternate parallel programming models at the exascale. GFDL has
been in extended discussions with IBM Watson Research about a research program on
extended parallelism semantics. These projects include:

  • Improve single core performance of the code. The areas for study include
    the implementation of: prefetch, transactional memory, and vectorization in
    the code.
  • Exploit additional forms of higher-level parallelism in the codes including
    improvements to our OpenMP implementation.
  • Examine the trade-offs between process and thread based parallelism.
  • Improve the current implementation of the I/O scheme.

Science Summary:

The purpose of the experiments proposed is to validate the global cloud-resolving
climate model via hurricane hindcasts. For this purpose, we propose to perform hurricane
verification studies for the 2008 Atlantic Season. These 2008 storms lasted a total
of 100-days and performing 5-day forecasts on each of the days. We propose to utilize the 12.5km global model
for these forecasts. In addition to these hydrostatic, moderate resolution studies,
we propose to perform high-resolution 3.5km non-hydrostatic model runs on five selected
storms, yet to be determined. We propose to perform high-resolution long-term climate
simulations with the 12.5km GFDL climate model for the year 2008. The year 2008
was chosen to correspond to Year of Tropical Convection research program which is
an initiative jointly organized by the World Climate Research Programme and the
World Weather Research Programme.

Application Summary:

The GFDL weather and climate models are built on the Flexible Modeling System
(FMS). FMS uses a hybrid MPI/OpenMP model. Parallel I/O is run from only the MPI
ranks. The parallel I/O layer allows single and multi-threaded I/O, as well as quilted
I/O from a subset of MPI ranks. Output data uses the netCDF4 library, including
its parallel I/O, chunking and deflation options.

Considerable progress has been made on improvements to the FMS infrastructure
which is needed to perform experiments on the IBM-BG/P. These improvements to FMS
have included:

  • Implementing a memory footprint that scales with increasing core counts,
    incorporating a high-level hybrid programming model and providing an I/O scheme
    that scales with increasing core counts.
  • Several strong-scaling studies for the atmospheric component model
    at 3.5km and 12.5km resolutions are encouraging as they show reasonable scaling
    characteristics on core counts typically available on the IBM-BG/P.

This development activity is intended to expand on the success of this work and
enable the entire FMS software which would include the ocean, ice, … component
models to run the experiments described below at ANL on the IBM-BG/P platform.

Research Team Members:

  • V. Balaji ? Principal Investigator
  • Isaac Held ? Senior Research Scientist
  • Christopher Kerr ? Computational Scientist
  • Shian-Jiann Lin ? Physical Scientist

Figure 1: The figure demonstrates the capabilities
of the GFDL’s prototype global cloud resolving model at 12.5 km resolution. The research is a key part of an extensive study to evaluate the prototype cloud resolving
model’s predictive capabilities with different spatial structures.