Overview
Research published on arXiv assesses the capacity of autoregressive, full-depth ocean emulators to accurately replicate forced responses over extended periods, ranging from months to centuries. This evaluation contrasts with AI weather emulators, which benefit from clearly defined objectives and metrics for rapid development and benchmark outperformance. Long-term climate emulators, conversely, rely on training objectives spanning limited model time steps. The study utilized data from a midHolocene experiment of a numerical climate model to examine emulator skill in responding to surface forcings derived from an in-distribution, out-of-sample climate.
Research Context
AI weather emulators have shown rapid development, often surpassing traditional benchmarks, due to their clear objectives and metrics. However, long-term climate emulators present a different challenge. Their primary requirement is to reliably reproduce forced responses across temporal scales from months to centuries. The training objectives for these climate emulators typically cover only a small number of model time steps, raising questions about their ability to generalize and accurately capture long-term climate dynamics.
Approach
The study employed data from the midHolocene experiment of a numerical climate model. This dataset was used to assess autoregressive, full-depth ocean emulators. The primary objective was to evaluate the emulators' skill in responding to surface forcings. The selected climate for this assessment was characterized as an in-distribution, out-of-sample climate, providing a specific testing environment for the emulators' generalization capabilities. As a comparative baseline, the study also considered methods that directly inferred the ocean state from boundary forcings.
Findings
- The assessed autoregressive, full-depth ocean emulators demonstrated generalization to new orbital forcings. They successfully reproduced the spatial structure of the large-scale response, along with changes in seasonal patterns and in the spatial structure of ocean variability. However, the amplitude of these responses was underestimated by the emulators.
- Baseline methods, which infer the ocean state directly from boundary forcings, were also capable of recovering much of the large-scale pattern. This recovery, however, was limited predominantly to near the surface. These baselines did not capture changes in seasonal patterns or ocean variability, suggesting that some representation of dynamics is necessary for these specific aspects.
- Despite their successes in generalizing to new orbital forcings and capturing large-scale spatial structures and seasonal variability, the emulators failed to reproduce the slow, internally driven evolution occurring within the ocean interior.
- The total forced response generated by the emulators was found to be well reconstructed through the linear composition of their independent responses to each individual forcing component.
- Analysis of response tracking across training epochs revealed that convergence on mean state metrics within the training climate did not guarantee that the emulators had acquired the necessary dynamics for a skillful response in the tested out-of-sample climate.
Why This Matters
These experiments establish the midHolocene as a controlled and ground-truthed setting. This specific paleoclimate scenario offers a valuable environment for diagnosing failures in forced-response reproduction by climate emulators. Identifying such failures within this controlled context can occur before these emulators are applied to more complex or out-of-distribution climates, potentially aiding in the refinement of emulator design and training methodologies.