Overview
This research evaluates the performance of two machine learning (ML) atmosphere models, GraphCast and FourCastNetV2, in the context of sub-seasonal prediction. The study specifically assessed their capacity to represent key climate drivers of variability, namely the Madden-Julian Oscillation and the Southern Annular Mode. Model skill was examined over two distinct hindcast periods: a 38-year period and a 2.5-year period. A comparison was drawn against the Bureau of Meteorology's physics-based seasonal prediction system, ACCESS-S2, and a more recent physics-based coupled model.
Approach
The evaluation involved a dual-period assessment approach to address the challenge of insufficient independent data for sub-seasonal prediction model validation. The 38-year hindcast period was chosen because it provides a larger sample size for evaluation, though it overlaps with the training windows of the ML models. The 2.5-year hindcast period, on the other hand, was selected for its independence from the ML model training periods. This dual methodology represents a compromise designed to robustly evaluate model performance.
The ML models GraphCast and FourCastNetV2 were compared against established physics-based systems. For the 38-year period, the comparison baseline was the Bureau of Meteorology's ACCESS-S2, a physics-based seasonal prediction system. For the shorter 2.5-year hindcast period, the ML models were compared against a more recent physics-based coupled model.
Findings
- Both GraphCast and FourCastNetV2 exhibited skill for sub-seasonal timescales, despite their design for medium-range forecasting.
- The ML models demonstrated skill that was generally comparable to the ensemble mean of the physical models at shorter lead times.
- At longer lead times, the ML models' skill was comparable to that of individual members within the physical model ensemble.
- The evaluation included an assessment of the ML models' ability to represent the Madden-Julian Oscillation and the Southern Annular Mode, identified as key climate drivers of variability.
Why This Matters
The observed skill of machine learning models like GraphCast and FourCastNetV2 for sub-seasonal prediction, despite their medium-range design, indicates their potential utility in forecasting beyond their originally intended scope. Their comparable performance to established physics-based models, particularly at certain lead times, suggests a valuable contribution to the field of meteorological prediction.