Which significance test performs the best in climate simulations?

Abstract

Climate change simulated with climate models needs a significance testing to establish the robustness of simulated climate change relative to model internal variability. Student's t-test has been the most popular significance testing technique despite more sophisticated techniques developed to address autocorrelation. We apply Student's t-test and four advanced techniques in establishing the significance of the average over 20 continuous-year simulations, and validate the performance of each technique using much longer (375–1000 yr) model simulations. We find that all the techniques tend to perform better in precipitation than in surface air temperature. A sizable performance gain using some of the advanced techniques is realised in the model Ts output portion with strong positive lag-1 yr autocorrelation (> + 0.6), but this gain disappears in precipitation. Furthermore, strong positive lag-1 yr autocorrelation is found to be very uncommon in climate model outputs. Thus, there is no reason to replace Student's t-test by the advanced techniques in most cases.

• autocorrelation
• temporal correlation
• internal variability
• climate noise
• significance test
• Student's t-test

Acknowledgements

This work was supported by the Korea Meteorological Administration Research and Development Program (ref: CATER 2012-7100) and by the Korean-Sweden Research Cooperation Program of the National Research Foundation of Korea (ref: 2012K2A3A1035889). We furthermore acknowledge the Model and Data Group at Max Planck Institut für Meteorologie for making their ECHO-G output available for analysis. Special thanks to Eduardo Zorita of the Helmholtz-Zentrum Geesthacht for pointing us to this data.