Validation of a skill prediction method

Abstract

A large experiment is performed to validate two predictors for the quality of ECMWF forecasts over Western Europe. One predictor yields the spread of the probability distribution for the error in the predicted 500 hPa geopotential height. It is determined by the trace of the covariance matrix for the geographically local forecast error. In addition the spread for the 500 hPa vorticity error at a location near the Netherlands is computed. The local covariance matrix, necessary for determining both predictors, is computed for 607 days, using the tangent linear and an adjoint version of a quasi-geostrophic 3-level model with truncation T21. We assume linear error growth and the absence of model errors. The forward reference orbit is obtained by interpolating actual ECMWF forecasts with the 3-level model. Small values of the predictor imply small error growth, and therefore accurate forecasts. Large values may or may not be associated with large actual forecast errors, depending on whether the initial error strongly projects on the fastest growing modes. How the uncertainty in the structure of the initial error influences the performance of the skill predictor is studied by considering three different covariance matrices for the initial error. Validation of the predicted variance with the 2-day and 3-day ECMWF forecast error shows that for all initial covariance matrices, both predictors provide significant information about the quality of the forecast. In case of small and large predicted variance, the probabilities for small and large prediction errors are 10% higher than the climatological probabilities. Projection of the observed forecast error onto the eigenvectors of the local covariance matrix indicates that a few eigenvectors already describe a large portion of the forecast error.