Performance evaluation of AR5-CMIP5 models for the representation of seasonal and multi-annual variability of precipitation in Brazilian hydropower sector basins under RCP8.5 scenario

Figures & data

Figure 1. Map of the Brazilian hydropower sector (NIS) basins studied.

Table 1. The CMIP5 models analysed in this study. The models are identified in some figures by the ID number in column 1.

ID	Models	Institution	Reference
1	ACCESS1-0	Commonwealth Scientific and Industrial Research Organization (CSIRO) and Bureau of Meteorology (BOM), Australia	Bi et al. (2013b), Dix et al. (2013)
2	ACCESS1-3
3	bcc-csm1-1	Beijing Climate Centre, China Meteorological Administration, China	Wu (2012), Xin et al. (2012, 2013)
4	bcc-csm1-1-m
5	BNU-ESM	College of Global Change and Earth System Science, Beijing Normal University, China	Dai et al. (2003, 2004)
6	CanESM2	Canadian Centre for Climate Modelling and Analysis, Canada	Arora et al. (2011), von Salzen et al. (2013)
7	CESM1-CAM5	Community Earth System Model Contributors, USA	Hurrell et al. (2013)
8	CMCC-CM	Centro Euro-Mediterraneo per i Cambiamenti Climatici, Italy	Fogli et al. (2009), Scoccimarro et al. (2011)
9	CMCC-CMS		Fogli et al. (2009)
10	CNRM-CM5	Centre National de Recherches Meteorologiques/Centre Europeen de Recherche et Formation Avancees en Calcul Scientifique, France	Voldoire et al. (2013)
11	CSIRO-Mk3-6–0	CSIRO in collaboration with Queensland Climate Change Centre of Excellence, Australia	Rotstayn et al. (2012)
12	GFDL-CM3	Geophysical Fluid Dynamics Laboratory – NOAA, USA	Delworth et al. (2006), Donner et al. (2011),
13	GFDL-ESM2G		Dunne et al. (2012, 2013)
14	GFDL-ESM2M
15	GISS-E2-H	NASA Goddard Institute for Space Studies, USA	Schmidt et al. (2006)
16	GISS-E2-H-CC
17	GISS-E2-R
18	GISS-E2-R-CC
19	HadGEM2-AO	National Institute of Meteorological Research/Korea Meteorological Administration, South Korea	Collins et al. (2011), Martin et al. (2011)
20	HadGEM2-CC	Met Office Hadley Centre, UK
21	HadGEM2-ES
22	inmcm4	Institute for Numerical Mathematics, Russian Academy of Sciences	Volodin et al. (2010)
23	IPSL-CM5A-LR	Institut Pierre-Simon Laplace, France	Dufresne et al. (2012)
24	IPSL-CM5A-MR
25	IPSL-CM5B-LR
26	MIROC-ESM	Japan Agency for Marine-Earth Science and Technology, Atmosphere and Ocean Research Institute (The University of Tokyo), and National Institute for Environmental Studies	Watanabe et al., 2011
27	MIROC-ESM-CHEM
28	MIROC5	Atmosphere and Ocean Research Institute (The University of Tokyo), National Institute for Environmental Studies, and Japan Agency for Marine-Earth Science and Technology	Watanabe et al., 2010
29	MPI-ESM-LR	Max Planck Institute for Meteorology, Germany	Stevens et al., 2012
30	MPI-ESM-MR
31	MRI-CGCM3	Meteorological Research Institute, Japan	Yukimoto et al., 2011; Yukimoto et al., 2012
32	NorESM1-M	Norwegian Climate Centre, Norway	Iversen et al., 2013

Figure 2. Scheme of evaluation of the CMIP5 models for the period 1950–1999. All indexes are explained in the text.

Figure 3. Comparison between GPCC and CRU datasets of the basins in the NIS for (a) mean total annual rainfall and (b) coefficient of variation.

Figure 4. (a) Annual precipitation time series and (b) monthly precipitation climatology over São Luiz do Tapajós basin.

Figure 5. Climatology in terms of precipitation (1950–1999) from six CMIP5 global models and GPCC data (dashed line) for the Furnas, Itaipú, Sobradinho and Tucuruí basins. The models are chosen according to the AVAL_s criterion. The three best and three worst models are shown.

Table 2. Values of the proposed statistical indices for the evaluation of the CMIP5 models at the seasonal scale for the Brazilian electricity sector (NIS) basins. Bold values of the Pearson correlation coefficient (CORREL) are not significant at the 95% level.

	CORREL			RMSE
Basin	Max	Med	Min	Max	Med	Min
Emborcação	0.992	0.974	0.874	5.177	1.699	0.912
Nova Ponte	0.995	0.980	0.908	4.301	1.437	0.764
Itumbiana	0.993	0.971	0.855	5.602	1.948	0.805
São Simão	0.997	0.975	0.885	5.192	1.896	0.491
Furnas	0.995	0.979	0.867	3.663	1.519	0.788
Água Vermelha	0.993	0.978	0.923	4.004	1.668	0.716
N. Avanhandava	0.989	0.967	0.868	4.273	1.930	0.814
Porto Primavera	0.965	0.939	0.744	5.663	2.849	1.251
Rosana	0.987	0.972	0.921	5.098	2.565	0.857
Itaipú	0.870	0.820	0.692	7.158	4.093	1.443
Santa Cecília	0.992	0.964	0.836	3.373	1.660	0.712
Salto Caxias	0.816	0.681	0.492	6.371	3.508	1.100
Itá	0.823	0.581	0.404	5.272	3.140	1.181
Dona Francisca	0.651	0.050	–0.401	6.302	3.731	1.498
Três Marias	0.997	0.979	0.872	4.085	1.520	0.596
Sobradinho	0.982	0.925	0.707	7.605	2.866	1.635
Xingó	0.978	0.926	0.797	6.318	3.715	1.549
Serra da Mesa	0.994	0.963	0.808	6.671	2.279	0.754
Lageado	0.993	0.958	0.774	7.354	2.762	0.831
Tucuruí	0.994	0.974	0.839	5.993	2.079	0.735
Belo Monte	0.989	0.963	0.760	5.209	1.921	1.043
Teles Pires	0.997	0.977	0.835	5.182	1.863	0.630
S. L. do Tapajós	0.996	0.972	0.713	4.525	1.747	0.515
Santo Antônio	0.997	0.983	0.928	3.780	1.626	0.404

Table 3. AVAL_s of the CMIP5 models tested for the Furnas, Itaipu, Sobradinho and Tucuruí basins.

Model	Basin
	Furnas	Itaipu	Sobradinho	Tucurui
ACCESS1-0	0.802	0.883	0.926	0.880
ACCESS1-3	0.967	0.747	0.921	0.824
bcc-csm1-1	0.850	0.591	0.993	0.947
bcc-csm1-1-m	0.803	0.544	0.995	0.719
BNU-ESM	0.951	0.576	0.923	0.923
CanESM2	0.877	0.780	0.607	0.655
CESM1-CAM5	0.986	0.653	0.790	0.950
CMCC-CM	0.517	0.660	0.655	0.809
CMCC-CMS	0.688	0.703	0.753	0.838
CNRM-CM5	0.857	0.863	0.800	0.991
CSIRO-Mk3-6–0	0.743	0.516	0.962	0.823
GFDL-CM3	0.605	0.395	0.276	0.461
GFDL-ESM2G	0.108	0.115	0.157	0.263
GFDL-ESM2M	0.307	0.117	0.291	0.359
GISS-E2-H	0.243	0.519	0.648	0.698
GISS-E2-H-CC	0.089	0.573	0.682	0.764
GISS-E2-R	0.818	0.633	0.689	0.636
GISS-E2-R-CC	0.799	0.652	0.680	0.657
HadGEM2-AO	0.979	0.973	0.971	1.000
HadGEM2-CC	0.851	0.734	0.940	0.943
HadGEM2-ES	0.864	0.929	0.869	0.922
inmcm4	0.796	0.406	0.499	0.715
IPSL-CM5A-LR	0.703	0.063	0.158	0.060
IPSL-CM5A-MR	0.608	0.000	0.000	0.000
IPSL-CM5B-LR	0.988	0.511	0.352	0.563
MIROC-ESM	0.777	0.644	0.862	0.722
MIROC-ESM-CHEM	0.772	0.632	0.843	0.717
MIROC5	0.947	0.781	0.997	0.876
MPI-ESM-LR	0.864	0.876	0.856	0.839
MPI-ESM-MR	0.821	0.909	0.908	0.811
MRI-CGCM3	0.917	0.564	0.736	0.857
NorESM1-M	0.126	0.466	0.816	0.783

Figure 6. Global wavelet spectrum of the best and worst CMIP5 models for Furnas, Itaipu, Sobradinho and Tucuruí basins in comparison with GPCC data. “o” indicates the best model, while “$\gamma$” indicates the worst; $\gamma$ is the observation data.

Table 4. Best and worst values obtained from the proposed statistical indexes for the seasonal CMIP5 models evaluation on a inter-annual scale, considering the Brazilian electricity sector basins. Bold values of Pearson’s Correlation coefficient are not significant at 95%.

	CORREL			DIST
Basin	Max	Med	Min	Max	Med	Min
Emborcação	0.929	0.798	0.594	1.160	1.084	0.971
Nova Ponte	0.956	0.795	0.418	1.170	1.091	0.949
Itumbiana	0.939	0.747	0.499	1.200	1.111	1.012
São Simão	0.949	0.844	0.628	1.171	1.095	0.917
Furnas	0.937	0.808	0.423	1.187	0.096	0.925
Água Vermelha	0.957	0.855	0.661	1.167	1.085	0.959
Nova Avanhandava	0.919	0.835	0.654	1.121	1.045	0.968
Porto Primavera	0.910	0.745	0.413	1.140	1.059	0.905
Rosana	0.960	0.784	0.452	1.170	1.091	0.950
Itaipú	0.897	0.787	0.591	1.218	1.137	1.023
Santa Cecília	0.966	0.845	0.498	1.157	1.074	0.981
Salto Caxias	0.921	0.750	0.454	1.168	1.109	1.007
Itá	0.943	0.809	0.547	1.145	1.088	0.992
Dona Francisca	0.951	0.857	0.481	1.134	1.060	0.948
Três Marias	0.935	0.778	0.381	1.154	1.069	0.870
Sobradinho	0.940	0.635	0.293	1.199	1.118	1.008
Xingó	0.957	0.696	0.101	1.122	1.024	0.836
Serra da Mesa	0.942	0.783	0.470	1.193	1.103	0.975
Lageado	0.908	0.746	0.366	1.124	1.064	0.932
Tucuruí	0.939	0.747	0.348	1.155	1.036	0.872
Belo Monte	0.948	0.828	0.684	1.096	0.963	0.843
Teles Pires	0.967	0.801	0.571	1.108	0.989	0.877
São Luiz do Tapajós	0.971	0.811	0.586	1.068	0.919	0.758
Santo Antônio	0.976	0.592	0.390	0.974	0.906	0.828

Table 5. AVAL_i of the CMIP5 models for the Furnas, Itaipú, Sobradinho and Tucuruí basins.

Model	Basin
	Furnas	Itaipu	Sobradinho	Tucurui
ACCESS1-0	0.615	0.811	0.371	0.386
ACCESS1-3	0.632	0.616	0.617	0.578
bcc-csm1-1	0.709	0.191	0.647	0.472
bcc-csm1-1-m	0.213	0.198	0.509	0.527
BNU-ESM	0.279	0.447	0.597	0.474
CanESM2	0.390	0.570	0.665	0.499
CESM1-CAM5	0.469	0.489	0.379	0.591
CMCC-CM	0.314	0.398	0.514	0.718
CMCC-CMS	0.496	0.776	0.453	0.630
CNRM-CM5	0.464	0.566	0.424	0.516
CSIRO-Mk3-6–0	0.693	0.366	0.234	0.436
GFDL-CM3	0.407	0.601	0.345	0.588
GFDL-ESM2G	0.351	0.443	0.862	0.743
GFDL-ESM2M	0.685	0.737	0.498	0.342
GISS-E2-H	0.454	0.272	0.327	0.553
GISS-E2-H-CC	0.410	0.348	0.485	0.783
GISS-E2-R	0.434	0.657	0.711	0.845
GISS-E2-R-CC	0.412	0.064	0.475	0.768
HadGEM2-AO	0.566	0.593	0.407	0.507
HadGEM2-CC	0.201	0.211	0.277	0.355
HadGEM2-ES	0.222	0.364	0.718	0.731
inmcm4	0.749	0.319	0.838	0.885
IPSL-CM5A-LR	0.547	0.503	0.858	0.714
IPSL-CM5A-MR	0.618	0.778	0.332	0.541
IPSL-CM5B-LR	0.686	0.311	0.342	0.507
MIROC-ESM	0.617	0.532	0.744	0.432
MIROC-ESM-CHEM	0.676	0.459	0.472	0.442
MIROC5	0.514	0.222	0.465	0.532
MPI-ESM-LR	0.538	0.792	0.310	0.539
MPI-ESM-MR	0.580	0.669	0.398	0.498
MRI-CGCM3	0.854	0.944	0.355	0.456
NorESM1-M	0.492	0.688	0.413	0.501

Figure 7. Taylor diagrams for the Sobradinho, Tucuruí, Itaipu and Furnas hydrographic basins. The concentric arcs labelled REF represent the RMSE normalized by the standard deviation.

Figure 8. Total AVAL_g of the major basins in the Southeast/Midwest of the Brazilian electricity sector (NIS).

Figure 9. AVAL_g of the major basins in (a) the South, (b) the Northeast and (c) the North of the Brazilian electricity sector.

Figure 10. Impact on the average annual rainfall in the twentieth century for CMIP5 models with a RCP8.5 scenario in relation to the reference period (1984–2003) for (a) 2015–2044; (b) 2045–2074 and (c) 2075–2098. The models with AVAL_g below 0.2 were removed from this analysis.

Figure 11. Seasonal projection (2070–2098 relative to 1984–2003) under the RCP8.5 scenario. The grey shaded area is computed by standard deviation anomaly of the models projected onto observational monthly precipitation for (a) Furnas, (b) Itaipú, (c) Sobradinho and (d) Tucuruí basins.

Bi, D., et al., 2013b. The ACCESS coupled model: description, control climate and evaluation. Australian Meteorological and Oceanographic Journal, 63, 41–64. doi:10.22499/2.6301.004

 Google Scholar

Dix, M., et al., 2013. The ACCESS coupled model: documentation of core CMIP5 simulations and initial results. Australian Meteorological and Oceanographic Journal, 63, 83–99. doi:10.22499/2.6301.006

 Google Scholar

Wu, T., 2012. A mass-flux cumulus parameterization scheme for large-scale models: description and test with observations. Climate Dynamics, 38, 725–744. doi:10.1007/s00382-011-0995-3

 Web of Science ®Google Scholar

Xin, X., et al., 2012. How well does BCC_CSM1.1 reproduce the 20th century climate change over China? Atmospheric and Oceanic Science Letters, 6, 21–26.

 Google Scholar

Xin, X., et al., 2013. Climate change projections over East Asia with BCC_CSM1.1 climate model under RCP scenarios. Journal of the Meteorological Society of Japan, 91, 413–429. doi:10.2151/jmsj.2013-401

 Web of Science ®Google Scholar

Dai, Y.J., et al., 2003. The common land model. Bulletin of the American Meteorological Society, 84, 1013–1023. doi:10.1175/BAMS-84-8-1013

 Web of Science ®Google Scholar

Dai, Y.J., Dickinson, R.E., and Wang, Y.P., 2004. A two-big-leaf model for canopy temperature, photosynthesis, and stomatal conductance. Journal of Climate, 17, 2281–2299. doi:10.1175/1520-0442(2004)017<2281:ATMFCT>2.0.CO;2

 Web of Science ®Google Scholar

Arora, V.K., et al., 2011. Carbon emission limits required to satisfy future representative concentration pathways of greenhouse gases. Geophysical Research Letters, 38, L05805. doi:10.1029/2010GL046270

 Web of Science ®Google Scholar

von Salzen, K., et al., 2013. The Canadian fourth generation atmospheric global climate model (CanAM4). Part I: representation of physical processes. Atmosphere-Ocean, 51 (1), 104–125. doi:10.1080/07055900.2012.755610

 Web of Science ®Google Scholar

Hurrell, J., et al., 2013. The community earth system model: A framework for collaborative research. Bulletin of the American Meteorological Society. doi:10.1175/BAMS-D-12–00121

 Web of Science ®Google Scholar

Fogli, P.G., et al.2009. INGV-CMCC carbon (ICC): a carbon cycle earth system model. CMCC Research Papers. Bologna, Italy: Euro-Mediterranean Centre on Climate Change, 31.

 Google Scholar

Scoccimarro, E., et al., 2011. Effects of tropical cyclones on ocean heat transport in a high resolution Coupled General Circulation Model. Journal of Climate, 24, 4368–4384. doi:10.1175/2011JCLI4104.1

 Web of Science ®Google Scholar

Voldoire, A., et al., 2013. The CNRM-CM5.1 global climate model: description and basic evaluation. Climate Dynamics, 40, 2091–2121. doi:10.1007/s00382-011-1259-y

 Web of Science ®Google Scholar

Rotstayn, L.D., et al., 2012. Aerosol- and greenhouse gas-induced changes in summer rainfall and circulation in the Australasian region: A study using single-forcing climate simulations. Atmospheric Chemistry and Physics, 12, 6377–6404. doi:10.5194/acp-12-6377-2012

 Web of Science ®Google Scholar

Delworth, T.L., et al., 2006. GFDL’s CM2 global coupled climate models. Part I: formulation and simulation characteristics. Journal of Climate, 19, 643–674. doi:10.1175/JCLI3629.1

 Web of Science ®Google Scholar

Donner, L.J., et al., 2011. The dynamical core, physical parameterizations, and basic simulation characteristics of the atmospheric component AM3 of the GFDL global coupled model CM3. Journal of Climate, 24, 3484–3519. doi:10.1175/2011JCLI3955.1

 Web of Science ®Google Scholar

Dunne, J.P., et al., 2012. GFDL’s ESM2 global coupled climate-carbon earth system models. Part I: physical formulation and baseline simulation characteristics. Journal of Climate, 25, 6646–6665. doi:10.1175/JCLI-D-11-00560.1

 Web of Science ®Google Scholar

Dunne, J.P., et al., 2013. GFDL’s ESM2 global coupled climate-carbon earth system models part II: carbon system formulation and baseline simulation characteristics. Journal of Climate. doi:10.1175/JCLI-D-12-00150.1

 Web of Science ®Google Scholar

Schmidt, G.A., et al., 2006. Present day atmospheric simulations using GISS ModelE: comparison to in-situ, satellite and reanalysis data. Journal of Climate, 19, 153–192. doi:10.1175/JCLI3612.1

 Web of Science ®Google Scholar

Collins, W.J., et al., 2011. Development and evaluation of an earth-system model-HadGEM2. Geoscientific Model Development, 4, 1051–1075. doi:10.5194/gmd-4-1051-2011

 Web of Science ®Google Scholar

Martin, G.M., et al., 2011. The HadGEM2 family of met office unified model climate configurations. Geophysical Model Development, 4, 723–757. doi:10.5194/gmd-4-723-2011

 Web of Science ®Google Scholar

Volodin, E.M., Dianskii, N.A., and Gusev, A.V., 2010. Simulating present-day climate with the INMCM4.0 coupled model of the atmospheric and oceanic general circulations. Izvestiya Atmospheric and Oceanic Physics, 46, 414–431. doi:10.1134/S000143381004002X

 Web of Science ®Google Scholar

Dufresne, J.-L., et al., 2012. Climate change projections using the IPSL-CM5 earth system model: from CMIP3 to CMIP5. Climate Dynamics. doi:10.1007/s00382-012-1636-1

 PubMed Web of Science ®Google Scholar

Watanabe, M., et al., 2011. Convective control of ENSO simulated in MIROC. Journal of Climate, 24, 543–562. doi:10.1175/2010JCLI3878.1

 Web of Science ®Google Scholar

Watanabe, M., et al., 2010. Improved climate simulation by MIROC5: mean states, variability, and climate sensitivity. Journal of Climate, 23, 6312–6335. doi:10.1175/2010JCLI3679.1

 Web of Science ®Google Scholar

Stevens, B., et al., 2012. The atmospheric component of the MPI-M Earth System Model: ECHAM6. Journal of Advances in Modeling Earth Systems. doi:10.1002/jame.20015

 Web of Science ®Google Scholar

Yukimoto, S., et al.2011. Meteorological research institute-earth system model v1 (MRI-ESM1)—model description. Technical Report of MRI. Ibaraki, Japan, 88 pp.

 Google Scholar

Yukimoto, S., et al., 2012. A new global climate model of the Meteorological Research Institute: MRI-CGCM3–model description and basic performance. Journal of the Meteorological Society of Japan, 90A, 23–64. doi:10.2151/jmsj.2012-A02

 Web of Science ®Google Scholar

Iversen, T., et al., 2013. The Norwegian earth system model, NorESM1–M. Part 2: climate response and scenario projections. Geoscience and Model Development, 6, 1–27.

 Web of Science ®Google Scholar