Assessing the impact of observations on ocean forecasts and reanalyses: Part 2, Regional applications

Abstract

The value of global (e.g. altimetry, satellite sea-surface temperature, Argo) and regional (e.g. radars, gliders, instrumented mammals, airborne profiles and biogeochemical) observation-types for monitoring the mesoscale ocean circulation and biogeochemistry is demonstrated using a suite of global and regional prediction systems and remotely-sensed data. A range of techniques is used to demonstrate the value of different observation-types to regional systems and the benefit of high-resolution and adaptive sampling for monitoring the mesoscale circulation. The techniques include Observing System Experiments, Observing System Simulation Experiments, adjoint sensitivities, representer matrix spectrum, observation footprints and spectral analysis. It is shown that local errors in global and basin-scale systems can be significantly reduced when assimilating observations from regional observing systems.

Introduction

High-resolution ocean forecasting, including models that resolve eddies and the mesoscale circulation, depends heavily on ocean observations. The generation and evolution of mesoscale structures in the ocean are often chaotic and so there are limitations to predictability. Without data assimilation, forecasts of the mesoscale circulation are inaccurate. The best one could hope for is an ensemble of forecasts that predict the likelihood of a particular mesoscale feature (e.g. eddy, front) developing or evolving. Data assimilation uses observations to initialise and constrain those features of the ocean that are unpredictable. But for high-resolution models, the demands on an observing system are significant. Simply stated, a forecast system cannot accurately predict a chaotic feature that is poorly observed.

The main conventional components of the Global Ocean Observing System (GOOS) include satellite altimetry and sea-surface temperature (SST), Argo floats, eXpendable BathyThermographs (XBT), surface drifting buoys and moorings. These platforms partly resolve the mesoscale ocean circulation (Oke & Schiller, 2007). Regional ocean observing systems are being developed around the world under the auspices of the GOOS Regional Alliance [GRA; www.iocgoos.org], which is comprised of national and institutional efforts that cooperate regionally to complement the GOOS. Regional observation platforms include mooring arrays, land-based high-frequency (HF) radar arrays and repeat glider deployments and are organised under projects such as EuroGOOS [www.eurogoos.org], USGOOS [www.ioc-goos.org/usgoos] or IOOS [www.ioos.noaa.gov], IOGOOS [www.incois.gov.in/Incois/iogoos/intro.jsp] and IMOS [www.imos.org.au]. For regional applications, such as shelf-scale models or even basin-scale models, observations delivered under the GRA are important for providing additional constraints for initialising ocean forecasts. Models on these scales often include different physics (e.g. tides) and resolve different processes (e.g. fronts, small eddies) and can be used for the design and assessment of regional observing systems. This review demonstrates the value of assimilating observations from HF radars, gliders, instrumented mammals, and XBT observations. Additionally, the impact of assimilating data from multiple satellite altimeters into an eddy-resolving model is demonstrated, along with the potential of in-situ biogeochemical observations to constrain a coupled physical-biological model.

This review article is intended to capture a broad crosssection of observation impact studies for regional applications, including studies using global, basin-scale and shelf-scale models and high-resolution observational databases. Applications include results from traditional Observing System Experiments (OSEs; e.g. Oke & Schiller, 2007), traditional Observing System Simulation Experiments (OSSEs; e.g. Milles, 1990), OSSEs with adaptive sampling (Bishop et al. 2001), the Representer Matrix spectrum (RMspec) using stochastic modelling (Le Hénaff, 2009) and observation footprints (Oke & Sakov, 2012).

Contributions from the GODAE community to observing system design and evaluation have been previously documented (Oke et al. 2009). This review article provides an update on this community’s contribution to regional observing system evaluation, under the auspices of GODAE OceanView, and is complemented by a companion paper describing global studies (Oke et al. 2015). In this paper, results from a series of studies are presented, followed by a discussion, conclusion, and a series of recommendations.

Results

It is important to note that the results from the types of experiments presented in this paper depend on the details of the model and assimilation system used. This includes the model resolution, model physics, assimilation method, estimated background error covariance, observation error estimate and method of initialisation. The results provide a meaningful representation of the impact of assimilated observations on each system, given their strengths, weaknesses, assumptions and limitations. Furthermore, it is anticipated that although the studies only represent observation impacts in specific regions, for specific times, the results are indicative of observation impacts in other regions with similar dynamics and for other systems using similar methods or approaches. Importantly, the range of approaches presented below gives a good indication of the state of play in observing system evaluation for the operational oceanography community.

Suppose the difference between two OSE simulations is compared, where OSE _X+Y assimilates observation types X and Y, and OSE_Y assimilates only observation type Y. The difference between OSE _X+Y and OSE _Y does not necessarily quantify the ‘improvement’ attributable to observation type X. However, it does faithfully quantify the ‘impact’ of observation type X. It is preferable to quantify the improvement, not just the impact, but the availability of sufficient independent observations is a common problem for systems that seek to assimilate all available observations. Analysis of the difference between two OSEs is common (Oke et al. 2009; Lea et al. 2013; Oke et al. 2015).

Bluelink altimeter OSEs around Australia

Results from a series of OSEs designed to quantify the impact of satellite altimeter data on a regional, eddy-resolving ocean reanalysis system; the Bluelink ReANalysis (BRAN; version 3) system (Oke et al. 2013) are presented in Figure 1. The BRAN configuration uses a near-global configuration of the Modular Ocean Model (Griffies et al. 2004) with 1/10° grid spacing around Australia. An Ensemble Optimal Interpolation (EnOI; Oke et al. 2002) data assimilation system is used to assimilate along-track sea-level anomaly (SLA) from altimeters, satellite sea surface temperature (SST) and in-situ profiles from Argo, XBT and moorings.

Figure 1. Profiles of the 90th percentile of the RMSD for temperature (left) and salinity (right) between OSEs performed using the Bluelink system using data from one and zero altimeters (1ALT – 0ALT; blue), two and one altimeters (2ALT – 1 ALT; green), three and two altimeters (3ALT – 2ALT; red).

The OSEs performed with the BRAN system include runs that assimilate data from zero, one, two and three altimeters, spanning a 12-month period beginning in March 2008. Figure 1 shows profiles of the 90th percentile of the root-mean-squared difference (RMSD) between temperature and salinity in the Australian region for different OSEs. The 90th percentile provides an indication of more extreme impacts of withholding different sub-sets of data. The results presented in Figure 1 show profiles of the RMSD of temperature and salinity in OSEs that assimilate data from a different number of altimeters. All experiments assimilate in-situ profiles and SST and each OSE assimilates SLA data from a different number of altimeters. The greatest impact of altimeter data is made with the addition of the first altimeter (1ALT – 0ALT), where the temperature and salinity change by up to 1.5°C and 0.4 psu, respectively. The addition of the second altimeter (2ALT – 1 ALT) results in changes of temperature and salinity of up to 1°C and 0.18 psu; and the addition of the third altimeter (3ALT – 2ALT) changes of temperature and salinity of up to 0.8°C and 0.14 psu.

The difference between reanalysed SLA and observed SLA (including assimilated and unassimilated data) in each OSE is compared in Table 1, showing the RMSD and correlation between reanalysed SLA and along-track SLA from Jason-1, Jason-2 and Envisat. Also shown in Table 1, is the percentage improvement in the RMSD relative to the OSE that assimilates no altimeter data. These results show that assimilation of data from three altimeters reduces the RMSD with observed SLA by 23%. Consistent with the results in Figure 1, the addition of the first altimeter has the largest impact, reducing the RMSD with observed SLA by 12% and the addition of the second and third altimeter improving SLA by a further 8% and 3% respectively. For comparison, Table 1 also includes an evaluation of the OSE with no data assimilation (with-holding SST and in-situ profiles as well). It is found that with-holding in-situ T/S and SST, in addition to withholding the altimeter data, degrades the modelled SLA by 11%. This indicates that the impact of one altimeter data is similar to the impact of assimilating in-situ T/S data from Argo and XBT.

Table 1. Area-averaged correlation and RMSD between SLA in each OSE and atSLA from all altimeters for the whole Australian region (90-180E, 60S-10N) from the Bluelink altimeter OSEs; and the percentage improvement of each OSE relative to the OSE with zero altimeters (% Improvement = [RMSD_0ALT – RMSD_NALT] / RMSD_0ALT). For reference, the standard deviation of the SLA signal from atSLA is about 15 cm in the region of interest. Note that some observations are assimilated, but statistics reported here are from model ‘forecasts’ – not analyses.

	3ALT	2ALT	1ALT	0ALT	NoDA
RMSD (cm)	6.1	6.3	6.7	7.9	8.8
Correlation	0.67	0.65	0.61	0.45	0.29
% Improvement	23	20	12	0	-11

It should be noted that the RMSD is about 6 cm when data from 3 altimeters are assimilated. Also to note is that the instrument error of altimetry is a couple of cm, the representation error (Oke & Sakov, 2008) is a couple of cm and the model and data assimilation are imperfect. So a difference of 6 cm is arguably about as good as one could expect, without over-fitting observations, given these errors and limitations.

This OSE study demonstrates the value of satellite altimeter data to an eddy-resolving ocean model. It shows that the addition of the first altimeter has the largest impact – but that there are quantitative improvements seen by the addition of a second and third altimeter. Further, it is found that the assimilation of in-situ profiles and SST improves the modelled sea-level (Table 1), demonstrating that observations provide meaningful information on ocean state variables other than those directly measured (e.g. temperature and salinity observations helps initialise SLA).

MFS altimeter OSEs in the Mediterranean Sea

Results from OSEs designed to quantify the impact of satellite altimeter data on a high-resolution ocean forecast system is presented in Figure 2 and Table 2, using the Mediterranean Forecast System (MFS; Tonani et al. 2009; Oddo et al. 2009). The MFS uses 3dVar (Dobricic & Pinardi, 2008) to assimilate data from satellite altimetry, Argo and XBT into a regional configuration of NEMO with 6–7 km resolution and 72 vertical levels. The OSEs performed with the MFS include runs that assimilate data from one (J2), two (J2+SA; J2+C2) and three (J2+SA+C2) altimeters, in addition to Argo and XBT, for the period 26 June to 5 October 2013. The results shown in Figure 2 and Table 2 report the unbiased RMSD between the analysed SLA computed for each assimilation cycle and along-track SLA (atSLA) from each (and all) altimeter. The mean difference of SLA for each track is removed prior to calculation of the RMSD (hence the reference to the unbiased RMSD above). For reference, the standard deviation of the SLA signal in the model domain is 4-1–4.8 cm (from atSLA from altimeters). The time-series shows that the RMSD typically ranges from about 3 cm to 4.5 cm. The OSE that assimilates data from 3 altimeters performs the best and the OSE that assimilates data from 1 altimeter performs the worst. The difference between 1 and 3 altimeters is typically about 10% - which is comparable to the results presented for the Bluelink altimeter OSEs (Table 1).

Table 2. RMSD of analysed SLA (cm) for each OSE with respect to along-track SLA from each altimeter and all altimeters (J2+SA+Cr) for the MFS altimeter OSEs. For reference, the standard deviation of the SLA signal from along-track altimeters is 4.1−4.4 cm in the model domain.

	3ALT (J2+SA+C2)	2ALT (J2+C2)	2ALT (J2+SA)	1ALT (J2)
Jason-2 (J2)	3.0	3.1	3.1	3.3
Saral-AltiKa (SA)	3.4	3.5	3.6	3.8
Cryosat-2 (C2)	3.8	3.9	3.9	4.1
J2+SA+C2	3.5	3.5	3.6	3.8

Figure 2. Time series of the RMSD between analysed SLA in each of the MFS altimeter OSEs during 2013.

The results presented in Table 2 also highlight the difference in the quality of data from different altimeters. Note that the RMSD is greatest for all OSEs for the comparisons with atSLA from Cryosat. This indicates that Cryosat data has larger errors than the other altimeters.

The OSE that assimilates data from only one altimeter (Jason-2) is the least reliable of all the OSEs, producing analyses with the largest RMSD (Table 2) and the most variability (the spikiest time-series) of RMSD (Figure 2). The latter point is particularly important for operational ocean forecasting. Results from these OSEs show that the forecast error (represented here by the RMSD statistics) when only one altimeter is used varies in time more than the forecast error when multiple altimeters are used. This means that decisions made (for search and rescue, for example) based on forecasts (or analyses) from 3ALT can be trusted more than 1ALT – both in terms of the magnitude of the error and the consistency of the error over time.

REMO GOOS OSEs in the Atlantic

The first version of the Brazilian Oceanographic Modeling and Observation Network (REMO) ocean data assimilation system for the Hybrid Coordinate Ocean Model [HYCOM; www.hycom.org] (Bleck, 2002; Chassignet et al. 2003; Halliwell, 2004) has been recently constructed (RODAS_H). At the time of writing, RODAS_H is under evaluation and only preliminary results have been produced so far (Tanajura, 2013; Tanajura, 2014). The system is based on the EnOI scheme mostly following the work of others (Oke et al. 2002; Evensen, 2003; Oke et al. 2007). REMO focuses on simulations, short-range forecasts and observations in the Atlantic Ocean and sub-regions of particular interest to Brazil (Lima et al. 2012) [www.rederemo.org]. The first version of RODAS_H was tested in a 1/4° grid with 21 vertical layers, covering the Atlantic Ocean region from 78S to 50N and from 100W to 20E, excluding the Pacific and the Mediterranean.

For this study, five OSEs are presented for the period January to June 2010. The first OSE is a control run without data assimilation. The other four OSEs assimilate different observations every 3 days, namely: (i) only along-track SLA (Jason-1, Jason-2 and Envisat; A_SLA); (ii) only ¼°-resolution Reynolds SST (A_SST); (iii) only vertical profiles of T/S from Argo (A_Argo); and (iv) all the aforementioned observation types (A_All). Argo data are assimilated using a 3-day observational window. For the other data, only a one-day window is used.

The impact of the observations is assessed by comparing the prior model state (before assimilation) with daily T/S data from 6,988 Argo profiles (Figure 3). These data are independent, since they were not assimilated at the assessment time. The control run attains maximum RMSD of about 2.2°C and 0.34 psu. A slight reduction of RMSD is obtained with the A_SLA OSE, showing that for the REMO system there is little impact of assimilation of SLA in the vertical profiles of T and S, with the impact limited to the top 500 m depth. Assimilation of Argo data only (A_Argo) reduces the maxima RMSD for T and S to about 1.7°C and 0.24 psu, respectively. Assimilation of SST only (A_SST) mostly constrains the temperature near the surface and it does not significantly impact in T and S below 500 m depth. Assimilation of all data (A_All) produces results comparable to the A_Argo, but it is not always better than the A_Argo along the profile. RMSD of T in the A_All experiment is higher than in the A_Argo in the top 250 m and between 600 m and 1400 m approximately. RMSD of S in the A_All experiment is slightly higher than in the A_Argo between 600 m and 1500 m. This is because A_All assimilates all data types, as mentioned above – so the challenge of ‘fitting’ observations of different type using EnOI is more difficult than it is for fitting a single data type. It is expected that this challenge to be met, with superior performance from equivalent A_All OSEs with future, more mature, versions of RODAS_H.

Figure 3. Profiles of the RMSD for temperature (left) and salinity (right) between about 6988 Argo profiles (before assimilation) and model fields from OSEs using the REMO system (RODAS_H) for the entire model domain (100W-20E, 78S-50N) from January 1 to June 30, 2010.

To further investigate the sensitivity of the prior model state in the present OSE, the RMSD with respect to Reynolds SST analyses and correlation with AVISO SLA gridded data is calculated for each run. The results are presented in Table 3, which also shows the mean RMSD for all T/S Argo data. A_All does not attain the smallest RMSD and the highest SLA correlation, for reasons described above. However, considering the overall performance, the A_All produced the best results. Assimilation in the A_All OSE consistently reduces the RMSD of SST, T and S and increases the SLA correlation by more than 80%.

Table 3. RMSD with respect to Reynolds SST analyses and 6,988 Argo T-S profiles and correlation with respect to AVISO SLA for each of the REMO GOOS OSEs in the whole model domain (100 W-20E, 78S-50N) from January 1 to June 30, 2010.

	RMSD		Correlation
	SST (°C)	Argo T (°C)	Argo S (psu)	AVISO SLA
Control	1.51	1.42	0.26	0.22
A_SST	0.81	1.30	0.27	0.23
A_Argo	1.36	1.12	0.18	0.22
A_SLA	1.49	1.40	0.25	0.47
A_All	0.92	1.16	0.18	0.40

It can be noted that the version of the REMO system used in this study is relatively immature, having only been recently developed. This is reflected in the results presented in Figure 3 and Table 3, showing relatively poor performance compared to other systems (e.g. Bluelink; Table 1 – SLA correlations are 0.4 for REMO and 0.67 for Bluelink). This highlights one of the benefits of this type of community OSEval activity, offering indirect inter-comparisons that help identify strengths and weaknesses in different systems. In this case, the impact of different data types is smaller than in other systems probably because the estimated background error covariances are relatively poor. It can be noted that RODAS_H is under development and new ensemble will be likely improve the system’s skill. For instance, it was found that assimilation of Argo profiles changed the model mean dynamical topography, mainly correcting the model temperature bias. This has led the developers to revisit the strategy for converting model sea-level into SLA for the assimilation of altimetry data.

Marine mammal T/S OSEs in the Southern Ocean

Results from a series of near-real-time (NRT) OSEs using the FOAM system are described in the published literature (Lea, 2012; Lea et al. 2013; Oke et al. 2015). Results presented here are from a series of more traditional OSEs, with a focus on the Southern Ocean and the region around the Kerguelen Plateau using the FOAM system (Lea et al. 2013; Blockley et al. 2013). Briefly, this version of Global FOAM uses the NEMO ORCA025L75 ocean model coupled to the CICE sea-ice model. Data assimilation is performed using the NEMOVAR 3D-Var system with profile observations from EN3 (Ingleby & Huddleston, 2007) as well as altimeter, SST and sea-ice data (Blockley et al. 2013).

Temperature and salinity profiles are obtained from instrumented marine mammals in NRT. The mammals, mostly elephant seals, sample to depths of up to 2000 m in high latitude regions where there are very few other in-situ observations. Two OSE integrations are performed using the UK Met Office’s Global FOAM v12 system: SealTS – where all mammal temperature and salinity profiles assimilated; and NoSeals – no mammal data assimilated.

Both experiments assimilated all of these observations, except the marine mammal profiles are withheld in the NoSeals OSE. The SealTS OSE assimilated 20,593 mammal profiles between 1 December 2010 and 31 December 2011, of which 13,579 were in the Southern Ocean.

Table 4 and Figure 4 show observation minus model background (OmB) statistics for 2011. These statistics provide a measure of the forecast skill of the system, representing comparisons with observations immediately before assimilation. Percentage RMSD indicate that assimilation of seal profiles improves model T and S fields, with improvements of 2.3% and 11.6% in temperature and salinity profiles for the global ocean and 4.3% and 33.1%, respectively, for the Southern Ocean where seal observations are most abundant. In all three regions, the assimilation of seal T & S data appears to reduce the bias and RMS errors. For temperature, the effect is small but significant in the upper 500 m of the Southern Ocean. The salinity impact is evident down to 2000 m in the Southern Ocean and Kerguelen regions and is notable in the upper 400 m of the global ocean.

Table 4. Temperature and salinity profile OmB results for three model hindcasts (no seal data assimilated, seal T and S assimilated, seal T assimilated), calculated using all available profile observations (the percentage improvement in the RMSD, %Imp = [RMSD_NoSeals - RMSD_SealXX]/RMSD_NoSeals) for the marine mammal T/S OSEs.

	Temperature (°C)				Salinity (psu)
	Bias	Corr	RMSD	%Imp	Bias	Corr	RMSD	%Imp
Global Ocean (Argo + Seals)
NoSe	−0.005	0.998	0.614	–	0.010	0.990	0.138	–
SealTS	0.004	0.998	0.600	2.3%	0.004	0.992	0.122	11.6%
SealT	0.004	0.998	0.601	2.1%	0.009	0.990	0.139	0.0%
Southern Ocean (Argo + Seals)
NoSe	−0.038	0.993	0.580	–	0.038	0.908	0.163	–
SealTS	−0.017	0.993	0.555	4.3%	0.011	0.956	0.109	33.1%
SealT	−0.017	0.994	0.548	5.5%	0.038	0.913	0.159	2.5%
Kerguelen region (Argo + Seals)
NoSe	−0.037	0.966	0.600	–	0.100	0.887	0.210	–
SealTS	−0.024	0.967	0.589	1.8%	0.025	0.929	0.149	29.1%
SealT	−0.022	0.969	0.565	5.8%	0.095	0.889	0.205	2.4%
Kerguelen region (Argo ony)
NoSe	0.008	0.963	0.801	–	0.003	0.931	0.146	–
SealTS	0.012	0.960	0.828	−3.4%	−0.026	0.923	0.155	−6.2%
SealT	0.003	0.963	0.796	0.6%	−0.017	0.930	0.148	−1.4%
Kerguelen region (Seals only)
NoSe	−0.063	0.951	0.449	–	0.168	0.903	0.238	–
SealTS	−0.044	0.960	0.394	12.3%	0.054	0.940	0.145	39.1%
SealT	−0.037	0.964	0.377	16.0%	0.160	0.904	0.232	2.5%

Figure 4. Profiles of the RMSD (solid lines) and bias (dashed lines) of the OmB for (a)-(e) temperature and (f)-(j) salinity observations for the (a), (f) global ocean, (b), (g) Southern Ocean and (c), (h) a region around the Kerguelen Isles (45-85E, 62-44S), compared to observations from both Seal and Argo observations; and (d), (i) compared to observations of Argo only; and (e), (j) compared to observations from Seals only. Obs-background statistics were calculated using all available T/S profiles, including the seal observations.

The results in Table 4 provide an optimistic estimate of the potential for marine mammal observations to improve FOAM system. Comparing OmB statistics in regions where the only profiles available for assimilation are from mammals make it appear that the mammal profiles have a positive impact on profile statistics. A lack of independent collocated T & S profiles makes it difficult to independently verify the impact of the mammal observations. It is clear that assimilation of the seal data makes a difference to model fields, but it cannot be certain that the analysed fields are an improvement.

A better understanding of the impact of seal observations is gained by focusing on a region with both mammal and Argo observations. Such a region is around the Kerguelen Islands, in the Indian sector of the Southern Ocean (45-85E, 62-44S). Table 4 and Figure 4(c) and (h) shows the OmB profiles using all profiles, from both Argo and mammals; Figure 4(d) and (i) shows the OmB profiles using only Argo profiles; and Figure 4(e) and (j) shows the OmB profiles using only mammal observations. The results are mixed. The comparison with only Argo profiles suggests that assimilation of mammal data degrades the model, by 3.4% and 6.2% for temperature and salinity respectively; but comparison with the assimilated marine mammal data indicates a significant improvement, reducing the RMSD by 12.3% and 39.1% for temperature and salinity respectively.

The reversal of the salinity impact between Argo-only and seal-only OmB RMS results and the corresponding large change in the bias value lead to the conclusion that some/most of the seal salinity data is biased high. This effect has been noted during laboratory tests of the CTD Satellite Relay Data Logger and is thought to be due to interference between the conductivity sensor’s electric field and the seal’s body. It is not currently possible to correct for this in NRT (Boehme et al. 2009). In addition, it is likely that biased salinity data may act to reduce the positive impact of seal temperature profile data due to the T-S balance relationships used by the data assimilation scheme (Boehme et al. 2009).

Work is continuing to find the most beneficial way to assimilate mammal data into FOAM (Carse et al. 2013). One option is to assimilate only the T data from the mammal database. To test this, a third OSE is performed to assimilate seal temperature profiles only. The SealT OSE is identical to SealTS, except all mammal salinity data is excluded from the assimilation.

SealT statistics show improvements compared with the SealTS OSE (Table 4). In the Southern Ocean, temperature profile RMSD improvements increase from 4.3 to 5.5% using the all-profiles method (Table 4). A larger positive impact is found in the Kerguelen region, with an increase from 1.8 to 5.8% using the all-profiles method and sign reversal to show a modest benefit using the Argo-only method (-3.4 to 0.6%, Table 4). The SealT OSE has no impact on the global ocean salinity statistics but causes an improvement of around 2.5% in the Southern Ocean and Kerguelen regions compared to the NoSeals OSE (Table 4). The Argo-only OmB salinity statistics in Table 4 show a reduction from -6.2 to -1.4% between SealTS and SealT OSEs.

The improvement in both temperature and salinity model fields shown in the SealT OSE strongly suggests that the mammal salinity data has a negative impact on FOAM T and S profiles. It is concluded that assimilating seal temperature data only is the pragmatic way to proceed, until the salinity bias in the observations can be corrected in near real time.

XBT-CTD OSSEs in the Gulf of Mexico

A new fraternal-twin ocean OSSE system (Halliwell et al. 2013) has been developed by the joint NOAA/AOML and University of Miami/RSMAS/CIMAS Ocean Modeling and OSSE Center [OMOC; www.ci-mas.org/omoc.html]. This OSSE system has been used to assess different design strategies for rapid-response airborne surveys. This fraternal twin system employs two different realisations of HYCOM for the Nature Run (NR) and the forecast model. Each model is configured to produce substantially different physics and truncation errors required of a credible OSSE system (Halliwell et al. 2013) by taking advantage of the multiple choices of numerical algorithms and subgrid-scale parameterizations contained in HYCOM.

OSSE system validation in the interior Gulf of Mexico (GoM) was achieved using an evaluation procedure that compared OSSEs to reference OSEs. Each OSE-OSSE experiment pair was identical, except for assimilating real (OSE) and synthetic (OSSE) observations, respectively. The key step was to demonstrate that similar impact assessments were consistently obtained between all corresponding OSE-OSSE pairs based on multiple error metrics calculated for several model variables. This design and validation exercise followed long-established procedures used in atmospheric OSSEs (Atlas et al. 1985; Atlas, 1997). The initial application of this OSSE system addressed the impact of different sampling strategies of rapid-response Airborne eXpendable profile surveys on a data assimilating model, including instrument type (shallow AXBTs versus deep AXCTDs) and horizontal resolution (Carse et al. 2013; Halliwell et al. 2014).

Example results from this OSSE study are presented in Figure 5. Figure 5(a) presents the locations where synthetic profiles were sampled from the NR for the two airborne sampling grids (0.5° and 1.0°), while Figures 5(b)-(e) provides an example impact assessment using RMS error maps calculated within this domain. Error maps are presented for Tropical Cyclone Heat Potential (TCHP), also known as Ocean Heat Content (OHC), which is an index of available thermal energy to support hurricane intensity (Mainelli et al. 2008). TCHP is the amount of thermal energy required to raise the temperature of all near surface water exceeding 26°C from that reference temperature to its actual temperature.

Figure 5. (a) Synthetic airborne profile sampling array for the OSSEs; (b) RMS error map for TCHP between the Nature Run (NR) and OSSE experiment AXCTDHR; change (increase) in RMS error (with respect to AXCTDHR) in experiment: (c) NOP3B, when all airborne profiles are denied; (d) AXCTDLR, showing the impact of lower profile resolution; and (e) AXBTHR, showing the impact of assimilating shallow AXBTs.

Figure 5(b) shows the RMS error field for experiment AXCTDHR, which assimilates 0.5° AXCTD profiles extending to 1000 m depth along with all other observing systems and serves as a reference, because it provides the greatest error reduction among all experiments. Figure 5(c) presents the increase in RMS error in experiment NOP3B, which denies only the airborne profiles, with respect to AXCTDHR. Large error increases are observed, demonstrating the positive impact of assimilating the airborne profiles.

The increase in RMS error in experiment AXCTDLR [assimilating profiles on the 1.0° grid – large dots in Figure 5(a)] with respect to AXCTDHR, is shown in Figure 5(d). Small-scale structure is observed in this error increase pattern, with the largest increases tending to occur at the 0.5° airborne sampling grid points that were denied in this experiment [small dots in Figure 5(a)]. This demonstrates that the higher-resolution sampling reduces RMS errors primarily by constraining smaller-scale ocean variability that is not well constrained by altimetry.

The increase in RMS error in experiment AXBTHR (assimilating AXBTs to 400 m depth) with respect to AXCTDHR is shown in Figure 5(e). The change of probes and profile depths has an insignificant impact on TCHP error because both probe types measure temperature profiles over the upper 400 m. By contrast, the assimilation of shallower probes that only measure temperature leads to somewhat larger errors in ocean dynamical fields, such as SSH (not shown).

From this study, it is concluded that assimilation of airborne profiles and that high-resolution sampling (in this case at 0.5° resolution, compared to 1°) constrains the small-scale circulation that is not well constrained by satellite altimetry. The deep CTD observations have relatively little impact on the upper OHC that is the focus of this study.

Glider OSSEs in the Ligurian Sea

A series of OSSEs was performed for the Ligurian Sea (Western Mediterranean) to evaluate the impact of the assimilation of glider data on forecast ocean temperature. The impact of a cooperative-coordinated glider fleet flying in a triangular formation maintaining a 25 km distance between the platforms is compared to a cooperative-unaware fleet where the individual platforms can freely move in the domain. Both configurations were optimised to minimise the temperature error between the surface and 100 m depth in a 1.8-km resolution regional simulation of the Ligurian Sea based on the ROMS (Shchepetkin & McWilliams, 2005).

Observations were simulated from a nature run provided by a reference ROMS simulation and assimilated in a slightly modified configuration of the model (a so-called fraternal twin-experiment) using an ensemble Kalman filter [EnKF; Evensen (2003)]. The control run is the mean of an ensemble of simulations generated from the same perturbations as those used in the EnKF, but without data assimilation. One of the most striking difference between the nature and control runs is the presence (absence) of a warm anticyclonic eddy travelling off the north-western coast of Corsica Island in the nature (control) run [denoted by ‘E’ in Figure 6(a) and (b)]. Figure 6(c) shows that the initial EnKF uncertainty field (ensemble spread) includes high uncertainty to the south-west of the domain and around the eddy ‘E’. The latter highlights the presence of travelling eddies (with similar characteristics to the eddy present in the nature run) off the north-western coast of Corsica in some of the ensemble simulations. This uncertainty estimate is used to pilot a fleet of three gliders in both unaware and coordinated configurations during four 2-day cycles. The initial glider-positions (on 21 August 2010) are indicated by the black cross in Figure 6(c). The fleet is redirected every 48 hours according to an optimal mission plan aimed at minimising the predicted model uncertainty for the next two days after assimilation of past observations. The algorithm constrains the platforms to fly in the Ligurian basin, defined in particular with a southern boundary at 43.15°N on the eastern side of Corsica Island. The details of the mission-planning algorithm are described elsewhere (Mourre & Alvarez, 2012; Alvarez & Mourre, 2014).

Figure 6. Initial 2-day averaged (21-23 August) temperature, averaged between the surface and 100 m depth, for the (a) nature run, (b) control run and (c) estimated uncertainty; and final (27-29 August) temperature for the (d) nature run, (e) cooperative unaware and (f ) coordinated glider fleet assimilated runs. The Corsica eddy location is marked by the letter E in (a) and (b). The glider trajectories are represented by fine black lines in panels (e) and (f).

The glider trajectories from 21 to 27 August are denoted by the fine black lines in Figures 6(e) and (f). The adaptive sampling procedure successfully directs the gliders (or at least one of them in the case of the cooperative-unaware scenario) towards the Corsica eddy. In the cooperative-unaware scenario, one of the platforms samples in the eddy area on 27 August, but without really crossing the core of the eddy. As a consequence, the model prediction, after assimilation of these data, does not properly reproduce the eddy. The improved coverage provided by the triangular formation leads to a better representation of the structure, even if not at exactly the right position. Note that none of the gliders are directed to the high-uncertainty region in the south-west of the domain. This is because that region is well constrained by remote observations, through the system’s ensemble-based background error covariances; so the region needn’t be sampled directly.

Statistically, no improvement is found with the cooperative-unaware configuration, owing to the chaotic nature of the high-resolution mesoscale circulation, however the coordinated glider configuration leads to a 44% reduction of the RMSE in the area of the eddy (Table 5). The slight performance degradation with the assimilation over the remaining area is attributed to spurious ensemble spatial correlations. This effect could be eliminated by the use of covariance localisation in the EnKF.

Table 5. Temperature RMSE (°C), of the control and assimilated solutions in a subdomain defined around the eddy and in the remaining area for the period 27-29 August for the Glider OSSEs.

	Control	Assimilated unaware	Assimilated coordinated
RMSE (eddy area)	1.40	1.42	0.79
RMSE (remaining area)	0.45	0.51	0.46

These results suggest that the use of gliders for adaptive sampling, where glider fleets are piloted towards model error hot spots, can be effective. This is particularly true for applications that resolve the mesoscale (10–20 km) circulation, in which case entire eddies and eddy systems may be missing from a model without appropriate initialisation. Furthermore, this study suggests that good results can be achieved when a glider fleet maintains a geometrical formation allowing the collection of information along different directions across the mesoscale structures [i.e. the coordinated glider fleet experiment; referred to as Assimilated Coordinated; Figure 6(f); Table 5].

Shelf-scale integrated observing system impacts off California

Results are presented from an observation impact study in the California Current system quantifying the impact of coastal high-frequency (HF) radar observations of surface velocities on alongshore transport estimates. The model used is the ROMS (Shchepetkin & McWilliams, 2005) configured for the west coast of North America, with 10 km resolution in the horizontal and 42 terrain-following levels in the vertical (Alvarez & Mourre, 2014). The dual formulation of the ROMS 4-dimensional variational (4D-Var) data assimilation system (Moore et al. 2011; Moore et al. 2011) is employed in the experiments presented here. ROMS 4D-Var has been used to compute a sequence of historical analyses, without tides, spanning the period 1980–2012 (Saraceno, 2008) and is also run routinely in NRT [oceanmodeling.ucsc.edu]. In the examples presented here the period March to December 2012 is focussed on. Data are assimilated into ROMS using 8-day assimilation cycles, overlapping every 4 days. Each cycle assimilates standard data types, including: SST from NOAA Coastwatch from multiple platforms (combined to form super observations as necessary), gridded SSH and T/S profiles from EN3 (Ingleby & Huddleston, 2007). Because of potential land contamination of the altimeter return (Saraceno et al. 2008), SSH observations are not assimilated within 50 km of the coast. Additionally, ROMS assimilates de-tided, 3-hourly gridded surface velocities from HF radar. HF radar observations are the dominant data type during each 4D-Var cycle when there are typically 4.3×10⁵ HF radar observations, 4.2×10⁴ SST observations, 1.8×10⁴ SSH observations, 740 vertical profiles of temperature and 630 profiles of salinity. The model is forced with surface fluxes of momentum, heat and freshwater from derived atmospheric fields from the Coupled Atmosphere-Ocean Mesoscale Prediction System (COAMPS; Doyle et al. 2009). Open boundary data are taken from the Simple Ocean Data Assimilation (SODA; Carton & Giese, 2008). During each 4D-Var cycle, the control vector comprises the initial conditions; surface forcing and open boundary conditions.

The impact of individual observations during each 4D-Var cycle is quantified following the approach of Langland and Baker (2004) who showed that the contribution of each observation to a function I(x) of the state-vector estimate x can be quantified using the adjoint of the Kalman gain matrix. In the example presented here I(x) is chosen to be the alongshore transport at 37°N over the upper 500 m from the coast to 127°W averaged over each 8 day assimilation cycle [indicated in Figure 7(b)]. This metric represents an average of the transport carried by three distinct circulation features: the California Current, the California Undercurrent and a coastal jet. Specifically, the transport increment ΔI=I(x _a)–I(x _b) is considered, where x_a and x_b are the analysis and background state-vector estimates respectively. The transport increment can be expressed as the dot-product ΔI=d^T g, where d=y–H(x _b) is the innovation vector, y is the vector of observations, H(x _b) is the time evolved background estimate evaluated at the observation points and H is the observation operator. The vector g=(k₂–k₁)^-1 K^TS_k(M^T)_k h _k, summed from k=k₁ … k₂, where k₁dt and k₂dt are the start and end times of the 4D-Var cycle and dt is the model time step; K^T is the adjoint of the Kalman gain matrix; M^T is the adjoint of the tangent linear model of ROMS, linearised about the time evolving background; and h is a vector such that h^T x defines the 37°N transport at a given time. Since each observation is uniquely related to each of the elements of the vectors d and g, the impact of each observation on ΔI=d^T g can be quantified by their individual contributions to the dot-product.

Figure 7. (a) Time series of the transport increment DI for each 4D-Var analysis cycle during 2012. The coloured segments of each vertical bar represent the contributions from the various different observation platforms indicated: SSH-Aviso SSH; SST-all SST platforms combined; u(v) HF-zonal (meridional) components of surface current from HF radar measurements; EN3 T and ENS3 S all in-situ temperature and salinity observations. (b) A map of the time-mean impact of each grid point location at which HF radar surface current observations are available. The location of the 37°N section (denoted by the black line), along which values in panel (a) are computed.

Figure 7(a) shows a time series of the transport increments DI for each 4D-Var cycle during 2012. The contribution of each observation platform to the total transport increment DI is indicated by the different colours and the impact of the HF radar observations is quite dramatic, particularly measurements of alongshore surface current (denoted as v HF radar) as might be anticipated. SST observations also often have a significant impact, in agreement with previous studies (Langland & Baker, 2004) that did not consider HF radar data. The HF radar network in the region considered is extensive and in the calculations presented here, the number of HF radar measurements assimilated into the model during each 4D-Var cycle typically outnumbers, by a factor 3, the total number of all satellite observations and hydrographic observations combined over the entire domain.

Figure 7(b) shows the impact of each gridded HF radar measurement location on DI averaged over all 4D-Var cycles. It is evident that radar observations both upstream and downstream of the 37°N section significantly impact the analysis increments of transport. This result can be understood in terms of equatorward advection of information from the observations by the prevailing circulation and the poleward propagation of information by coastally trapped waves. During each 4D-Var cycle, observational information is dynamically interpolated both forward and backward in time via the tangent linear and adjoint versions of ROMS respectively. Therefore sensitivity maps like that of Figure 7(b) can yield significant insight into the dynamical mechanisms that are involved in the flow of observation information through the data assimilation system. Figure 7(b) shows that, while on average HF radar observations upstream of the 37°N have the greatest influence on transport, observations downstream are important as well, a result that may not be anticipated a priori.

The observation impact calculations presented here are now commonplace and routine at some meteorological centers and they provide complementary information to the more traditional OSEs and OSSEs, described above. The calculations presented here demonstrate the impact of HF radar observations on an important aspect of the circulation in the vicinity of Monterey Bay, supporting the argument for the continued maintenance of the coastal array.

Representer matrix spectrum (RMspec) using polynomial chaos in the Gulf of Mexico

The RMspec (Le Hénaff, 2009) aims at assessing the performance of an observational array without assimilating data by characterizing the part of the system’s error subspace that can be detected by an observational network (real or virtual). As in data assimilation, a major issue is estimating the background error (BGE). This method uses stochastic modelling (Le Hénaff, 2009), whereby uncertain inputs are perturbed randomly and O(10–100) ensemble simulations are performed to generate possible model states. However, the small ensemble size may misrepresent error statistics. Alternatively, Polynomial Chaos (PC) methods explicitly construct the dependence of the model output on the uncertain inputs through a spectral series in the uncertain variables; the series can then be used as a model surrogate and can be mined efficiently for more accurate statistical information (Gelaro & Zhu, 2009; Le Maître & Knio, 2010).

Here the impact of using PC-based and ensemble-based covariances in the RMspec approach are compared the capacity of an altimeter track over the GoM to characterize BGEs associated with the frontal dynamics of the Loop Current (LC) is studied. For this study a NRT version of HYCOM is used [hycom.org/ocean-prediction]. The initial conditions uncertainties, the sources of BGEs, are represented using two stochastic variables (p=2): the amplitudes of two variability modes, defined by an empirical orthogonal function analysis of the model fields in the two weeks preceding the experiment. These modes are added as perturbations to the initial conditions. Using a polynomial decomposition of degree 6 (k=6), the number of simulations required to estimate the polynomial series is (k+1)^p=49. Since the PC ensemble samples the edges of the distribution more densely than the center, only 25 of those simulations for the conventional ensemble calculation are retained.

The RMspec analysis requires the eigenvalue decomposition of the scaled representer matrix x, which is equivalent to the BGE covariance matrix, at the observation location, normalized by the observation error covariance matrix. The performance of an observational array is assessed by the number of eigenvalues larger than 1. The spectrum [Figure 8(a)] from the PC approach has generally larger values than the spectrum estimated with the ensemble. In particular, the number of eigenvalues larger than 1, is 4 with the PC and 3 with the ensemble. This difference in the number of significant modes from both approaches supports the need for advanced techniques for characterizing BGEs; the PC approach can provide a detailed estimate with a limited number of simulations.

Figure 8. (a) Representer matrix spectra, estimated with the PC approach (blue) and the ensemble approach (red). (b), (c), (d) Modal representers in SSH (m) for the first three eigenmodes of the representer matrix eigenvalue decomposition. The satellite track over which the observations are considered is thick black line, estimated with the PC approach. The magenta line defines the location of the Loop Current and an anticyclonic ring in the western Gulf of Mexico.

The RMspec technique also provides tools for qualitative interpretation of the signal associated with each eigenmode of x, through the estimation of the modal representers, which project the eigenmodes, defined in observation space, onto model space. The first 3 modes, which are detected and have similar signatures in both cases, show a stronger signal in the frontal area of the LC [Figures 8(b)-(d)]. They clearly show ocean mesoscale patterns with spatial scales decreasing with the mode’s rank. The limited number of detected modes illustrates that a single altimeter track can only detect rather large ocean mesoscale features. Furthermore, this result demonstrates the need for a more comprehensive observational array to properly observe mesoscale ocean variability; likely comprised of multiple altimeters and other observational platforms.

Biogeochemical footprints in the Great Barrier Reef

A new project, called eReefs (Schiller et al. 2014) [www.ereefs.org.au/] has been established within Australia to develop a modeling, hindcasting and forecasting capability for the ocean circulation and water quality on the Australian Great Barrier Reef (GBR). Observations over the GBR are sparse. Satellite altimetry is unreliable, because the reef is wide and shallow (so tidal corrections are poor); satellite SST is often isothermal and therefore uninformative; and Argo and XBT programs do not cover the shallow reef or lagoons. The motivation is, therefore, to assess the degree to which a limited number of in-situ moorings, deployed and planned as part of the Australian Integrated Marine Observing System [IMOS; www.imos.org.au], can monitor the physical and biogeochemical properties of the GBR region.

Here, the ‘footprint’ of an ocean observation as the region that is well correlated to the observed variable at zero timelag is defined (Oke & Sakov, 2012). The footprint of observations from an observation array provides an indication of the region that is effectively monitored by that array. To assess the footprint of an observation of Chlorophyll-a (Chl-a) at different mooring locations on the Australian Great Barrier Reef (GBR), the cross-correlation between anomalies at an observation location with anomalies elsewhere over the GBR using ensembles that represent different time-scales of Chl-a variability are computed (Jones et al. 2013). Specifically, a 9-km resolution composite satellite image, based on MODIS data is used [modis.gsfc.nasa.gov], to construct time-series of Chl-a. The weatherband (10–30 days), intra-monthly (30–90 days) and intra-seasonal (90–180 day) time-scales in the Chl-a data are assembled by constructing an ensemble of anomalies. For the weatherband timescale, the ensemble is constructed by subtracting 10-day means fields from 30-day mean fields; and similarly for intra-monthly and intraseasonal timescales. The correlation footprints of locations where long-term moorings are either currently deployed or planned, under IMOS are then computed.

The Stradbroke Island National Reference Station [NRS; Figures 9(a)-(b)] is located in a region that is strongly influence by the East Australian Current (EAC), where the MODIS data reliably represent biological variability. On weatherband time scales, the footprint of this mooring is closely aligned with the principal direction of the EAC, with short de-correlation length scale in the cross-flow direction. The intra-monthly footprint is noisy, is not obviously related to any specific isolated dynamical process and indicates that this mooring represents variability within about 200 km of its location. At intra-seasonal timescales, the Stradbroke Island mooring is representative of a broad area to the north and south of the mooring location. The spatially coherent pattern to the south of the intra-seasonal footprint represents the EAC separation from the coast at around 32.5°S. There is no statistically significant correlation south of 35°S, as expected, since the variability in that region is believed to be largely independent of the variability near Stradbroke Island.

Figure 9. Correlation footprint of Chl-a for locations (denoted by stars) at (a)-(c) Stradbroke Island and (d)-(f ) Yangola (denoted by the star) at the (a), (d) weatherband (b), (e), intra-monthly and (c), (f) intra-seasonal time-scale. The colour denotes correlation, which has been truncated at plus/minus 0.5 (statistically insignificant correlations, at the 95% level).

The Yongala NRS [Figures 9(d)-(f)] is located within the GBR lagoon system. There are a number of rivers that regularly discharge large amounts of freshwater into the system, along with optically active components that will be evident in the MODIS data. At intra-weekly timescales, the Yongala NRS is representative of a large proportion of the central and southern GBR. However, at these short time scales, measurements at the Yongala mooring are not representative of the northern GBR. On intra-monthly time scales, the Yongala NRS is representative of the whole central GBR, but with weaker correlations in the northern and southern regions. At intra-seasonal time scales, the NRS footprint is spatially coherent and explains more than 50% of the Chl-a variability over the whole GBR. It is hypothesised that this footprint is representative of the large-scale intra-seasonal dynamics of the region that is dominated by the wet-dry seasonal cycle and is related to freshwater discharge from the numerous rivers within the region.

At intra-weekly and intra-monthly time scales the two NRS sites considered here explain greater than 40% of the variability near the coast for central and southern GBR. At longer intra-seasonal timescales, the footprints expand to include the entire GBR and also a large portion of the western Tasman Sea. It is clear that the Chl-a footprints are closely related to the ocean circulation in the region of interest. These results indicate that a data-assimilating model of the GBR can expect to be reasonably well constrained by a relatively small number of moorings – carefully placed – over the GBR.

Bluelink western equatorial Pacific study

The operational ocean forecast system developed under Bluelink, called Ocean Modelling Analysis and Prediction System (OceanMAPS; Brassington, 2013) uses a 4-cycle ensemble approach to deliver multiple hindcasts for the last 9 days and multiple forecasts for the next 4-7 days with the intention of providing routine estimates of hindcast and forecast skill that depends on the state of the current ocean, atmosphere and the observing system. The 4-cycle mean has been shown to be superior to each individual forecast (Brassington, 2013), consistent with previously documented studies using ensemble prediction systems (O’Kane et al. 2011). In this study, the power spectrum of temperature and salinity in the upper ocean near the equator (between 145–170°E) is analysed, where the Bluelink model has 1/10° resolution. Each power spectrum [Figures 10(a)-(b)] is averaged over time (between 1 March to 31 August 2012) and space (within 1° latitude of the equator).

Figure 10. (a) The power-spectrum for sea surface height anomaly in the Tasman Sea composed of the average of six power spectra from zonal sections (38S:1:32S). Each zonal section power spectra is based on a temporal average from the 1st March-31st August 2012. The black (red) lines represent the 0-lag forecast and the ensemble mean respectively. Mean power spectrum are shown for the forecast hours 096 (solid), 048 (dashed), 000 (dash-dot) and 048 (dotted). (b) the difference in power between the 0-lag forecast and the ensemble mean for the forecast 096 (solid), 048 (dashed), 000 (dash-dot) and 048 (dotted), (c) same as (a) but the south east Indian Ocean and (d) same as (b) but the south east Indian Ocean.

The spectra shown in Figures 10(a)-(b) are for hindcasts of temperature and salinity, valid at 4-days behind real-time (black) and 4-cycle means (red) for different depths. For each case, the 4-cycle mean has less power than the individual hindcasts because the noisy aspects of each hindcast are averaged out in the 4-cycle mean. It is suggested that the difference between the power of individual hindcasts and the 4-cycle mean hindcasts provides an indication of the scales and signals that are poorly constrained in the individual hindcasts. This provides an indication of the limitations of the observing systems that underpin the operational forecast system (Brassington et al. 2014). The differences between the power spectrum of the individual hindcasts and the 4-cycle means are shown in Figures 10(c)-(d) for temperature and salinity at different depths.

The differences in power spectra [Figures 10(c)-(d)] are similar for both temperature and salinity at all depths (except temperature at 5 m depth). Most striking the difference spectra is the high power for high wavenumbers (low wave-lengths). It is interpreted that this means that the forecast system is poorly constrained at those scales (i.e. < 1–2° zonally). The key observation types used to initialise ocean forecasts in the region of interest are satellite altimetry, TAO, XBT and Argo. None of these observation types properly resolve scales that are less than 1–2° zonally. It is found that only SST is well constrained for most scales (OceanMAPS assimilates a 4-km resolution SST product) – but even SST shows some differences between the mean and individual forecasts for shorter wavelengths.

Discussions and conclusions

This review article describes a range of different techniques for quantifying the impact of assimilated observations on ocean forecast systems. Techniques described here include traditional OSEs, traditional OSSEs, OSSEs with adaptive sampling; and analysis of RMspec using PC and observation footprints. Applications described in this paper include global and basin-scale studies using eddy-resolving and eddy-permitting models; shelf-scale studies using high-resolution models that resolve the mesoscale and an observation-based study to explore the potential impacts of assimilated mooring observations. The studies presented here quantify the impacts of a range of different observation types, including the conventional data types (altimetry, SST, in-situ T/S); plus newer platforms, such as gliders, HF radars, instrumented marine mammals and in-situ biogeochemical platforms. This paper also includes a demonstration of the benefits of adaptive sampling, where a carefully ‘flown’ fleet of gliders, or a high-resolution array of CTD measurements, can constrain the mesoscale variability in a high-resolution model.

It is clear from this community paper, that forecast systems developed under GODAE and GODAE OceanView are maturing to the point where they can benefit from a range of different observation types in addition to the conventional data types that are associated with the GOOS. Different systems have been developed at global, regional and coastal scales. For each of these scales, different observing systems are used. That is, additional observations are assimilated in regional systems compared to global systems; and in coastal systems compared to regional systems. The higher-resolution models require denser observing systems to constrain their circulation. The results presented in this paper demonstrate the positive impact of those regional datasets on ocean forecasts. Conclusions from this review include the following:

• data from HF radars, gliders, instrumented mammals and airborne profiles contain independent information to the conventional observation platforms – providing additional constraints on systems that better resolve the mesoscale;

• regional observation platforms may have limited impact globally, but very significant impacts locally – reducing local errors by as much as 50%. It is recommended that global forecast systems assimilate regional data sets in addition to the those normally associated with the GOOS;

• carefully designed regional in-situ observation arrays (e.g. optimized glider fleets, or high-resolution XBT surveys) can add significant constraint to high-resolution models – with up to 40% improvement;

• OSEval studies are sensitive to details of the underpinning systems – and can be used to identify strengths and weaknesses of data assimilation systems. It is recommended that the GOV OSEval-TT coordinate community OSSEs, as well as OSEs to further assess the GOOS as well as contributing to the assessment and improvement of the underpinning data assimilation systems;

• the suite of tools available to routinely assess observing systems for operational and quasi-operational ocean forecast has expanded (e.g. RMspec; spectral analysis; adjoint sensitivities); and

• the GOV OSEval-TT is well-placed to assess and improve the quality of OSEval experiments conducted by various groups.

As well as show-casing a range of conventional and emerging techniques for assessing the impact of observations on the ocean models assimilating them, this paper demonstrates the value of regional observation platforms, such as HF radar, gliders and instrumented marine mammals. These platforms are often overlooked by operational forecast centers, owing to their small spatial footprint. But the local impact of those observations could be critical, particularly if the applications for which the operational forecasts are being used are focused on those regions. Consider, for example, the case of someone who is lost at sea with only a life jacket. Their fate could be influenced by the quality of ocean forecasts at their precise location. So delivering the most accurate forecasts possible, for as many regions as possible is imperative. This consideration puts some of the observation impact studies into perspective. Perhaps a global metric, like global RMS of OmB, reduces only by a small fraction of a degree when a localised dataset is assimilated. One might naively dismiss this dataset as unimportant. However, if that dataset improves a forecast by a substantial amount in the vicinity of the measurements, then for an application like search and rescue, that dataset might be difference between life and death.

It is important for those additional data types to be maintained and for operational centers to assimilate their data into forecast systems routinely. Of course, a key step towards the uptake of these regional datasets into operational systems is the NRT availability of data.

Acknowledgements

The authors acknowledge the constructive comments of three anonymous reviewers that led to significant improvements in this paper. P. Oke acknowledges the contributions of the Bluelink science team. V. Kourafalou acknowledges support from NOAA (NA13OAR4830224). C.A.S. Tanajura acknowledges support of the CAPES Foundation, Ministry of Education of Brazil (Proc. BEX 3957/13-6). A. M. Moore acknowledges the support of the ONR (N00014-10-1-0476, N00014-10-1-0322) and NSF (OCE-1061434). Satellite altimetry is provided by NASA, NOAA and CNES. Drifter data are provided by NOAA-AOML and SST observations are provided by NOAA and Remote Sensing Systems. Argo data were collected and made freely available by the International Argo Program and the national programs that contribute to it [www.argo.ucsd.edu, argo.jcommops.org]. The Argo Program is part of the Global Ocean Observing System. XBT data are made freely available on the Atlantic Oceanographic and Meteorological Laboratory, funded by the NOAA Office of Climate Observations and by the Scripps High Resolution XBT program [www-hrx.ucsd.edu]. AVHRR data from the Pathfinder program were provided by GHRSST and the US National Oceanographic Data Center, supported in part by a grant from the NOAA Climate Data Record (CDR) Program for satellites.