Medium-Range Forecast Sensitivity to the Background State and Background Error Statistics in a Global Data Assimilation and Prediction System

The increase in the medium-range forecast skill of global models is attributable to the improvement of the initial condition, model dynamics, and physics. In this study, we evaluated the role of the background state and background error statistics (BES) on a medium-range forecast testbed, using the Global/Regional Integrated Model system (GRIMs) that incorporates the National Center for Environmental Prediction (NCEP) Gridpoint Statistics Interpolation (GSI). In addition to the control run, in which the initial condition is obtained from the NCEP Global Data Assimilation System (GDAS) analysis, four additional experiments with different background states and BES are executed to evaluate the impacts of the background state and the BES on forecast skills. The standard background fields are produced from the sophisticated, higher resolution NCEP Global Data Assimilation System (GDAS), whereas the other background fields are produced from the GRIMs through cycle runs every 6 hours. Further, the two kinds of BES are calculated from the NCEP Global Forecast System (GFS) and the GRIMs, respectively. Evaluations are performed in August 2010, with the focus on the 500-hPa geopotential height and precipitation. The experiment simulated with results from high-quality background fields performs better than when using results of low-quality background fields with respect to the 500-hPa geopotential height Anomaly Correlation (AC). This is true for both North and South Hemisphere results. The impact of BES dose not responds much towards primitive forecast skills, but it influences the forecast skill after day 7. In contrast to the large-scale features, the forecast skills of precipitation show the overall improvement of land precipitation with the support of the cycle run.


Introduction
With the increase in computing power, improvements to the numerical weather prediction (NWP) models are being actively pursued for initial fields, model physics, and increased spatial resolution.The high-quality analysis produced by the use of more accurate methods of data assimilation, along with the increased availability of data, especially satellite and aircraft data over the seas, critically increased the predictability skill (Simmons & Hollingsworth, 2002).The performance of a variety of Observing System Experiments studies manifests the contribution of a more frequent usage of observation (Kelly, 2004;Kelly & Thépaut, 2007;Hwang et al., 2010;Laroche & Sarrazin, 2010;Hwang & Hong, 2012).
In the context of NWP models, the finer grid resolution and the improvement in parameterization of subgrid-scale physics, incorporated into explicit model dynamics, has increased forecast skills.Clearly, analysis fields generated by any data assimilation (DA) system are important, due to their role as the best possible estimates of nature and in the initial states for NWP weather forecasts.Associated error relates with future forecast error as the former evolves during the forecast time interval, directly determining the quality of the forecast.This relationship may result from various sources of error associated with observations, a forecast model, and DA systems, such as observation screening, observation data error, quality control, background information, background error variances, the choice of control variables, imposed constraint, and so on.
Background information may be one of the most essential factors because the DA system provides substantial corrections, based on the background field, which is derived from a short numerical forecast.Thus, background error shares properties with forecast error.Some studies on the contribution of model physics to DA system are found.Hwang and Hong (2010) demonstrated that a revision in the physics algorithm improves the quality of initial data and compensate the defect of a DA system.Hong et al. (2011) showed that differences in the physics package with the identical data assimilation matrix solver could result in significant differences in the analyzed data.
Observation data have inevitable subjective, instrumental, or other errors, from which error statistics corresponding to the observation type, is assigned in the data assimilation module.These errors are generally fixed once observations are newly introduced.However, the chaotic behaviors of nonlinear systems shall change forecast errors, in the evolution of the atmosphere in a sequence of analysis steps.The 4-dimensional variational method (4D-Var) possesses flow-dependent forecast errors, unlike the 3-dimensional variational method (3D-Var).Therefore, the adaptation of background error statistics (BES), which is calculated from the forecast error, matters more in the 3D-Var system.
Over the last decade or so, having an accurate knowledge of BES has been fundamental to achieving success in the assimilation process.Numerous scholars and experts on the BES aspired to unravel its nature and subtle ways of affecting the assimilation, as well as ways to measure, and model the BES for use in a data assimilation setting (Bannister, 2008a).Regarding the impact of using a different BES, Ajjaji et al. (2007) reported that when different ingredients' positive impacts are associated together, forecasts initiated from the Weather Research and Forecasting numerical weather prediction model (WRF) with the 3D-Var initial states in a warm cycling mode, outperform those based on the Global Forecast System (GFS) Gridpoint Statistical Interpolation (GSI ; Derber et al., 1991;Parrish & Derber, 1992;Wu et al., 2002;Derber et al., 2003;Kleist et al., 2009).
In this study, we investigated the role of BES and the background field in the medium-range forecast skill, using the global data assimilation and prediction system, whereas previous studies were carried out for either regional models or in low spatial resolution with insufficient observation data.Moreover, the relative importance between BES and the background field has been assessed.The Global/Regional Integrated Model system (GRIMs) (Hong et al., 2013) is used.Evaluations were performed for August 2010, focusing on the 500-hPa geopotential height and precipitation.We first introduce the forecast model, the BES, and the experimental setup in Section 2. Results are discussed in Section 3. Summary and concluding remarks follow in the final section.

Model Description
The GRIMs is a multi-scale atmospheric modeling system with unified physics, created for use in numerical weather prediction, seasonal simulations, and climate research from global to regional scales.Model physics include long-and short-wave radiation, cloud-radiation interaction, deep and shallow convection, large-scale condensation, gravity wave drag, land surface processes, and vertical diffusion including PBL processes.Table 1 summarizes the physics parameterization.Several physics schemes are included from the GFS model, as of July 2010: for example, the revised deep and shallow convections described by Han and Pan (2011).Han and Pan (2011) introduced revisions in the SAS scheme, including an entrainment rate parameterization with the rate dependent upon environmental moisture.The shallow convection scheme employs a mass flux parameterization against the eddy-diffusivity based approach of Tiedke (1983).They demonstrated that the revised convection algorithms improved the forecast skill of the 500-hPa geopotential height, vector wind over the globe, and precipitation over the continental US.The major component of the YSU PBL is the inclusion of an explicit treatment of the entrainment processes at the top of the PBL, which is formulated following the LES data from Noh et al. (2003).For a stable boundary layer of the YSU PBL, enhanced vertical mixing is applied, based on the bulk Richardson number between the surface layer and the top of the boundary layer (Hong, 2010).Table 1.A summary of the physics parameterizations Short-wave radiation Chou and Lee (1996) Long-wave radiation Chou and Suarez (1994) Land surface model Chen and Dudhia (2001) Ocean mixed layer Kim and Hong (2010) Vertical

Background Error Statistics (BES)
The BES is designed to determine the distribution of the information in the data in space and between variables.
The BES needs to be created, due to the newly coordinated forecast model.In this study, the background error covariances calculated from GRIMs were estimated in grid space by the NMC method (Parrish & Derber, 1992;Rabier et al., 1998), which is widely used in operational centers.Error statistics were estimated with the differences between 6-and 30-h forecasts, valid at the same time for 103 cases from 1 June to 22 July 2010.
Estimates for the balance projection matrices, standard deviations, and correlation length scales, were computed.Standard deviations of the background error, for the analysis variables as a function of latitude and height, are shown in Figure 1.The amplitudes of the stream function, unbalanced temperature, and unbalanced surface pressure, are larger in the mid-latitudes than in the tropics, and larger in the Southern Hemisphere than in the Northern Hemisphere.These results correspond with those reported by Wu et al. (2002).Refer to Kleist et al. (2009) for a detailed description of the GSI configuration.
Single-observation test is common and straightforward way to get some ideas of how the background error covariances work in the model variable space.The horizontal and vertical impacts on the temperature increments of a single 850 hPa temperature observation at 35.1°N 126.8°E are examined.The vertical impact on the temperature increments is illustrated by the vertical cross-section in the west-east direction in Figure 2a.The temperature increments are isotropic on the horizontal with a relatively coherent length scale (~200 km).Even though the observation measures the values of temperature at one point, the data assimilation disperses this information.The winds have a dipole structure consistent with the expected near-geostrophic, anti-cyclonic response to the adjustment of the temperature field (Figure 2b).
Figure 2. Vertical cross section of the temperature analysis increments (left), and the analysis increments of the U-component (bold) and V-component (plain) at the 14 th vertical layer (right)

Experimental Setup
The horizontal resolution of the GRIMs in this study is T254, which corresponds to a grid spacing of approximately 50 km.A total of 64 model levels (hybrid sigma-pressure coordinates) were considered on the vertical, up to 0.3 hPa.As the surface boundary condition, the observed SST, snow and ice data with a resolution of 1 o x 1 o at the initial time, are utilized.Ten-day forecasts were carried out at every 00 and 12 UTC from 1 to 31 August 2010.One of the reasons for selecting these months is that the GFS model output format was changed in July 2010.The other reason is that this summer period had typical characteristics, including major precipitation over the tropics, and monsoonal precipitation in East Asia.
All experiments designed in this study are summarized in Table 2. CNTL, not incorporated into any assimilation system, is a cold-start run of GRIMs T254 with the GDAS analysis.The analysis fields used in CNTL were produced from the NCEP Global Data Assimilation System (GDAS) with a T574 grid (approximately 23 km).GDAS analysis fields are reproduced with a smooth spectral truncation in transforming T574 to T254 resolution, with which we only run the GRIMs forecast model.In addition, we performed two pairs of experiments with different background fields and BES.They have the same DA system and forecast model.A new estimate of the atmospheric state (analysis) in the DA system was created with the GSI on the model vertical levels, and on a horizontal Gaussian-linear grid that corresponds to the T254 spectral definition: i.e. 512 x 256 grid points at every 00 and 12 UTC.The three-dimensional First Guess at the Appropriate Time (3D-FGAT) was configured in the DA system; therefore the background fields used for each analysis at the time of the previous 3-, 6-, and 9-h forecast are reproduced.The EXP1 and EXP2 experiments were simulated with the initial fields assimilated by using the background fields reconfigured from GFS background fields, where the BES of GFS and the BES of GRIMs were set up, respectively.Note that the analysis fields of CNTL were produced by the DA system combined with the T574 operational system, but those of EXP1 and EXP2 by the coarser T254 DA system.Therefore, the primary difference between CNTL and EXP1 lies in analysis field.In contrast with above experiments assimilating imported background field, the EXP3 and EXP4 experiments were performed with their own assimilated analysis fields in GRIMs, by using the corresponding background fields with the BES from GFS and GRIMs, respectively.These experiments are so-called warm start, which are done at four analysis cycles per day (at 0000, 0600, 1200, and 1800UTC).They use a total system where the DA and forecast model are fully cooperated.The difference between EXP1 and EXP2 in addition to that between EXP3 and EXP4 can show the impact of BES, while the difference between the group of EXP1 and EXP2 and the group of EXP3 and EXP4 can explain the impact of background field.Two groups have the same resolution background field, but the group of EXP1 and EXP2 could have more information in the background field than the latter group, because of the interpolation from fields of higher-resolution forecast model and the DA system.We expect EXP3 and EXP4 to determine the sensitivity of the medium-range forecast to the quality of the initial conditions produced by the warm cycle.
The set of observational data used in this study consisted of all GTS conventional data (TEMP, PILOT, SYNOP, SHIP, BUOY, AIREP, AMDAR, METAR, GEOAMV, and SATEM) and non-conventional observation data like QuickSCAT, HIRS and AMSU radiances.Unlike CNTL, however, EXP1 and EXP2 used less observation data due to the restriction in importing available data.The impossibility to import all of the observation data used in the GDAS T574 system, which work internally in NCEP, caused such limitation.

Large-Scale Features
To estimate the synoptic quality of the analyses, not only the departures between observed values and corresponding analysis and background values, but also the verification against observation were studied.Figure 3 shows the time-series of background and analysis fit to the radiosonde temperature in the global region for August 2010.We found that EXP1 and EXP2 (Group A) are higher-quality fields than EXP3 and EXP4 (Group B) in terms of observation verification.All of the experiments showed significant decreases in the root mean square of analysis departures, compared to the background field.Even though the background departures of Group A were less than that of Group B at 0.2, the difference in the analysis departures between the two groups was much smaller.This result implies that the DA system works well in producing the analysis field.The more marked descent from the Root Mean Square of Difference (RMSD) of the Background to the RMSD of analysis has been more remarkable between Group B rather than Group A, demonstrating that the incorporated DA system within the warm cycle could create the optimal state of the atmosphere with even a background field produced from lower resolution forecast model.Within the Groups, the use of GRIMs BES decreased the RMSD of analysis further than the use of GFS BES, although this result seems statistically insignificant.Figure 4 contains the time-series of background and analyzed temperature profile differences (EXP4-EXP2), which are averaged over the Northern Hemisphere (NH; 20°N to 90°N) and the Southern Hemisphere (SH; 90°S to 20°S).The time-series of background temperature profile's difference over the NH (Figure 4a) shows that the differences at the level below 0.5 sigma level of model decreased gradually during the experiment period.The EXP2 experiment assimilated into the background fields from the GFS, un-affected from GRIMs forecast.However, the EXP4 assimilated into the background fields from GRIMs, of which the system could adjust background field to its own DA/forecast model.Thus, we deduce that the warm-start cycle lessens the difference of the background fields from GFS and GRIMs.The reduction of analyzed temperature differences are seen in Figure 4c, which means that the observation data give beneficial information to temperature fields and further reduce the differences.Over the SH, the differences of the background temperature profile decrease in the lower atmosphere; however the differences increase in case of the analyzed temperature profile (Figures 4b and 4d).
The impacts of the satellite observation data may explain this speciously inexplicable phenomenon.There are many kinds of observation data like radiosonde, aircraft and satellite data in NH, of which radiosonde and aircraft data provide much information in the lower atmosphere.Over the SH, because there are few radiosonde and aircraft data, satellite data can affects much more than other observation data in the lower atmosphere.Anomaly correlation (AC) and bias of the predicted large-scale fields against the ERA-interim reanalysis data were computed to provide objective forecast skills.These scores were computed for the NH and SH.Skill scores for the 500-hPa geopotential height are standard measures of the medium-range forecast skill suggested by World Meteorological Organization (WMO; 1992).Figure 5 compares the 500-hPa geopotential height AC and bias computed from the experiments.These scores are the average of 62 cases in August 2010.For the AC (Figures 5a and 5c), the forecast skill for CNTL shows the highest of all experiments.This result indicates that the analysis fields created by the system, consisting of higher-resolution DA and the forecast model, is better than the others' analysis fields.This superiority could be due to the especially advanced GDAS DA system in the CNTL, based on high resolution and integrated benefits of a more diversified set of non-conventional observation data, such as SSMI, AIRS, and QuickSCAT radiances, despite virtual similarities with that in other experiments.The more effective way to gain observation on the higher resolution and sophisticated DA framework could relate with a higher AC.
Moreover, notably, the background fields in the CNTL were simulated from the higher-resolution GFS forecast model with revised physics package.This noteworthy fact reflects that the superiority of CNTL arises from a high-quality background field as well as a sophisticated DA system.The EXP1 and EXP2 experiments simulated in cold-start mode, using the analysis created from assimilation with higher-resolution of background fields, clearly outperformed the warm cycle (EXP3 and EXP4), in terms of AC in both hemispheres, throughout the 5-day forecast period.The similar kind of features appears in the bias, too (Figures 5b and 5d).The EXP1 and EXP2 showed the smaller bias than EXP3 and EXP4 for both hemispheres.The differences in AC and bias, between the group of EXP1 and EXP2 and the group of EXP3 and EXP4, depend on the background field and choice of cycle.Such differences' connection to the background field and choice of cycle was particularly marked in the SH, which exhibited superior background fields generated by advanced DA/forecast system, with the usage of diverse observation.Although EXP3 and EXP4 ran in warm-start mode, e.g. more stable analysis and forecast sequence, the group showed poor forecast skill.Therefore, the difference in the quality of background field could result in a distinct difference between two groups.The background fields used in EXP1 and EXP2 were produced by the interpolation of the higher-resolution background field, but they still possess the memory of assimilated information with highly resolved forecast field over the fine-grid system, which highlights the importance of background field in the DA and forecast system.
The impact of BES was also found in AC and bias.The ACs and biases of EXP1 and EXP2 showed consistently better scores than those of EXP3 and EXP4 over the globe during the entire forecast time.In addition to the comparison between groups, both the comparison of EXP1 with EXP2, and the comparison of EXP3 with EXP4 demonstrated mixed results, and statistically insignificant differences.These results imply that the DA system that employs its own BES, cannot show better forecast skill than one that employs a BES that is not created from its own forecast error.This finding supports the notion that the impact of the BES is negligible, compared to the impact of background fields.Meanwhile, a relatively poor skill of the GRIMs forecast system with its own BES (EXP4) seems to be due to a relatively short period of BES computation, as compared to the typical period carried out in operational centers.
It is helpful to investigate the impact of the BES and the background field by means of another analysis tool such as the RMSE.RMSE distributions against the ERA interim analysis over the NH were calculated according to the forecast time (Figure 6).At the initial time, the RMSEs from the CNTL, EXP1 and EXP2 experiments were concentrated at lower value than those from the EXP3 and EXP4 experiments.These differences relatively decreased during the forecast period.For the 5-day forecast, the peaks of RMSE distribution in all experiments seemed to converge, but EXP3 and EXP4 still showed the high distribution over the greater sections of RMSE.This phenomenon indicates that the impact of background fields on forecast skill had been dominant throughout the forecast period, but its impact gradually decreased during forecasts.Compared to the highest distribution of EXP1, that of EXP2 was shifted to the lower sections of RMSE, during the analysis time, to improve the quality of the analysis field.Moreover, EXP4 coincided with EXP3 in the highest distribution; however EXP3 was more skewed to the lower sections of RMSE, indicating the decrease of higher RMSE.These features manifest the generally better quality of the analysis fields that are carried out by DA system with its own BES, into which dynamical and physical relationships of the forecast model are built.Such DA system demonstrates superiority to the alternative, in terms of RMSE.The impact of the BES gradually diminished for the 5-day forecast.Despite the BES's passivity to primitive forecast skills, the BES clearly influences the medium-range forecast skills after day 7.  (Huffman et al., 2007).The TRMM provides a calibration-based sequential scheme for combining precipitation estimates from multiple satellites, as well as rain gauge analyses on fine scales.An overall increase in the total precipitation in the western Pacific and maritime tropical islands is observed in CNTL (Figure 7d).Apparently, the precipitation of the EXP1 and EXP2 are quite similar to that of the CNTL experiment, but the amounts of precipitation over the ITCZ from EXP1 and EXP2 are smaller than that from the CNTL data.The precipitation in maritime tropical islands increases, according to the EXP3 and EXP4 experiments (Figures 7c and 7f).This, precipitation verification in qualitative terms, emphasizes the larger impact of background fields, compared with that of BES.Increases in total precipitation based on data from EXP3 and EXP4 for the maritime tropical islands presumably rely on the significant intensification of large scale precipitation rather than the convective one (not shown).
Verification of daily accumulated precipitation in the day 5 forecast, especially over land, NH (20°N to 50°N), tropics (20°S to 20°N) and SH (50°S to 20°S), against the observed rain data, was conducted to evaluate the skill in precipitation forecasts (Table 3).Refer to Koo et al. (2009) for detailed descriptions of these indices such as the bias, RMSE, and pattern correlation coefficient (PC).Table 3 confirms the overall improvement of land precipitation in the NH and SH, due to implementing the cycle run (EXP3 and EXP4).PCs from those experiments are larger than those from the CNTL, in the NH and tropics.The physical parameter such as precipitation, unlike large-scale parameter, in warm-start run corresponded more with the observation than the cold-start run.Although EXP3 and EXP4 utilized a lower-resolution background field than EXP1 and EXP2, the sequence of analysis steps has the advantage to provide slightly better physical or hydrological balance.Physical processes in EXP1 and EXP2 employing dissimilar background field out of other forecast models at every analysis step could not ensure the enforcement of the hydrological balance.The usage of the different kind of BES did not show marked difference.Like the previous qualitative comparison of precipitation over the globe, this quantitative verification over land area manifest the stronger impact of the choice of background field in a DA/forecast system, than that of the choice of BES.

Summary and Concluding Remarks
The resulting medium-range forecast sensitivities of a global forecast model, from applying different initial fields and BES to the data assimilation were examined.To evaluate the sensitivity to the initial field, two experiments were simulated with the initial data that are assimilated with the high-resolution GDAS background fields, where BES of GFS and GRIMs are set up, respectively.Another pair of experiments was designed with the GRIMs own assimilated analysis fields with the BES from GFS and GRIMs, respectively.Evaluations were performed in August 2010, focusing on the 500-hPa geopotential height and precipitation.
According to the verification against the ERA-interim reanalysis data of 500-hPa geopotential height, results from the GDAS analysis and background fields appear to outperform those from the GRIMs background fields in terms of AC scores over both hemispheres.Comparisons of the forecast skill scores with the GFS BES and the GRIMs BES show the improved predictability in the NH when the GRIMs BES is used.This indicates that the BES determined by the forecast model impacts initial data, in which the initial data's attributes derived from data assimilation sustain during the simulation.RMSE distributions according to the forecast time show that the shape of the error distribution from the GDAS analysis, background data, and GRIMs background distinguishes itself at the initial time, but this distinction blurs during the simulation.This indicates that the impact of initial and background fields on the forecast skill is dominant throughout the forecast period.Throughout the overall evaluation, the impact of background field is larger than BES.
Considering the monthly average of the daily accumulated total precipitation at the 120-hour forecast time, all experiments reproduced several peaks over the Andaman Basin and ITCZ in the Pacific and Atlantic oceans.The verification results show that the simulation with higher resolution of background fields approximates the observation, unlike the other.Verification of the daily accumulated precipitation in the day 5 forecast confirmed the overall improvement of the PC in the NH and tropics, due to the support of the cycle run.In the context of the balance in DA/forecast system, observation verification shows that the consecutive process of the warm-cycle run can ensure the physical consistency in the system.
In conclusion, background fields have a much more significant impact on medium-range forecast skill than the choice of BES.Despite its lack of response to primitive forecast skills, the BES influences the forecast skill after day 7.A relatively poor skill of the GRIMs forecast system with its own BES seems to be due to a relatively short period of BES computation, as compared to the typical period carried out in operational centers.

Figure 3 .
Figure 3.Time series of the globally averaged and vertically integrated root mean square differences (RMSD) of temperature of the background field (bold) and analysis (plain), computed from (a) EXP1, (b) EXP3, (c) EXP2 and (d) EXP4 experiments.They are verified against radiosonde observation.Values denote the average for the entire experiment period

Figure 5 .
Figure 5. Monthly mean 500-hPa geopotential height verification scores of the (a) and (c) anomaly correlation, (b) and (d) bias of the NH (left panels) and SH (right panels) against ERA-interim reanalysis, computed from the CNTL (bold, solid line), EXP1 (black solid line), EXP2 (black dashed line), EXP3 (red solid line) and EXP4 (red dashed line) experiments in August 2010

Figure 6 .Figure 7 .
Figure 6.The distribution of 500-hPa geopotential height RMSE distribution at (a) initial time and (b) five-day forecast against ERA-interim reanalysis over the NH, computed from the CNTL (bold, black solid line), EXP1 (black solid line), EXP2 (black dashed line), EXP3 (red solid line) and EXP4 (red dashed line) experiments in August 2010

Table 2 .
Summary of numerical experiments configuration to assess the impact of the background field and background error statistics (BES)

Table 3 .
Bias, RMSE, and pattern correlation coefficient (PC) for monthly averaged 24-hour accumulated precipitation of the day 5 forecasts from 1 to 31 August 2010 over land in the NH, tropics and SH simulated by the experiments and, compared to the TMPA data.The values in parentheses are those over the ocean.The lowest RMSE and highest PC in respective regions are in boldface