Simulations of the Asian summer monsoon in the sub‐seasonal to seasonal prediction project (S2S) database

Several monsoon indices have been applied to multiple models from the sub‐seasonal to seasonal (S2S) prediction project database during the period May to October 1999–2010 to assess their ability to simulate the Asian monsoon. The bivariate anomaly correlation (BAC) of the Boreal Summer IntraSeasonal Oscillation (BSISO) index suggests that the operational models can predict the BSISO1 and BSISO2 events up to 6–24.5 and 6.5–14 days in advance respectively, although the models tend to underestimate the amplitude of BSISO as the lead time increases. For the strong BSISO events, BSISO1 (BSISO2) display lower skill mostly in phases 3–5 for all the models, suggesting that the BSISO1 (BSISO2) is not easy to predict when it is located over India and the Maritime Continent (South China Sea and Bay of Bengal). On the other hand, the higher skills appear in different phases for different models. For instance, the limit of predictive skill of strong BSISO1 and BSISO2 events in phases 6–7 for the ECMWF ensemble forecast could exceed 30 and 28 days, respectively. The comparisons of the BSISO life cycle among the ECMWF, NCEP and CMA models also indicate that the ECMWF model can better predict the evolution of strong BSISO events. Predictions of additional monsoon circulation indices including the Webster–Yang index (WY), Indian Summer Monsoon index (ISM), South Asian Monsoon index (SAM), and SouthEast Asian Monsoon index (SEAM) in the S2S models have statistically significant skill over the corresponding monsoon regions up to 9–31, 3–17, 7–13 and 7–14 days, respectively. However, the significant skill of summer monsoon precipitation over the SAM and SEAM regions varies significantly among the models, with the skill ranging from 2 days to 2 weeks lead time.


Introduction
Following the success of short-to-medium-range weather prediction and long-range or seasonal forecasts, recent attempts have been made to produce skilful sub-seasonal to seasonal forecasts. The sub-seasonal to seasonal (S2S) time range has been considered as a 'desert of predictability' for a long time (Waliser et al., 2003;Vitart et al., 2017). However, prediction at this time range remains challenging, because this time range is believed to be too long for the atmosphere to still have a memory of its initial conditions and too short for the boundary forcing (e.g. ocean, land and sea ice) to vary enough to add skill beyond simple persistence.
To bridge the gap between weather and seasonal prediction and meet the demands in user communities, the World Weather Research Program (WWRP) and the World Climate Research Program (WCRP) established a Sub-seasonal to Seasonal (S2S) Prediction Project (lasting 5 years; Vitart et al., 2017) in November 2013. An important outcome of this project has been the establishment of an extensive database, containing sub-seasonal to seasonal re-forecasts (up to 60 days) and near-real-time forecasts (3 weeks behind) from 11 operational centres (available to the research community since May 2015). This database represents an important tool for the research community and offers a good opportunity to assess and improve the performance of operational model systems, and advance our understanding of the S2S time range.
Major topics, such as monsoons, Madden-Julian Oscillation (MJO), Africa, extremes, teleconnections and verification, are investigated in the S2S project. It is generally known that monsoons are closely related to many other weather events (e.g. tropical cyclones; heavy rainfall) and climate phenomena (e.g. El Niño/Southern Oscillation (ENSO)), and show enormous social and economic impacts all over the world, especially in Asia (e.g. Wang et al., 2008Wang et al., , 2014Liu et al., 2013a;Jie et al., 2014Jie et al., , 2015. However, monsoons are difficult to simulate because of the complexity of these events over the Asian regions (Liu et al., 2013b). The forecast skill of monsoons has significantly improved in recent decades (Liu et al., 2013a). It is due to the improvements in the development of dynamical models including better initial conditions, increased resolution, better representation of physical processes and ensemble generation.
Many studies have demonstrated that state-of-the-art models not only are able to simulate the interannual or climatological variability of the Asian monsoon (Goswami, 1999;Kang et al., 2002;Wang et al., 2008), but are also able to predict the Asian monsoon features on seasonal-to-interannual time-scales several months in advance, such as heavy monsoon rainfall centres and large-scale circulation patterns (Yang et al., 2008;Jiang et al., 2013). However, the prediction of the amplitude, frequency and propagation of the monsoon remains difficult. While most of Asian monsoon studies focus on long-range predictions, the number of studies on sub-seasonal to seasonal variability which include the prediction of the monsoon onset and retreat has increased in recent years. For instance, Sperber et al. (2001) examined the intraseasonal variability of the Asian summer monsoon (ASM) in atmospheric general circulation models (AGCMs). They found that many models are able to simulate the dominant dynamical pattern of sub-seasonal variability, but have difficulty in simulating the precipitation patterns and higherorder modes of sub-seasonal variability. Waliser et al. (2003) evaluated the simulations of intraseasonal variability associated with the ASM from ten AGCM models. The results showed that several models displayed spatial patterns of intraseasonal variability consistent with observation. Fu et al. (2007Fu et al. ( , 2009 indicated that ocean-atmosphere coupling significantly increased the predictive skill of monsoon intraseasonal oscillations rainfall in almost the entire Asian-western Pacific region by about a week, when the monsoon intraseasonal oscillation signals in NCEP reanalysis used as initial conditions were similar to those in satellite observations and in situ observations. These forecast skills reach 25 days for 850 hPa zonal winds and 15 days for rainfall over the Tropics and Southeast Asia. Thereafter, Fu et al. (2013) compared the intraseasonal forecast of the ASM in four operational models. They showed that the ECMWF and University of Hawaii (UH) models had a better representation of the northward propagation of Boreal Summer IntraSeasonal Oscillation (BSISO) than the National Centers for Environmental Prediction (NCEP) Climate Forecast System version 1 and version 2 (CFSv1 and CFSv2). Liu et al. (2013aLiu et al. ( , 2013b further indicated that the NCEP CFSv2 showed reasonable skill of sub-seasonal prediction of summer monsoon rainfall over several tropical Asian ocean domains. Abhilash et al. (2014) evaluated the intraseasonal prediction skill of the Indian summer monsoon by using an ensemble prediction system based upon NCEP CFSv2. They found that useful probabilistic forecasts could be generated up to the fourth pentad lead. Lee et al. (2015) examined the predictability and prediction skill of BSISO over the ASM region in the IntraSeasonal Variability Hindcast Experiment (ISVHE). They found the multi-model mean BSISO predictability and prediction skill with strong initial amplitude (10% higher than the mean initial amplitude) are about 45 and 22 days, respectively. It indicates that there is considerable room for improvement of BSISO prediction. Recently, Neena et al. (2017) evaluated the prediction skills of BSISO for 27 general-circulation models (GCMs). Many of the GCMs can capture BSISO propagation, but significant biases are still seen in precipitation simulations. Although these previous works have shown the potential for subseasonal to seasonal prediction of the Asian monsoon in several models, the predictive skill level in the current major operational centres has still not been assessed. The S2S project database provides a great opportunity for researchers to further explore the prediction of the Asian monsoon in the state-of-the-art operational S2S models, and thereby help to improve the development of operational models. This study analyses the Asian monsoon predictions at the sub-seasonal and seasonal time-scale in ten models from the S2S database (the upload of Korea Meteorological Administration data is not complete). This article is organized as follows: the S2S database and observational data are introduced in section 2; the verification methods are provided in section 3; the results are explored in section 4; and section 5 provides a summary and discussion.

S2S database and observations
The S2S database contains re-forecasts and near-real-time forecasts (only available with a 3-week delay) with a forecast lead-time up to 60 days from 11 operational centres: Australian Bureau of Meteorology (BoM), China Meteorological Administration (CMA), European Centre for Medium-range Weather Forecasts (ECMWF), Environment and Climate Change Canada (ECCC), the Institute of Atmospheric Sciences and Climate (CNR-ISAC), Hydrometeorological Centre of Russia (HMCR), Japan Meteorological Agency (JMA), Korea Meteorological Administration (KMA), Météo-France/Centre National de Recherche Météorologiques (CNRM), National Centers for Environmental Prediction (NCEP) and the United Kingdom's Met Office (UKMO). Each re-forecast or forecast includes a control forecast (only using a single non-perturbed initial value) and a number of perturbed forecasts which are used to produce probabilistic forecasts or ensemble means. Table 1 shows the main characteristics of the S2S models only for the re-forecasts (see details in Vitart et al. (2017) or https://software.ecmwf.int/wiki/display/ S2S/home). Because of the differences in re-forecast time ranges and re-forecast periods among different operational centres, the common lead time 0-31 days and the period May to October in 1999-2010 were chosen in this study. Due to the different re-forecast frequencies, the re-forecast start dates are not identical in most models (ECMWF is initialized twice weekly from 30 April; NCEP and CMA daily from 30 April; JMA three times a  month from 10 May; CNRM twice a month from 1 May; HMCR weekly from 1 May; BoM five daily from 6 May; CNR-ISAC five daily from 1 May; UKMO eight daily from 1 May; ECCC five daily from 5 May). The different horizontal resolutions of atmospheric models were uniformly interpolated to a 1.5 • × 1.5 • grid. In addition, the ensemble size and whether or not the atmospheric model is coupled to an ocean or sea-ice model are not consistent among the various models. Despite these differences, there are enough commonalities in the S2S models to make intercomparisons, as will be shown in this work. The observation data used in this study include: daily horizontal winds at 850 and 200 hPa from ECMWF's ERA-Interim reanalysis (Dee et al., 2011) at 1.5 • × 1.5 • horizontal resolution and NCEP reanalysis  at 2.5 • × 2.5 • resolution; daily precipitation from the Global Precipitation Climatology Project (GPCP: Huffman et al., 2001;Prakash et al., 2015; http://precip.gsfc.nasa.gov/) at version 1DD (One-Degree Daily) 1.2; and daily outgoing long-wave radiation (OLR) with 2.5 • × 2.5 • horizontal resolution from the National Oceanic and Atmospheric Administration (NOAA) polar-orbiting satellites (Liebmann and Smith, 1996).

Boreal summer ISO and monsoon indices
The tropical IntraSeasonal Oscillation (ISO) plays an extremely important role in the evolution of the ASM (Waliser et al., 2003). The BSISO index suggested by Lee et al. (2013) is based on multivariate empirical orthogonal function (MV-EOF) analysis of daily anomalies of OLR and zonal wind at 850 hPa (U850) in the region 10 • S-40 • N, 40 • -160 • E during the period 1 May to 31 October (e.g. the observed EOF modes for 1999-2010 as shown in Figure 1). The OLR and U850 anomalies were obtained by removing the slow annual cycle (mean and first three harmonics of climatological annual variation) and interannual variability (the mean of the last 120 days) as in Wheeler and Hendon (2004). While BSISO1, defined by the first two principal components (PCs) of the MV-EOF, represents the canonical northward-propagating BSISO which often occurs in conjunction Table 2. List of monsoon indices defined by atmospheric circulation and precipitation.

Type Index Definition
Atmospheric circulation with the eastward-propagating MJO, BSISO2 characterized by the second two PCs mainly reflects the northward/northwestwardpropagating BSISO during pre-monsoon and monsoon onset periods. In this work, BSISO1 and BSISO2 indices have been computed for each S2S model over the period 1999-2010 in order to assess the skill of the S2S models over the ASM region. The PC data from each S2S model are obtained by projecting OLR and U850 anomalies onto the observed EOF structures. As shown in Table 2, additional monsoon indices including the Webster-Yang index (WY: Webster and Yang, 1992); Indian summer monsoon index (ISM: Wang et al., 2001); the South Asian Monsoon index (SAM: Goswami et al., 1999); and the SouthEast Asian Monsoon index (SEAM: Wang and Fan, 1999), have been applied to the S2S models to provide additional measures of the prediction skill of atmospheric circulation or precipitation over ASM regions. The WY index (zonal windshear anomaly between 850 and 200 hPa levels averaged over the region 0-20 • N, 40-110 • E) is a good indicator of the Indian summer monsoon , although it has low correlation with Indian monsoon rainfall (Goswami et al., 1999); the ISM (difference of U850 between 5-15 • N, 40-80 • E and 20-30 • N, 70-90 • E) not only represents well the monsoon anomalies averaged over the Bay of Bengal, India and the eastern Arabian Sea, but it is also highly correlated with the Indian rainfall anomalies; the SAM index (meridional wind-shear anomaly between 850 and 200 hPa In addition, the SAM and SEAM monsoon rainfall indices, calculated by averaging rainfall over 10-30 • N, 70-110 • E, and 5-25 • N, 100-130 • E, respectively, will also be used to validate rainfall predictions over these regions (Lau et al., 2000;Liu et al., 2013a).

Results
In this section, the skill of sub-seasonal to seasonal forecasts from ten S2S models has been assessed as a function of lead time using different monsoon indices over the ASM regions. According to Table 1, only the hindcasts of NCEP and CMA have the same re-forecast frequency as the ECMWF hindcasts in the S2S database. Each re-forecast has been verified up to day 31 against the corresponding observation during May to October for the 12 years of the common re-forecast period 1999-2010. Figure 2 shows the bivariate anomaly correlation (BAC) and the root-mean-squared error (RMSE) between ERA-Interim reanalysis/NOAA and forecasts as a function of lead time for BSISO1 and BSISO2. The BACs for BSISO1 (Figure 2(a)) indicate that control forecasts (CF; dashed lines) from ten models can predict the BSISO1 event up to 6-17.5 days in advance as the corresponding correlations exceed 0.5 which is taken as the threshold of useful skill (black dashed horizontal line). The ensemble mean skill scores (ENS; solid lines) show an obvious enhancement in the ECMWF, UKMO, BoM, CNRM and ECCC models compared to the CF, whereas only slight improvements are displayed in the NCEP, CMA and JMA models. This improvement is possibly related to the perturbation methods and the ensemble size for different models. These skilful ensemble forecasts display skill between 10 and 24.5 days. In particular, the ECMWF ensemble forecast can increase the BSISO1 prediction skill by nearly one week compared to CF. Similarly, the BSISO2 event can be predicted by the CFs 6.5-12 days in advance, depending on the model, and by the ENS between 7 and 14 days in advance (Figure 2(b)). In particular, the BSISO2 prediction skill of ECMWF, UKMO and CNRM models increases by up to 2 days when using ENS rather than CF. These results are confirmed by the RMSE (Figures 2(c) and (d)) of CF and ENS for BSISO1 and BSISO2 as the RMSE values exceed 1.414 (black dashed horizontal lines) between 6 and 24.5 days and between 6.5 and 14 days, respectively.

BSISO index
To explore the impact of frequency of model initialization, re-forecast period, and observation on the BSISO BAC, ECMWF re-forecasts have been scored using an initialization frequency of twice weekly and once weekly, for the periods of 1999-2010 and 1995-2013, and using ERA-Interim and NCEP reanalyses. Figure 3(a) shows that the BACs of BSISO1 for the CFs at frequencies of twice weekly (purple dashed line) and weekly (green dashed line) are quite similar during 1999-2010 when the BACs are above 0.5. However, with the same forecast frequency (twice weekly), the BAC of BSISO1 in the period 1995-2013 (black dashed line) is clearly higher than that in the period 1999-2010 (green dashed line). These results are also supported by the ENS (solid lines), with a prediction skill limit exceeding 32 days for the period 1995-2013 (red solid line). For BSISO2 prediction (Figure 3(b)), the impact of frequency of initialization  (d), although there is a slightly lower skill for the ECMWF model in the first 3 days for both BSISO1 and BSISO2. This is likely due to the fact that the ECMWF re-forecasts have been initialized with ERA-Interim. The above results indicate that the intercomparisons among the ten available models are valid even if the frequencies of initialization are different, and the impact of using ERA-Interim instead of NCEP reanalysis on the validations can be ignored in this study. Figure 4 shows the errors of BSISO amplitude ( √ PC1 2 + PC2 2 for BSISO1; √ PC3 2 + PC4 2 for BSISO2) as a function of lead time during May-October (MJJASO) 1999-2010, which are calculated by A model −A obs A model (A model and A obs represent the amplitudes of forecast and observation, respectively). The BSISO1 and BSISO2 amplitude errors computed from the ten CFs (dashed lines in Figure 4) increase gradually during the first 10 days. This result suggests that the amplitude of BSISO is underestimated by all the S2S models, particularly at long lead times, with the amplitude 30-70% weaker than in ERA-Interim at a 30-day lead time. The ECMWF, UKMO and NCEP models show better performances in the prediction of BSISO1 amplitude in the first 2 weeks. The ECMWF and UKMO models are also skilful in predicting BSISO2. Generally, there is no significant difference between the averaged amplitude error of perturbed forecasts (solid lines) and the corresponding CF (dashed lines).
The EOF modes of OLR and U850 anomalies from the ECMWF CFs are further examined. Figure 5 shows that the patterns of the first four EOF modes from the control forecasts at lead time day 11-15 (left panel) are basically similar to the corresponding observations (not shown). The differences of OLR and U850 EOF (middle panel) are mainly located in the western Pacific, Indian Ocean and Indian peninsula, and generally become greater as the EOF mode increases. In addition, the biases of the EOF modes at lead time of day 1 (right panel) are relatively small, and their patterns are inconsistent with the biases at days 11-15 ( Figure 5, middle panel). This suggests that the main biases of EOF mode may not originate from the model initial value, but from the model dynamical or physical processes. Figures 6 and 7 show the life cycles for the strong BSISO1 and BSISO2 events (i.e. the amplitude is greater than 1.5) predicted by the ECMWF, NCEP and CMA models with the common initialization dates at the lead times of day 1, 5, 10 and 15 during MJJASO for the period 1999-2010. The three models capture reasonably the characteristic of the observed life cycle (blue dashed lines) in the eight phases of BSISO1: the corresponding colour lines denoting the biases between CFs and observation are generally short on day 1 and 5 (Figures 6(a) and (b)). From day 10 to day 15, the ECMWF (purple curves) model can still predict the evolution of strong BSISO1 events, although the intensity of BSISO1 decreases with lead time. However, the NCEP (green curves) and CMA (orange curves) models start to show low skill in some of the eight phases on day 10 (e.g. the large biases shown in phases 3 and 4), and no skill in most phases beyond day 15. Likewise, while the NCEP and CMA models have difficulty    predicting the occurrence of strong BSISO2 events in most phases after 10 days, the ECMWF model can still forecast strong BSISO2 events, although their amplitudes are underestimated. Beyond day 15, the forecast amplitudes are generally lower than 1.0 (Figure 7(d)). The BACs between the observed strong events and forecasts have been calculated as a function of lead time and target phase for the ten S2S models (Figures 8 and 9). For each target phase, the BAC is computed by using the BSISO events belonging to the target phase with amplitude greater than 1.5. The CFs (shadings) from ECMWF can generally predict the strong BSISO1 events more than 18 days in advance since the corresponding BACs are greater than 0.5 (Figure 8(a)). The ECMWF seems to be more skilful in predicting phases 1-3 and 6-7 than in predicting phases 4-5 and 8. This suggests that the BSISO1 is more difficult to predict when it is located over East Asia, India and the western North Pacific (the location corresponding to each phase can be identified in Figure 6). Similar results were obtained with the NCEP, HMCR, ISAC and UKMO models, although their predictive skills were relatively lower (Figures 8(b), (f), (g) and (i)). The CMA and BoM models show higher skills in phases 3-4 and 6-8, but relatively lower skills in phases 1 and 5 (Figures 8(c) and (h)); the JMA and CNRM display better scores around phase 6 (Figures 8(d) and (e)); and the ECCC displays better skills around phases 3-4 (Figure 8(j)). After averaging the ensemble forecasts (contours), the BSISO1 predictions are generally improved in each phase, especially for the ECMWF model. The useful prediction skill limit (BACs dropping to 0.5) of the BSISO1 in phases 6-7 in ENS exceeds 31 days with the ECMWF model. However, these forecasts often display lower skill around phase 5. This suggests that the BSISO1 is not easy to forecast when it is located over India and the Maritime Continent, which involves complex and intense variation of convection. Figure 9 shows the skill as a function of lead time and phase for strong BSISO2 events for the ten S2S models. The CF (shadings) and ENS (contours) from ECMWF have better skills in phases 3-4 and 6-7, and display predictive skill (BACs ≥ 0.5) up to 28 days (Figure 9(a)). The CMA, JMA, CNRM, ISAC, BoM and ECCC models show a similar distribution of forecast skill which is generally lower around phases 3-5 and relatively higher in front and rear phases, whereas the NCEP, HMCR and UKMO models display a relatively uniform distribution of skill for each phase.
The above results indicate that most models lack skill when the strong BSISO2 events occur over India, the South China Sea and the Bay of Bengal (the location corresponding to a given phase can be identified in Figure 7).

Monsoon indices
The seasonal variation of ASM is explored with different monsoon indices. Figure 10 shows the variations of the WY index (described in section 3) for the ten S2S models' CFs at different lead pentad means. In the first pentad, all S2S models capture well the main features of the Indian summer monsoon as the predicted WY indices (red lines) are generally very close to the observation (black lines). However, as the lead pentad increases (other colour lines), the biases between CFs and observation increase. Basically, the NCEP, ECMWF, CMA and JMA models underestimate the intensity of the Indian monsoon, particularly during the period July to October. This suggests that model biases are stronger during the monsoon retreat. However, the WY index is overestimated by the CNRM, HMCR and BoM models after two pentads; predicted too early by the CNR-ISAC model and too late by the UKMO and ECCC models. The verifications of SEAM, SAM and ISM indices (not shown) suggest that the seasonal variations of summer monsoon over different regions can be generally reproduced by most models during pentads 1-4, although the SEAM index is always overestimated by all models (except NCEP and JMA); the SAM index is strongly underestimated by all models (except ISAC); the ISM index is overestimated by six models (excluding the NCEP, JMA, UKMO and ECCC).
After removing the seasonal cycle, the sub-seasonal prediction skill of the monsoon indices is investigated by computing the correlation between observation and forecasts over the corresponding monsoon regions. Figure 11 shows the temporal correlations between the S2S re-forecasts and observations as a function of lead time for the dynamical monsoon (Figures 11(a)-(d)) and precipitation indices (Figures 11(e)-(f)) listed in Table 2. While 0.5 is taken as the BAC correlation threshold of useful skill for BSISO, as far as we know, many studies mainly use a statistical significance test in the prediction assessment of a monsoon index (Lau et al., 2000;Xue et al., 2010;Liu et al., 2013a). Over the Indian monsoon region, the CF skill of the WY index is in the range of 9-20 days as the correlations are above the 99% confidence level (black dashed horizontal line), and 2-7.5 days when the correlations exceed 0.5 (red dashed horizontal line). It could be respectively enhanced to 12.5-31 and 2.5-10 days by using the ENS, particularly for ECMWF and UKMO models. The CF correlations of the ISM index are higher than the 99% confidence level (the correlation 0.5) up to 3-12 (0-8) days, and can be enhanced by about 0.5-5 (0.5-2) days after ensemble averaging except for UKMO (Figure 11(d)). Over   south and southeastern Asia, the skills of the SAM and SEAM indices are lower than the 99% level and the correlation equal to 0.5 after about 10 and 5 days, respectively. The ENS performs generally better than the CFs (Figures 11(b) and (c)). For the summer monsoon rainfall, there is a significant difference of forecast skills among the ten S2S models (Figures 11(e) and (f)). For example, the HMCR CF skills for the SAM and SEAM indices drop below the 99% level beyond 2 days, compared to 9 days with the ECMWF model. The ENS shows better performance (solid lines) than the CFs, particularly with the ECMWF and UKMO forecasts of the SAM index as the correlations are significant at 99% level in about 2 weeks. However, in the criterion of correlation 0.5, there are only a few models that show the precipitation skills of SAM and SEAM in about 2.5 days. It should be noted that an obvious drop at day 10 for the ECMWF model is due to the change of model resolution. This could be resolved with full access to the ECMWF archive, but this is unavailable in the S2S project. The verification of daily precipitation and 850 hPa winds averaged from 25 May to 25 June (monsoon transition period) during 1999-2010 over the ASM region is shown in Figure 12. At the 10-day lead time, the ECMWF model can reproduce accurately the observed patterns of precipitation, although there are, to some extent, positive biases over the tropical western Pacific, southeastern Asia, the Bay of Bengal, Arabian Sea and the Tibetan Plateau, and negative biases over the Indo-China Peninsula, south of China and subtropical western Pacific (Figure 12(c)). Generally, the NCEP, JMA, CNRM, BoM and UKMO models display similar bias patterns but with a larger amplitude (Figures 12(b), (e), (f), (i) and (j). However, the CMA, HMCR, ISAC and ECCC models show significant negative biases over the eastern Indian Ocean and part of southeastern Asia, and positive biases over the Arabian Sea (except the ISAC) (Figures 12(d), (g), (h) and (k)). For the monsoon circulation, while the ECMWF and UKMO models have relatively small biases, most models present large anticyclonic biases over the Arabian Sea, Indian peninsula, the Bay of Bengal (except the ISAC) and the western Pacific. During the mature monsoon phase (from 25 June to 25 July; as shown in Figure 13), the negative biases of precipitation and circulation increase over India, the Indo-China Peninsula and Philippine Sea in the ECMWF, NCEP, JMA CNRM, HMCR, ISAC, BoM and UKMO models, but the biases over subtropical western Pacific   decrease in most models (except the ECCC) as compared to that in the monsoon transition period.

Summary and discussion
The sub-seasonal to seasonal predictions of Asian monsoon in multiple operational models including the BoM, CMA, ECMWF, ECCC, CNR-ISAC, HMCR, JMA, CNRM, NCEP and UKMO models are assessed and explored using the S2S database and by using the BSISO, WY, SAM, SEAM, ISM monsoon indices during May to October for the 12 years of 1999-2010. The BACs and RMSE for BSISO1 and BSISO2 indices indicate that the operational models with a single/perturbed initial value can predict the BSISO1 and BSISO2 events, respectively, up to 6-24.5 and 6.5-14 days in advance, although the amplitude of the BSISO is underestimated as the lead time increases. The BSISO skill scores are generally improved when using ensemble means. Further diagnostics demonstrate that the impact of different model initialization frequencies or using different observations for the validation is limited except when using a different forecast period. The patterns of the first four EOF modes of OLR and U850 anomalies are well reproduced by the ECMWF model.
The comparisons of the BSISO life cycle among the ECMWF, NCEP and CMA models suggests that the ECMWF model can better predict the evolution of strong BSISO1 and BSISO2 events than the other models, although the amplitudes are seriously underestimated beyond day 15. The lower skills of the ten S2S models for BSISO1 (BSISO2) happen mostly around phase 5 (phases 3-5). This suggests that the BSISO1 (BSISO2) is not easy to reproduce if it is located over India and the Maritime Continent (the South China Sea and Bay of Bengal). For the ECMWF ensemble mean, the limit of skilful predictions of BSISO1 and BSISO2 in phases 6-7 exceeds 30 and 28 days, respectively. The seasonal variations of WY, SAM, SEAM and ISM indices can be reproduced by the ten models even at the pentad 4 lead time.
The skills of sub-seasonal variation of monsoon circulation can be significant at the 99% correlation level up to 9-31 days over the WY index region (up to 3-17 days over ISM, up to 7-13 days over SAM and up to 7-14 days over SEAM) by the CFs and ENS. However, the skills of summer monsoon precipitation over the SAM and SEAM index regions are different among the ten S2S models (significant skill ranging from 2 days to more than 2 weeks). Using 0.5 as a correlation criterion, the predictive skills for monsoon circulation are up to about 10 days over WY, ISM and SAM index regions and 5 days over the SEAM region, but the skills of precipitation over SAM and SEAM are only up to about 2.5 days. The climatology of daily precipitation and 850 hPa winds predicted by ten S2S models displays generally relatively large biases over the western Pacific, southeastern Asia, the Bay of Bengal, the Indo-China Peninsula and the Arabian Sea.
Overall, although some of the S2S models show encouraging skill in predicting the Asian monsoon at the sub-seasonal to seasonal time-scale, there is still a gap between the skills of the various operational models and between the different ASM regions. This intercomparison among the S2S models should help us identify the advantages and disadvantages of each model which could lead to further model improvements in the future. For example, why do low skills always appear around phase 5 of BSISO in most models? Why does the ECMWF model display such high skill for dynamical circulation and precipitation? These topics need to be further explored to improve the operational models and extend the S2S predictability.