Evaluation of cloud base height in the North American Regional Reanalysis using ceilometer observations

Future climate change predictions by global climate models or earth system models diverge significantly, most likely due to their different cloud responses to global warming. There is an uncertainty as to how the cloud frequency (or cloud fraction) and height will change, in turn, affecting the sign, and amount of cloud feedbacks. While satellite observations have been very useful in augmenting information on clouds, it is mostly related to cloud tops, and there is a lack of information on cloud base height (CBH). In this study, a unique record of CBH information was collected at 706 Automated Surface Observing System (ASOS) ceilometer stations to evaluate the ability of the North American Regional Reanalysis (NARR) model to correctly simulate similar information. It was found that NARR can capture the geographical distribution and the seasonal variation of CBH and cloud base frequency (CBF). On average, the CBF values of NARR were 7% fewer and the CBH values of NARR were 631 m lower than those from observations that span the distance from the surface to 7,600 m. NARR simulates CBH better in arid area in the west of the contiguous United States (CONUS) than in humid areas, where NARR frequently predicted cloud bases too low compared with observations. In the west coast area of the CONUS, the discontinuity between high cloud bases over arid inland areas and low marine cloud layers from the Pacific Ocean over coastal areas produced especially large deviations between the NARR‐simulated and ASOS‐observed CBHs.


| INTRODUCTION
The Earth's climate system is driven by the energy supplied to it at the Earth' surface (Trenberth et al., 2009). An important component of this energy is the surface radiation budget composed both of shortwave (SW) radiation from the sun and longwave (LW) radiation from the atmosphere. Estimation of the LW component is the more complex one since it requires detailed information on the vertical structures of temperature, absorbing species, and cloud properties, including their location in the atmosphere and in particular, their base height (CBH) (Niemela et al., 2001;Stephens et al., 2012;Wang and Dickinson, 2013). Studies have shown that an error of 100 m in CBH may induce an error of 1.5 W m −2 in the downward LW radiation (Viudez-Mora et al., 2015). Therefore, CBH is a crucial parameter for determining the sign of cloud radiative forcing and its response to global warming (Slingo and Slingo, 1988;Stephens, 2005). Accordingly, its accurate simulation is important for the projection of future climate change (Boucher et al., 2013).
Evaluation is essential to ensure that models can accurately simulate the vertical and horizontal distributions of clouds due to the complicated nature of cloudclimate interactions and the lack of understanding of cloud development (Baker, 1997;Chepfer et al., 2014). Moreover, cloud-related processes span scales from under a micrometre for cloud condensation nuclei up to a mega-meter for cloud systems and thus, cannot be fully resolved by numerical simulations at present (Boucher et al., 2013). Up to now, cloud processes and related feedbacks remain the leading sources of uncertainties in the prediction of future climate (Webb et al., 2013;Sherwood et al., 2014;Bony et al., 2015). Therefore, observationally based information is needed to evaluate and improve the performance of cloud processes in numerical weather/climate models.
Evaluation of clouds in reanalysis data or models has been performed for decades following projects such as the Atmospheric Radiation Measurement (ARM) program (Stokes and Schwartz, 1994), the Global Energy and Water Cycle Experiment Cloud System Study (Randall et al., 2003), and the Cloudnet Project (Illingworth et al., 2007), using both satellite-based (Yoo and Li, 2012;J. Zhang et al., 2014) and ground-based observations (Hogan et al., 2001;Wu et al., 2012). Passive satellite sensors can provide information on clouds at global scale but are limited to sensing cloud top properties (Stubenrauch et al., 2013). Active satellite sensors such as the Cloud Profiling Radar on board the CloudSat satellite and Cloud-Aerosol Lidar with Orthogonal Polarization on board the CALIPSO satellite can provide the necessary information on the cloud vertical structure (Hunt et al., 2009;Winker et al., 2010); however, the measurements are spatially sparse and lack diurnal variations of clouds (Rossow and Zhang, 2010). Ground-based radars and lidars can provide continuous and objective observations with high accuracy and have been used to evaluate cloud vertical structure (Barrett et al., 2009;Mittermaier, 2012;Arbizu-Barrena et al., 2015;Nuijens et al., 2015). However, most of these comparisons focused on a few stations or a small region owing to the expensive cost of these ground-based instruments.
Composition of cloud types differ greatly between divergent climate regions and thus present different features. Cloud parameterizations in models treat different cloud processes (e.g., convective and stratiform) differently (Lopez, 2007). Therefore, evaluating clouds in reanalysis data or models in different climate zones may yield totally different results. A recent study (An et al., 2017) showed that the ceilometers in the Automated Surface Observing System (ASOS) provide valid observations of CBHs with high temporal and spatial densities over a large area, covering multiple climatic regions in the contiguous United States (CONUS). The World Weather Research Programme (WWRP) of the World Meteorological Organization (WMO) noted that ceilometer observations offered a reliable method for evaluating cloud forecasts (WMO/WWRP, 2012). In this study, we use information from ASOS and from the Active Remote Sensing of Clouds (ARSCL) project (Clothiaux et al., 2000), which produced the best estimated CBH at the ARM Southern Great Plains (SGP) central facility site.
The North American Regional Reanalysis (NARR) (Mesinger et al., 2006) is evaluated here. It is a highresolution climatological reconstruction of North America and surrounding area. Its products are widely used in research and applications (Markovic et al., 2009;Walsh et al., 2009;Kennedy et al., 2011), where CBH errors could have some impact. It assimilates a great deal of observational data (Shafran et al., 2004), uses an improved land-surface model, and is considered to represent better the hydrological cycle and land-atmosphere interactions than the earlier NCEP/NCAR global reanalysis (Mesinger et al., 2006). Clouds are not assimilated into this reanalysis but are instead predicted by the Eta model (now referred to as North American Meso [NAM]) (Black, 1994) and as such, they are subject to the physical parameterizations of the model. The modelpredicted clouds are further used to predict precipitation and are incorporated into the model's radiation calculations (Zhao et al., 1997).
With the advantages of ASOS observation, the evaluation of NARR was performed for different seasons, times of days, and climatic regions. Both CBH and cloud base frequency (CBF) defined as the frequency of occurrence of cloud base are considered in this study. The rest of the paper is organized as follows: The data and methods are described in Section 2. The results of the evaluation are presented in Section 3. A discussion is made in Section 4 and conclusions are provided Section 5.

| DATA AND METHODS
In this part, we describe NARR, its cloud prediction scheme and post-processing code for CBH, the groundbased observations and cloud detection algorithms, the method for matching NARR simulations to observations, and the geographic classification.
2.1 | NARR, its cloud prediction scheme and post-processing for CBH NARR provides analyses covering North America from 1979 at 3-hourly resolution (Mesinger et al., 2006). It is based on the Eta model Data Assimilation System (EDAS) of 2003 when NARR was frozen; it has a 32-km grid spacing and 45 vertical layers. The cloud prediction scheme used by NARR in the Eta model is described in Zhao et al. (1997). In the scheme, the three-dimensional field of the cloud water/ice mixing ratio is predicted. Stratiform clouds are produced through the large-scale condensation scheme, while convective clouds are produced through the modified Betts-Miller convective adjustment scheme (Rogers et al., 1996).
NARR has a post-processing code for calculating CBH: it checks each grid point starting from the bottom for the first layer with a cloud water/ice mixing ratio ≥ CLIMIT (1×e −6 kgÁkg -1 ). Once it finds a layer with a cloud water/ice mixing ratio ≥ CLIMIT, the CBH will be set to the height of that interface. If the CLIMIT check finds no clouds, it checks whether the column has any convective clouds and sets the CBH to the bottom height of the convective cloud. CBH in the NARR output files is used in this study as provided in the GRIB file format of daily NARR reanalysis data.
2.2 | Ground-based observations and the cloud detection algorithms

| Automated Surface Observing System
The ASOS is composed of automated sensor suites and serves as a primary observing network in the United States. It gradually replaced the manned weather stations starting in the early 1990s (there are still a significant number of man-augmented stations). In this study, only automatic observations from ceilometers at ASOS were used in the evaluation.
The ASOS employs standard laser ceilometers (Vaisala models CT12K or CL31;Vaisala Inc.,Woburn,Massachusetts) to observe CBH. The first-generation ceilometer used in ASOS is CT12K, which has a detection range of 12,000 ft (approximately 3,600 m) above the ground (NWS, 1998). In a few ASOS stations (approximately 130 or more), the CT12K ceilometers have been replaced by CL31 ceilometers, which observe clouds to a range of 25,000 ft (approximately 7,600 m). The replacement has been tested, and the results show that the CL31 sensors were comparable with the CT12K sensors under all conditions (Poyer and Lewis, 2009).
The ASOS sky condition algorithm (NWS, 1998) processes the ceilometer signals into 30-s sample "hits." It determines CBH values using the most recent 30 min of the 30-s sample "hits," and data in the last 10 min are double-weighted. In this study, the ASOS 5-min dataset (NWS, 1998) was used.
CBHs during precipitation are included because the ceilometer and micro-pulse lidar provide relatively accurate CBHs under large but sparse precipitation conditions (Clothiaux et al., 2000). When the intensity of precipitation is high enough that it obscures the CBH signal, the ASOS reports a vertical visibility (VV) calculated from the attenuation coefficient (Vaisala, 1989), and the proportion of VV cases is very low (less than 1%) over the CONUS (An et al., 2017). Besides, the precipitation identifications in the ASOS and NARR are different. ASOS determines different types of precipitation (e.g., rain, snow, and frozen rain) and intensity (e.g., light, moderate, and heavy) using Precipitation Identification Sensors based on the latest 10-min of observations (NWS, 1998), while NARR output 3-hourly accumulated total precipitation. Therefore, removing the precipitation cases would introduce more uncertainties because of the mismatch between observations and model.

| Active Remote Sensing of Clouds
We also use the ARSCL value added product (VAP) best estimate CBH from ARM SGP to test the reliability of ceilometer observations as reference data. The ARSCL VAP combines data from several active remote sensors to produce an objective determination of the vertical distribution of hydrometeors. The best estimated CBHs in ARSCL VAP use the Belfort laser ceilometer and the micro-pulse lidar to produce a single estimate of the CBH. The millimetre-wave cloud radar (MMCR) observations were not applied in the best estimated CBH because the MMCR fails to detect some clouds, especially when the clouds are composed of small hydrometeors, and the MMCR observations are problematic under circumstances involving sparse precipitation, virga, and insect targets (Clothiaux et al., 2000). Data from 2008 to 2010 over the SGP site were used because the ARSCL product at SGP site ends in 2010.
The nearest ASOS station to the ARM SGP station lies at a distance of 37.17 km. The cloud regimes would hardly be the same over such long distance. However, the ARM SGP station also has ceilometer observations. This station uses a CT25K model ceilometer from the Vaisala Company, a version between the old generation CT12K and the new generation CL31. CT25K uses the same single lens technology as CL31 and comparison between CT25K and CL31 shows that these instruments capture a similar pattern of cloud features, while CL31 has a better vertical resolution (Morris, 2012). Here, we compare the 130-min average of observations at NARR output times from CT25K and ARSCL at the SGP station for 3 years, as shown in Figure 1. The observations from CT25K generally agree well with those of ARSCL. The correlation coefficient is 0.79 (p = .00) between instantaneous cases of CBH observed by CT25K and reported by ARSCL.

| Matching samples
The unit of CBH and surface height in NARR are both in geopotential height above sea level. Because the height range is limited by the height of the ASOS observations (lower than 7,600 m), we assume that the geopotential height (gpm) is equal to the geometric height (meters). Initially, the model fixed field of surface geopotential height is subtracted from CBH in NARR at every grid point to obtain the CBH above ground level (AGL). Then, CBHs AGL from NARR are interpolated to the ASOS stations using nearest neighbour interpolation (CBH is a discontinuous variable). Since all the ASOS stations used in this study are located over land, model grid points over water were eliminated. The outputs from NARR at every 3 hr (0000, 0300,…, 2100) (UTC) have been converted into local standard time.
Ground-based active remote sensing instruments, such as cloud radar, lidar, and ceilometers, observe only a narrow field of view directly above their beams. The cloud statistics from such two-dimensional height-time transects are not directly comparable to the threedimensional model grid box (Hogan et al., 2001 T A B L E 1 Criteria used to determine geographic regions, number of ASOS stations with sufficient valid data within the surface-3,600 m level and as well as within the surface-7,600 m level in each region No.

Geographic regions Criterion
Number of sites selected for Section 3.  Appropriate averaging time window centred at NARR output time is chosen for observational data to match the two-dimensional slice to the threedimensional model grid box. A sensitivity test using time windows of 10, 20, 30,…, 180 min was carried out. We determined a valid ASOS observation sample if the number of missing ASOS 5-min reports during a time window is no more than one, because two missing ASOS 5-min reports may mix results calculated by time windows at 10-min interval. CBH for a valid ASOS observation sample is calculated by the mean value of the CBH reports from ASOS during a certain time window. CBF is a binary value, where if a single cloud base is reported by ASOS during the time window, the frequency is set to 1. Figure 2a shows the mean correlation coefficients between CBH simulated by NARR and observed by ASOS using different time windows. It is shown that, the correlation coefficient increases as time window increases and reached a maximum at 130-min (changes very little there-after). Therefore 130-min time window was used to average ASOS observations in the following text.

| The division into geographic regions and selection of ASOS stations
We selected 706 stations from ASOS with abundant data that met the following criteria: sufficient data for each season and each time period (proportion of valid observation samples ≥70%) for more than 5 F I G U R E 6 Seasonal mean CBF (%) simulated by NARR (a1-a4) and observed by ASOS (b1-b4), as well as the bias between NARR and ASOS (c1-c4) within the surface-7,600 m over the CONUS during the period from 2008 to 2013. The four rows "1-4" indicate four seasons: MAM, JJA, SON, and DJF stations have been replaced with CL31 ceilometers. These selected 706 stations were used to study the spatial distribution within the surface-3,600 m layer. To compare CBH in a larger vertical extent (surface-7,600 m), 126 stations with CL31 ceilometers were used. The distributions of the selected 706 stations and 126 stations that have CL31 ceilometers are shown in Figure 3. For climatic considerations and land-sea impact on clouds, we divided the CONUS into four geographic regions as follows (Table 1 and Figure 3): West Coast, Arid West, Humid East, and East Coast regions. In coastal areas, CBH may be affected by both land and ocean, showing larger variability than over inland areas. Figure 2b shows that the correlation coefficient between monthly mean CBH simulated by NARR and observed by ASOS is generally lower and even negative in coastal area. We label an area as coastal if the station is within 100 km of a shoreline and has an elevation of less than 100 m above sea level according to Small and Nicholls (2003). Since CBH is highly correlated with relative humidity throughout the CONUS (An et al., 2017), the Arid West and Humid East are separated by the longitude of 100 W, according to Seager et al. (2018).

| RESULTS
3.1 | Evaluation of multi-year averages of CBF and CBH against observations from surface to 3,600 m As shown in Figure 4, the spatial distribution of CBF within the surface-3,600 m layer simulated by NARR is generally similar to that observed by ASOS, that is, F I G U R E 7 Seasonal mean CBH (m) simulated by NARR (a1-a4) and observed by ASOS (b1-b4), as well as the bias between NARR and ASOS (c1-c4) within surface-7,600 m over the CONUS during the period from 2008 to 2013. The four rows "1-4" indicate four seasons: MAM, JJA, SON, and DJF higher in the northwest coastal area and east of the CONUS but lower in the west of the CONUS, especially in the southwestern desert area. On average, CBF simulated by NARR is 3% higher than the ASOS observations for the 706 stations within the surface-3,600 m layer. NARR captures the features of seasonal variations of CBF within the surface-3,600 m layer. As shown in Figure 4a1-a4, the CBF simulated by NARR is higher in the east of the CONUS in JJA and higher along the Great Lakes area in DJF, consistent with observations ( Figure 4b1-b4), which are reasonable because that frequency of cumulus are generally higher the southeast of the CONUS in summer (Warren et al., 1986), and cold air passing over relative warm water surface causes an enhance formation of clouds in the Great Lakes area in winter . The bias of CBF between NARR and ASOS (NARR minus ASOS) is 2, 5, 0, and 4% in MAM, JJA, SON, and DJF, respectively. In JJA, the overestimation of CBF by NARR is more noticeable in the southeast of CONUS. In DJF, the overestimation of CBF by NARR is more noticeable in the northeast of CONUS around the Great Lakes area.
NARR also captures the spatial distribution of CBH within the surface-3,600 m layer (Figure 5a1-a4,b1-b4). Similar to that from the ASOS, CBH is generally higher in the west of the CONUS (except in the West Coast area where CBH is very low) and lower in the east of the CONUS, especially in the coastal areas along the Gulf of Mexico and the Great Lakes area. However, NARR yields much lower CBH values than the ASOS observations. On average, the CBH simulated by NARR is 501 m lower than the ASOS observations for the 706 stations within the surface-3,600 m layer. In the coastal area, the annual mean CBH simulated by NARR is generally lower than 700 m (which is rare in ASOS observations). The underestimation is larger in the east of the CONUS and smaller in the centre of the CONUS. Some stations in the centre of the CONUS present higher CBH simulated by NARR than ASOS observations. The annual mean underestimations for the West Coast, Arid West, Humid East, and East Coast regions within surface-3,600 m layer are −370, −383, −551, and − 670 m, respectively.
Model simulations and observations both show the same seasonal variability in CBH; namely, the CBH is generally higher in summer and lower in winter F I G U R E 8 Frequency of CBH at different levels from 0 to 500, 500 to 1,500, 1,500 to 2,500, 2,500 to 3,500, 3,500 to 4,500, 4,500 to 5,500, 5,500 to 6,500, and 6,500 to 7,500 m in four geographic regions (West Coast, Arid West, Humid East, and East Coast) in JJA (a-d) and DJF (e-h) during the period 2008-2013 for ASOS and NARR layer simulated by NARR are similar to those of the ASOS observations (Figure 6a1-a4, b1-b4). Unlike the results within the surface-3,600 m layer, in most part of the CONUS, CBF simulated by NARR is lower than ASOS observations. And the underestimation is most noticeable in the west of the CONUS (Figure 6c1-c4). Such differences in the geographic distributions of CBF and CBH between results within surface-3,600 m layer and results within surface-7,600 m layer, for example, some stations have shown underestimates of CBH in surface-3,600 layer but overestimates of CBH in surface-7,600 m layer, are due to the inhomogeneity in the vertical distributions of the cloud base. Figure 8 shows the frequency distributions of CBH at different vertical levels in JJA and DJF at the 126 stations. As shown, NARR simulated too many lowlevel cloud bases than ASOS especially in the Humid East, and East Coast regions. Over 1,500 m, the frequencies of  F I G U R E 9 Comparison of monthly mean CBF from the surface-7,600 m simulated by NARR and observed by ASOS in different seasons, times of day (JJA: 0000-0600; JJA: 1200-1800; DJF: 0000-0600; DJF: 1200-1800), over the four different geographic regions (West Coast, Arid West, Humid East, and East Coast) during the period from 2008 to 2013. The colour bar indicates the scatter density, which is defined as the number of data dots in 100 × 100 axis grids. The solid lines are the best fit lines calculated using least squares regression the cloud bases observed by ASOS surpass those simulated by NARR. In the 6,500-7,500 m level, the frequencies of the NARR-simulated cloud base are higher than those of ASOS in DJF in all four geographic regions. In the West Coast region in summer, NARR simulated low-level cloud bases are much fewer than ASOS observations (Figure 8a). A large portion of cloud bases were above 3,600 m, especially in JJA and in the Arid West region. The seasonal variations of CBF and CBH simulated by NARR (Figures 6a1-a4 and 7a1-a4) are generally consistent with those observed by ASOS within the surface-7,600 m layer (Figures 6b1-b4 and 7b1-b4). Model simulations and observations both show higher CBFs in the east of the CONUS in JJA and higher CBFs in the Great Lake area in DJF ( Figure 6). And they both show higher CBH in JJA and lower CBH in DJF (Figure 7). However, the seasonal variations of CBF and CBH within the surface-7,600 m layer are less distinct than those within the surface-3,600 m layer shown in Figures 4 and 5. 3.3 | Evaluation of monthly means CBF and CBH against observations from surface to 7,600 m Since the range surface-3,600 m cuts off a portion of valid cloud base information (Figure 8), the comparison between the monthly means and instantaneous cases was conducted only for the range of surface-7,600 m. Monthly mean CBF and CBH were not included in the comparison if the proportion of valid observation samples is less than 70% in the month. The density scatterplots and corresponding statistics of the monthly mean CBF from the surface-7,600 m for NARR and ASOS in different seasons, times of day, and geographic regions are shown in Figure 9 and Table 2. The correlation coefficients between the CBFs simulated by NARR and observed by ASOS vary in different time periods and geographic regions although they are all significant (Table 2). Correlation coefficients are generally higher, and the RMSE are also higher in the Arid West and West Coast regions than in the Humid East and East Coast regions. Figure 9 shows that the scatterplots are more discrete in the Arid West and West Coast regions. In the Humid East and East Coast regions, though the scatterplots aggregate at top right corners of the panels, the monthly CBFs simulated by NARR and observed by ASOS present weak linear relationships. The correlation coefficient is even negative in JJA: 0000-0600. Possible reason may be that NARR simulate too many low cloud bases (under 1,500 m) in these two regions (Figure 8). For a given region, the correlation coefficients are generally higher and the RMSE are generally lower in DJF: 0000-0600 than in other time period. The reason may be that stratus clouds predominate in DJF during 0000-0600 period and span a large horizontal extent that can be captured by both the model simulations and ceilometer observations. The scatterplots between monthly mean CBHs simulated by NARR and observed by ASOS in different seasons, times of day, and geographic regions are shown in Figure 10, and the corresponding statistics are listed in Table 3. The correlation coefficients are significant in the Arid West, Humid East, and East Coast regions (Table 3). In the West Coast region and in the time period of DJF: 0000-0600, the p value of Student's t test for the correlation coefficient is 0.3; thus, the correlation is not significant. The correlation coefficients are negative in the West Coast region in JJA: 0000-0600 and DJF: 0000-0600.
As shown in Figure 10, there are many cases when ASOS observed a very low monthly mean CBH, while NARR simulated a monthly mean CBH higher than 4 km; this discrepancy does not occur in other geographic regions. Coastal low cloudiness and fog are persistent seasonal features on the west coast of the United States, where the low stratus clouds generally occur in summer during May-September, while thick fog generally occurs in winter from September to March (Leipper, 1994;Clemesha et al., 2016). However, high CBHs in inland areas were observed due to the arid climate, and these inland clouds are discontinuous with the low cloudiness  F I G U R E 1 0 Comparison of monthly mean CBH from the surface-7,600 m simulated by NARR and observed by ASOS in different seasons, times of day (JJA: 0000-0600; JJA: 1200-1800; DJF: 0000-0600: DJF: 1200-1800), and over the four different geographic regions (West Coast, Arid West, Humid East, and East Coast) during the period from 2008 to 2013. The colour bar indicates the scatter density, which is defined as the number of data dots in 100 × 100 axis grids. The solid lines are the best fit lines calculated using least squares regression and fog of the coast (see figure 5 of An et al., 2017). The weak correlation between monthly mean CBH simulated by NARR and observed by ASOS in the West Coast region is primarily caused by the decoupling of clouds and surface conditions. The complicated landforms and the land and sea breezes make CBH difficult to predict. In addition, the resolution of the model limits its ability to resolve such local meteorology. Coastal low cloudiness and fog exhibits a strong diurnal cycle with maximum occurrence in the morning and minimum occurrence in the afternoon (Lundquist and Bourcy, 2000), thus larger discrepancies and even negative correlations between NARR and ASOS have been seen in the mornings (JJA: 0000-0600; DJF: 0000-0600) compared to the afternoons (JJA: 1200-1800; DJF: 1200-1800). The correlation coefficients are much higher, and the RMSEs and biases are generally smaller in the Arid West than in other regions, suggesting that NARR simulate CBH better in the Arid West region (Table 3). Except for the West Coast region, where have shown large discrepancies between CBH simulated by NARR and observed by ASOS, for a given region, monthly mean CBHs simulated by NARR generally correlate better and present lower RMSE with those observed by ASOS in JJA: 1200-1800. These results suggest NARR can more effectively simulate the CBH of cumuliform clouds at monthly scale.
3.4 | Evaluation of instantaneous CBH against observations from surface to 7,600 m Instantaneous samples of CBHs from the 3-hourly NARR outputs are evaluated using the matched 130-min average ASOS observations at the 126 stations. For instantaneous cases, the results of comparison are shown in Figure 11 and Table 4. Evaluation of instantaneous cases considered the Percent Correct (PC) based on a 2 × 2 contingency table that accounts for (a) the number of hits (i.e., both NARR and ASOS report valid CBH values); (b) false alarms (i.e., NARR simulates valid CBH while ASOS does not observed it); (c) misses (i.e., ASOS observed valid CBH while NARR does not simulate it) and (d) correct non-events (i.e., both NARR and ASOS agree there is no cloud base). The PC value is the percent of simulations that are correct, calculated as PC = (a + d)/ (a + b + c + d). As shown in Table 4, PC values are generally higher in the Humid East and East Coast regions than in the Arid West and West Coast regions, and higher in DJF than in JJA. Cloud cover is considered to be larger in humid area and in DJF, thus clouds are more likely to be simulated by NARR and observed by ceilometers simultaneously.
Pairs of CBH samples are compared when both NARR and ASOS report valid CBH values. Correlation T A B L E 3 The number of samples (N), correlation coefficient (CC), root-mean-square deviation (RMSE), and bias (NARR -ASOS) between monthly mean CBH simulated by NARR and observed by ASOS during the period from 2008 to 2013 in different seasons, times of day (JJA: 0000-0600; JJA: 1200-1800; DJF: 0000-0600; DJF: 1200-1800), and in the four different geographic regions (West Coast, Arid West, Humid East, and East Coast) coefficients between instantaneous CBH from NARR and ASOS are significant in different regions and time periods ranging from 0.2 to 0.66. There are large RMSEs in the West Coast region (up to 260%). Figure 11 shows that in the West Coast region, the observed CBH is very low while CBH simulated by NARR can be as high as 7 km. In general, the correlation coefficients are higher, and the RMSEs and biases are lower in the Arid West region. Meanwhile, the slopes for the best fit lines are more closed to 1. These results are consistent with the comparison of monthly mean CBH that NARR simulate CBH better in the Arid West region. In the Humid East and East Coast regions, it is shown that NARR frequently simulated CBH much lower than ASOS. Stripes of concentrated scatterplots occur near the X axis in all the time periods in the Humid East and East Coast regions ( Figure 11). This phenomenon is not obvious in the Arid West region. The frequently predicted low CBHs caused the relatively lower correlation between NARR and ASOS in the humid area. For instantaneous cases, the correlation coefficients present lower values in JJA than in DJF, which is different from the comparison of monthly CBH. Possible reason may be that, in summer, convective clouds with shorter lifetimes and smaller horizontal extents more often occur.

| DISCUSSION
This study shows that NARR predicts too low CBHs in the humid area in the east of the CONUS. In a study that evaluated the Eta model (Sengupta et al., 2004) between March 2000 and July 2002 within surface-3,000 m layer also showed that the cloud bases were much lower (on average 429 m) than observations at the ARM SGP site. This was also confirmed in other global and regional climate models and is considered to be related to the too shallow, cold, and moist boundary layers in the models (Barrett et al., 2009;Hannay et al., 2009). Possible reasons for the too low cloud bases predicted by NARR in humid area will be explored in the future.
Monthly mean CBH simulated by NARR correlates better with observations in JJA: 1200-1800, suggesting that CBH from convective scheme performs better than those from large-scale condensation scheme at monthly scale. For instantaneous cases, since convective clouds generally have a smaller horizontal range that could be hardly resolved by both model and observations, correlations coefficients are higher in DJF than JJA.
Limitations of ceilometers may affect the comparison between NARR and ASOS. Since ceilometer is a vertically pointing laser, there are times when ceilometer miss clouds in mostly clear or partly cloudy conditions, usually under conditions of widely scattered fair weather cumulus (NWS, 1998). However, model in a 32-km box will catch these cloud bases, thus resulting in an overestimation of CBF. The large overestimation (over 30%) of CBF in the southeast of the CONUS in JJA may be caused by such issue. In addition, a very sensitive laser may detect invisible moist layers as cloud base occasionally. Such different configurations between models and T A B L E 4 The number of samples (N), percent correct (PC), correlation coefficient (CC), root-mean-square deviation (RMSE), and bias (NARR -ASOS) between instantaneous CBH simulated by NARR and observed by ASOS during the period from 2008 to 2013 in different seasons, times of day (JJA: 0000-0600; JJA: 1200-1800; DJF: 0000-0600; DJF: 1200-1800), and in the four different geographic regions (West Coast, Arid West, Humid East, and East Coast). The number in the brackets are the p value of Student's t test, the normalization of RMSE and bias JJA: 0000-0600 JJA: 1200-1800 DJF: 0000-0600 DJF: 1200-1800 observations may have potential influences on the results. Other factors may impact on the comparison results of CBHs. For example, CBH simulated by NARR have a vertical resolution of 200 m. CBH from NARR are in geopotential height, which will cause an error of less than 10 m within the surface-7,600 m. Comparisons between the best estimated CBH from ARSCL and observations from a ceilometer at ARM SGP shows the ceilometer underestimated CBH for about 89 m compared to the best estimate of CBH within the surface-7,600 m layer in 2008-2010 ( Figure 1). All these factors may have an influence on the bias between CBH simulated by NARR and observed by ASOS. However, these factors are not likely the main causes for the underestimation of NARR CBH.

| CONCLUSIONS
Using observations from ceilometers at 706 ASOS stations during 2008-2013, the performance of the NARR simulations of CBF and CBH is evaluated. Results show that the geographic distributions and seasonal cycles of CBF and CBH were captured by NARR. On average, the CBF values of NARR were 7% lower and the CBH values of NARR were 631 m lower than those of the observations from the 126 selected ASOS stations from the surface to 7,600 m. The underestimation of CBF is more noticeable in the west of the CONUS while the underestimation of CBH is more noticeable in the east of the CONUS.
CBH simulated by NARR correlated better with observations in the Arid West region of the CONUS. The reasons for the less accurate NARR simulations of CBH in the West Coast, Humid East, and East Coast regions are different. In the West Coast regions, coastal low clouds from ocean and inland high clouds from the mountainous area coexist, making CBH difficult to predict. In the Humid East and East Coast regions, NARR frequently simulates cloud bases too low (under 500 m) compared to ASOS, causing large underestimations of CBH compared to ASOS. This phenomenon may be related to the boundary layer scheme in NARR.
This study uses a unique record of observations to evaluate the performance of CBH and CBF in NARR. The high spatial and temporal resolution of ASOS provide an opportunity to evaluate comprehensively NARR in different climatic regions and time periods. Different results have been seen for different climate regions, seasons, and times of day. Based on the findings of this study, future work will focus on obtaining estimates on how much the differences in CBH may affect the downwelling LW radiation reaching the surface.