Journal list menu

Volume 141, Issue 689 p. 1294-1305
Research Article
Open Access

Assessing the quality of active–passive satellite retrievals using broad-band radiances

Howard W. Barker

Corresponding Author

Howard W. Barker

Environment Canada, Toronto, Ontario, Canada

Correspondence to: H. Barker, Cloud Physics and Severe Weather Research Section, Environment Canada, 4905 Dufferin St., Toronto, Ontario M3H 5T4, Canada. E-mail: [email protected]Search for more papers by this author
Jason N. S. Cole

Jason N. S. Cole

Environment Canada, Toronto, Ontario, Canada

Search for more papers by this author
Carlos Domenech

Carlos Domenech

Institute for Space Sciences, Free University of Berlin, Germany

Search for more papers by this author
Mark W. Shephard

Mark W. Shephard

Environment Canada, Toronto, Ontario, Canada

Search for more papers by this author
Christopher E. Sioris

Christopher E. Sioris

Centre for Research in Earth and Space Science, York University, Toronto, Ontario, Canada

Search for more papers by this author
Florian Tornow

Florian Tornow

Institute for Space Sciences, Free University of Berlin, Germany

Search for more papers by this author
Tobias Wehr

Tobias Wehr

European Space Agency, Noordwijk, the Netherlands

Search for more papers by this author
First published: 21 August 2014
Citations: 7
Reproduced with the permission of the Minister of Environment Canada
[The copyright line in this article was changed on 17 April 2015 after original online publication.]


Included in the Earth, Clouds, Aerosols, and Radiation Explorer (EarthCARE) satellite's array of instruments is a multi-view broad-band radiometer (BBR). BBR data will facilitate a radiative closure assessment of cloud and aerosol properties inferred from data gathered by EarthCARE's other passive and active sensors. The closure assessment will consist, in part, of comparisons between BBR radiances and radiances computed by three-dimensional (3D) radiative transfer models (RTM) that act on narrow 3D domains that derive from, and include, the retrieved cross-section of cloud and aerosol properties. Assessment domains D will be ∼100 km2. Following a brief outline of the closure experiment, a method is proposed for estimating the likelihood of BBR radiances providing a meaningful closure assessment of cloud and aerosol properties in D. The method capitalizes on the ability of Monte Carlo RTMs to compute contributions to radiances from any constituent in any given D. While this methodology introduces some circularity into the closure test, it might, nevertheless, be tolerable, given that the method's purpose is simply to identify, and thus avoid, assessments that are likely to be fruitless or misleading. A 3000 km long stretch of A-Train satellite data was used in this initial demonstration of the proposed methodology. Only results for solar radiation are shown. All radiative quantities used here were computed by a 3D Monte Carlo RTM. A control simulation provided proxy BBR measurements. Random ‘errors’ were introduced into the A-Train field to produce experimental fields that roughly mimic retrievals. Experimental and control radiances were compared in mock-closure assessments. Arbitrarily assuming that a fruitful assessment requires ∼75% of a BBR radiance to result from cloud and aerosol scattering events inside D, ∼70% of the (11 km)2 domains were flagged as reliably testable for this example.

1 Introduction

The Earth, Clouds, Aerosols, and Radiation Explorer (EarthCARE) satellite mission is a collaboration between the European Space Agency (ESA) and Japan Aerospace Exploration Agency (JAXA) and is expected to be launched no sooner than the end of 2016 (ESA, 2001). It will make synergistic use of data from its active radar and lidar and passive multi-spectral imager (MSI) to retrieve profiles of cloud and aerosol properties for each ∼(1 km)2 nadir column. As stated formally (ESA, 2001, 2006), EarthCARE's ultimate goal is to retrieve cloud and aerosol properties well enough that, when acted upon by suitable broad-band (BB) radiative transfer models, estimated top-of-atmosphere (TOA) fluxes for most domains with areas ∼100 km2 will be within ±10 W m−2 of fluxes inferred from radiances measured by its multi-view BB radiometer (BBR). The BBR will measure nadir and two along-track oblique views with viewing zenith angles of ∼50–55° for a common swath of width ∼10 km (Wallace et al., 2009).

A notable difference between EarthCARE and its A-Train progenitors—Aqua, CloudSat and CALIPSO (Wielicki et al., 1996; Stephens et al., 2002; Winker et al., 2003)—is that, from its inception, BBR data that are not to be used for retrievals were intended to form the basis of a radiative closure experiment (cf. Oreopoulos et al., 2012), the aim of which is to assess the verisimilitude of retrieved cloud and aerosol properties (ESA, 2001). Basically, 3D BB solar and thermal radiative transfer models will be applied to 3D atmospheres constructed around the retrieved cross-section (see Barker et al., 2011, 2014), with estimated radiances and TOA fluxes compared with the corresponding BBR radiances and inferred TOA fluxes (e.g. Loeb et al., 2005; Domenech and Wehr, 2011). With the aid of an error budget for measured and modelled quantities, the probability of measured and modelled values differing by less than 10 W m−2 will be determined for each ∼100 km2 assessment domain D. For domains this size, closure assessment faces several complications, some of which are addressed in this study.

It could be argued that closure assessment should progress through comparisons of the three BB radiances first and then on to TOA fluxes, the latter being computed by Earth Radiation Budget style angular distribution models (Domenech and Wehr, 2011). A fundamental issue, however, when using spectrally integrated radiances, either directly or integrated to fluxes, to assess the retrieved characteristics of particular constituents in any particular D involves the amount of pertinent information in the radiances. For instance, if an oblique BBR's lines of sight passed through D and yet the magnitudes of measured radiances were effectively insensitive to significant fluctuations of variables in D, those radiances should not be used to assess the quality of retrievals in D. The purpose of this study is to present a method for deciding what BBR radiances to use for EarthCARE's closure assessments. The method is based on estimation of the likelihood that measured radiances consist of sufficiently large proportions of photons that underwent interactions with constituents of concern inside D.

The following section discusses the nature of some issues that are faced when using BBR radiances for the purpose of radiative closure assessment. Section 3 presents a method that attempts to identify which BBR radiances to use for closure purposes, so as to maximize the integrity and usefulness of EarthCARE's closure assessment experiment. The data and models used are presented in section 4, along with a description of the experiments. Results are discussed in section 5 and conclusions are provided in the final section.

2 Radiative closure assessment for EarthCARE

The first part of this section provides a brief overview of EarthCARE's planned radiative closure process. The second part describes the problem of identifying useful radiances to perform closure assessments. The context into which closure experiments fit is discussed by A. J. Illingworth, 2014; personal communication.

2.1 Overview of EarthCARE's radiative closure assessment

Each EarthCARE level 2 (L2) data file will contain retrieved profiles of cloud and aerosol properties, profiles of atmospheric state variables and surface optical properties on a Joint Standard Grid (JSG) of ∼1 km2 pixels for along-track segments of ∼5000 km. EarthCARE's radiative closure experiment aims to assess how well measured BBR radiances and fluxes agree with their modelled counterparts; the latter are computed by applying 3D radiative transfer models to 3D domains constructed around the L2 cross-section by Barker et al.'s (2011) scene construction algorithm, which, in turn, uses MSI narrow-band radiances. Hence, modelled BB radiances will be defined on the JSG for frames measuring ∼30 × 5000 km2. Measured and modelled radiances will then be averaged over assessment domains D, each of which will cover ∼100 km2 and be centred on the L2 cross-section.

Figure 1 shows a simplified flowchart of EarthCARE's proposed closure assessment, which begins with measured BBR nadir I0, fore-viewing I+ and back-viewing I radiances and their uncertainties (i.e. standard deviations) urn:x-wiley:00359009:media:qj2438:qj2438-math-0144 on the JSG, along with their modelled counterparts urn:x-wiley:qj:media:qj2438:qj2438-math-0001 and urn:x-wiley:qj:media:qj2438:qj2438-math-0002. Define an assessment domain D as a sequence of contiguous columns having an across-track width of Δy, stretching along-track from column urn:x-wiley:qj:media:qj2438:qj2438-math-0003 to urn:x-wiley:qj:media:qj2438:qj2438-math-0004 and extending from the surface to altitude hD (see Figure 2(a)). The magnitude of hD should exceed the altitude of most clouds and aerosols and can vary from domain to domain. For this study, hDwas defined as the altitude of the highest cloud in D. Next, decide if radiances are available to perform a trustworthy assessment of retrieved cloud and aerosol properties in D. This last step is the main point of this study and is discussed at length in section 3.

Details are in the caption following the image
Flowchart showing the basic structure of the radiative closure process as planned for EarthCARE. This study is concerned primarily with the boxes with bold outlining. Details pertaining to production of data represented by the shaded boxes can be found in A. J. Illingworth, 2014; personal communication.
Details are in the caption following the image
(a) Schematic diagram showing how an assessment domain D is defined using nadir radiances I0 (between JSG columns urn:x-wiley:qj:media:qj2438:qj2438-math-0014 and urn:x-wiley:qj:media:qj2438:qj2438-math-0015. It also shows the ranges of oblique radiances I± proposed for use in the closure assessment. zD signifies the top of D and zM is the top of the modelling domain (satellite altitude). The image is a CloudSatCALIPSO cloud-mask, where white is for ice and grey for liquid clouds. (b) Six regions in which scattering (or emission) events can occur and result in photons being received by the BBR whilst viewing in the backward direction. See the text for a discussion of these regions.
If radiances exist that are likely to provide a trustworthy assessment, JSG values are averaged over D, giving mean measured and modelled radiances of L±,0 and urn:x-wiley:qj:media:qj2438:qj2438-math-0005 with uncertainties denoted, respectively, as urn:x-wiley:00359009:media:qj2438:qj2438-math-0145 and urn:x-wiley:qj:media:qj2438:qj2438-math-0006. Assuming that these quantities define Gaussian distributions of possible measured and modelled radiances, their difference is also Gaussian and the probability that they differ by less than ±ΔL is
and urn:x-wiley:qj:media:qj2438:qj2438-math-0009 is a pooled variance based on urn:x-wiley:qj:media:qj2438:qj2438-math-0010 and urn:x-wiley:qj:media:qj2438:qj2438-math-0011. In practice, Eq. 1 can be evaluated as

where erf(⋯) is the error function.

The value urn:x-wiley:00359009:media:qj2438:qj2438-math-0146 provides a quantitative indication of the likelihood that the L2 retrievals in D have satisfied EarthCARE's general goal. To date, EarthCARE's closure activities have been discussed exclusively in terms of TOA fluxes in W m−2. To remain consistent when using radiances to perform closure assessments, simply multiply the above-mentioned radiances by π (cf. Figure 1).

The current plan is for closure assessments to progress through increasingly demanding tests from nadir radiance to oblique radiances to upwelling flux at zD. For reasons explained in the following subsection, not all BB radiances will provide trustworthy and meaningful closure tests. The method presented in section 3 offers an automated means of establishing, and hence avoiding, such situations.

2.2 BBR radiances and closure assessment

As shown in Figure 2(a), when attempting to assess retrievals in D, the most relevant I±,0 are likely to consist of photons that arrived along lines of sight that passed through D. As mentioned above and hereinafter, I±,0 and their modelled counterparts urn:x-wiley:qj:media:qj2438:qj2438-math-0013 are integrated across-track to the width of D. It is proposed that for the closure assessment all oblique radiances with lines of sight through D be used, as opposed to just those with lines of sight through the top of D. This is because the purpose of the closure assessment is to gauge the quality of retrievals through the entirety of D, not just near its top.

Considering, for example, back-viewing radiances, Figure 1(b) shows that photons making up I last underwent a scattering, or emission, event in one of six distinct regions. Photons that experience an event in region 1 and travel toward the radiometer must traverse part of region 1 and (the broad-band-integrated, near-transparent) region 6 to be received by the radiometer. In other words, the radiometer has a clear view of D along these lines and so these radiances are often likely to carry much information about the constituents of concern in D. However, these constituents could reside in the lower reaches of region 1 and have much extinction immediately aloft, due to extraneous constituents also in region 1, and so these radiances might at times carry little applicable information. Events in region 2 are similar, but photons emerging from it have the added deterrent of having to traverse regions 5 and 6, which are beyond D. Note that the thin high cloud in region 6 was not actually present; it was inserted into this figure to illustrate that the top of region 1 need not necessarily extend across the entire top of D.

Photons arising in region 3 (surface + atmosphere) have less immediate connection to D, but they do have to traverse region 1 before gaining easy access to the radiometer. Thus, these contributions to applicable radiances are influenced by attenuation in D, of which only a portion of that attenuation need be due to constituents of concern to the closure assessment. Photons from region 4 are like those from 3, except that they have to cross regions 2 and 5 and thus will generally carry less importance in the assessment process than those from region 3. Photons arising from events in regions 3 and 4 can be thought of as back-lighting of D. If their optical properties can be included accurately in the radiative transfer models, then there should be little concern about their contributions to radiances used to assess D. Failing this, however, it is unfortunate that they are inextricably mixed in with contributions from regions 1 and 2.

Measured photons stemming from events in region 5 can be expected to carry relatively little information about the state of constituents of concern in regions 1 and 2. The worst case is a very dense region 5 that effectively shrouds regions 1 and 2, thereby rendering most oblique radiances, with lines of sight through D, nugatory for assessment of retrievals in D.

To be sure, some information about relevant constituents in D will be translated via photons scattered horizontally from regions 1 and 2 that encounter attenuators in regions 3, 4 and 5 and subsequently result in contributions to measured radiances (cf. Davis et al., 1997; Várnai and Marshak, 2003; Gabriel et al., 2008). While such events lead to radiances outside of, but close to, urn:x-wiley:qj:media:qj2438:qj2438-math-0016 being influenced by attenuators in D, they are neglected here, given that, for EarthCARE, geometric distances from urn:x-wiley:qj:media:qj2438:qj2438-math-0017 to urn:x-wiley:qj:media:qj2438:qj2438-math-0018 will range from ∼12 to 35 km.

For nadir radiances, regions 2, 4 and 5 vanish, region 3 collapses to the surface beneath D and virtually all relevant contributions to radiances stem from events in region 1. Hence, through this dissection of radiances, it would appear that nadir radiances are the most relevant and least ambiguous for closure assessment of constituents in D. Unfortunately, however, BB nadir radiances are often determined much by, and hence correlated very well with, narrow-band nadir (MSI) radiances (Barker et al., 2014), which can play important roles in active-passive retrieval of cloud and aerosol properties (Delanoë and Hogan, 2010). This reduces the ability of nadir BB radiances to provide completely independent closure assessments.

To illustrate these points, if low-level liquid clouds in D are overlayed by dense upper-level clouds, it is likely that I±, and possibly I0 too, should be excluded from assessments of liquid clouds in D. If, however, the upper clouds are optically thin, some I±,0 could have large contributions from liquid clouds in D and thus facilitate high-quality assessments, provided the liquid clouds were sufficiently dense and the surface not excessively reflective. In general, the most appropriate solution (i.e. choice of which radiances to use) will lie between these extremes. The problem is how to identify which I±,0 facilitate the most useful assessments when only portions of them are relevant.

The underlying assumption here has been that photon events that occur in D (regions 1 and 2) potentially carry the most information about the attenuating characteristics of D and thus, if their contributions to radiances could be isolated, those contributions should form the basis of the closure assessment. However, this is not possible with spectrally integrated measurements. Moreover, back-lighting of D by regions 3 and 4, or fore-lighting by region 5, can be expected to often dominate radiances that have lines of sight through D, thereby rendering near-fruitless their use for assessment of retrievals in D.

It is important to note that the above-mentioned assumption is not always best. For instance, if absorbing aerosol is the only constituent being assessed, it is likely that their greatest impact on BB radiances will be via attenuation of back-lighting. However, if one has low confidence in optical properties used for regions 3 and 4, it would seem that there is little hope for a credible assessment of retrieved aerosol properties in D.

Unlike passive-only retrievals, EarthCARE will have 3D fields of cloud and aerosol (A. J. Illingworth, 2014; personal communication). While the purpose of the closure experiment is to assess the integrity of these very fields, it is proposed, in the following section, that BB radiances computed by radiative transfer models applied to these fields be used to estimate whether measured I±,0 are likely to contain enough suitable information to allow for confident assessment of retrieved cloud and aerosol properties in D. Admittedly, this sounds like gold-bricking, for what is being proposed is that the variables to be assessed be used to help guide their own assessment. However, because there will be literally millions of potential assessments over the mission's life and because the proposed method aims only to identify and avoid potentially meaningless assessments, the violation may be tolerable.

3 Closure screening: fractional radiances

Drawing on the discussion in the previous section, this section introduces the idea of fractional radiances. It explains how to compute them with a Monte Carlo model and how they might be used to help avoid potentially misleading assessments of retrieved quantities. The Appendix outlines an extension to fractional fluxes.

As discussed above, urn:x-wiley:qj:media:qj2438:qj2438-math-0019 (and I±,0) consist of photons with trajectories that ranged from having much to nothing to do with constituents of concern in urn:x-wiley:qj:media:qj2438:qj2438-math-0020 (the model counterpart of D). As such, only a fraction of any given urn:x-wiley:qj:media:qj2438:qj2438-math-0021 (and I±,0) is useful for closure analysis. While one could begin to parse useful contributions using spectral radiances, it is hypothesized that, by applying Monte Carlo radiative transfer models to retrieved atmospheres, relative contributions to urn:x-wiley:qj:media:qj2438:qj2438-math-0022 arising from photons that were scattered, emitted, or reflected by relevant attenuators within urn:x-wiley:qj:media:qj2438:qj2438-math-0023 can be estimated and used to decide when I±,0 are likely to contain adequate amounts of information to furnish meaningful closure assessments.

For this study, only short-wave BB radiances were considered. Here, as in EarthCARE's official algorithm, the local estimation Monte Carlo method (Marchuk et al., 1980; Evans and Marshak, 2005) was used to compute short-wave radiances and fluxes. The idea of fractional radiances is, therefore, presented in terms of local estimation terminology. Extension to backward Monte Carlo techniques, as used to compute long-wave BB radiances for EarthCARE (see A. J. Illingworth, 2014; personal communication), is fairly straightforward and is discussed in a separate study.

Begin by letting ζji,k(n) be the nth contribution to radiance at the top of the ith column arising from a scattering event involving the kth constituent in the jth column at altitude z. Thus,
where pk(θ,ϕ;θ′,ϕ′,z) is the kth constituent's phase function, which describes the probability of radiation travelling in direction (θ′,ϕ′) being scattered at altitude z(j) in the jth column, with single-scattering albedo of ω0(n), into direction (θ,ϕ) toward the sensor at the top of the ith column at altitude zM(i),w(n) is the weight the photon cluster had upon arriving at z(j), and β(z) is extinction due to all constituents along the path from z(j) to zM(i). An expression similar to Eq. 4 describes contributions from surface reflection. As in section 2.1, horizontal dimensions of columns are fixed at Δy across track, with variable Δx(i) along track. Summing over all contributions, the simulated radiance at zM(i) is
where Ncell is the number of photon clusters injected on to the top of each column. Implicit in Eq. 5 is spectral integration, meaning that terms in Eq. 4 are actually spectrum-dependent. BBR measured counterparts to Eq. 5 are denoted as I±,0(i).
The portion of urn:x-wiley:qj:media:qj2438:qj2438-math-0026 arising from photons that underwent scattering events inside urn:x-wiley:qj:media:qj2438:qj2438-math-0027 is
The portion of urn:x-wiley:qj:media:qj2438:qj2438-math-0030 arising from photons that were scattered by specific retrieved constituents within urn:x-wiley:qj:media:qj2438:qj2438-math-0031 is
in which {k′} denotes the set of constituents being assessed. For EarthCARE, there are five possible constituents: cloud ice crystals; cloud droplets; aerosol particles; air molecules; and Earth surfaces. If so desired, these categories could be subdivided into size and habit classes. Within a Monte Carlo algorithm, it is straightforward to keep track of contributions from any number of specific constituents to radiances in any number of viewing directions (cf. Barker et al., 1998, 2003; Iwabuchi, 2006). In fact, one could go as far as to keep track of contributions to a radiance from all constituents in all cells (e.g. Faure et al., 2001; Várnai and Marshak, 2003), but only a small subset of that information is required here.
Summing urn:x-wiley:qj:media:qj2438:qj2438-math-0034 over all radiances with lines of sight that pass through urn:x-wiley:qj:media:qj2438:qj2438-math-0035 gives
with urn:x-wiley:qj:media:qj2438:qj2438-math-0037 and urn:x-wiley:qj:media:qj2438:qj2438-math-0038 denoting the beginning and end indices for radiances that span urn:x-wiley:qj:media:qj2438:qj2438-math-0039 when viewed from one of EarthCARE's BBR directions (see Figure 2(a)). The total radiance counterpart to Eq. 10 is simply
with its measured counterpart denoted as L±,0.
Now, define fractional radiance as
which is the ratio of the contribution from constituents {k′} in urn:x-wiley:qj:media:qj2438:qj2438-math-0042 to total radiance for all radiances with lines of sight that pass through urn:x-wiley:qj:media:qj2438:qj2438-math-0043. For nadir view, urn:x-wiley:qj:media:qj2438:qj2438-math-0044, where ɛ represents the (typically) very small contribution from region 6. For oblique views, the behaviour of urn:x-wiley:qj:media:qj2438:qj2438-math-0045 is more complicated. For instance, if urn:x-wiley:qj:media:qj2438:qj2438-math-0046 is filled with relevant attenuators with optical depth approaching urn:x-wiley:qj:media:qj2438:qj2438-math-0047, the surface is dark and regions 5 and 6 are almost transparent, urn:x-wiley:qj:media:qj2438:qj2438-math-0048. If for the same situation, however, the relevant attenuators form a plane-parallel layer,
where urn:x-wiley:qj:media:qj2438:qj2438-math-0050 is the ratio of along-track geometric length of urn:x-wiley:qj:media:qj2438:qj2438-math-0051 to total along-track length of all radiances used to define averaged oblique radiances used above (e.g. urn:x-wiley:qj:media:qj2438:qj2438-math-0052 is along-track length of the ith pixel and θv is viewing zenith angle. To harmonize these ratios across θv, define
with urn:x-wiley:qj:media:qj2438:qj2438-math-0054 for nadir view. Subsequent discussions of fractional radiance pertain to urn:x-wiley:qj:media:qj2438:qj2438-math-0055.

One can now set a critical ratio, <1, such that when urn:x-wiley:qj:media:qj2438:qj2438-math-0056 assessment might best be avoided on account of it being too likely that the signal in L±,0 arising from constituents {k′} in D is, for whatever reason, too small to provide a reliable closure test. Moreover, as the objective of EarthCARE's closure process is to assess the quality of cloud and aerosol retrievals, hereinafter, unless stated otherwise, urn:x-wiley:qj:media:qj2438:qj2438-math-0057 and urn:x-wiley:qj:media:qj2438:qj2438-math-0058 represent collective contributions from crystals, droplets and aerosols over [urn:x-wiley:qj:media:qj2438:qj2438-math-0059] and below domain-specific urn:x-wiley:qj:media:qj2438:qj2438-math-0060.

Since retrieval of underlying surface properties has always been outside EarthCARE's purview, their impacts on measured radiances should, ideally, be avoided by cloud and aerosol closure exercises. It is straightforward for the Monte Carlo model to accumulate, in a single simulation, both urn:x-wiley:qj:media:qj2438:qj2438-math-0061 and a second set of contributions to radiances that excludes both direct contributions from surface reflection and contributions from photons scattered by cloud and aerosol after their trajectories have encountered the surface; that is, the black-surface counterpart to a regular simulation. Since these additional sums require very little computation, one obtains regular radiances, partial radiances and their black-surface counterparts for the price of approximately a single regular simulation.

4 Data, models and simulations

This section discusses the data, radiative transfer model and simulations used to demonstrate the idea of employing partial radiances in the context of EarthCARE closure assessment.

4.1 A-Train satellite data

A 3000 km long sample from the merged A-Train dataset of Kato et al. (2010) was used to demonstrate this method. Assembly of this product is explained briefly here. For details see Kato et al. (2010) and Barker et al. (2011).

The merged A-Train dataset consists of CloudSat and CALIPSO retrieved profiles mapped to 1 km resolution and associated with MODIS pixels. MODIS imagery extends on both sides of the profiles for ∼40 km. A merged cloud-mask was created by interpolating CloudSat and CALIPSO cloud-masks on to the CERES vertical grid (Kato et al., 2005). For each column, effective visible optical depth τe, effective particle size re and particle phase near the cloud top were retrieved based on inversion of a 1D radiative transfer model (Minnis et al., 2010a). If CloudSat's columnar classification (Sassen and Wang, 2008) was cirrus, altostratus, altocumulus or deep convection, those cells in the column identified as cloud with temperatures < 273 K were classed as ice and rewas set uniformly to the MODIS-inferred value (Minnis et al., 2008, 2010a, 2010b). Warmer cells in the column were set to liquid with re=10 μm. When the columnar classification was stratus, stratocumulus, cumulus or nimbostratus, the column was assumed to consist of liquid only with re set uniformly to the MODIS-inferred value.

Based on the cloud optical property parametrization used for radiation calculations in this study (see section 4.2) and re as defined above, values of cloud water content were assigned to each cloudy cell in the merged cloud-mask field such that total cloud depth equalled τe.

The 3000 km segment used here was derived from measurements made on 2 January 2007 over the Tropical Western Pacific (TWP) starting at 15.00°S, 123.23°E at 0537 UTC and ending at 10.00°N, 117.86°E at 0544 UTC (see A. J. Illingworth, 2014; personal communication). The retrieved cross-section was expanded to a ∼31 km wide swath using MODIS imagery and Barker et al.'s (2011) algorithm.

4.2 Monte Carlo radiative transfer model

Broad-band solar radiances were calculated by a 3D Monte Carlo algorithm that employs cyclic horizontal boundary conditions (Barker et al., 1998, 2003). Gaseous transmittances (H2O, CO2, O3) were computed using the correlated k-distribution method with 31 quadrature points in cumulative probability space (Li and Barker, 2005; von Salzen et al., 2013). Optical properties for liquid droplets (Wiscombe, 1980) and ice crystals (Yang et al., 2013) were resolved into four bands. Rayleigh scattering and attenuation by aerosols were also included. Two types of surfaces were used: Cox and Munk's (1956) distribution of wave slopes coupled with Fresnel reflection; and Lambertian (Barker and Davies, 1992). For more details, see Barker et al. (2012).

4.3 Simulations

Four simulations were performed and these are described in this subsection. The computation grid was 31 × 3400 km2, but only their inner 11 × 3000 km2 assessment grid was analyzed (i.e. Δy= 11 km for definition of urn:x-wiley:qj:media:qj2438:qj2438-math-0063 and urn:x-wiley:qj:media:qj2438:qj2438-math-0064. The outer portion of the computation grid buffered the assessment grid from unwanted effects affected by cyclic horizontal boundary conditions. Continental tropospheric aerosol (Deepak and Gerber, 1983) blanketed the domain with optical depth 0.15 at wavelength 0.55 μm and extinction decreased exponentially with altitude above the surface at a scale height of 1 km.

4.3.1 Experiment 0: Control

This simulation used the A-Train constructed atmosphere with an underlying ocean surface defined by Cox and Munk (1956) forced by local surface winds (Kato et al., 2010). The computation grid received 450 × 106 simulated photons, comparable to that expected for EarthCARE, implying 4.27 × 103photons per 1 km column or 5.16 × 105 photons per 121 km2 assessment domain urn:x-wiley:qj:media:qj2438:qj2438-math-0065. Given the variance reduction techniques employed, which are described in a separate study, this resulted in negligible Monte Carlo errors for radiances and partial radiances once averaged over urn:x-wiley:qj:media:qj2438:qj2438-math-0066. BB radiances computed by this simulation were meant to represent BBR measurements.

4.3.2 Experiment I: Cloud perturbation

This was the same as the control simulation, except that cloud properties were invested with random fluctuations in a simplistic attempt to mimic retrieval errors. For every 106 incident photons, the control cloud field was perturbed with unbiased and uncorrelated Gaussian noise added to layer values of cloud water content and particle size with relative standard deviations of ∼30% at the bases of deep clouds and decreasing with altitude to almost noiseless by 15 km (e.g. Ebell et al., 2010; Protat et al., 2010).

While this representation of retrieval error or uncertainty is almost a pasquinade, it is only meant to demonstrate the case of very accurate, but not perfect, cloud retrieval against the backdrop of having perfectly set boundary conditions, such as surface reflection. Thus, radiance differences between this and the control are meaningful to closure assessments of cloud retrievals.

4.3.3 Experiment II: Surface perturbation

This is the same as the control, except that the surface is a Lambertian reflector with spectrally independent albedo αs of 0.06. This value corresponds to the 90% quantile of the distribution of surface albedos in the control simulation (the median of which was ∼0.038). As such, this case represents perfect cloud retrieval with an erroneous description of the lower boundary condition. Thus, radiance differences between this and the control are meaningless to closure assessments of clouds.

4.3.4 Experiment III: Cloud + surface perturbation

This experiment combines the perturbations described in experiments I and II.

5 Results

This section consists of two parts. The first demonstrates the behaviour of fractional radiances urn:x-wiley:qj:media:qj2438:qj2438-math-0067 using data from the control simulation. The second shows the results of closure assessments between the control and the experiments and highlights how urn:x-wiley:00359009:media:qj2438:qj2438-math-0148 and urn:x-wiley:qj:media:qj2438:qj2438-math-0068, as defined in Eqs 1 and 14, can be used together. In all cases, assessment domains D were (11 km)2 and centred on the nadir cross-section. Each D overlapped its neighbour in the along-track direction by 10 km, thus yielding 2999 assessment domains along the 3000 km frame.

5.1 Characteristics of urn:x-wiley:qj:media:qj2438:qj2438-math-0069

Figure 3 shows urn:x-wiley:qj:media:qj2438:qj2438-math-0070 pertaining to contributions from clouds and aerosols, including contributions that have had interactions with the surface, for a 300 km stretch over the TWP that contained a wide variety of cloud forms. Note first that, while urn:x-wiley:qj:media:qj2438:qj2438-math-0071 are typically ∼1, their uncertainties due to Monte Carlo noise σ are just ∼0.001. This is primarily due to the large number of photons per D, but the high correlation between numerator and denominator in Eq. 12 helps too. Indeed, using
with typical values for this simulation of urn:x-wiley:qj:media:qj2438:qj2438-math-0073 and correlation coefficient urn:x-wiley:qj:media:qj2438:qj2438-math-0074, the expected urn:x-wiley:qj:media:qj2438:qj2438-math-0075 is ∼0.0012.
Details are in the caption following the image
The middle panel (b) shows a CloudSat–CALIPSO cloud-mask for a 300 km stretch of atmosphere over the TWP in which white is for ice and grey for liquid. This image has the correct aspect ratio. The top panel (a) shows the corresponding mean optical depths (including clear-sky fraction, which was usually 0) for each (11 km)2 assessment domain. The left axis of the lower plot (c) is for the control simulation's fractional radiances urn:x-wiley:qj:media:qj2438:qj2438-math-0062 for three BBR views as indicated. The right axis is for their uncertainties due to Monte Carlo noise. Shaded areas and locations marked with a letter are discussed in the text. Over this domain, solar zenith angles ranged from 37°–39.5° and the Sun shone in from the southwest direction.

The marked locations in Figure 3 highlight the behaviour of urn:x-wiley:qj:media:qj2438:qj2438-math-0076. They are addressed in alphabetical order. The range marked by A had thick, extensive convective cloud and so all urn:x-wiley:qj:media:qj2438:qj2438-math-0077, indicating that most photons that contributed to radiances with lines of sight that passed through an assessment domain D came from scattering by clouds within D. Hence, assessments for these domains can be carried out confidently. However, as the cloud mask in Figure 3 shows, the assessment will be overwhelmingly of upper-level ice cloud, because liquid clouds, while fairly thick, lie beneath ice clouds with optical depths typically between 20 and 40. At B, however, urn:x-wiley:qj:media:qj2438:qj2438-math-0078 are all ∼0.5, indicating that radiance lines of sight through these D had many contributions from outside D. This is because all clouds in these D were very tenuous; but also, with respect to urn:x-wiley:qj:media:qj2438:qj2438-math-0079, upper-level clouds in columns adjacent to D (i.e. region 5 in Figure 2(b)) were not negligible, as seen in the upper plot of Figure 3, and thus effectively blocked oblique views of low-level clouds in D. It should be pointed out that ɛ < 0.01 (see Eq. 13) across the entire domain.

The large values of urn:x-wiley:qj:media:qj2438:qj2438-math-0080 at C near distance 2395 km stem from fairly strong signals from low-level liquid clouds, but also from the irradiated sides of ice clouds that were detected easily through the modest upper-level clouds near 2375 km. Note that corresponding values of urn:x-wiley:qj:media:qj2438:qj2438-math-0081 at C are small. This is because the back views of the dense low clouds near C were blocked much by quite dense mid-level ice clouds between 2400 and 2415 km. Values of urn:x-wiley:qj:media:qj2438:qj2438-math-0082 exceed 1 when the signal coming from the cloud (edge) is much larger than the signal coming from the rest of the radiances with lines of sight through D. Arguably, cases with urn:x-wiley:qj:media:qj2438:qj2438-math-0083 should provide the most solid closure assessments.

At D the situation is the reverse of that at C. Back-view radiances have strong signals from mid-level clouds easily seen through the thin upper-level clouds located near 2500 km (as evident from the diminishing of both urn:x-wiley:qj:media:qj2438:qj2438-math-0084 and cloud optical depth beyond ∼2500 km). Meanwhile, the fore views offer little to assessments of domains near D, due to effective shielding by thick extensive clouds in the A region coupled with thinning clouds near D. For the sample section shown in Figure 3 there are several locations, other than C and D, where urn:x-wiley:qj:media:qj2438:qj2438-math-0085 for reasons similar to those just explained.

Figure 4 indicates that for all views through this section, which represents the entire 3000 km domain quite well, the signal received from liquid clouds only rivals that from ice clouds in a very small percentage of cases. This is simply because liquid clouds, the local optical depths of which often exceed those for ice clouds, occur beneath fairly thick ice clouds (see Figure 3). The signal from aerosol is generally very weak despite dark ocean beneath. Figure 5 shows frequency distributions of urn:x-wiley:qj:media:qj2438:qj2438-math-0086 for aerosols, liquid clouds and ice clouds over the entire 3000 km domain. While virtually all urn:x-wiley:qj:media:qj2438:qj2438-math-0087 for liquid clouds are< 0.5, over 50% of urn:x-wiley:qj:media:qj2438:qj2438-math-0088 for ice clouds exceed 0.8. Viewed this way, it is clear that for this frame it will be very difficult to say much, if anything, about the quality of aerosol and liquid cloud retrievals, especially with oblique radiances. Rather, the assessment will be relevant almost entirely to upper-level ice cloud retrievals.

Details are in the caption following the image
(a)–(c) Ice, liquid and aerosol components of fractional radiances (and their sums, which are labelled as total) for three BBR views, as listed at the top of each plot, for the same section of atmosphere as used to produce Figure 3. The middle plot's total curve is a re-plot of that shown in Figure 3. Note that the vertical axis in (b) spans half that of the other panels.
Details are in the caption following the image
Cumulative frequency distributions of ice, liquid and aerosol components of fractional radiances for three BBR views for a 3000 km long atmosphere inferred from A-Train data, of which only 300 km worth were shown in Figures 3 and 4.

5.2 Using urn:x-wiley:qj:media:qj2438:qj2438-math-0089 in closure assessments

Consider now how urn:x-wiley:qj:media:qj2438:qj2438-math-0090 and urn:x-wiley:00359009:media:qj2438:qj2438-math-0148 can be brought together in a closure assessment. Closure tests for (11 km)2 assessment domains for the 3000 km A-Train field, as used in the previous subsection, were performed using simulated radiances multiplied by π so as to resemble fluxes in magnitude. This is because EarthCARE's goal is expressed formally in W m−2. Hence, the assessment statistic computed by Eq. 1 is now denoted as urn:x-wiley:00359009:media:qj2438:qj2438-math-0148.

Figure 6 shows ‘flux’ differences πΔL along with pooled standard deviations σp for the inner 2500 km of the domain. It also shows urn:x-wiley:qj:media:qj2438:qj2438-math-0091 and urn:x-wiley:00359009:media:qj2438:qj2438-math-0149 using πΔL=10 W m−2. Values of urn:x-wiley:qj:media:qj2438:qj2438-math-0092 were actually unrealistically small. This is because they were determined only by Monte Carlo noise and not the full range of input uncertainties. Hence, 100 times fewer photons were assumed for the experimental simulations; the controlurn:x-wiley:qj:media:qj2438:qj2438-math-0093, better thought of as urn:x-wiley:00359009:media:qj2438:qj2438-math-0150, were left alone (i.e. small), thereby mimicking the anticipated small uncertainties in measured radiances. This enhanced experimentalurn:x-wiley:qj:media:qj2438:qj2438-math-0094 and thus σp by a factor of roughly 10 and made for more interesting and perhaps more realistic assessments. Had this adjustment not been applied, the majority of urn:x-wiley:00359009:media:qj2438:qj2438-math-0148 would have been very close to either 0 or 1.*

Details are in the caption following the image
The top panel (a) shows the CloudSat–CALIPSO cloud-mask for liquid and ice clouds for a 2500 km stretch of atmosphere inferred from the same A-Train data as used all along. Below it is mean cloud optical depth (including clear-sky fraction). The left axes of the two lower plots (b) and (c) are for π·nadir radiance (‘flux’) differences, πΔL, between an experiment, as listed on top of each plot, and the control, as well as corresponding pooled standard deviations σp (see text for details). The right axes are for both nadir fractional radiances urn:x-wiley:qj:media:qj2438:qj2438-math-0101 for the control simulation and the probability urn:x-wiley:00359009:media:qj2438:qj2438-math-0151 that |πΔL| is smaller than πΔL=10 W m−2. Over this domain, solar zenith angles ranged from 26.5° at 200 km to 41.5° at 2700 km. The Sun shone in from almost due west at 200 km and gradually swung to the southwest at 2700 km.

Like Figure 3, key features in Figure 6 are marked with a letter and discussed in alphabetical order. At A, both liquid and ice clouds were dense and overcast and so urn:x-wiley:qj:media:qj2438:qj2438-math-0095 are large. However, experiment I exhibits much random error, due to random perturbations in cloud properties, with πΔL often exceeding 10 W m−2, and so with relatively large σp the chances of |πΔL|<πΔL=10 W m−2 are quite small, as seen by urn:x-wiley:00359009:media:qj2438:qj2438-math-0151 generally being less than ∼0.3. Conversely, as experiment II had perfect cloud retrievals, with errors in αs being largely obscured by clouds, πΔL are usually smaller than 4 W m−2. However, since Monte Carlo errors are artificially large, urn:x-wiley:00359009:media:qj2438:qj2438-math-0151 are typically near 0.75; in other words, it is still not obvious whether the goal has been reached despite small πΔL. There are several regions on the right side of the plot that resemble conditions at A (with results near 2450 km being visible in Figures 3 and 4).

B is one of the only locations where dense low clouds dominated radiances; liquid and ice optical depths were ∼10 and ∼3, respectively. Values of urn:x-wiley:qj:media:qj2438:qj2438-math-0096 and urn:x-wiley:qj:media:qj2438:qj2438-math-0097 for liquid cloud contributions only (not shown) reach 0.36 and 0.84, respectively and for a small number of assessment domains exceed the corresponding urn:x-wiley:qj:media:qj2438:qj2438-math-0098 for ice clouds. Values of urn:x-wiley:qj:media:qj2438:qj2438-math-0099 are small on account of thick high cloud uptrack at distance 390 km. The overall situation does not differ much from that at A: excellent viewing of clouds, but with fairly large σp the chances of reaching the goal of |πΔL|<πΔL=10 W m−2 are low. What the results at this location do suggest, however, is the potential benefit of EarthCARE reporting urn:x-wiley:qj:media:qj2438:qj2438-math-0100 for all constituents, thereby allowing users to identify reliable assessments of constituents that they are interested in.

At C and D clouds were thin and contributed weakly to radiances. Indeed, with urn:x-wiley:qj:media:qj2438:qj2438-math-0102 well below 0.5, this alerts one that assessments with BB radiances for this stretch are not going to reveal much about the quality of cloud and aerosol retrievals. So, with no surface albedo errors in experiment I, πΔL are very small with urn:x-wiley:00359009:media:qj2438:qj2438-math-0151≈1, but this excellent closure result is due to αs having been set perfectly—it says little about the quality of cloud retrievals. On the other hand, experiment II has perfect cloud retrieval with poor specification of αs and so πΔL are near 20 W m−2 and, rightly, urn:x-wiley:00359009:media:qj2438:qj2438-math-0151≈0.

Figure 7 shows a scatter plot of urn:x-wiley:qj:media:qj2438:qj2438-math-0103 versus urn:x-wiley:00359009:media:qj2438:qj2438-math-0151 for the three experiments. It is clear from this that urn:x-wiley:00359009:media:qj2438:qj2438-math-0151 behaves quite differently depending on whether errors arise from cloud retrievals or specification of αs. When errors are with clouds (experiment I), there is, for obvious reasons, a marked tendency to have small probabilities of agreement as views of clouds improve. Conversely, when errors are with αs(experiment II), the odds of good agreement increase as views of clouds improve. When errors are allowed in both clouds and αs(experiment III), however, there is almost a complete elimination of cases with urn:x-wiley:00359009:media:qj2438:qj2438-math-0151>0.75 and many instances of good views of clouds with quite poor agreement with the control. Results for the two oblique views are similar (see Table 1) but, relative to nadir-view, they tend to have more occurrences of good views (urn:x-wiley:qj:media:qj2438:qj2438-math-0104 with good performance (urn:x-wiley:00359009:media:qj2438:qj2438-math-0148>0.5) and fewer occurrences of poor views with poor performance.

Details are in the caption following the image
(a), (b) Scatter plots of fractional nadir radiances urn:x-wiley:qj:media:qj2438:qj2438-math-0105 versus assessment statistic urn:x-wiley:00359009:media:qj2438:qj2438-math-0151 for comparisons of the control experiment and experiments I, II and III as listed on the top of the plots. The data used to produce these plots were the same as those used to produce Figures 5 and 6 as well as Table 1. Assuming, for example, that satisfactory views of clouds have urn:x-wiley:qj:media:qj2438:qj2438-math-0106 and that satisfactory closure performances have urn:x-wiley:00359009:media:qj2438:qj2438-math-0151>0.5, results that fall in the shaded region can be flagged as retrievals, averaged to the scale of D, that achieved EarthCARE's overall goal, subject to estimates of measurement, retrieval and computational uncertainties.
Table 1. Fraction of (11 km)2 assessment domains for the scene shown in Figure 6 that had good (urn:x-wiley:qj:media:qj2438:qj2438-math-0107 and poor (urn:x-wiley:qj:media:qj2438:qj2438-math-0108 viewing of clouds and aerosols for forward, nadir and backward viewing, with corresponding high (urn:x-wiley:00359009:media:qj2438:qj2438-math-0148>0.5) and low (urn:x-wiley:00359009:media:qj2438:qj2438-math-0148≤0.5) probabilities of radiances, scaled by π to resemble fluxes, being less than πΔL<10 W m−2. Results are for comparisons between the control simulation and experiment I (cloud perturbation), experiment II (surface perturbation) and experiment III (cloud + surface perturbation)
Forward Nadir Backward
urn:x-wiley:00359009:media:qj2438:qj2438-math-0151>0.5 urn:x-wiley:00359009:media:qj2438:qj2438-math-0151≤0.5 urn:x-wiley:00359009:media:qj2438:qj2438-math-0151>0.5 urn:x-wiley:00359009:media:qj2438:qj2438-math-0151≤0.5 urn:x-wiley:00359009:media:qj2438:qj2438-math-0151>0.5 urn:x-wiley:00359009:media:qj2438:qj2438-math-0151≤0.5
Experiment I—control
urn:x-wiley:qj:media:qj2438:qj2438-math-0109 0.67 0.05 0.60 0.14 0.67 0.06
urn:x-wiley:qj:media:qj2438:qj2438-math-0110 0.25 0.02 0.24 0.02 0.24 0.03
Experiment II—control
urn:x-wiley:qj:media:qj2438:qj2438-math-0111 0.61 0.10 0.55 0.14 0.65 0.07
urn:x-wiley:qj:media:qj2438:qj2438-math-0112 0.07 0.22 0.00 0.31 0.11 0.17
Experiment III—control
urn:x-wiley:qj:media:qj2438:qj2438-math-0113 0.40 0.31 0.27 0.43 0.45 0.27
urn:x-wiley:qj:media:qj2438:qj2438-math-0114 0.03 0.27 0.00 0.31 0.05 0.23

For the case considered here, if one were to limit oneself to performing assessments of clouds and aerosols when urn:x-wiley:qj:media:qj2438:qj2438-math-0115, about 70% of the (11 km)2 assessment domains would be admitted (see Table 1). Since ∼99% of the assessment domains had some cloud in them, there would still be over 2000 cloud closure assessments performed. Conversely, the EarthCARE mission could assess all domains, report urn:x-wiley:qj:media:qj2438:qj2438-math-0116, and tabulate performance statistics as functions of .

6 Conclusion and discussion

The overall goal of the EarthCARE satellite mission is to retrieve profiles of cloud and aerosol properties well enough that, when acted upon by BB radiative transfer models, estimated TOA fluxes for each ∼100 km2 domain D will be within 10 W m−2 of fluxes inferred from measurements made by its three-view BBR (ESA, 2001, 2006). Attaining this requires tackling many technical and scientific challenges, including development of a sound radiative closure experiment, to assess when, where and under what conditions the goal is met. While many details of the closure process remain to be worked out (A. J. Illingworth, 2014; personal communication), the basic plan was explained briefly in section 2.1. The main point of this exploratory study, however, was to present a new technique that attempts to quantity, for each D, the likelihood of BBR data furnishing a meaningful assessment of specific retrieved quantities. Ideally this technique will improve the usefulness of EarthCARE's closure exercise.

For the closure assessment, 3D Monte Carlo radiative transfer models act on 3D scenes constructed around the retrieval cross-section (Barker et al., 2011) and yield BB radiances. Both modelled and measured BBR radiances are then averaged over D. Using estimates of their uncertainties and assuming that their underlying distributions are Gaussian, the probability urn:x-wiley:00359009:media:qj2438:qj2438-math-0151 that their difference πΔL is <πΔL is computed and reported, as opposed to a simple binary answer to the question ‘is |πΔL|<πΔL?’. Details regarding estimation of radiances and uncertainties for measured and modelled radiances will be discussed, along with additional EarthCARE radiative products, in a separate report.

As discussed throughout the text, it might often be that comparing measured with modelled BB radiances actually makes for rather weak closure assessments of specific retrieved constituents in D. This is because radiances might be governed only weakly by the attenuators being assessed. By using BB radiances to assess the quality of retrieved cloud and aerosol properties within D, one is banking on those radiances carrying adequate amounts of information about the character of attenuators in D. In actuality, however, photons that contribute to a radiance measurement have underlying spectrally dependent conditional distributions of the number of scattering events with any number of atmospheric constituents, number of encounters with various underlying surfaces and path lengths through absorbing gases.

Presented like this the task sounds daunting, especially as the size of D decreases and radiances come to depend increasingly on conditions outside D. EarthCARE's anticipated areas of D are ∼100 km2 and so difficulties, most notably for short-wave radiation, are expected. Moreover, EarthCARE's BBR radiances are irreducible BB integrals that lack the flexibility of high spectral resolution radiances, such as those from O2 A-band spectrometers (e.g. Stephens et al., 2005).

Therefore, it was hypothesized here that by applying Monte Carlo radiative transfer models to retrieved atmospheres, fractional contributions to BB radiances from photons that are scattered, emitted or reflected by relevant attenuators in D can be computed and used to estimate the trustworthiness of closure assessments. Basically, when the fractional contribution to a radiance arising from particular retrieved constituents in D, denoted here as urn:x-wiley:qj:media:qj2438:qj2438-math-0117 (see Eqs 12-14), is small, for whatever reason, it is likely that a comparison of the relevant measured and modelled BB radiances will be a poor indicator of the quality of the constituents in D; vice versa for large urn:x-wiley:qj:media:qj2438:qj2438-math-0118. However, this proposal should raise concerns, because it requires retrieved fields to be used within their own closure assessment. This circularity might be tolerable, given that urn:x-wiley:qj:media:qj2438:qj2438-math-0119 are intended to be used only to alert users of closure assessments that might best be avoided. The hope is that the benefits outweigh any detriments.

The closure procedure and its use of urn:x-wiley:qj:media:qj2438:qj2438-math-0120 were demonstrated using A-Train satellite data and a 3D solar radiative transfer model similar to the one to be used for EarthCARE. While the tests were highly idealized and not indicative of EarthCARE's ultimate performance, they served the purpose of illustrating how urn:x-wiley:qj:media:qj2438:qj2438-math-0121 could be used as an aid in the proposed closure process. In defining urn:x-wiley:qj:media:qj2438:qj2438-math-0122 for oblique views, it was proposed that all radiances passing through D be used. This acknowledges that the aim of the closure experiment is to assess, as much as possible, the quality of the entire retrieved column.

For cases such as clouds over snow or ice, urn:x-wiley:qj:media:qj2438:qj2438-math-0123 will often be small. Hence, the methodology presented here will tend to suggest that assessments under these conditions be avoided on account of the potential dominance of potentially uncertain background boundary conditions, which are formally outside EarthCARE's purview, on TOA radiances. If, however, one has confidence in model boundary conditions, warnings arising from urn:x-wiley:qj:media:qj2438:qj2438-math-0124 could be taken less seriously. The same goes for the case when (relatively) bright clouds constitute the background to D.

Very small values of urn:x-wiley:qj:media:qj2438:qj2438-math-0125 can result from D lacking constituents of interest (e.g. being almost cloud-free). In some of these cases, other data will yield essentially the same information as urn:x-wiley:qj:media:qj2438:qj2438-math-0126. At other times, such as for liquid cloud beneath ice cloud when liquid cloud is the target of the assessment, it might not be so straightforward. Nevertheless, if one simply avoids closure tests when the relevant urn:x-wiley:qj:media:qj2438:qj2438-math-0127 are small, it makes little difference whether the test is being avoided because nothing of interest was present, D was obscured by clouds outside D or backlighting was overly dominant. The important point is that the trustworthiness of the assessment is suspect and that EarthCARE's instrumental configuration cannot pass judgement on the retrievals, notwithstanding the possibility that the retrievals might be excellent.

The highly idealized experiments performed here were intended to demonstrate EarthCARE's radiative closure assessment process and a technique that is being considered to estimate the trustworthiness of assessments performed on individual assessment domains. As EarthCARE's retrieval algorithms mature, it will be possible, indeed imperative, to perform full end-to-end simulations of the entire processing chain. Those simulations will provide a much clearer indication of the utility of the technique proposed here.


This study was supported by a contract issued by the European Space Agency, under the EarthCARE component of its Living Planet Programme, to the Free University of Berlin, which was subsequently subcontracted to Environment Canada and Atmospheric and Climate Applications, Inc. Monte Carlo simulations were performed on the facilities of the Shared Hierarchical Academic Research Computing Network (SHARCNET:

A. Appendix A: Extension to fractional fluxes

The idea of a signal from specific constituents that reside in a specific region could be extended to fluxes too by defining a fractional flux for the kth constituent's contribution as
where fractional radiances urn:x-wiley:qj:media:qj2438:qj2438-math-0129, as defined in Eq. 10, have been averaged for each viewing direction. For all intents and purposes, this is an angular distribution model (ADM) with anisotropy factors that are buried in urn:x-wiley:qj:media:qj2438:qj2438-math-0130. However, unlike a conventional ADM which seeks to estimate actual flux at some level using weighted TOA radiance(s), urn:x-wiley:qj:media:qj2438:qj2438-math-0131 is best thought of as a quadrature approximation of an immeasurable flux-like signal received at TOA that stems from select attenuators that reside in a volume urn:x-wiley:qj:media:qj2438:qj2438-math-0132 of the Earth–atmosphere system. Clearly, urn:x-wiley:qj:media:qj2438:qj2438-math-0133 serves only for something like a closure assessment, as it is meaningless from an energetic perspective.
Like Eq. 12, one could define
and urn:x-wiley:qj:media:qj2438:qj2438-math-0136 are sums of radiances with lines of sight that intersect urn:x-wiley:00359009:media:qj2438:qj2438-math-0147. As with Eqs 12 or 14, if f<f it could be assumed that there is likely too little ‘flux’ arising from the constituents of concern in urn:x-wiley:00359009:media:qj2438:qj2438-math-0147to warrant an assessment between urn:x-wiley:qj:media:qj2438:qj2438-math-0137 and its measured counterpart, the latter computed by Eq. A3 but replacing urn:x-wiley:qj:media:qj2438:qj2438-math-0138 with L±,0 (sums of measured radiances across appropriate columns).
Alternatively, as urn:x-wiley:qj:media:qj2438:qj2438-math-0139 come from the model, their measured counterparts, using mean measured BBR radiances, could be defined as
Clearly, when modelled and measured radiances are equal, urn:x-wiley:qj:media:qj2438:qj2438-math-0141, as defined in Eq. A1, is recovered from Eq. A4. One could envisage, therefore, a closure test that compares urn:x-wiley:qj:media:qj2438:qj2438-math-0142 with urn:x-wiley:qj:media:qj2438:qj2438-math-0143 (either individually or summed over some k).

  • * To reiterate, the point of these tests was to demonstrate the proposed methodology. They were not intended to be indicative of EarthCARE's ultimate performance level.