Temperature, precipitation and sunshine across China, 1912‐51: A new daily instrumental dataset

This project compiled 463,530 instrumental observations of daily temperature, precipitation and sunshine at up to 319 stations in China between 1912 and 1951. The principal sources are monthly reports of the Institute of Meteorology, Nanjing, observatories in Japanese‐occupied Manchuria, and the Japanese Army in North China. Gross errors, rounding errors and inhomogeneities are identified. The new dataset is temporally and spatially consistent with existing datasets of monthly temperature, but reports higher precipitation and sunshine.


| SOURCES
Referring to Table 1, the new dataset was compiled from various historical sources. The largest source, accounting for over 40 per cent of the data, is the Monthly Meteorological Bulletin, published by the Institute of Meteorology of the National Research Institute (now called the Academia Sinica) at Nanjing, then capital of China.
The January 1929 issue published weather reports from 44 stations of the Chinese Maritime Customs Service (which established China's first nationwide meteorological network (Bickers, 2016)) and 10 other stations. The Bulletin published various information on a daily basis, including temperature (mean, minimum and maximum), barometric pressure, absolute and relative humidity, cloud cover, duration of sunshine, precipitation, and wind direction and force, with the coverage differing by station. The geographical coverage of the Bulletin expanded progressively to a peak of 89 stations in January-November 1936. The November 1936 issue is the last that this project has been able to locate. 1 Figure 1 (left) presents a cover of the Bulletin.
Until this project, partial collections of the Bulletin were publicly available, at the China National Library and various university libraries within mainland China, Central Weather Bureau, Taipei, Taiwan, and University of Pennsylvania library. To assist in data rescue, the National University of Singapore Libraries purchased and published an almost complete collection of the Bulletin (Chen et al., 2017).
The next largest source, accounting for around 11 per cent of the data, is the monthly report published by the Manchuria Central Observatory at present-day Changchun from 1935 until 1941, covering 55 stations across Manchuria. 2 Partial collections of Manchuria weather reports are dispersed across the Hiroshima University Library, Japan Center for Asian Historical Records, and the Japan Meteorological Agency.
The third largest source of data is monthly reports of the North China Meteorological Section of the Japanese Army (see Figure 1, right). 3 The first report this project has found reported daily weather in January 1938 at 4 stations across the present-day provinces of Inner Mongolia, Shanxi and Hebei. The hand-written document reports average pressure, temperature (mean, maximum and minimum), wind (average velocity and most frequent direction), humidity, cloud, precipitation and visibility. The issue for March 1944 is the last found. The U.S. Library of Congress holds the largest collections of the reports, with smaller holdings at the Japan Meteorological Agency and Central Weather Bureau, Taipei, Taiwan.
In total, the project compiled weather reports from 64 sources stored at 36 libraries and online archives. The copyright of these publications, if any, has expired. For the convenience of future researchers, the National University of Singapore Libraries published much of the corpus on its portal (URL https://doi.org/10.25541 /2nb3-3x47). 4 The records are listed by geographical region and chronologically within region, and the dataset is organized for search through a map.

| DATASET
This project aims to compile a dataset of temperature, precipitation and sunshine in China between 1912 and 1951, from historical records. 5 Historical observations that were not recorded according to modern standards and techniques require quality control and homogenization (Brázdil et al., 2010). Due to the quantity and heterogeneity of the data, these processes required considerable manual effort.
Generally, discrepancies and inhomogeneities arise for two reasons. One is peculiarities or changes in the observation procedure, personnel, instruments or monitoring station. The other is typographical errors in the original records or mistakes in the digitization process. This project applied various procedures to identify discrepancies and inhomogeneities, which were noted, with the corresponding observations 1 Apparently, the China Meteorological Administration archive holds issues of the Bulletin up to January 1937 (Zhongguo Qixiangju Qixiang Danganguan, 2003, p. 34), but the project was not able to access them. These start and end dates are based on the monthly reports found by this project. 3 Kobayashi and Yamamoto, 2013, This online corpus does not include the bulletins of the North China Meteorological Section of the Japanese Army, which the U.S. Library of Congress will scan and publish. 5 With the limited available resources, the project focused on temperature, precipitation and sunshine, leaving other parameters for future work. retained. Table 1 reports summary statistics of the errors and inhomogeneities.

| Gross errors
Two checks were applied to detect gross errors. One checked whether the daily maximum temperature exceeded the minimum. Referring to Table 2, 847 observations failed this condition and were tagged accordingly. The other check compared each observation in a series to the observation for the previous day. If the difference between the observations exceeded two standard deviations of the monthly series, the observation was tagged as a jump.
Referring to Table 2, the proportion of jumps ranged from 3 to 4 per cent in minimum and maximum temperature to about 10.5 per cent in precipitation, and about 13.5 per cent in sunshine hours.

| Rounding
Figure 2 (left) depicts the distributions of the first decimal of the daily minimum and maximum temperatures. Evidently, there are excessive zeros and fives. The main reason for excessive zeros and fives is rounding, due to either recording practices or conversion from Fahrenheit scale to Celsius and re-rounding. The outcome of re-rounding varies according to the precision of the Fahrenheit scale, whether 0.1°F, 1.0°F or 2.0°F. Rhines et al. (2015) propose a precision-decoding algorithm which models the unobserved precision states of each observation in a time series as latent variables, which are then inferred using a Hidden Markov Model. The algorithm decodes globally the time sequence and estimates the most probable states given all of the observations for each station. The original distribution of the first decimal is restored with the application of jitters by generating uniform random values consistent with the precision level and then adding to the sample before rounding to the first decimal.
The present temperature dataset does not seem to exhibit rounding from an original precision of 1.0°F. Such data would exhibit relatively few fives (Rhines et al., 2015), which contradicts the pattern of excessive fives depicted in Figure 2 (left). Nevertheless, for completeness, the Rhines et al. (2015) procedure was applied with 0.1°F, 1.0°F and 2.0°F precision.
Referring to Table 2, around 22 per cent of minimum and maximum temperatures were identified as being rounded. Figure 2 (right) depicts the distributions of the first decimal of the daily minimum and maximum temperatures after correction for rounding. Evidently, the first decimals are more evenly distributed, consistent with the original data being rounded. In principle, the Rhines et al. (2015) method could be applied to correct for possible rounding in records of precipitation and sunshine. However, their code is written to check records of temperature, specifically looking for conversion from Fahrenheit to Celsius. Accordingly, we did not check for rounding in precipitation or sunshine.

| Inhomogeneities
A standard procedure in climatology is to check time series for inhomogeneities that indicate undocumented change points, which are possibly due to changes in observation procedure, personnel, instruments or station. Figure 3 depicts the frequency of changes of observing organization and Table S1 reports the changes by station. Figure 4 depicts the frequency of movement of station as identified by a change in geo-coordinate or elevation in consecutive issues of the source. Table S2 reports the changes by station. On average, 8.0 per cent of stations changed observer and 6.0 per cent moved. For completeness, Table S3 lists stations whose names were changed, of which there were 13. Wang (2008aWang ( , 2008b proposes the penalized maximal F test (PMFT) to detect change points, or undocumented mean shifts that are not associated with any sudden change in the linear trend of a time series. Wang and Feng (2013) implemented the PMFT in the statistical software, R, and published the code as the package, RHtestsV4, to process a single time series. However, the present project involves four series (minimum and maximum temperatures, precipitation and sunshine hours) at up to 319 stations. To simplify the test for inhomogeneity, the RHtestsV4 code was adapted to process multiple time series.
In constructing a monthly dataset, Ashcroft et al. (2014) checked inhomogeneities in series of at least 3 years. The present data are daily, and for a similar degree of statistical power, the adapted RHtestsV4 code was applied to series of at least 60 consecutive days. The output of the adapted code was validated with the original RHtestsV4 package for minimum and maximum temperatures at Abaga.
Referring to Table 2, application of the PMFT method to the records of minimum temperature, maximum temperature, precipitation and sunshine yielded less than 0.05 per cent inhomogeneity. Apparently, inhomogeneities were very rare.

| VALIDATION
To validate the new dataset, it was compared with three existing datasets by temporal and spatial variation. Two are instrumental datasets (Tao et al., 1997), one of 65 stations (S65), 6 which includes monthly air temperature, precipitation and sunshine duration (SSD), and another of 205 stations (S205), which includes monthly mean temperature and total precipitation. The third dataset is the Climatic Research Unit Time Series (CRU TS) v. 4.03 (Harris et al., 2014), from which daily maximum and minimum temperatures and monthly precipitation were retrieved. S65 and S205 are listed in the World Meteorological Organization historical record, and so, may have been assimilated into the CRU TS. 6 The original dataset of 60 stations was augmented to 65 stations in 1996 by the Chinese Academy of Sciences. For completeness and to avoid obvious errors, the dataset subject to validation excluded records with gross errors in temperature (daily minimum temperature exceeding maximum), or sunshine hours less than 0.5 (21.4% of the total) owing to the large number of records with SSD = 0, or missing geo-coordinates.

F I G U R E
Referring to Figure 5a,b, both maximum (Tmax) and minimum (Tmin) temperatures in the new dataset correspond  However, referring to Figure 5c, the seasonal cycle of precipitation in the new dataset, while exhibiting a similar shape, seems to over-estimate except in July. Sunshine hours follow a similar seasonal cycle but range between 1.5 and 4 hr higher than S65 (Figure 5d). The disparity might be partly due to the S65 sunshine data being geographically smoothed. Next, the datasets were compared by region-northern, middle and southern China, and the Sichuan basin, which has a distinct climatology. Figure 6 depicts the daily anomaly temperature, defined as the difference of the daily mean of Tmax and Tmin and the monthly mean, over the climatology period from 1912 to 1951 in CRU TS and the new dataset. The higher frequency of data (indicated by the grey lines) between 1930 and 1945 likely suggests a higher reliability of analysis during this period. Apparently, the new dataset exhibits stronger monthly variation compared to the CRU TS, while the trajectories of the two present a certain degree of similarity over the period in all regions. Figure 7 depicts the spatial pattern of the datasets. To better compare with CRU TS, the daily mean temperatures of the new dataset and monthly temperatures of S65 F I G U R E 3 Observer changes. Observer change is recorded at the level of observing organization Number of observer changes 1 9 1 2 1 9 1 3 1 9 1 4 1 9 1 5 1 9 1 6 1 9 1 7 1 9 1 8 1 9 1 9 1 9 2 0 1 9 2 1 1 9 2 2 1 9 2 3 1 9 2 4 1 9 2 5 1 9 2 6 1 9 2 7 1 9 2 8 1 9 2 9 1 9 3 0 1 9 3 1 1 9 3 2 1 9 3 3 1 9 3 4 1 9 3 5 1 9 3 6 1 9 3 7 1 9 3 8 1 9 3 9 1 9 4 0 1 9 4 1 1 9 4 2 1 9 4 3 1 9 4 4 1 9 4 5 1 9 4 6 1 9 4 7 1 9 4 8 1 9 4 9 1 9 5 0 1 9 5 1 Year F I G U R E 4 Station movements. Movement refers to a recorded change in either latitude, longitude or elevation in consecutive issues of the source. Information on latitude, longitude and elevation is extracted from available digitized primary source files. Any record for which the primary source file does not include the information is excluded from comparison and S205 were interpolated to 0.5 × 0.5 degree grids. The spatial temperature distribution displays strong agreement among the datasets, although the high similarity of the new dataset with S65 and S205 is due in part to interpolation. Nevertheless, the general pattern is similar, with higher temperatures in the south and lower in the north, and the Sichuan Basin being warmer. Figure 8 presents dot plots of annual precipitation and sunshine hours. Precipitation in northern China is clearly higher than in the CRU TS and S65 and S205 datasets. Sunshine hours are higher overall than S65.
Overall, the new dataset is similar to the S65, S205 and CRU TS datasets in temporal and spatial characteristics of temperature, which suggests its reliability. Yet, the discrepancies between the new dataset and the S65, S205 and CRU TS datasets in precipitation and sunshine suggest that these data should be used with caution.

| DISCUSSION
While the past few decades have seen considerable progress in our scientific understanding and assessment of climate change, numerical models of climate simulation and reanalysis still heavily rely on high-resolution quantitative climate data for multiple purposes including training models, cross-check and verification (Brönnimann et al., 2018). Analysis of extreme events, and their behaviour and impact, F I G U R E 5 Averaged seasonal cycle over all records in (a) Tmax, unit: °C, (b) Tmin, unit: °C, (c) precipitation, unit: mm/ month, and (d) sunshine, unit: hour/day. Black solid line represents CRU TS, blue and green lines represent S65 and S205 (Tao et al., 1997), and red dashed line represents the new dataset to support future climate risk analysis depends on the availability of more historical data. This motivates further work on historical weather and climate data rescue projects, which have gained recent attention through substantive international efforts (Allan et al., 2011). Data rescue mainly refers to recovery of non-digitized records which pre-date the era of electronic recording. Data of the late nineteenth century and early twentieth century are the most integral for extending our existing instrumental data series to improve and extend reanalysis for a better understanding of past climate variability and change.
Although some long and important terrestrial data series, particularly covering Europe, have been digitized and processed for academic and practical uses, work on a large proportion remains outstanding (Brönnimann et al., 2018). Data rescue in East Asia started at least a decade ago with Japan, and the associated work for mainland China is proceeding in collaboration with ACRE (Atmospheric Circulation Reconstructions over the Earth) and the wider CSSP (Climate Science for Service Partnership) (Williamson et al., 2018).
Contributing to these efforts, this project constructed a dataset of daily temperature, precipitation and sunshine in China between 1912 and 1951. The dataset of 463,530 daily instrumental records of up to 319 stations adds substantially to work by others such as the China Meteorological Administration (Williamson et al., 2017). Figure 9 depicts the geographical coverage of the dataset at its peak in the year 1936, covering 145 stations across 29 present-day provinces of China. As Figure 10 illustrates, the temporal coverage of the dataset varies by year while the functional coverage is primarily temperature with relatively less data on precipitation and sunshine.
To facilitate further research, a package of the raw data, code in Stata, R, and MATLAB, and processed datasets in CSV and Stata format will be made available through the Zenodo 7 and Scholarbank@NUS, named as 'NUS Republican China Weather Database'. All of the historical records have been scanned and compiled into a repository, made accessible through an intuitive online interface. pressure, wind and humidity. Also, future work can digitize other Nationalist era weather records in the China Meteorological Administration archive (Zhongguo Qixiangju Qixiang Danganguan, 2003). While large, the dataset presented here is limited in three important ways. The major limitation is the restricted temporal coverage as the Chinese meteorological network took time to develop and was then severely disrupted by the Japanese invasion and even more so, the Civil War (Fei, 2018). The second limitation is variation in the functional coverage as many sources reported only maximum and minimum temperatures. The third limitation is errors in typesetting and digitization, biases in recording, rounding and inhomogeneities, owing to which the data should be interpreted with caution.
Despite the limitations, the new dataset is valuable in several ways. First, compared to other instrumental datasets such F I G U R E 9 Geographical coverage in 1936, the peak year of the Monthly Meteorological Bulletin published by the Institute of Meteorology: 138 stations across 29 present-day provinces of China. Indicative mainland present-day China boundary (source: GADM.org)