This post will be updated from time to time as further GHCN-M data is archived.
Last updated December 17th 2015 (plots brought up to date to December 10th)
Temperatures quoted by NOAA and others are “what would have been measured in the past by systems in use today”, so it is not unreasonable for the numbers to change frequently.
It does however seem unreasonable that these past numbers should change quite drastically overnight, then in a short time again change quite drastically overnight, then in another short time …
That green box shown early in 2015 corresponds to a sudden jump in the GHCN-M adjusted value for January 1978, from 5.78°C on January 5th 2015 to 6.41°C on January 6th 2015, followed by a similar drop, from 6.49°C on February 9th 2015 to 5.77°C on February 10th 2015. Note how the January 1978 temperature changes frequently in GHCN-M v3, even overnight, and that the changes may include correction for both “urban heating” and “urban cooling”, seen when the adjusted value become greater than or less than the unadjusted value [My understanding is that Pairwise Homogeneity Adjustment may also include adjustment for urban heating/cooling as well as instrument changes, relocation, TOBS correction, etc]. Anyone willing to suggest station relocations take place this frequently? Problems with Pairwise Homogeneity Adjustment seem more likely.
As adding values for the recently introduced GHCN-M v4 beta has made recent data points more difficult to distinguish above, here is a similar plot, but showing data only from the introduction of the current GHCN-M v3 version, v3.3.0 onward:
While v3.3.0 had fairly stable adjustments, these become more volatile with v4 beta. The unadjusted data is also slightly lowered in value.
This behaviour is not a peculiarity related to the January 1978 temperature. The other 1978 monthly temperatures behave similarly, as seen below (each month is shifted slightly down, in a different colour, for visibility – without this offset all would coincide)
This post examines the behaviour of GHCN-M adjustments for past temperatures at Marseille and nearby (in the climate data sense) stations, using saved GHCN-M data sets. The choice of Marseille arises from a blog post LE GISS ET LES SÉRIES LONGUES DE TEMPÉRATURES. I have seen similar behaviour closer to home with past data for Irish stations, but illustration using an Irish station would have restricted the choice of nearby stations to a generally easterly direction, while choice of Marseille provides nearby stations around the compass in France, Spain, Italy and Switzerland. A supplementary post with examples of stations in other countries with similar “volatile” adjustments can be found at GHCN-M: Stations similar to Marseille/Marignane.
It may help to visualise the changing adjustments by looking at the actual adjusted temperature records for a few dates:
and focusing on the recent years:
The dates chosen are January 5th and 9th 2015 (see green box above), and the two dates exhibiting the extremes of the range of adjustments under GHCN-M v3.x.x (or at least the extremes of the data I have archived). The unadjusted GHCN-M v3 record is also shown, as well as the trends (°C per decade) from 1980 and from 1966. I have also marked the range of adjusted values for 1978 and 1880.
There is no special reason for choosing the past temperature values for January 1978 – I simply took a year from the plotted temperature records for Marseille in that blog post, and used that year for other nearby stations as well. I chose January as the first month in each record, along with December easiest to locate when performing manual checks. In retrospect, a later month in the year might have been a better choice – working through stations I noticed some without January and/or December values. I’ll look at these again later more carefully. My guess now is that these may be stations not manned in winter. I have pointed out to the owners of that blog (the post itself did not provide an opportunity to add comments) that GISS are using the adjusted GHCN-M data as input, and that this, rather than the Gistemp processing, may be the source of the variations discussed in that blog post. Whether this choice of input is wise is another question. Recent values do not display this instability. January 2015 for example remains 7.60°C in GHCN-M adjusted files from February 10th, the date the value first became available, until now. It will be interesting to see whether version 3.0.0 has reduced this instability in past adjusted values or not. As these jumps seem to occur in the first part of the month, I will update here around mid-July. In both cases, the second day is the day an additional item of raw data becomes available. These additional items are not outliers for Marseille. On January 6th the value for December 2014, 8.90°C, arrives; on February 12th, the value for January 2015, 7.60°C. 8.90°C is 0.86 standard deviation units above the mean of December values, 7.60°C is 0.57 standard deviation units above the mean of January values. Relatively few values for other stations are added or changed on either of these dates. (discussion of these added or changed values coming) One noticeable feature of the adjusted data is that values are missing from May 1970 to March 1971, although values for these months are present in the unadjusted data. With GHCN-M v3.0.0 these values have quality control flag X (= pairwise algorithm removed the value because of too many inhomogeneities). This information was not shown for previous GHCN-M versions. The temperature values in the GHCN-M data sets are shown and discussed here, rather than anomalies. The changing base means which would be subtracted to calculate anomalies are tabulated below for the four dates discussed and for four base periods.
Abrupt overnight changes are not simply an artifact of one station, Marseille. (Gistemp values will be added to the images below. Only the first, Marseille, image shows these for now. The data for v3.0.0 which I discovered I had retained will also be added to all images) The period 1971-1980 is missing from the Salon record, which is not shown in the GISS list above. Toulon has stable adjusted values. Mont Ventoux has data from 1949 to 1968 only. Nimes/Courbes has stable adjusted values, Montpellier is missing data from 1898 to 2000. Nice has stable adjusted values. Mont Aigoual is the closest station displaying jumps in the Pairwise Homogeneity adjusted values for January 1978. (More comments coming here and below between station images)
Mont Aigoual shows different behaviour for v4 beta when compared with v3.3.0 − v3.3.0 remained unadjusted. v4 beta has a lower unadjusted value, is now adjusted, and these adjustments are moderately volatile.
Torino/Bric does not appear in the GISS list above as it is dropped during Gistemp processing. I have included it here as it shows a station which is generally unadjusted, but with two dates where an adjusted value departs from the unadjusted value. This would not seem unreasonable behaviour for an automated adjustment process – two “glitches” among more than 500 data sets examined here. There are also more than 160 dates on which an adjusted value for January 1978 is missing.
In v4 beta the January unadjusted temperature for January 1978 appears to have been adjusted down. Later months show smaller changes.
Genova Sestri appears to have been dropped in v4 beta, replaced by Genova University
Ajaccio also appears to have been dropped in v4 beta
Milano/Malpensa was unadjusted in v3.x.x, but is now adjusted in v4 beta.
Alghero is missing January 1978 values, so December 1978 values are shown below instead. v4 beta values are shown, but these are for Alghero Fertilia, which may not be the same station as in v3.x.x, although nearby.
Milano/Linate appears to have been dropped in v4 beta
and going further than the GISS list above, since Pairwise Homogenization uses more stations: GHCN-M: Stations near Marseille/Marignane but beyond the GISS list
For anyone wondering about the possible effects of metadata location errors, this post is concerned with GHCN Pairwise Homogeneity adjustments rather than Gistemp adjustments. There are location errors in the metadata for some of the stations shown above. A commenter called Harry at another blog wrote:
I found the computer code for the Pairwise Homogeneity Adjustment (PHA) algorithm they use. It is on the NOAA website
My response to that was:
The code on the NOAA website appears to be v3.0.0, not the code currently used. I was tempted to download and run this code to try to determine the cause of these erratic adjustments, but thought better of it in the absence of current code. Having downloaded and recoded the Gistemp code with additional diagnostic output, I am aware of the scale of such an undertaking. It may come as a surprise to Harry to find that some of us have “had the energy” to do this, and have contributed by notifying GISS of bugs found in their code – another good reason for making the code available. You can verify that I have done so by looking for my name at http://data.giss.nasa.gov/gistemp/updates/
So I cannot comment on any possible effect of location errors on PHA. Here are two of the location errors illustrated (green pushpin corrected, yellow pushpin GHCN-M metadata). In this case these two errors have minimal effect on Gistemp. Even though Marseille/Marignane by its very name confirms that it cannot previously have been located at the GHCN-M location, in the 4th or 5th arondissement of Marseille, the nightime luminance at both locations is clearly urban. Similarly, although it would literally require a mountain to be moved to relocate the Puy de Dôme station at the GHCN-M location, both locations are clearly rural (and for anyone still in doubt I can confirm that I have stood beside the Puy de Dôme station, GPS in hand, and found no evidence of such a move!)
Note that this post is based on GHCN-M v3.x.x, not the recent Karl et al paper. For anyone who may want to know who is involved with both, I’ve indicated common authorship below. Karl et al uses the same Pairwise Homogenization for land temperatures, but with many additional station records. GHCNM (version 3): J. H. Lawrimore, M. J. Menne, B. E. Gleason, C. N. Williams, D. B. Wuertz, R. S. Vose, and J. Rennie (2011), An overview of the Global Historical Climatology Network monthly mean temperature data set, version 3, J. Geophys. Res., 116, D19121, doi:10.1029/2011JD016187. Possible artifacts of data biases in the recent global surface warming hiatus Thomas R. Karl, Anthony Arguez, Boyin Huang, Jay H. Lawrimore, James R. McMahon, Matthew J. Menne, Thomas C. Peterson, Russell S. Vose, Huai-Min Zhang