(now subtitled) The Little Known Tropical Rain-forest of Ireland
From the latest two GHCN-M v3 status.txt files:
User feedback indicated a problem with some mean temperature data for select stations in Ireland. The problems were traced to a particular data source (MCDW), and for the time being until that source is corrected, the data are now being sourced to the UK Met Office “Climat” data (“K” source flag”), which are believed to be the correct values. The data changeover to the UK Met Office has occurred, but the source flag (“K”) for the corrected values was inadvertently left out. Those source flags should be added within the next production cycle.
“select” stations in Ireland meaning those stations for which GHCN-M v3 has continued to include data in recent years. Although how NOAA can be confident that the problem is confined to stations in Ireland without discovering the cause of these errors in MCDW (one of their own products) is something which escapes me.
Normally you might expect that some care would be taken to get the correction right. Not however here. They have however botched it again, leaving many rogue values unchanged. Some, but not all of these are flagged as probably erroneous and to be omitted from further analysis. Others however pass their quality control. And they have not reverted to the (correct) values which they had earlier shown as received from the UK Met Office.
One absurd record caught my eye. Cork Airport in 2013 has been corrected, but in 2014 still has six identical rogue values including July, August and December. Five of these are flagged, but August still slips through as the rogue value of 10.4°. This value is wrong, but not sufficiently outlying to be caught by their quality control. I asked myself how probable six identical monthly means would be. Not impossible for a tropical rain-forest climate zone I thought. Singapore for example has monthly means of daily mean temperatures lying throughout the year within a range of less than 2°C, but could not match the record six identical monthly means of Cork Airport. Examining the complete GHCN-M v3 raw data file I found 24 stations with six or more identical monthly means, all well within the tropics. So book your next holiday in the tropical rain-forest climatic zone of Cork. Just beware of the crocodiles, and be aware that no guarantee is given that Cork will match the temperatures of the other 24 stations.
Their quality control procedures allow for manual flagging of erroneous values not caught by their automated procedure. Somehow I think it would have been prudent to take more care having admitted their own MCDW values were wrong, and if necessary resort to manual flagging until the cause of this MCDW problem had been determined and corrected.
END OF UPDATE
Errors are not confined to 2013 to 2016, and not confined to this one station.
The longer (not necessarily full length record however) data can be found at Historical Data (I’ll return to add navigation advice here when I have completed other sections of this post. Navigation on this section of the Met Eireann site may not be intuitively obvious)
The GHCN-M version used above was ghcnm.tavg.v220.127.116.1161230.qcu.dat (which of course had not had a December 2016 value added, whereas Met Eireann calculates and shows a month-to-date mean, 7.4°C up to December 30th)
As shown below, the correct 2013 values were shown by GHCN-M for a time in 2013, corrupted for a time later in 2013, and briefly reappeared again in 2014, before settling down again as corrupted values.
Now follow the history of the April 2013 value (7.4°C according to Met Eireann). In the first GHCN-M file below (dated May 19th 2013) it is correctly recorded, and attributed to a CLIMAT report as source (740 P). This value is the most recent value to reach GHCN-M, and in this case the CLIMAT report has been correctly decoded. I will return to this question of correct or incorrect decoding of CLIMAT reports below).
By July 9th the still correct value has as data source “received by the UK Met Office” (740 K). This has been the usual change of data source, first CLIMAT report, then the UK Met Office. As seen on May 19th the March 2013 value had already been processed in this way (430 K).
On (or before) the 8th November the data source changed to “Monthly Climatic Data of the World (MCDW) QC completed but value is not yet published” (1040 WC). The value had now become the rogue value 10.4°C. the “W” quality flag indicates “monthly value is duplicated from the previous month, based upon regional and spatial criteria”. My experience of my region would suggest that duplicating the mean temperature of the previous month would very rarely produce a correct estimate for the following month. What “regional and spatial criteria” have required the replacement of a recorded monthly mean my a rogue value?
After that this rogue value has been retained, except for a brief return to the correct value and UK Met Office as source on (and possibly around) June 28th 2014. In mid 2015 the data source changed to “Final (Published) Monthly Climatic Data of the World (MCDW)”.
Each monthly value above is followed by either one or two letters. A single letter, or the second of two letters, gives the data source:
|C = Monthly Climatic Data of the World (MCDW) QC completed but value is not yet published|
|K = received by the UK Met Office|
|M = Final (Published) Monthly Climatic Data of the World (MCDW)|
|P = CLIMAT (Data transmitted over the GTS, not yet fully processed for the MCDW)|
|W = World Weather Records (WWR), 9th series 1991 through 2000|
The first letter of two letters is a quality control flag:
|S = monthly value has failed spatial consistency check. Any value found to be between 2.5 and 5.0 bi-weight standard deviations from the bi-weight mean, is more closely scrutinized by examining the 5 closest neighbors (not to exceed 500.0 km) and determine their associated distribution of respective z-scores. At least one of the neighbor stations must have a z score with the same sign as the target and its z-score must be greater than or equal to the z-score listed in column B, where column B is expressed as a function of the target z-score ranges (column A). See GHCN-M README for table.|
|W = monthly value is duplicated from the previous month, based upon regional and spatial criteria and is only applied from the year 2000 to the present.|
More detail to be added at some future date when other work permits. You can find the CLIMAT reports in the archive at www.ogimet.com (select “CLIMAT monthly summaries from the left hand menu, then visualization mode: By country or territory, Country or territory: Ireland, and the appropriate Year and Month).
As noted above, the April 2013 value for Cork Airport was initially decoded correctly from a CLIMAT report. But this correct decoding has not always been the case.
When Valentia Observatory (62103953000) became an AWS station in April 2012 the first April values entering GHCN-M from decoded CLIMAT reports were rogue values.