Valentia, Met Éireann and NOAA

Placeholder for now. Discussion, explanations and further details following.

Image settings have been updated so that images can be opened enlarged in a new tab or window.

There will be a short (I hope) pause while I investigate the stations used by GHCN to adjust Valentia. Complicated by the not uncommon situation in climate science that things are not quite what they seem. In this case the downloadable code dates from 2012, which should correspond to v3.2.0 — but in fact it seems to be code specially modified for a dataset for Peter Thorne, now of Maynooth. When reconfigured for GHCN input and run with GHCN-M v3.2.0 raw data the output is close to, but does not quite match, the corresponding GHCN-M adjusted data. Whether this is due to some hard-coded parameter or parameters which were not placed in the documented configuration files, or whether the modified basic code was frozen in time before all the modifications for v3.2.0 were completed, remains for the moment a matter of conjecture.

As the tutorial material I am adding will be excessive for those already familiar with some topics, I will finally shorten the post by moving out some expanded explanations to linked secondary posts.

This post started out to bring together a Twitter conversation:

Twitter2018-07-21

The remaining corrupted Valentia GHCN-M data, dealt with further below, is relatively minor. But the facts that even after notification and ‘correction’ any such errors remain, and that 17 months later the nature of their errors in MCDW still seems to elude them, as evidenced by the absence of a code correction, can hardly inspire confidence in their claim that the problem lies just with ‘some mean temperature data for select stations in Ireland’.

This post now also provides an opportunity to clarify some common misunderstandings regarding surface temperature series as well as discussing the specific question above.

Owen M seems sound on energy, but has fallen into the trap of straying from the area he is familiar with and accepting what seems at face value an authoritative analysis, but one by someone else who has also strayed from the area they were familiar with, and not understood the source data they were dealing with. This is the reason I try to confine my own blog posts to Surface Temperature data sets, and in particular Gistemp and GHCN. It is only too easy to make errors when accepting other peoples work at face value without checking for yourself.

I dealt with Ewert’s errors in November 2015 when blogs picked up his material. At Not a Lot of People Know That (1), Not a Lot of People Know That (2), Not a Lot of People Know That (3) and NoTricksZone. At NoTricksZone unfortunately a flood of comments from one David Appell, although he was correct on this occasion, seems likely to have caused the substantive issue to become lost.

Continue reading

Advertisements
Posted in Uncategorized | Tagged , , , , , | Leave a comment

The Irish Times no longer allows comments without subscription

IrishTimesNoComments

And here is the comment I would have liked to make regarding a suggestion “any chance of bringing back John Gibbons’s regular column?”. The context for the first two paragraphs is a Gibbons piece published November 3rd 2017:  Ireland showing reckless disregard for climate justice

Continue reading

Posted in Uncategorized | Tagged , | Leave a comment

Through Giant Green Goggles – the An Taisce edition

Green goggles are somewhat akin to beer goggles, but the wearer sees what he or she wants to see rather than reality. A frequent synptom of the green goggles wearer is a willingness to accept at face value and redistribute misinformation without a thought as to whether the “information” in question defies common sense. The “timeline” below, circulated widely, illustrates the phenomenon with misinformation at almost every timestep. This blog post dissects it for you by allowing you to compare the “predictions” in this timeline with the claimed original sources.

Update (November 22, 2016): You may notice that the link below for the Copenhagen Diagnosis report is to an Australian university rather than to the Copenhagen Diagnosis website. This is the link provided at http://www.copenhagendiagnosis.org/press.html , this page having now replaced the old Copenhagen Diagnosis homepage.

This post is a reworking of an earlier post, with some added material, drawing attention in particular to the use of a very misleading article by An Taisce. Not, unfortunately, an isolated aberration on the part of An Taisce.

ATezineA

Continue reading

Posted in Uncategorized | Tagged , , , , | 1 Comment

GHCN-M Raw Data from Ireland

(now subtitled) The Little Known Tropical Rain-forest of Ireland

Updated:

From the latest two GHCN-M v3 status.txt files:

*************************************************************

03/11/2017

User feedback indicated a problem with some mean temperature data for select stations in Ireland.  The problems were traced to a particular data source (MCDW), and for the time being until that source is corrected, the data are now being sourced to the UK Met Office “Climat” data (“K” source flag”), which are believed to be the correct values.  The data changeover to the UK Met Office has occurred, but the source flag (“K”) for the corrected values was inadvertently left out.  Those source flags should be added within the next production cycle.

*************************************************************

“select” stations in Ireland meaning those stations for which GHCN-M v3 has continued to include data in recent years. Although how NOAA can be confident that the problem is confined to stations in Ireland without discovering the cause of these errors in MCDW (one of their own products) is something which escapes me.

Normally you might expect that some care would be taken to get the correction right. Not however here. They have however botched it again, leaving many rogue values unchanged. Some, but not all of these are flagged as probably erroneous and to be omitted from further analysis. Others however pass their quality control. And they have not reverted to the (correct) values which they had earlier shown as received from the UK Met Office.

One absurd record caught my eye. Cork Airport in 2013 has been corrected, but in 2014 still has six identical rogue values including July, August and December. Five of these are flagged, but August still slips through as the rogue value of 10.4°. This value is wrong, but not sufficiently outlying to be caught by their quality control. I asked myself how probable six identical monthly means would be. Not impossible for a tropical rain-forest climate zone I thought. Singapore for example has monthly means of daily mean temperatures lying throughout the year within a range of less than 2°C, but could not match the record six identical monthly means of Cork Airport. Examining the complete GHCN-M v3 raw data file I found 24 stations with  six or more identical monthly means, all well within the tropics. So book your next holiday in the tropical rain-forest climatic zone of Cork. Just beware of the crocodiles, and be aware that no guarantee is given that Cork will match the temperatures of the other 24 stations.

Their quality control procedures allow for manual flagging of erroneous values not caught by their automated procedure. Somehow I think it would have been prudent to take more care having admitted their own MCDW values were wrong, and if necessary resort to manual flagging until the cause of this MCDW problem had been determined and corrected.

END OF UPDATE

corkcomposite

Errors are not confined to 2013 to 2016, and not confined to this one station.

You can easily verify this. The Met Eireann most recent four year monthly data can be found at Monthly Data (new Met Eireann site) or at the old site:  Monthly Data

The longer (not necessarily full length record however) data can be found at Historical Data  (I’ll return to add navigation advice here when I have completed other sections of this post. Navigation on this section of the Met Eireann site may not be intuitively obvious)

The GHCN-M version used above was ghcnm.tavg.v3.3.0.20161230.qcu.dat (which of course had not had a December 2016 value added, whereas Met Eireann calculates and shows a month-to-date mean, 7.4°C up to December 30th)

Station history

As shown below, the correct 2013 values were shown by GHCN-M for a time in 2013, corrupted for a time later in 2013, and briefly reappeared again in 2014, before settling down again as corrupted values.

Now follow the history of the April 2013 value (7.4°C according to Met Eireann). In the first GHCN-M file below (dated May 19th 2013) it is correctly recorded, and attributed to a CLIMAT report as source (740  P). This value is the most recent value to reach GHCN-M, and in this case the CLIMAT report has been correctly decoded. I will return to this question of correct or incorrect decoding of CLIMAT reports below).

By July 9th the still correct value has as data source “received by the UK Met Office” (740  K). This has been the usual change of data source, first CLIMAT report, then the UK Met Office. As seen on May 19th the March 2013 value had already been processed in this way (430  K).

On (or before) the 8th November the data source changed to “Monthly Climatic Data of the World (MCDW) QC completed but value is not yet published” (1040 WC). The value had now become the rogue value 10.4°C. the “W” quality flag indicates “monthly value is duplicated from the previous month, based upon regional and spatial criteria”. My experience of my region would suggest that duplicating the mean temperature of the previous month would very rarely produce a correct estimate for the following month. What “regional and spatial criteria” have required the replacement of a recorded monthly mean my a rogue value?

After that this rogue value has been retained, except for a brief return to the correct value and UK Met Office as source on (and possibly around) June 28th 2014. In mid 2015 the data source changed to “Final (Published) Monthly Climatic Data of the World (MCDW)”.

corkcomposite2013

 

Each monthly value above is followed by either one or two letters. A single letter, or the second of two letters, gives the data source:

C = Monthly Climatic Data of the World (MCDW) QC completed but value is not yet published
K = received by the UK Met Office
M = Final (Published) Monthly Climatic Data of the World (MCDW)
P = CLIMAT (Data transmitted over the GTS, not yet fully processed for the MCDW)
W = World Weather Records (WWR), 9th series 1991 through 2000

The first letter of two letters is a quality control flag:

S = monthly value has failed spatial consistency check. Any value found to be between 2.5 and 5.0 bi-weight standard deviations from the bi-weight mean, is more closely scrutinized by examining the 5 closest neighbors (not to exceed 500.0 km) and determine their associated distribution of respective z-scores.  At least one of the neighbor stations must have a z score with the same sign as the target and its z-score must be greater than or equal to the z-score listed in column B, where column B is expressed as a function of the target z-score ranges (column A). See GHCN-M README for table.
W = monthly value is duplicated from the previous month, based upon regional and spatial criteria and is only applied from the year 2000 to the present.

CLIMAT reports

More detail to be added at some future date when other work permits. You can find the CLIMAT reports in the archive at www.ogimet.com (select “CLIMAT monthly summaries from the left hand menu, then visualization mode: By country or territory, Country or territory: Ireland, and the appropriate Year and Month).

As noted above, the April 2013 value for Cork Airport was initially decoded correctly from a CLIMAT report. But this correct decoding has not always been the case.

valentia2012a

 

 

When Valentia Observatory (62103953000) became an AWS station in April 2012 the first April values entering GHCN-M from decoded CLIMAT reports were rogue values.

Posted in Uncategorized | 10 Comments

An Taisce’s Dahr Jamail timeline : source comparison

This is a short comparison of the “timeline” published by An Taisce in its January 2014 ezine with the sources claimed for these “timeline” items. For a more exhaustive (and exhausting!) treatment of this “timeline” see Through Giant Green Goggles

Update (November 22, 2016): You may notice that the link below for the Copenhagen Diagnosis report is to an Australian university rather than to the Copenhagen Diagnosis website. This is the link provided at http://www.copenhagendiagnosis.org/press.html , this page having now replaced the old Copenhagen Diagnosis homepage.

Continue reading

Posted in Uncategorized | Tagged , , , | Leave a comment

GHCN and Gistemp Data Quality

Post to be completed when the post Self-adjusting the Adjustments is complete. Posted now as placeholder to allow linkage from previous post.

Location metadata

 

Temperature data

Posted in Uncategorized | Tagged , , , , , , , | 1 Comment

Self-adjusting the Adjustments

This post is unfinished. I’m posting it now in order to refer to it in a comment on another blog, and intend to finish it in the near future.

April 14: finishing delayed while recoding data extraction to speed it up and enable further exploration of the effects of large GHCN adjusted value ranges.

If the title of this post seems familiar you may be recalling a Climate Audit post from 2010 NASA GISS – Adjusting the Adjustments which suggested that:

It is entirely possible that the change in GISS US since August 2007 is primarily due to the replacement of USHCN v1 methodology (TOBS and that sort of thing that we discussed in the past) with Menne’s changepoint methodology used in USHCN v2

This present post examines the volatility of the adjustments due to that changepoint methodology in the Global Historical Climatology Network-Monthly (GHCN-M v3) and the U.S. Historical Climatology Network (USHCN v2), and in NASA Gistemp, which uses the adjusted GHCN-M data as input.

The Climate Audit post considered changes resulting from a major change in methodology. Here I examine the changes resulting from the “less major” (I hesitate to use the word “minor” in the absence of disclosure of the updated code) changes in methodology from subversion to subversion, and the day-to-day changes in adjusted station records using the same subversion – hence the change in title to “Self-adjusting the Adjustments”. And while the focus of the Climate Audit post was primarily on the US, my focus is primarily global, with only peripheral discussion of USHCN. Continue reading

Posted in Uncategorized | Tagged , , , , | 2 Comments