Code and Data

Code

My Gistemp port is a port to C#, implementing Step0 through Step3 in a single application, with various enhancements, as shown below. Many additional options are selectable in an associated properties file. Features which are shown on this screenshot but which may not be immediately obvious on inspection include:

  • Option to take input from downloaded Gistemp intermediate files (big-endian) or as generated by a previous step (little-endian). Selectable for each step in the properties file, and Gistemp intermediate file input indicated by “G” on the appropriate button.
  • Option to take a list of rural stations which should not be allowed as “adjusting” stations, and a list of stations which should be excluded altogether. The “Step2/3+” and “Step2/3-” buttons and the count control to the right allow exploration of the effect of subsets of the “excluded rural adjusters” list on gridded output in a manner similar to forward and backward stepwise multiple regression.
  • Option to compare last modified dates for inputs and Gistemp source code with local versions, and download any updated versions. This screenshot does not show file dates as this option must be specified as a switch when running. The default is no check to save time.
  • Indication of Gistemp intermediate files which will be used (2010_05 here)
  • Option to specify proxy server when remote inputs are checked or downloaded.
  • Option to display all input, output and log file entries related to a station or gridcell, optionally including the corresponding information from the Gistemp intermediate files for comparison with locally generated results.

A large number of other parameters and options can be set in the properties file.

The screenshot below shows the output when a station is selected for display. When the information is saved (as a text file) an R script is also saved to generate plots.

The Step3 output for the gridcell containing a latitude/longitude pair can also be displayed.

A separate utility takes Step3 output, either locally generated or as a downloaded Gistemp intermediate file, and produces the R script and text input file to plot the time series of zonal mean anomalies. This plot can also be restricted to a given longitude range if desired.

A utility to compare the main Step3 binary outputs, SBBXgrid.1880.GHCN.CL.PA.1200 or BX.Ts.GHCN.CL.PA.1200 (or the 250 km equivalents) with the corresponding downloaded Gistemp intermediate files. as locally generated results may differ slightly from the original Gistemp versions due to machine precision/roundoff etc., this comparison verifies that all missing month values correspond, and finds the maximum positive and negative differences for non-missing values. (The lower part of the dialog shown can be used instead to extract data relating to a particular gridcell or to a number of gridcells).

Data

All standard Gistemp output files are generated, some with additional information, as well as various additional logging files. Unlike the standard Gistemp, no intermediate files are deleted, and where Gistemp modifies a file already generated, both versions are retained with different names. So for example we have SBBXgrid.1880.GHCN.CL.PA.1200 and SBBXgrid.1880.GHCN.CL.PA.1200.trim, the first generated by to.SBBXgrid, the second by trimSBBX. In addition, I have an option to generate big-endian versions in the “FORTRAN” binary format as well as my local little-endian versions for certain binary output files, for comparison with the Gistemp original versions.

Code and Data availability

As mentioned in my response to Nick Barnes in my Effect of Station “Dropout” on Gistemp post, code and data are in principle available, but if there is anything more than an occasional request which can be satisfied via e-mail, setting up a new FTP server will probably not be possible before August because of other work and holiday plans. For the moment, post any request here, using a valid e-mail address (it is not visible to other visitors). Please do not request the input files which you can obtain from the NASA server. Before requesting an output file, have a look at the file size on the NASA FTP server before making the request – many of these files are large, and you need to know you may be requesting a very large file, and that you can handle an e-mail attachment that large if you make a request now before I have set up a server. Before requesting any of the binary output files make sure you know how to read such files!

Source code files are also large (the NASA FORTRAN/Python sources are for example included as comments in my source for reference as well as my C# code) and not very elegantly coded (I have retained the original code structure when porting).

Current status of my port: Step0 through Step3 ported and tested. However, Step0 and Step1 have not necessarily been brought fully up to date, as I have been running only Step2 and Step3 recently, taking the NASA archived output from Step1 (Ts.txt) as input, my interest being mainly in the UHI adjustment.

Advertisements
This entry was posted in Asides and tagged . Bookmark the permalink.

One Response to Code and Data

  1. drj11 says:

    Have you thought about putting your code on googlecode? Or one of the other public source code repositories? As you probably know, our ccc-gistemp project has been publicly available on googlecode for ages. I wouldn’t say it’s brilliant, but the convenience factor is good. We can offer advice if you want to think about it.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s