So.. one week to go before handover, and I'm just STARTING the Sun/Cloud parameter, the one I thought would cause the most
trouble! Oh, boy. Let's try and work out the scenario.
Historically, we've issued Cloud:
crua6[/cru/cruts/fromtyn1/data/cru_ts_2.10/data_all] gunzip -c cru_ts_2_10.1901-2002.cld.Z |head -10
Tyndall Centre grim file created on 22.01.2004 at 13:52 by Dr. Tim Mitchell
.cld = cloud cover (percentage)
CRU TS 2.1
[Long=-180.00, 180.00] [Lati= -90.00, 90.00] [Grid X,Y= 720, 360]
[Boxes= 67420] [Years=1901-2002] [Multi= 0.1000] [Missing=-999]
Grid-ref= 1, 148
725 750 750 700 638 600 613 613 663 675 713 725
..so data is in % x10.
Then, of course, there's the relevant read_me text (from /cru/cruts/fromdpe1a/
code/idl/pro/read_me_GRIDDING.txt):
"Bear in mind that there is no working synthetic method for cloud, because Mark New
lost the coefficients file and never found it again (despite searching on tape
archives at UEA) and never recreated it. This hasn't mattered too much, because
the synthetic cloud grids had not been discarded for 1901-95, and after 1995
sunshine data is used instead of cloud data anyway."
So that's alright then! See also the earlier attempts to recreate TS 2.10 cloud.
The main gridding prog for cloud appears to be cal_cld_gts_tdm.pro:
pro cal_cld_gts_tdm,dtr_prefix,outprefix,year1,year2,info=info
; calculates cld anomalies using relationship with dtr anomalies
; reads coefficients from predefined files (*1000)
; reads DTR data from binary output files from quick_interp_tdm2.pro (binfac=1000)
; creates cld anomaly grids at dtr grid resolution
; output can then be used as dummy input to splining program that also
; includes real cloud anomaly data
As for converting sun hours to cloud cover.. we only appear to have interactive, file-by-file
programs. Herewith all the relevant progs I can find:
IDL
./idl/pro/cal_cld_gts_tdm.pro (synthetic cloud from DTR)
./idl/pro/cloudcorr.pro (construct cloud correlation coefficients with DTR)
./idl/pro/cloudcorrspc.pro (construct cloud correlation coefficients with sunshine %)
./idl/pro/cloudcorrspcann.pro (construct cloud correlation coefficients with sunshine %)
./idl/pro/cloudcorrspcann9196.pro (construct cloud correlation coefficients with sunshine %)
(the 'ann' versions above include the assumption that the relationships remain constant through the year)
F77
./f77/mnew/sh2cld_tdm.for (this one needs to be modded as for sp2cldp_m.for I think)
./f77/mnew/Hsp2cldp_m.for (one I wrote last year which seems to almost do what we need)
./f77/mnew/sp2cld_m.for (this one needs to be modded as for sp2cldp_m.for I think)
./f77/mnew/sh2sp_m.for
./f77/mnew/sh2sp_normal.for
./f77/mnew/sh2sp_tdm.for
Aaaand - another head-banging shocker! The program sh2cld_tdm.for, which describes itself thusly:
program sunh2cld
c converts sun hours monthly time series to cloud percent (n/N)
Does NO SUCH THING!!! Instead it creates SUN percentages! This is clear from the variable names and
user interactions.
So.. if I add the sunh -> sun% process from sh2cld_tdm.for into Hsp2cldp_m.for, I should end up with a
sun hours to cloud percent convertor. Possibly. Except that the sun% to cld% engine looks like it's
creating oktas instead:
do im=1,12
ratio = (real(sunp(im))/100)
if (ratio.ge.0.95) cldp(im) = 0
if (ratio.lt.0.95.and.ratio.ge.0.35)
* cldp(im) = (0.95-ratio)*100
if (ratio.lt.0.35.and.ratio.ge.0.15)
* cldp(im) = ((0.35-ratio)*50)+60
if (ratio.lt.0.15) cldp(im) = ((0.15-ratio)*100)+70
if (cldp(im).gt.80.0) cldp(im) = 80.0
if (ratio.lt.0) cldp(im) = -9999
enddo
Added the previous '*12.5' mod to approximate true percentages (*10).
Looking back I see we found cloud and sunpercent databases (line counts shown):
228936 cld.0301081434.dtb
104448 cld.0312181428.dtb
111989 combo.cld.dtb
57395 spc.0301201628.dtb
51551 spc.0312221624.dtb
51551 spc.94-00.0312221624.dtb
And agreed a strategy:
<BEGIN_QUOTE>
AGREED APPROACH for cloud (5 Oct 06).
For 1901 to 1995 - stay with published data. No clear way to replicate
process as undocumented.
For 1996 to 2002:
1. convert sun database to pseudo-cloud using the f77 programs;
2. anomalise wrt 96-00 with anomdtb.f;
3. grid using quick_interp_tdm.pro (which will use 6190 norms);
4. calculate (mean9600 - mean6190) for monthly grids, using the
published cru_ts_2.0 cloud data;
5. add to gridded data from step 3.
This should approximate the correction needed.
<END_QUOTE>
This is confusing. I can only use one (observed) cloud database in the final gridding. The above
agreement seems to assume that all data after 1996 will come from sun. But dtbstat.for reports:
<BEGIN_QUOTE>
Report for: spc.0312221624.dtb (it's similar for the other spcs, except the earlier one goes to 2002)
Stations in Northern Hemisphere: 1750
Stations in Southern Hemisphere: 350
Total: 2100
Maximum Timespan in Northern Hemisphere: 1889 to 2003
Maximum Timespan in Southern Hemisphere: 1944 to 2003
Global Timespan: 1889 to 2003
Minimum Data Value: 0
Maximum Data Value: 1000
<END_QUOTE>
So the Sun Percent databases run for long periods. Similarly, for cloud:
<BEGIN_QUOTE>
Report for: cld.0312181428.dtb
Stations in Northern Hemisphere: 3286
Stations in Southern Hemisphere: 319
Total: 3605
Maximum Timespan in Northern Hemisphere: 1905 to 1996
Maximum Timespan in Southern Hemisphere: 1959 to 1996
Global Timespan: 1905 to 1996
Minimum Data Value: 0
Maximum Data Value: 1000
<END_QUOTE>
Not as long a run, and it sure ends at 1996! So 1901 to 1995 will, as agreed, remain untouched.
Well.. let's try converting the MCDW and CLIMAT Sun hours to Sun percents, then adding to the
SPC database (spc.0312221624.dtb). Modified Hsh2cld .for to save sun percent too. Lots of debugging..
eventually dug out:
Doorenbos, J., Pruitt, W.O., 1977. Guidelines for predicting crop water requirements. FAO irrigation
and drainage paper no. 24. Food and Agriculture Organization of the United Nations, Rome.
This was used to inform the Fortran conversion programs by indicating the latitude-potential_sun and
sun-to-cloud relationships. It also assisted greatly in understanding what was wrong - Tim was in
fact calculating Cloud Percent, despite calling it Sun Percent!! Just awful.
And so..
<BEGIN_QUOTE>
crua6[/cru/cruts/version_3_0/db/cld] ./Hsh2cld
Hsh2cld - Convert a Sun Hours database to a Cloud Percent one
Please enter the Sun Hours database: sun.0709111032.dtb
Data Factor detected: *1.000
Completed - 1693 stations converted.
Sun Percentage Database: spc.0711271420.dtb
Cloud Percentage Database: cld.0711271420.dtb
crua6[/cru/cruts/version_3_0/db/cld] ./Hsh2cld
Hsh2cld - Convert a Sun Hours database to a Cloud Percent one
Please enter the Sun Hours database: sun.0710151817.dtb
Data Factor detected: *0.100
Completed - 2020 stations converted.
Sun Percentage Database: spc.0711271421.dtb
Cloud Percentage Database: cld.0711271421.dtb
crua6[/cru/cruts/version_3_0/db/cld]
<END_QUOTE>