WET looks better, but variability is still too low. It's complicated by the synthetic elements in
combination with percentage anomalies!
I found that I could examine the 'mark New binary files' with Matlab, using (ie):
>> fid = fopen('glo25.rd0.6190','r');
>> [d,c] = fread(fid,inf,'int16');
>> whos
Name Size Bytes Class
c 1x1 8 double array
d 124416x1 995328 double array
fid 1x1 8 double array
Grand total is 124418 elements using 995344 bytes
>> c
c =
124416
>> min(d)
ans =
0
>> max(d)
ans =
303
>> hmean(d)
ans =
123.7939
So we can deduce that the rd0 2.5 degree normals are in days*10. Similarly for the others of interest:
FILE MIN MAX MEAN UNITS
glo25.rd0.6190 0 303 106 days*10
glo25.pre.6190 0 391 21 mm?
glo.rd0.norm 0 310 124 days*10
glo.pre.norm 0 1244 58 mm
clim.6190.lan.wet.grid 0 3090 1018 days*100
clim.6190.lan.pre.grid 0 12430 574 mm*10
My guess is that glo25.pre.6190 has a lower max because the wider coverage of each cell is squashing
the peaks. So.. an experimental run with half-degree synthetics!
RIGHT, stop all that.. Tim O has recalculated the 2.5-degree binary normals for PRE (from half degree)
and WET (from TS 2.1). So.. time to try out the OTHER synthetic rd0 generator, the one that reads
precip anomalies:
crua6[/cru/cruts/version_3_0/secondaries/wet] ./glo2abs
Welcome! This is the GLO2ABS program.
I will create a set of absolute grids from
a set of anomaly grids (in .glo format), also
a gridded version of the climatology.
Enter the path and name of the normals file: clim.6190.lan.wet
Enter a name for the gridded climatology file: clim.6190.lan.wet.grid7
Enter the path and stem of the .glo files: rd0glo/rd0.
Enter the starting year: 1901
Enter the ending year: 2006
Enter the path (if any) for the output files: rd0abs/
Now, CONCENTRATE. Addition or Percentage (A/P)? P
Do you wish to limit the output values? (Y/N): Y
1. Set minimum to zero
2. Set single minimum and maximum values
3. Set minima and maxima based on days in month
4. Set integer values >=1, (ie, positive)
5. Changed my mind, no limits
Choose: 3
Right, erm.. off I jolly well go!
rd0glo/rd0.01.1901.glo
rd0glo/rd0.1901.01.glo
rd0.01.1901.glo
(etc)
rd0.12.2006.glo
uealogin1[/cru/cruts/version_3_0/secondaries/wet] ./makegrids
Welcome! This is the MAKEGRIDS program.
I will create decadal and full gridded files,
in both ASCII text and NetCDF formats, from
the output files of (eg) glo2abs.for.
Enter a gridfile with YYYY for year and MM for month: rd0abs/rd0.YYYY.MM.glo.abs
Enter Start Year: 1901
Enter Start Month: 01
Enter End Year: 2006
Enter End Month: 12
Please enter a sample OUTPUT filename, replacing
start year with SSSS and end year with EEEE, and
ending with '.dat', eg: cru_ts_3_00.SSSS.EEEE.tmp.dat : cru_ts_3_00.SSSS.EEEE.wet.dat
Now please enter the 3-ch parameter code: wet
Enter a generic title for this dataset, eg:
CRU TS 3.00 Mean Temperature : CRU TS 3.00 Wet Days
Writing: cru_ts_3_00.1901.1910.wet.dat
cru_ts_3_00.1901.1910.wet.nc
Writing: cru_ts_3_00.1911.1920.wet.dat
cru_ts_3_00.1911.1920.wet.nc
Writing: cru_ts_3_00.1921.1930.wet.dat
cru_ts_3_00.1921.1930.wet.nc
Writing: cru_ts_3_00.1931.1940.wet.dat
cru_ts_3_00.1931.1940.wet.nc
Writing: cru_ts_3_00.1941.1950.wet.dat
cru_ts_3_00.1941.1950.wet.nc
Writing: cru_ts_3_00.1951.1960.wet.dat
cru_ts_3_00.1951.1960.wet.nc
Writing: cru_ts_3_00.1961.1970.wet.dat
cru_ts_3_00.1961.1970.wet.nc
Writing: cru_ts_3_00.1971.1980.wet.dat
cru_ts_3_00.1971.1980.wet.nc
Writing: cru_ts_3_00.1981.1990.wet.dat
cru_ts_3_00.1981.1990.wet.nc
Writing: cru_ts_3_00.1991.2000.wet.dat
cru_ts_3_00.1991.2000.wet.nc
Writing: cru_ts_3_00.2001.2006.wet.dat
cru_ts_3_00.2001.2006.wet.nc
Wrong again! The saga continues.. actually I'm beginning to wonder if it'll still be going when I JOIN SAGA.
This time, the 'real' areas have variability 10x too low, and the 'synthetic' areas have variability sqrt(10)
too low. The latter can be explained by the binary precip being in %age anoms *10, so rd0_gts_anom.pro modified
to divide by 1000 when calculating (instead of 100). Example (from the normals calculation):
Before:
pregrd(nland)=((pregrd(nland)/100.0)+1.0)*prenorm(nland) ; make pre anom into abs
After:
pregrd(nland)=((pregrd(nland)/1000.0)+1.0)*prenorm(nland) ; make pre anom into abs (mm)
'Synthfac=10' will also not be needed in the final gridding, that should take care of the 'real' area variability.
crua6[/cru/cruts/version_3_0/secondaries/wet] ./glo2abs
Welcome! This is the GLO2ABS program.
I will create a set of absolute grids from
a set of anomaly grids (in .glo format), also
a gridded version of the climatology.
Enter the path and name of the normals file: clim.6190.lan.wet
Enter a name for the gridded climatology file: clim.6190.lan.wet.grid9
Enter the path and stem of the .glo files: rd0glo/rd0.
Enter the starting year: 1901
Enter the ending year: 2006
Enter the path (if any) for the output files: rd0abs/
Now, CONCENTRATE. Addition or Percentage (A/P)? P
Do you wish to limit the output values? (Y/N): Y
1. Set minimum to zero
2. Set single minimum and maximum values
3. Set minima and maxima based on days in month
4. Set integer values >=1, (ie, positive)
5. Changed my mind, no limits
Choose: 3
Right, erm.. off I jolly well go!
rd0glo/rd0.01.1901.glo
rd0glo/rd0.1901.01.glo
rd0.1901.01.glo
(etc)
rd0.2006.12.glo
uealogin1[/cru/cruts/version_3_0/secondaries/wet] ./makegrids
Welcome! This is the MAKEGRIDS program.
I will create decadal and full gridded files,
in both ASCII text and NetCDF formats, from
the output files of (eg) glo2abs.for.
Enter a gridfile with YYYY for year and MM for month: rd0abs/rd0.YYYY.MM.glo.abs
Enter Start Year: 1901
Enter Start Month: 01
Enter End Year: 2006
Enter End Month: 12
Please enter a sample OUTPUT filename, replacing
start year with SSSS and end year with EEEE, and
ending with '.dat', eg: cru_ts_3_00.SSSS.EEEE.tmp.dat : cru_ts_3_00.SSSS.EEEE.wet.dat
Now please enter the 3-ch parameter code: wet
Enter a generic title for this dataset, eg:
CRU TS 3.00 Mean Temperature : CRU TS 3.00 Rain Days
Writing: cru_ts_3_00.1901.1910.wet.dat
cru_ts_3_00.1901.1910.wet.nc
Writing: cru_ts_3_00.1911.1920.wet.dat
cru_ts_3_00.1911.1920.wet.nc
Writing: cru_ts_3_00.1921.1930.wet.dat
cru_ts_3_00.1921.1930.wet.nc
Writing: cru_ts_3_00.1931.1940.wet.dat
cru_ts_3_00.1931.1940.wet.nc
Writing: cru_ts_3_00.1941.1950.wet.dat
cru_ts_3_00.1941.1950.wet.nc
Writing: cru_ts_3_00.1951.1960.wet.dat
cru_ts_3_00.1951.1960.wet.nc
Writing: cru_ts_3_00.1961.1970.wet.dat
cru_ts_3_00.1961.1970.wet.nc
Writing: cru_ts_3_00.1971.1980.wet.dat
cru_ts_3_00.1971.1980.wet.nc
Writing: cru_ts_3_00.1981.1990.wet.dat
cru_ts_3_00.1981.1990.wet.nc
Writing: cru_ts_3_00.1991.2000.wet.dat
cru_ts_3_00.1991.2000.wet.nc
Writing: cru_ts_3_00.2001.2006.wet.dat
cru_ts_3_00.2001.2006.wet.nc
Hmmm.. still some problems. In several areas, including a swathe of Russia, the mean values drop
around 1991 - just when MCDW comes in.
Please define the bounding box in tenths of a
degree, ie 55 degrees North would be 550:
Southern Edge (-900 to 899): 570
Northern Edge ( 570 to 900): 630
Western Edge (-1800 to 1799): 900
Eastern Edge ( 900 to 1800): 1050
Enter the CRU TS Database file: wet.0710161148.dtb
And finally, an output filename: wet.0710161148.57_63N.90_105E.dat
Done. Found 7 matching stations out of 6143
2388400 6160 9000 63 BOR RUSSIA (ASIA) 1936 2007 -999 -999
2389100 6167 9637 261 BAJKIT RUSSIA (ASIA) 1936 2007 -999 -999
2490800 6033 10227 260 VANAVARA RUSSIA (ASIA) 1936 2007 -999 -999
2926300 5845 9215 78 JENISEJSK RUSSIA (ASIA) 1936 2007 -999 -999
2928200 5842 9740 134 BOGUCANY RUSSIA (ASIA) 1936 2007 -999 -999
2398600 6047 9302 521 SEVERO-JENISEJSK RUSSIA (ASIA) 2004 2007 -999 0
2937900 5720 9488 147 TASEJEVA RIVER RUSSIA (ASIA) 2004 2007 -999 0
The last two are too short to have any meaning. The second and third have missing data over
the entire period of concern. That leaves BOR, JENISEJSK and BOGUCANY, the latter of which
we'll examine closer. Here's the series, lifted directly from wet.0710161148.dtb:
You can see that the data after 1990 are for some months significantly lower than the
period before.. which would be the period the normals would be based on! I used Matlab
to calculate the normals for this series:
They aren't percentage anomalies! They are percentage anomalies /10. This could explain why
the real data areas had variability 10x too low. BUT it shouldn't be - they should be
regular percentage anomalies! This whole process is too convoluted and created myriad
problems of this kind. I really think we should change it.
Back on the case. I need to find where the post-1990 data came from for these three stations. I already know
the geneaology of the database:
wet.0311061611.dtb
+
rdy.0709111032.dtb (MCDW composite)
+
rdy.0710151817.dtb (CLIMAT composite with metadata added)
V
V
wet.0710161148.dtb
I was going to do further backtracing, but it's been revealed that the same issues were in 2.1 - meaning that
I didn't add the duff data. The suggested way forward is to not use any observations after 1989, but to allow
synthetics to take over. I'm not keen on this approach as it's likely (imo) to introduce visible jumps at 1990,
since we're effectively introducing a change of data source just after calculating the normals. My compromise is
to try it - but to also try a straight derivation from half-degree synthetics.
So, first, we need synthetic-only from 1990 onwards, that can be married with the existing glos from pre-1990.
Actually, we might as well produce a full series of gridded syn-only rd0. Hell, we can do both options in one go!
No point in using the final gridding routine, rd0_gts_anom can produce glo files itself, let's give it a go.
Well - not straightforward. rd0_gts_anom.pro is quite resistant to the idea that it might produce half-degree
synthetics, to the point where I'm really not sure what's left to modify! Eventually found it.. the .glo saving
routine takes a second argument which is a code for the grid size. Because just giving it the grid size just
wouldn't be the same, would it? Here it is:
Now that 23 is the key, but you have to look in quick_interp_tdm2.pro to decode it:
if (gs[0] eq 0.5) then SaveGrid=12
if (gs[0] eq 2.5) then SaveGrid=22
if (gs[0] eq 5.0) then SaveGrid=23
So actually, this was saving with a gridsize of 5 degrees! Disquietingly, this isn't born out by the file sizes,
but we'll gloss over that. So, with '23' changed to '12', we have rd0_gts_anom_05.pro.
crua6[/cru/cruts/version_3_0/secondaries/wet] ./glo2abs
Welcome! This is the GLO2ABS program.
I will create a set of absolute grids from
a set of anomaly grids (in .glo format), also
a gridded version of the climatology.
Enter the path and name of the normals file: clim.6190.lan.wet
Enter a name for the gridded climatology file: clim.6190.lan.wet.grid14
Enter the path and stem of the .glo files: rd0syn05glo/rd0syn05.
Enter the starting year: 1901
Enter the ending year: 2006
Enter the path (if any) for the output files: rd0syn05abs/
Now, CONCENTRATE. Addition or Percentage (A/P)? P
Do you wish to limit the output values? (Y/N): Y
1. Set minimum to zero
2. Set single minimum and maximum values
3. Set minima and maxima based on days in month
4. Set integer values >=1, (ie, positive)
5. Changed my mind, no limits
Choose: 3
Right, erm.. off I jolly well go!
rd0syn05glo/rd0syn05.01.1901.glo
rd0syn05glo/rd0syn05.1901.01.glo
rd0syn05.1901.01.glo
(etc)
rd0syn05.2006.12.glo
There was then some copying around of decades' worth chunks of .abs files, to make a set with obs/syn to 1989
and syn from 1990 onwards. Then:
uealogin1[/cru/cruts/version_3_0/secondaries/wet] ./makegrids
Welcome! This is the MAKEGRIDS program.
I will create decadal and full gridded files,
in both ASCII text and NetCDF formats, from
the output files of (eg) glo2abs.for.
Enter a gridfile with YYYY for year and MM for month: rd0abs_mixed/rd0.YYYY.MM.glo.abs.gz
Enter Start Year: 1901
Enter Start Month: 01
Enter End Year: 2006
Please enter a sample OUTPUT filename, replacing
start year with SSSS and end year with EEEE, and
ending with '.dat', eg: cru_ts_3_00.SSSS.EEEE.tmp.dat : cru_ts_3_00.SSSS.EEEE.wet.dat
Now please enter the 3-ch parameter code: wet
Enter a generic title for this dataset, eg:
CRU TS 3.00 Mean Temperature : CRU TS 3.00 Rain Days synth 1990 on
Writing: cru_ts_3_00.1901.1910.wet.dat
cru_ts_3_00.1901.1910.wet.nc
Writing: cru_ts_3_00.1911.1920.wet.dat
cru_ts_3_00.1911.1920.wet.nc
Writing: cru_ts_3_00.1921.1930.wet.dat
cru_ts_3_00.1921.1930.wet.nc
Writing: cru_ts_3_00.1931.1940.wet.dat
cru_ts_3_00.1931.1940.wet.nc
Writing: cru_ts_3_00.1941.1950.wet.dat
cru_ts_3_00.1941.1950.wet.nc
Writing: cru_ts_3_00.1951.1960.wet.dat
cru_ts_3_00.1951.1960.wet.nc
Writing: cru_ts_3_00.1961.1970.wet.dat
cru_ts_3_00.1961.1970.wet.nc
Writing: cru_ts_3_00.1971.1980.wet.dat
cru_ts_3_00.1971.1980.wet.nc
Writing: cru_ts_3_00.1981.1990.wet.dat
cru_ts_3_00.1981.1990.wet.nc
Writing: cru_ts_3_00.1991.2000.wet.dat
cru_ts_3_00.1991.2000.wet.nc
Writing: cru_ts_3_00.2001.2006.wet.dat
cru_ts_3_00.2001.2006.wet.nc