Now, we're on to STATISTICS. Specifically, the ones needed for the paper. They include:
Cell coverage - the percentage of cells in each region that have stations, are near stations, or are not covered;
Station counts - the number of contributing stations in each region;
Data sources - the observed/synthetic split for secondary parameters.
All statistics will be required at a yearly timestep and broken down by region (ie, Europe, Africa, Asia..).
Cell coverage would ideally come from the IDL gridder - however, as we know, the approach of quick_interp_tdm2.pro
does not lend itself to revealing such information! The best solution will be to use the paired cell/cdd station
counts, as produced by stncounts.for.
Station counts should be straightforward to derive from the anomaly files (.txt), as output by anomdtb.f90. This,
however, will only work for Primary parameters, since Secondaries are driven from synthetic data as well. Further,
the synthetic element in this is usually at 2.5 degrees, so a direct relationship with half-degree coverage will
be hard to establish.
Data sources will not be easy (see Station counts above). One approach could be to analyse the anomaly files for
the Primary parameter(s), and make the assumption that their half-degree coverage will carry through (via the
2.5-degree synthetic stage and the gridding) to the final gridded data.
Actually, I think the most logical approach is to produce secondary station files that just record the observed
contributions (as opposed to the derived ones). Users will be free to use these in tandem with the appropriate
primary counts, which they can assume will have 'contributed' to the unfilled cells but to a less reliable extent
(both because of the indirect derivation and the lower resolution).
So - about time we had a drains-up on the update procedures. I've already established a logical hierarchy:
/cru/cruts/final_structure
/cru/cruts/final_structure/database * Repository for databases, might need subdirs
/cru/cruts/final_structure/incoming
/cru/cruts/final_structure/incoming/BOM
/cru/cruts/final_structure/incoming/CLIMAT
/cru/cruts/final_structure/incoming/MCDW
/cru/cruts/final_structure/incoming/other
/cru/cruts/final_structure/primary
/cru/cruts/final_structure/primary/tmp
/cru/cruts/final_structure/primary/tmp/txt
/cru/cruts/final_structure/primary/tmp/glo
/cru/cruts/final_structure/primary/tmp/abs
/cru/cruts/final_structure/primary/tmp/stn
/cru/cruts/final_structure/primary/tmp/stn/cdd0
/cru/cruts/final_structure/primary/tmp/stn/cddn
/cru/cruts/final_structure/primary/pre
/cru/cruts/final_structure/primary/pre/txt
/cru/cruts/final_structure/primary/pre/glo
/cru/cruts/final_structure/primary/pre/abs
/cru/cruts/final_structure/primary/pre/stn
/cru/cruts/final_structure/primary/pre/stn/cdd0
/cru/cruts/final_structure/primary/pre/stn/cddn
/cru/cruts/final_structure/primary/tmn
/cru/cruts/final_structure/primary/tmn/txt
/cru/cruts/final_structure/primary/tmn/glo
/cru/cruts/final_structure/primary/tmn/abs
/cru/cruts/final_structure/primary/tmn/stn
/cru/cruts/final_structure/primary/tmn/stn/cdd0
/cru/cruts/final_structure/primary/tmn/stn/cddn
/cru/cruts/final_structure/primary/tmx
/cru/cruts/final_structure/primary/tmx/txt
/cru/cruts/final_structure/primary/tmx/glo
/cru/cruts/final_structure/primary/tmx/abs
/cru/cruts/final_structure/primary/tmx/stn
/cru/cruts/final_structure/primary/tmx/stn/cdd0
/cru/cruts/final_structure/primary/tmx/stn/cddn
/cru/cruts/final_structure/primary/dtr
/cru/cruts/final_structure/primary/dtr/txt
/cru/cruts/final_structure/primary/dtr/glo
/cru/cruts/final_structure/primary/dtr/abs
/cru/cruts/final_structure/primary/dtr/stn
/cru/cruts/final_structure/primary/dtr/stn/cdd0
/cru/cruts/final_structure/primary/dtr/stn/cddn
/cru/cruts/final_structure/secondary
/cru/cruts/final_structure/secondary/vap
/cru/cruts/final_structure/secondary/vap/syn
/cru/cruts/final_structure/secondary/vap/txt
/cru/cruts/final_structure/secondary/vap/glo
/cru/cruts/final_structure/secondary/vap/abs
/cru/cruts/final_structure/secondary/vap/stn
/cru/cruts/final_structure/secondary/vap/stn/observed_only
/cru/cruts/final_structure/secondary/vap/stn/observed_only/cdd0
/cru/cruts/final_structure/secondary/vap/stn/observed_only/cddn
/cru/cruts/final_structure/secondary/vap/stn/all * might not do these
/cru/cruts/final_structure/secondary/vap/stn/all/cdd0
/cru/cruts/final_structure/secondary/vap/stn/all/cddn
/cru/cruts/final_structure/secondary/wet
/cru/cruts/final_structure/secondary/wet/syn
/cru/cruts/final_structure/secondary/wet/txt
/cru/cruts/final_structure/secondary/wet/glo
/cru/cruts/final_structure/secondary/wet/abs
/cru/cruts/final_structure/secondary/wet/stn
/cru/cruts/final_structure/secondary/wet/stn/observed_only
/cru/cruts/final_structure/secondary/wet/stn/observed_only/cdd0
/cru/cruts/final_structure/secondary/wet/stn/observed_only/cddn
/cru/cruts/final_structure/secondary/wet/stn/all * might not do these
/cru/cruts/final_structure/secondary/wet/stn/all/cdd0
/cru/cruts/final_structure/secondary/wet/stn/all/cddn
/cru/cruts/final_structure/secondary/frs
/cru/cruts/final_structure/secondary/frs/syn
/cru/cruts/final_structure/secondary/frs/txt
/cru/cruts/final_structure/secondary/frs/glo
/cru/cruts/final_structure/secondary/frs/abs
/cru/cruts/final_structure/secondary/frs/stn
/cru/cruts/final_structure/secondary/frs/stn/observed_only
/cru/cruts/final_structure/secondary/frs/stn/observed_only/cdd0
/cru/cruts/final_structure/secondary/frs/stn/observed_only/cddn
/cru/cruts/final_structure/secondary/frs/stn/all * might not do these
/cru/cruts/final_structure/secondary/frs/stn/all/cdd0
/cru/cruts/final_structure/secondary/frs/stn/all/cddn
/cru/cruts/final_structure/secondary/cld
/cru/cruts/final_structure/secondary/cld/syn
/cru/cruts/final_structure/secondary/cld/txt
/cru/cruts/final_structure/secondary/cld/glo
/cru/cruts/final_structure/secondary/cld/abs
/cru/cruts/final_structure/secondary/cld/stn
/cru/cruts/final_structure/secondary/cld/stn/observed_only
/cru/cruts/final_structure/secondary/cld/stn/observed_only/cdd0
/cru/cruts/final_structure/secondary/cld/stn/observed_only/cddn
/cru/cruts/final_structure/secondary/cld/stn/all * might not do these
/cru/cruts/final_structure/secondary/cld/stn/all/cdd0
/cru/cruts/final_structure/secondary/cld/stn/all/cddn
/cru/cruts/final_structure/static
/cru/cruts/final_structure/static/climatology
/cru/cruts/final_structure/static/mask
Then, there's the list of procedures (which probably need checking):
Now, in terms of methodology, we obviously need this to be portable. So either a system parameter or a text file
with local info is going to be needed. Since reading system parameters is less than easy (syntax differs between
platforms) a text file might be the way forward. At a pinch, all it would need to contain would be the root to
the hierarchy (ie, '/cru/cruts/final_structure/' in the above example). Might be an idea to provide a sandbox for
users to compile into.. and a file of compile lines? I wonder how far off I am from a makefile? That would help with
the frightening anomdtb.f linkages. Tried 'make' with anomdtb and it doesn't automatically find the includes, even
when they're in the same directory!
I guess I need to finish the fortran gridder program. That would allow steamlining. Notes on that
work are mainly in the file 'gridder.sandpit'. Suffice to say, it works :-) Needs tweaking, and
a few philosophical questions resolving, but apart from that..