create_OBSnetcdf
Python script to create a netCDF file from observational ASCII files

netCDF files are generated from a given ASCII file with observational data. Observational data can be of different types:

Transformation is controlled by an ASCII file with the description of the data. As example a template of this file called description.dat.template is provided.

Observational ASCII file is expected to contain measurements of different variables at a same time like:

[Time1] [Var1] [Var2] [Var3] ... [varN]
(...)
[TimeM] [Var1] [Var2] [Var3] ... [varN]

According to the type of observation file might contain observations at different locations, fixed or time-changing

diagnostics.py -c [comvals] -d [descobsfilen] -e [endcol] -f [ASCIIobsfilen] -g [dbg] -k [obskind] -s [stvals] -f [cftimeval]   Script has the following attributes to be used with:     [comvals]':' separated list of characters used for comments inside the
      ASCII observational file
    [descobsfilen]name of the file with the description of the ASCII file with the
      observations to transform
    [endcol]character to indicate end of the column for each observed
        variable. There are two special cases:
      'space', for spaces
      'None',[fixcoljumps] for files with the measurements provided by fixed size columns
        [fixcoljumps]: ',' separated list of positions of column ends within the line
    [ASCIIobsfilen] ASCII file with the observations
    [dbg] whether debug is required ('false', 'true')
    [obskind]kind of observations one of: 'stations-map', 'multi-points',
      'single-station', 'trajectory'
    [stvals]=[stname],[stlong],[stlat],[stheight] Values to describe the station,
         only valid for 'single-station':
      [stname] name of the station ('!' for spaces)
      [stlon] longitude of the station
      [stlat] latitude of the station
      [stheight] height of the station
    [cftimeval]=[Refdate],[tunits] values to generate the time following
      CF conventions with reference date in [YYYY][MM][DD][HH][MI][SS] format and
      time units: 'weeks', 'days', 'hours', 'miuntes', 'seconds'
  
  Inside the description file, user has to specify:
    institution=Institution who creates the data
    department=Department within the institution
    scientists=names of the data producers
    contact=contact of the data producers
    description=description of the observations
    acknowledgement=sentence of acknowlegement
    comment=comment for the measurements
         MissingValue='|' list of ASCII values for missing values within the data
      (as they appear!, 'empty' for no value at all)
    comment=comments
         varN='|' separated list of variable names (CF conventions recommended)
    varLN='|' list of long variable names
    varU='|' list units of the variables (CF conventions recommended)
       NOTE: in case observation is tabulled following a WMO code,
        user can use wmo_code_[NUM] as units and if
        a given file called wmo_[NUM].code exists (already available:
        0200, 020012, 0509, 4561, 0513, 4677, 0500, 0515) additional dimensions and
        variables will be added (see below):
          wmo_code_[NUM]: Values of the code
          wmo_code_[NUM]_meaning: meaning of each code
    varBUFR='|' list BUFR code of the variables (optional, can not be provided)
    varTYPE='|' list of variable types ('D', 'F', 'I', 'I64', 'S')
    varOPER='|' list of operations to do to the variables to meet their
      units ([oper],[val]) (optional)
      [oper]:
        -, nothing
        sumc, add [val]
        subc, rest [val]
        mulc, multiply by [val]
        divc, divide by [val]
        rmchar,[val],[pos], remove [val] characters from [pos]='B', beginning, 'E', end
    NAMElon=name of the variable (from varN, if applicable) with the longitude
      (x position)
    NAMElat=name of the variable (from varN, if applicable) with the latitude
      (y position)
    NAMEheight=name of the variable (from varN, if applicable) with the height
      (z position)
    NAMEtime=name of the varibale with the time (from varN)
      NOTE: It can be a combination of different variables from varN separated
        by '@' character (e.g.: date@time)     FMTtime=format of the time (as in 'C' (e.g.: %Y-%m-%d@%H:%M:%S), or use
      'CFtime' for observational times which are already in CF-like format)
  ------- ------ ----- ---- --- -- -
    wmo_[NUM].code file with structure:
      reference|web page, document reference with the code
      short_description|main description of the code wmo_code|[NUM]
      long_description|long description of the code
      codeTYPE|type of the code (andy of 'D', 'F', 'I', 'S')
      @| Valuesline giving the start of the values
      [val1]|[meaning of first value]
      (...)
      [valN]|[meaning of last value]
* Observations at a single station:
  python create_OBSnetcdf.py -c '#' -e None,8,17,22,28,30,33,35,40,45,51,59,67,72,78,82,87,92,94,96,99,102
    -d description_sfc_DCAO_horarios.txt -f 1995_01_SMN_HORARIOS_87576.txt
    -k single-station -s 'Ezeiza!airport,-58.534,-34.817,20.' -t 19490101000000,minutes
* Observations following a trajectory:
  python create_OBSnetcdf.py -c '#' -d description.ATRCore.txt -e space
    -f ATR_1Hz-HYMEXBDD-SOP1-v3_20121018_as120051.txt -k trajectory
* Observations at multiple points:
  python create_OBSnetcdf.py -c '#' -d description.ATDnet.txt -e space
    -f ATDnet_data_121018.csv -t 19491201000000,seconds -k 'multi-points'