source: lmdz_wrf/trunk/tools/documentation/other/obs_netcdf.html @ 1958

Last change on this file since 1958 was 1958, checked in by lfita, 7 years ago

Adding documentation for U. Wyoming transformer
Fixing typos in OBS netcdf documentation
File size: 11.1 KB
Line 
1<HTML>
2  <HEAD>
3    <META HTTP-EQUIV="Content-Type" CONTENT="text/html; CHARSET=utf-8"/>
4    <LINK REL="stylesheet" TYPE="text/css" HREF="../PyNCplot.css"/>
5  </HEAD>
6
7  </HEAD>
8  <BODY>
9    <DIV CLASS="valheader">
10      create_OBSnetcdf
11    </DIV>
12    <DIV CLASS="valimg">
13      Python script to create a netCDF file from observational ASCII files<BR>
14      <BR>
15      <P STYLE="text-align:left;font-size:0.9em;">
16      netCDF files are generated from a given ASCII file with observational data. Observational data can be of different types:<BR></P>
17      <UL>
18      <P STYLE="text-align:left;font-size:0.9em;">
19        <LI><B>stations-map:</B> multiple individual punctual observations at no fixed locations and not fixed times (e.g., lightning strikes)</LI> 
20        <LI><B>multi-points:</B> multiple observations at fixed locations and fixed times</LI>
21        <LI><B>single-station:</B> multiple observations on a single station at different times</LI>
22        <LI><B>trajectory:</B> multiple observations following a trajectory</LI>
23      </UL></P>
24      <P STYLE="text-align:left;font-size:0.9em;">
25      Transformation is controlled by an ASCII file with the description of the data. As example a template of this file called <CODE>description.dat.template</CODE> is provided.<BR></P>
26      <P STYLE="text-align:left;font-size:0.9em;">
27      Observational ASCII file is expected to contain measurements of different variables at a same time like:<BR></P>
28      <P STYLE="text-align:left;font-size:0.9em;">
29      <CODE>[Time1] [Var1] [Var2] [Var3] ... [varN]<BR>
30      (...)<BR>
31      [TimeM] [Var1] [Var2] [Var3] ... [varN]</CODE><BR></P>
32      <P STYLE="text-align:left;font-size:0.9em;">
33      According to the type of observation file might contain observations at different locations, fixed or time-changing</P>
34      </P>
35    </DIV>
36    <DIV CLASS="valhelp">
37      diagnostics.py -c [comvals] -d [descobsfilen] -e [endcol] -f [ASCIIobsfilen] -g [dbg] -k [obskind] -s [stvals] -f [cftimeval]
38      &nbsp;&nbsp;Script has the following attributes to be used with:
39      &nbsp;&nbsp;&nbsp;&nbsp;<CODE>[comvals]</CODE>':' separated list of characters used for comments inside the <BR>
40      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;ASCII observational file<BR>
41      &nbsp;&nbsp;&nbsp;&nbsp;<CODE>[descobsfilen]</CODE>name of the file with the description of the ASCII file with the <BR>
42      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;observations to transform<BR>
43      &nbsp;&nbsp;&nbsp;&nbsp;<CODE>[endcol]</CODE>character to indicate end of the column for each observed <BR>
44      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;variable. There are two special cases:<BR>
45      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<CODE>'space'</CODE>, for spaces <BR>
46      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<CODE>'None',[fixcoljumps]</CODE> for files with the measurements provided by fixed size columns<BR>
47      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<CODE>[fixcoljumps]</CODE>: ',' separated list of positions of column ends within the line<BR>
48      &nbsp;&nbsp;&nbsp;&nbsp;<CODE>[ASCIIobsfilen]</CODE> ASCII file with the observations<BR>
49      &nbsp;&nbsp;&nbsp;&nbsp;<CODE>[dbg]</CODE> whether debug is required ('false', 'true')<BR>
50      &nbsp;&nbsp;&nbsp;&nbsp;<CODE>[obskind]</CODE>kind of observations one of: <CODE>'stations-map'</CODE>, <CODE>'multi-points'</CODE>,<BR>
51      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<CODE>'single-station'</CODE>, <CODE>'trajectory'</CODE><BR>
52      &nbsp;&nbsp;&nbsp;&nbsp;<CODE>[stvals]=[stname],[stlong],[stlat],[stheight]</CODE> Values to describe the station, <BR>
53      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; only valid for <CODE>'single-station'</CODE>:<BR>
54      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<CODE>[stname]</CODE> name of the station ('!' for spaces)<BR>
55      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<CODE>[stlon]</CODE> longitude of the station<BR>
56      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<CODE>[stlat]</CODE> latitude of the station<BR>
57      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<CODE>[stheight]</CODE> height of the station<BR>
58      &nbsp;&nbsp;&nbsp;&nbsp;<CODE>[cftimeval]=[Refdate],[tunits]</CODE> values to generate the time following <BR>
59      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;CF conventions with reference date in <CODE>[YYYY][MM][DD][HH][MI][SS]</CODE> format and <BR>
60      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;time units: 'weeks', 'days', 'hours', 'miuntes', 'seconds'<BR>
61      &nbsp;&nbsp;<BR>
62      &nbsp;&nbsp;Inside the description file, user has to specify:<BR>
63      &nbsp;&nbsp;&nbsp;&nbsp;<CODE>institution=</CODE>Institution who creates the data<BR>
64      &nbsp;&nbsp;&nbsp;&nbsp;<CODE>department=</CODE>Department within the institution<BR>
65      &nbsp;&nbsp;&nbsp;&nbsp;<CODE>scientists=</CODE>names of the data producers<BR>
66      &nbsp;&nbsp;&nbsp;&nbsp;<CODE>contact=</CODE>contact of the data producers<BR>
67      &nbsp;&nbsp;&nbsp;&nbsp;<CODE>description=</CODE>description of the observations<BR>
68      &nbsp;&nbsp;&nbsp;&nbsp;<CODE>acknowledgement=</CODE>sentence of acknowlegement<BR>
69      &nbsp;&nbsp;&nbsp;&nbsp;<CODE>comment=</CODE>comment for the measurements<BR>
70      &nbsp;&nbsp;&nbsp;&nbsp;
71      &nbsp;&nbsp;&nbsp;&nbsp;<CODE>MissingValue=</CODE>'|' list of ASCII values for missing values within the data <BR>
72      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;(as they appear!, 'empty' for no value at all)<BR>
73      &nbsp;&nbsp;&nbsp;&nbsp;<CODE>comment=</CODE>comments<BR>
74      &nbsp;&nbsp;&nbsp;&nbsp;
75      &nbsp;&nbsp;&nbsp;&nbsp;<CODE>varN=</CODE>'|' separated list of variable names (<A CLASS="lc" HREF="http://cfconventions.org/" TARGET="_blank">CF conventions</A> recommended)<BR>
76      &nbsp;&nbsp;&nbsp;&nbsp;<CODE>varLN=</CODE>'|' list of long variable names<BR>
77      &nbsp;&nbsp;&nbsp;&nbsp;<CODE>varU=</CODE>'|' list units of the variables (<A CLASS="lc" HREF="http://cfconventions.org/" TARGET="_blank">CF conventions</A> recommended)<BR>
78      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <B>NOTE:</B> in case observation is tabulled following a WMO code, <BR>
79      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;user can use <CODE>wmo_code_[NUM]</CODE> as units and if <BR>
80      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;a given file called <CODE>wmo_[NUM].code</CODE> exists (already available: <BR>
81      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0200, 020012, 0509, 4561, 0513, 4677, 0500, 0515) additional dimensions and <BR>
82      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;variables will be added (see below):<BR>
83      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<CODE>wmo_code_[NUM]</CODE>: Values of the code<BR>
84      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<CODE>wmo_code_[NUM]_meaning</CODE>: meaning of each code<BR>
85      &nbsp;&nbsp;&nbsp;&nbsp;<CODE>varBUFR=</CODE>'|' list BUFR code of the variables (optional, can not be provided)<BR>
86      &nbsp;&nbsp;&nbsp;&nbsp;<CODE>varTYPE=</CODE>'|' list of variable types ('D', 'F', 'I', 'I64', 'S')<BR>
87      &nbsp;&nbsp;&nbsp;&nbsp;<CODE>varOPER=</CODE>'|' list of operations to do to the variables to meet their <BR>
88      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;units ([oper],[val]) (optional)<BR>
89      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[oper]:<BR>
90      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;-, nothing<BR>
91      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;sumc, add [val]<BR>
92      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;subc, rest [val]<BR>
93      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;mulc, multiply by [val]<BR>
94      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;divc, divide by [val]<BR>
95      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;rmchar,[val],[pos], remove [val] characters from [pos]='B', beginning, 'E', end<BR>
96      &nbsp;&nbsp;&nbsp;&nbsp;<CODE>NAMElon=</CODE>name of the variable (from <CODE>varN</CODE>, if applicable) with the longitude <BR>
97      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;(x position)<BR>
98      &nbsp;&nbsp;&nbsp;&nbsp;<CODE>NAMElat=</CODE>name of the variable (from <CODE>varN</CODE>, if applicable) with the latitude <BR>
99      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;(y position)<BR>
100      &nbsp;&nbsp;&nbsp;&nbsp;<CODE>NAMEheight=</CODE>name of the variable (from <CODE>varN</CODE>, if applicable) with the height <BR>
101      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;(z position)<BR>
102      &nbsp;&nbsp;&nbsp;&nbsp;<CODE>NAMEtime=</CODE>name of the varibale with the time (from <CODE>varN</CODE>) <BR>
103      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<B>NOTE:</B> It can be a combination of different variables from <CODE>varN</CODE> separated <BR>
104      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;by '@' character (e.g.: date@time)
105      &nbsp;&nbsp;&nbsp;&nbsp;<CODE>FMTtime=</CODE>format of the time (as in 'C' (e.g.: %Y-%m-%d@%H:%M:%S), or use <BR>
106      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<CODE>'CFtime'</CODE> for observational times which are already in CF-like format)<BR>
107
108      &nbsp;&nbsp;------- ------ ----- ---- --- -- -<BR>
109      &nbsp;&nbsp;&nbsp;&nbsp;<CODE>wmo_[NUM].code</CODE> file with structure: <BR>
110      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<CODE>reference|</CODE>web page, document reference with the code<BR>
111      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<CODE>short_description|</CODE>main description of the code <CODE>wmo_code|</CODE>[NUM]<BR>
112      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<CODE>long_description|</CODE>long description of the code<BR>
113      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<CODE>codeTYPE|</CODE>type of the code (andy of 'D', 'F', 'I', 'S')<BR>
114      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<CODE>@| Values</CODE>line giving the start of the values<BR>
115      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<CODE>[val1]|</CODE>[meaning of first value]<BR>
116      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<CODE>(...)</CODE><BR>
117      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<CODE>[valN]|</CODE>[meaning of last value]<BR>
118    </DIV>
119    <DIV CLASS="valins">
120      * Observations at a single station:<BR>
121      &nbsp;&nbsp;python create_OBSnetcdf.py -c '#' -e None,8,17,22,28,30,33,35,40,45,51,59,67,72,78,82,87,92,94,96,99,102 <BR>
122      &nbsp;&nbsp;&nbsp;&nbsp;-d <A CLASS="lc" HREF="docs/description_sfc_DCAO_horarios.txt" TARGET="_blank">description_sfc_DCAO_horarios.txt</A> -f <A CLASS="lc" HREF="docs/1995_01_SMN_HORARIOS_87576.txt">1995_01_SMN_HORARIOS_87576.txt</A> <BR>
123      &nbsp;&nbsp;&nbsp;&nbsp;-k single-station -s 'Ezeiza!airport,-58.534,-34.817,20.' -t 19490101000000,minutes<BR>
124      * Observations following a trajectory:<BR>
125      &nbsp;&nbsp;python create_OBSnetcdf.py -c '#' -d <A CLASS="lc" HREF="docs/description.ATRCore.txt" TARGET="_blank">description.ATRCore.txt</A> -e space <BR>
126      &nbsp;&nbsp;&nbsp;&nbsp;-f <A CLASS="lc" HREF="docs/ATR_1Hz-HYMEXBDD-SOP1-v3_20121018_as120051.txt" TARGET="_blank">ATR_1Hz-HYMEXBDD-SOP1-v3_20121018_as120051.txt</A> -k trajectory<BR>
127      * Observations at multiple points:<BR>
128      &nbsp;&nbsp;python create_OBSnetcdf.py -c '#' -d <A CLASS="lc" HREF="docs/description.ATDnet.txt" TARGET="_blank">description.ATDnet.txt</A> -e space <BR>
129      &nbsp;&nbsp;&nbsp;&nbsp;-f <A CLASS="lc" HREF="docs/ATDnet_data_121018.csv" TARGET="_blank">ATDnet_data_121018.csv</A> -t 19491201000000,seconds -k 'multi-points'
130    </DIV>
131  </BODY>
132</HTML>
133
Note: See TracBrowser for help on using the repository browser.