Changeset 4171 for branches/2013/dev_UKMO_2013/DOC
- Timestamp:
- 2013-11-08T18:29:37+01:00 (11 years ago)
- Location:
- branches/2013/dev_UKMO_2013/DOC/TexFiles/Chapters
- Files:
-
- 2 edited
-
Chap_DIA.tex (modified) (9 diffs)
-
Chap_OBS.tex (modified) (4 diffs)
Legend:
- Unmodified
- Added
- Removed
-
branches/2013/dev_UKMO_2013/DOC/TexFiles/Chapters/Chap_DIA.tex
r3976 r4171 362 362 \subsubsection{Use of Groups} 363 363 364 Groups can be used for 2 purposes. Firstly, the group can be used to define common attributes to be shared by the elements of the group through theinheritance. In the following example, we define a group of field that will share a common grid ''grid\_T\_2D''. Note that for the field ''toce'', we overwrite the grid definition inherited from the group by ''grid\_T\_3D''.364 Groups can be used for 2 purposes. Firstly, the group can be used to define common attributes to be shared by the elements of the group through inheritance. In the following example, we define a group of field that will share a common grid ''grid\_T\_2D''. Note that for the field ''toce'', we overwrite the grid definition inherited from the group by ''grid\_T\_3D''. 365 365 \vspace{-20pt} 366 366 \begin{alltt} {{\scriptsize … … 386 386 \end{verbatim} 387 387 }}\end{alltt} 388 that can be directly include in a file through the following syntax:388 that can be directly included in a file through the following syntax: 389 389 \vspace{-20pt} 390 390 \begin{alltt} {{\scriptsize … … 466 466 \end{verbatim} 467 467 }}\end{alltt} 468 However it is often very convienent to define the file name with the name of the experi ence, the output file frequency and the date of the beginning and the end of the simulation (which are informations stored either in the namelist or in the XML file). To do so, we added the following rule: if the id of the tag file is ''fileN''(where N = 1 to 99) or one of the predefined section or mooring(see next subsection), the following part of the name and the name\_suffix (that can be inherited) will be automatically replaced by:\\468 However it is often very convienent to define the file name with the name of the experiment, the output file frequency and the date of the beginning and the end of the simulation (which are informations stored either in the namelist or in the XML file). To do so, we added the following rule: if the id of the tag file is ''fileN''(where N = 1 to 99) or one of the predefined sections or moorings (see next subsection), the following part of the name and the name\_suffix (that can be inherited) will be automatically replaced by:\\ 469 469 \\ 470 470 \begin{tabular}{|p{4cm}|p{8cm}|} … … 474 474 \hline 475 475 \centering @expname@ & 476 the experi encename (from cn\_exp in the namelist) \\476 the experiment name (from cn\_exp in the namelist) \\ 477 477 \hline 478 478 \centering @freq@ & … … 590 590 file\_definition & 591 591 encapsulates the definition of all the files that will be outputted & 592 enabled, min\_digits, name, name\_suffix, output\_level, split\_f ormat, split\_freq, sync\_freq, type, src &592 enabled, min\_digits, name, name\_suffix, output\_level, split\_freq\_format, split\_freq, sync\_freq, type, src & 593 593 context & 594 594 file or file\_group \\ … … 596 596 file\_group & 597 597 encapsulates a group of files that will be outputted & 598 enabled, description, id, min\_digits, name, name\_suffix, output\_freq, output\_level, split\_f ormat, split\_freq, sync\_freq, type, src &598 enabled, description, id, min\_digits, name, name\_suffix, output\_freq, output\_level, split\_freq\_format, split\_freq, sync\_freq, type, src & 599 599 file\_definition, file\_group & 600 600 file or file\_group \\ … … 602 602 file & 603 603 define the contents of a file to be outputted & 604 enabled, description, id, min\_digits, name, name\_suffix, output\_freq, output\_level, split\_f ormat, split\_freq, sync\_freq, type, src &604 enabled, description, id, min\_digits, name, name\_suffix, output\_freq, output\_level, split\_freq\_format, split\_freq, sync\_freq, type, src & 605 605 file\_definition, file\_group & 606 606 field \\ … … 775 775 field family \\ 776 776 \hline 777 split\_f ormat&778 date format used in the name of splitted output files. can be spécified using the following syntaxe: \%y, \%mo, \%d, \%h \%mi and \%s&779 split\_f ormat= "\%yy\%mom\%dd" &777 split\_freq & 778 frequency at which to temporally split output files. Units can be ts (timestep), y, mo, d, h, mi, s. Useful for long runs to prevent over-sized output files.& 779 split\_freq="1mo" & 780 780 file family \\ 781 781 \hline 782 split\_freq & 783 split output files frequency. units can be ts (timestep), y, mo, d, h, mi, s. & 784 split\_freq="1mo" & 782 split\_freq\-\_format & 783 date format used in the name of temporally split output files. Can be specified 784 using the following syntaxes: \%y, \%mo, \%d, \%h \%mi and \%s & 785 split\_freq\_format= "\%y\%mo\%d" & 785 786 file family \\ 786 787 \hline … … 812 813 \hline 813 814 type (1)& 814 specify if the output files must be split(multiple\_file) or not (one\_file) &815 specify if the output files are to be split spatially (multiple\_file) or not (one\_file) & 815 816 type="multiple\_file" & 816 817 file familly \\ -
branches/2013/dev_UKMO_2013/DOC/TexFiles/Chapters/Chap_OBS.tex
r3294 r4171 5 5 \label{OBS} 6 6 7 Authors: D. Lea, M. Martin, K. Mogensen, A. Vidard, A. Weaver ... % do we keep that ?7 Authors: D. Lea, M. Martin, K. Mogensen, A. Vidard, A. Weaver, A. Ryan, ... % do we keep that ? 8 8 9 9 \minitoc … … 42 42 where to obtain data and how to setup the namelist. Section~\ref{OBS_details} introduces some 43 43 more technical details of the different observation types used and also shows a more complete 44 namelist. Section~\ref{OBS_theory} introduces some of the theoretical aspects of the 45 observation operator including interpolation methods and running on multiple processors. 46 Section~\ref{OBS_obsutils} introduces some utilities to help working with the files produced 47 by the OBS code. 44 namelist. Section~\ref{OBS_theory} introduces some of the theoretical aspects of the observation 45 operator including interpolation methods and running on multiple processors. 46 Section~\ref{OBS_ooo} describes the offline observation operator code. 47 Section~\ref{OBS_obsutils} introduces some utilities to help working with the files 48 produced by the OBS code. 48 49 49 50 % ================================================================ … … 786 787 \newpage 787 788 789 % ================================================================ 790 % Offline observation operator documentation 791 % ================================================================ 792 793 %\usepackage{framed} 794 795 \section{Offline observation operator} 796 \label{OBS_ooo} 797 798 \subsection{Concept} 799 800 The obs oper maps model variables to observation space. It is possible to apply this mapping 801 without running the model. The software which performs this functionality is known as the 802 \textbf{offline obs oper}. The obs oper is divided into three stages. An initialisation phase, 803 an interpolation phase and an output phase. The implementation of which is outlined in the 804 previous sections. During the interpolation phase the offline obs oper populates the model 805 arrays by reading saved model fields from disk. 806 807 There are two ways of exploiting this offline capacity. The first is to mimic the behaviour of 808 the online system by supplying model fields at regular intervals between the start and the end 809 of the run. This approach results in a single model counterpart per observation. This kind of 810 usage produces feedback files the same file format as the online obs oper. 811 The second is to take advantage of the offline setting in which multiple model counterparts can 812 be calculated per observation. In this case it is possible to consider all forecasts verifying 813 at the same time. By forecast, I mean any method which produces an estimate of physical reality 814 which is not an observed value. In the case of class 4 files this means forecasts, analyses, persisted 815 analyses and climatological values verifying at the same time. Although the class 4 file format 816 doesn't account for multiple ensemble members or multiple experiments per observation, it is possible 817 to include these components in the same or multiple files. 818 819 %-------------------------------------------------------------------------------------------------------- 820 % offline_oper.exe 821 %-------------------------------------------------------------------------------------------------------- 822 823 \subsection{Using the offline observation operator} 824 825 \subsubsection{Building} 826 827 In addition to \emph{OPA\_SRC} the offline obs oper requires the inclusion 828 of the \emph{OOO\_SRC} directory. \emph{OOO\_SRC} contains a replacement \textbf{nemo.f90} and 829 \textbf{nemogcm.F90} which overwrites the resultant \textbf{nemo.exe}. This is the approach taken 830 by \emph{SAS\_SRC} and \emph{OFF\_SRC}. 831 832 %-------------------------------------------------------------------------------------------------------- 833 % Running 834 %-------------------------------------------------------------------------------------------------------- 835 \subsubsection{Running} 836 837 The simplest way to use the executable is to edit and append the \textbf{ooo.nml} namelist to 838 a full NEMO namelist and then to run the executable as if it were nemo.exe. 839 840 \subsubsection{Quick script} 841 842 A useful Python utility to control the namelist options can be found in \textbf{OBSTOOLS/OOO}. The 843 functions which locate model fields and observation files can be manually specified. The package 844 can be installed by appropriate use of the included setup.py script. 845 846 Documentation can be auto-generated by Sphinx by running \emph{make html} in the \textbf{doc} directory. 847 848 %-------------------------------------------------------------------------------------------------------- 849 % Configuration section 850 %-------------------------------------------------------------------------------------------------------- 851 \subsection{Configuring the offline observation operator} 852 The observation files and settings understood by \textbf{namobs} have been outlined in the online 853 obs oper section. In addition there are two further namelists wich control the operation of the offline 854 obs oper. \textbf{namooo} which controls the input model fields and \textbf{namcl4} which controls the 855 production of class 4 files. 856 857 \subsubsection{Single field} 858 859 In offline mode model arrays are populated at appropriate time steps via input files. 860 At present, \textbf{tsn} and \textbf{sshn} are populated by the default read routines. 861 These routines will be expanded upon in future versions to allow the specification of any 862 model variable. As such, input files must be global versions of the model domain with 863 \textbf{votemper}, \textbf{vosaline} and optionally \textbf{sshn} present. 864 865 For each field read there must be an entry in the \textbf{namooo} namelist specifying the 866 name of the file to read and the index along the \emph{time\_counter}. For example, to 867 read the second time counter from a single file the namelist would be. 868 869 \begin{alltt} 870 \tiny 871 \begin{verbatim} 872 !---------------------------------------------------------------------- 873 ! namooo Offline obs_oper namelist 874 !---------------------------------------------------------------------- 875 ! ooo_files specifies the files containing the model counterpart 876 ! nn_ooo_idx specifies the time_counter index within the model file 877 &namooo 878 ooo_files = "foo.nc" 879 nn_ooo_idx = 2 880 / 881 \end{verbatim} 882 \end{alltt} 883 884 \subsubsection{Multiple fields per run} 885 886 Model field iteration is controlled via \textbf{nn\_ooo\_freq} which specifies 887 the number of model steps at which the next field gets read. For example, if 888 12 hourly fields are to be interpolated in a setup where 288 steps equals 24 hours. 889 890 \begin{alltt} 891 \tiny 892 \begin{verbatim} 893 !---------------------------------------------------------------------- 894 ! namooo Offline obs_oper namelist 895 !---------------------------------------------------------------------- 896 ! ooo_files specifies the files containing the model counterpart 897 ! nn_ooo_idx specifies the time_counter index within the model file 898 ! nn_ooo_freq specifies number of time steps between read operations 899 &namooo 900 ooo_files = "foo.nc" "foo.nc" 901 nn_ooo_idx = 1 2 902 nn_ooo_freq = 144 903 / 904 \end{verbatim} 905 \end{alltt} 906 907 The above namelist will result in feedback files whose first 12 hours contain 908 the first field of foo.nc and the second 12 hours contain the second field. 909 910 %\begin{framed} 911 \textbf{Note} Missing files can be denoted as "nofile". 912 %\end{framed} 913 914 It is easy to see how a collection of fields taken fron a number of files 915 at different indices can be combined at a particular frequency in time to 916 generate a pseudo model evolution. As long as all that is needed is a single 917 model counterpart at a regular interval then namooo is all that needs to 918 be edited. However, a far more interesting approach can be taken in which 919 multiple forecasts, analyses, persisted analyses and climatologies are 920 considered against the same set of observations. For this a slightly more 921 complicated approach is needed. It is referred to as \emph{Class 4} since 922 it is the fourth metric defined by the GODAE intercomparison project. 923 924 %-------------------------------------------------------------------------------------------------------- 925 % Class 4 file section 926 %-------------------------------------------------------------------------------------------------------- 927 \subsubsection{Multiple model counterparts per observation a.k.a Class 4} 928 929 A generalisation of feedback files to allow multiple model components per observation. For a single 930 observation, as well as previous forecasts verifying at the same time there are also analyses, persisted 931 analyses and climatologies. 932 933 934 The above namelist performs two basic functions. It organises the fields 935 given in \textbf{namooo} into groups so that observations can be matched 936 up multiple times. It also controls the metadata and the output variable 937 of the class 4 file when a write routine is called. 938 939 %\begin{framed} 940 \textbf{Note: ln\_cl4} must be set to \emph{.TRUE.} in \textbf{namobs} 941 to use class 4 outputs. 942 %\end{framed} 943 944 \subsubsection{Class 4 naming convention} 945 946 The standard class 4 file naming convention is as follows. 947 948 \noindent 949 \linebreak 950 \textbf{\$\{prefix\}\_\$\{yyyymmdd\}\_\$\{sys\}\_\$\{cfg\}\_\$\{vn\}\_\$\{kind\}\_\$\{nproc\}.nc} 951 952 \noindent 953 \linebreak 954 Much of the namelist is devoted to specifying this convention. The 955 following namelist settings control the elements of the output 956 file names. Each should be specified as a single string of character data. 957 958 \begin{description} 959 \item[cl4\_prefix] 960 Prefix for class 4 files e.g. class4 961 \item[cl4\_date] 962 YYYYMMDD validity date 963 \item[cl4\_sys] 964 The name of the class 4 model system e.g. FOAM 965 \item[cl4\_cfg] 966 The name of the class 4 model configuration e.g. orca025 967 \item[cl4\_vn] 968 The name of the class 4 model version e.g. 12.0 969 \end{description} 970 971 \noindent 972 The kind is specified by the observation type internally to the obs oper. The processor 973 number is specified internally in NEMO. 974 975 \subsubsection{Class 4 file global attributes} 976 977 Global attributes necessary to fulfill the class 4 file definition. These 978 are also useful pieces of information when collaborating with external 979 partners. 980 981 \begin{description} 982 \item[cl4\_contact] 983 Contact email for class 4 files. 984 \item[cl4\_inst] 985 The name of the producers institution. 986 \item[cl4\_cfg] 987 The name of the class 4 model configuration e.g. orca025 988 \item[cl4\_vn] 989 The name of the class 4 model version e.g. 12.0 990 \end{description} 991 992 \noindent 993 The obs\_type, 994 creation date and validity time are specified internally to the obs oper. 995 996 \subsubsection{Class 4 model counterpart configuration} 997 998 As seen previously it is possible to perform a single sweep of the 999 obs oper and specify a collection of model fields equally spaced 1000 along that sweep. In the class 4 case the single sweep is replaced 1001 with multiple sweeps and a certain ammount of book keeping is 1002 needed to ensure each model counterpart makes its way to the 1003 correct piece of memory in the output files. 1004 1005 \noindent 1006 \linebreak 1007 In terms of book keeping, the offline obs oper needs to know how many 1008 full sweeps need to be performed. This is specified via the 1009 \textbf{cl4\_match\_len} variable and is the total number of model 1010 counterparts per observation. For example, a 3 forecasts plus 3 persistence 1011 fields plus an analysis field would be 7 counterparts per observation. 1012 1013 \begin{alltt} 1014 \tiny 1015 \begin{verbatim} 1016 cl4_match_len = 7 1017 \end{verbatim} 1018 \end{alltt} 1019 1020 Then to correctly allocate a class 4 file the forecast axis must be defined. This 1021 is controlled via \textbf{cl4\_fcst\_len}, which in out above example would be 3. 1022 1023 \begin{alltt} 1024 \tiny 1025 \begin{verbatim} 1026 cl4_fcst_len = 3 1027 \end{verbatim} 1028 \end{alltt} 1029 1030 Then for each model field it is necessary to designate what class 4 variable and 1031 index along the forecast dimension the model counterpart should be stored in the 1032 output file. As well as a value for that lead time in hours, this will be useful 1033 when interpreting the data afterwards. 1034 1035 \begin{alltt} 1036 \tiny 1037 \begin{verbatim} 1038 cl4_vars = "forecast" "forecast" "forecast" "persistence" "persistence" 1039 "persistence" "best_estimate" 1040 cl4_fcst_idx = 1 2 3 1 2 3 1 1041 cl4_leadtime = 12 36 60 1042 \end{verbatim} 1043 \end{alltt} 1044 1045 In terms of files and indices of fields inside each file the class 4 approach 1046 makes use of the \textbf{namooo} namelist. If our fields are in separate files 1047 with a single field per file our example inputs will be specified. 1048 1049 \begin{alltt} 1050 \tiny 1051 \begin{verbatim} 1052 ooo_files = "F.1.nc" "F.2.nc" "F.3.nc" "P.1.nc" "P.2.nc" "P.3.nc" "A.1.nc" 1053 nn_ooo_idx = 1 1 1 1 1 1 1 1054 \end{verbatim} 1055 \end{alltt} 1056 1057 When we combine all of the naming conventions, global attributes and i/o instructions 1058 the class 4 namelist becomes. 1059 1060 \begin{alltt} 1061 \tiny 1062 \begin{verbatim} 1063 !---------------------------------------------------------------------- 1064 ! namooo Offline obs_oper namelist 1065 !---------------------------------------------------------------------- 1066 ! ooo_files specifies the files containing the model counterpart 1067 ! nn_ooo_idx specifies the time_counter index within the model file 1068 ! nn_ooo_freq specifies number of time steps between read operations 1069 &namooo 1070 ooo_files = "F.1.nc" "F.2.nc" "F.3.nc" "P.1.nc" "P.2.nc" "P.3.nc" "A.1.nc" 1071 nn_ooo_idx = 1 1 1 1 1 1 1 1072 / 1073 !---------------------------------------------------------------------- 1074 ! namcl4 Offline obs_oper class 4 namelist 1075 !---------------------------------------------------------------------- 1076 ! 1077 ! Naming convention 1078 ! ----------------- 1079 ! cl4_prefix specifies the output file prefix 1080 ! cl4_date specifies the output file validity date 1081 ! cl4_sys specifies the model counterpart system 1082 ! cl4_cfg specifies the model counterpart configuration 1083 ! cl4_vn specifies the model counterpart version 1084 ! cl4_inst specifies the model counterpart institute 1085 ! cl4_contact specifies the file producers contact details 1086 ! 1087 ! I/O specification 1088 ! ----------------- 1089 ! cl4_vars specifies the names of the output file netcdf variable 1090 ! cl4_fcst_idx specifies output file forecast index 1091 ! cl4_fcst_len specifies forecast axis length 1092 ! cl4_match_len specifies number of unique matches per observation 1093 ! cl4_leadtime specifies the forecast axis lead time 1094 ! 1095 &namcl4 1096 cl4_match_len = 7 1097 cl4_fcst_len = 3 1098 cl4_fcst_idx = 1 2 3 1 2 3 1 1099 cl4_vars = "forecast" "forecast" "forecast" "persistence" "persistence" 1100 "persistence" "best_estimate" 1101 cl4_leadtime = 12 36 60 1102 cl4_prefix = "class4" 1103 cl4_date = "20130101" 1104 cl4_vn = "12.0" 1105 cl4_sys = "FOAM" 1106 cl4_cfg = "AMM7" 1107 cl4_contact = "example@example.com" 1108 cl4_inst = "UK Met Office" 1109 / 1110 \end{verbatim} 1111 \end{alltt} 1112 1113 \subsubsection{Climatology interpolation} 1114 1115 The climatological counterpart is generated at the start of the run by restarting 1116 the model from climatology through appropriate use of \textbf{namtsd}. To override 1117 the offline observation operator read routine and to take advantage of the restart 1118 settings, specify the first entry in \textbf{cl4\_vars} as "climatology". This will then 1119 pipe the restart from climatology into the output class 4 file. As in every other 1120 class 4 matchup the input file, input index and output index must be specified. 1121 These can be replaced with dummy data since they are not used but they must be 1122 present to cycle through the matchups correctly. 1123 1124 \subsection{Advanced usage} 1125 1126 In certain cases it may be desirable to include both multiple model fields per 1127 observation window with multiple match ups per observation. This can be achieved 1128 by specifying \textbf{nn\_ooo\_freq} as well as the class 4 settings. Care must 1129 be taken in generating the ooo\_files list such that the files are arranged into 1130 consecutive blocks of single match ups. For example, 2 forecast fields 1131 of 12 hourly data would result in 4 separate read operations but only 2 write 1132 operations, 1 per forecast. 1133 1134 \begin{alltt} 1135 \tiny 1136 \begin{verbatim} 1137 ooo_files = "F1.nc" "F1.nc" "F2.nc" "F2.nc" 1138 ... 1139 cl4_fcst_idx = 1 2 1140 \end{verbatim} 1141 \end{alltt} 1142 1143 The above notation reveals the internal split between match up iterators and file 1144 iterators. This technique has not been used before so experimentation is needed 1145 before results can be trusted. 1146 1147 1148 1149 1150 \newpage 1151 788 1152 \section{Observation Utilities} 789 1153 \label{OBS_obsutils} … … 801 1165 handling observation files and the feedback file output from the NEMO observation operator. 802 1166 The utilities are as follows 1167 1168 \subsubsection{c4comb} 1169 1170 The program c4comb combines multiple class 4 files produced by individual processors in an 1171 MPI run of NEMO offline obs\_oper into a single class 4 file. The program is called in the following way: 1172 1173 \begin{alltt} 1174 \footnotesize 1175 \begin{verbatim} 1176 c4comb.exe outputfile inputfile1 inputfile2 ... 1177 \end{verbatim} 1178 \end{alltt} 803 1179 804 1180 \subsubsection{corio2fb}
Note: See TracChangeset
for help on using the changeset viewer.
