Changeset 4063 for branches/2013/dev_r3987_UKMO4_OBS/DOC/TexFiles
- Timestamp:
- 2013-10-15T18:35:30+02:00 (11 years ago)
- File:
-
- 1 edited
Legend:
- Unmodified
- Added
- Removed
-
branches/2013/dev_r3987_UKMO4_OBS/DOC/TexFiles/Chapters/Chap_OBS.tex
r3294 r4063 5 5 \label{OBS} 6 6 7 Authors: D. Lea, M. Martin, K. Mogensen, A. Vidard, A. Weaver ... % do we keep that ?7 Authors: D. Lea, M. Martin, K. Mogensen, A. Vidard, A. Weaver, A. Ryan, ... % do we keep that ? 8 8 9 9 \minitoc … … 42 42 where to obtain data and how to setup the namelist. Section~\ref{OBS_details} introduces some 43 43 more technical details of the different observation types used and also shows a more complete 44 namelist. Section~\ref{OBS_theory} introduces some of the theoretical aspects of the 45 observation operator including interpolation methods and running on multiple processors. 46 Section~\ref{OBS_obsutils} introduces some utilities to help working with the files produced 47 by the OBS code. 44 namelist. Section~\ref{OBS_theory} introduces some of the theoretical aspects of the observation 45 operator including interpolation methods and running on multiple processors. 46 Section~\ref{OBS_ooo} describes the offline observation operator code. 47 Section~\ref{OBS_obsutils} introduces some utilities to help working with the files 48 produced by the OBS code. 48 49 49 50 % ================================================================ … … 786 787 \newpage 787 788 789 % ================================================================ 790 % Offline observation operator documentation 791 % ================================================================ 792 793 %\usepackage{framed} 794 795 \section{Offline observation operator} 796 \label{OBS_ooo} 797 798 \subsection{Concept} 799 800 The obs oper maps model variables to observation space. It is possible to apply this mapping 801 without running the model. The software which performs this functionality is known as the 802 \textbf{offline obs oper}. The obs oper is divided into three stages. An initialisation phase, 803 an interpolation phase and an output phase. The implementation of which is outlined in the 804 previous sections. During the interpolation phase the offline obs oper populates the model 805 arrays by reading saved model fields from disk. 806 807 There are two ways of exploiting this offline capacity. The first is to mimic the behaviour of 808 the online system by supplying model fields at regular intervals between the start and the end 809 of the run. This approach results in a single model counterpart per observation. This kind of 810 usage produces feedback files the same file format as the online obs oper. 811 The second is to take advantage of the offline setting in which multiple model counterparts can 812 be calculated per observation. In this case it is possible to consider all forecasts verifying 813 at the same time. By forecast, I mean any method which produces an estimate of physical reality 814 which is not an observed value. In the case of class 4 files this means forecasts, analyses, persisted 815 analyses and climatological values verifying at the same time. Although the class 4 file format 816 doesn't account for multiple ensemble members or multiple experiments per observation, it is possible 817 to include these components in the same or multiple files. 818 819 %-------------------------------------------------------------------------------------------------------- 820 % offline_oper.exe 821 %-------------------------------------------------------------------------------------------------------- 822 823 \subsection{Using the offline observation operator} 824 825 \subsubsection{Building} 826 827 In addition to the code required to build nemo.exe the offline obs oper requires the inclusion 828 of the \emph{OOO\_SRC directory} and \emph{offline\_oper.exe} to be specified as a target. 829 Depending on your build system this can be achieved in several ways. But for users of 830 the \textbf{makenemo} script, a flag has been added which adds OOO\_SRC to the 831 \textbf{config.txt} and which also adds a secondary target to the \textbf{bld.cfg}. 832 833 %-------------------------------------------------------------------------------------------------------- 834 % Running 835 %-------------------------------------------------------------------------------------------------------- 836 \subsubsection{Running} 837 838 The simplest way to use the executable is to edit and append the \textbf{ooo.nml} namelist to 839 a full NEMO namelist and then to run the executable as if it were nemo.exe. 840 841 %\subsubsection{Scripting} 842 % 843 %Not yet implemented. 844 845 %-------------------------------------------------------------------------------------------------------- 846 % Configuration section 847 %-------------------------------------------------------------------------------------------------------- 848 \subsection{Configuring the offline observation operator} 849 The observation files and settings understood by \textbf{namobs} have been outlined in the online 850 obs oper section. In addition there are two further namelists wich control the operation of the offline 851 obs oper. \textbf{namoff} which controls the input model fields and \textbf{namcl4} which controls the 852 production of class 4 files. 853 854 \subsubsection{Single field} 855 856 In offline mode model arrays are populated at appropriate time steps via input files. 857 At present, \textbf{tsn} and \textbf{sshn} are populated by the default read routines. 858 These routines will be expanded upon in future versions to allow the specification of any 859 model variable. As such, input files must be global versions of the model domain with 860 \textbf{votemper}, \textbf{vosaline} and optionally \textbf{sshn} present. 861 862 For each field read there must be an entry in the \textbf{namoff} namelist specifying the 863 name of the file to read and the index along the \emph{time\_counter}. For example, to 864 read the second time counter from a single file the namelist would be. 865 866 \begin{alltt} 867 \tiny 868 \begin{verbatim} 869 !---------------------------------------------------------------------- 870 ! namoff Offline obs_oper namelist 871 !---------------------------------------------------------------------- 872 ! off_files specifies the files containing the model counterpart 873 ! nn_off_idx specifies the time_counter index within the model file 874 &namoff 875 off_files = "foo.nc" 876 nn_off_idx = 2 877 / 878 \end{verbatim} 879 \end{alltt} 880 881 \subsubsection{Multiple fields per run} 882 883 Model field iteration is controlled via \textbf{nn\_off\_freq} which specifies 884 the number of model steps at which the next field gets read. For example, if 885 12 hourly fields are to be interpolated in a setup where 288 steps equals 24 hours. 886 887 \begin{alltt} 888 \tiny 889 \begin{verbatim} 890 !---------------------------------------------------------------------- 891 ! namoff Offline obs_oper namelist 892 !---------------------------------------------------------------------- 893 ! off_files specifies the files containing the model counterpart 894 ! nn_off_idx specifies the time_counter index within the model file 895 ! nn_off_freq specifies number of time steps between read operations 896 &namoff 897 off_files = "foo.nc" "foo.nc" 898 nn_off_idx = 1 2 899 nn_off_freq = 144 900 / 901 \end{verbatim} 902 \end{alltt} 903 904 The above namelist will result in feedback files whose first 12 hours contain 905 the first field of foo.nc and the second 12 hours contain the second field. 906 907 %\begin{framed} 908 \textbf{Note} Missing files can be denoted as "nofile". 909 %\end{framed} 910 911 It is easy to see how a collection of fields taken fron a number of files 912 at different indices can be combined at a particular frequency in time to 913 generate a pseudo model evolution. As long as all that is needed is a single 914 model counterpart at a regular interval then namoff is all that needs to 915 be edited. However, a far more interesting approach can be taken in which 916 multiple forecasts, analyses, persisted analyses and climatologies are 917 considered against the same set of observations. For this a slightly more 918 complicated approach is needed. It is referred to as \emph{Class 4} since 919 it is the fourth metric defined by the GODAE intercomparison project. 920 921 %-------------------------------------------------------------------------------------------------------- 922 % Class 4 file section 923 %-------------------------------------------------------------------------------------------------------- 924 \subsubsection{Multiple model counterparts per observation a.k.a Class 4} 925 926 A generalisation of feedback files to allow multiple model components per observation. For a single 927 observation, as well as previous forecasts verifying at the same time there are also analyses, persisted 928 analyses and climatologies. 929 930 931 The above namelist performs two basic functions. It organises the fields 932 given in \textbf{namoff} into groups so that observations can be matched 933 up multiple times. It also controls the metadata and the output variable 934 of the class 4 file when a write routine is called. 935 936 %\begin{framed} 937 \textbf{Note: ln\_cl4} must be set to \emph{.TRUE.} in \textbf{namobs} 938 to use class 4 outputs. 939 %\end{framed} 940 941 \subsubsection{Class 4 naming convention} 942 943 The standard class 4 file naming convention is as follows. 944 945 \noindent 946 \linebreak 947 \textbf{\$\{prefix\}\_\$\{yyyymmdd\}\_\$\{sys\}\_\$\{cfg\}\_\$\{vn\}\_\$\{kind\}\_\$\{nproc\}.nc} 948 949 \noindent 950 \linebreak 951 Much of the namelist is devoted to specifying this convention. The 952 following namelist settings control the elements of the output 953 file names. Each should be specified as a single string of character data. 954 955 \begin{description} 956 \item[cl4\_prefix] 957 Prefix for class 4 files e.g. class4 958 \item[cl4\_date] 959 YYYYMMDD validity date 960 \item[cl4\_sys] 961 The name of the class 4 model system e.g. FOAM 962 \item[cl4\_cfg] 963 The name of the class 4 model configuration e.g. orca025 964 \item[cl4\_vn] 965 The name of the class 4 model version e.g. 12.0 966 \end{description} 967 968 \noindent 969 The kind is specified by the observation type internally to the obs oper. The processor 970 number is specified internally in NEMO. 971 972 \subsubsection{Class 4 file global attributes} 973 974 Global attributes necessary to fulfill the class 4 file definition. These 975 are also useful pieces of information when collaborating with external 976 partners. 977 978 \begin{description} 979 \item[cl4\_contact] 980 Contact email for class 4 files. 981 \item[cl4\_inst] 982 The name of the producers institution. 983 \item[cl4\_cfg] 984 The name of the class 4 model configuration e.g. orca025 985 \item[cl4\_vn] 986 The name of the class 4 model version e.g. 12.0 987 \end{description} 988 989 \noindent 990 The obs\_type, 991 creation date and validity time are specified internally to the obs oper. 992 993 \subsubsection{Class 4 model counterpart configuration} 994 995 As seen previously it is possible to perform a single sweep of the 996 obs oper and specify a collection of model fields equally spaced 997 along that sweep. In the class 4 case the single sweep is replaced 998 with multiple sweeps and a certain ammount of book keeping is 999 needed to ensure each model counterpart makes its way to the 1000 correct piece of memory in the output files. 1001 1002 \noindent 1003 \linebreak 1004 In terms of book keeping, the offline obs oper needs to know how many 1005 full sweeps need to be performed. This is specified via the 1006 \textbf{cl4\_match\_len} variable and is the total number of model 1007 counterparts per observation. For example, a 3 forecasts plus 3 persistence 1008 fields plus an analysis field would be 7 counterparts per observation. 1009 1010 \begin{alltt} 1011 \tiny 1012 \begin{verbatim} 1013 cl4_match_len = 7 1014 \end{verbatim} 1015 \end{alltt} 1016 1017 Then to correctly allocate a class 4 file the forecast axis must be defined. This 1018 is controlled via \textbf{cl4\_fcst\_len}, which in out above example would be 3. 1019 1020 \begin{alltt} 1021 \tiny 1022 \begin{verbatim} 1023 cl4_fcst_len = 3 1024 \end{verbatim} 1025 \end{alltt} 1026 1027 Then for each model field it is necessary to designate what class 4 variable and 1028 index along the forecast dimension the model counterpart should be stored in the 1029 output file. As well as a value for that lead time in hours, this will be useful 1030 when interpreting the data afterwards. 1031 1032 \begin{alltt} 1033 \tiny 1034 \begin{verbatim} 1035 cl4_vars = "forecast" "forecast" "forecast" "persistence" "persistence" 1036 "persistence" "best_estimate" 1037 cl4_fcst_idx = 1 2 3 1 2 3 1 1038 cl4_leadtime = 12 36 60 1039 \end{verbatim} 1040 \end{alltt} 1041 1042 In terms of files and indices of fields inside each file the class 4 approach 1043 makes use of the \textbf{namoff} namelist. If our fields are in separate files 1044 with a single field per file our example inputs will be specified. 1045 1046 \begin{alltt} 1047 \tiny 1048 \begin{verbatim} 1049 off_files = "F.1.nc" "F.2.nc" "F.3.nc" "P.1.nc" "P.2.nc" "P.3.nc" "A.1.nc" 1050 nn_off_idx = 1 1 1 1 1 1 1 1051 \end{verbatim} 1052 \end{alltt} 1053 1054 When we combine all of the naming conventions, global attributes and i/o instructions 1055 the class 4 namelist becomes. 1056 1057 \begin{alltt} 1058 \tiny 1059 \begin{verbatim} 1060 !---------------------------------------------------------------------- 1061 ! namoff Offline obs_oper namelist 1062 !---------------------------------------------------------------------- 1063 ! off_files specifies the files containing the model counterpart 1064 ! nn_off_idx specifies the time_counter index within the model file 1065 ! nn_off_freq specifies number of time steps between read operations 1066 &namoff 1067 off_files = "F.1.nc" "F.2.nc" "F.3.nc" "P.1.nc" "P.2.nc" "P.3.nc" "A.1.nc" 1068 nn_off_idx = 1 1 1 1 1 1 1 1069 / 1070 !---------------------------------------------------------------------- 1071 ! namcl4 Offline obs_oper class 4 namelist 1072 !---------------------------------------------------------------------- 1073 ! 1074 ! Naming convention 1075 ! ----------------- 1076 ! cl4_prefix specifies the output file prefix 1077 ! cl4_date specifies the output file validity date 1078 ! cl4_sys specifies the model counterpart system 1079 ! cl4_cfg specifies the model counterpart configuration 1080 ! cl4_vn specifies the model counterpart version 1081 ! cl4_inst specifies the model counterpart institute 1082 ! cl4_contact specifies the file producers contact details 1083 ! 1084 ! I/O specification 1085 ! ----------------- 1086 ! cl4_vars specifies the names of the output file netcdf variable 1087 ! cl4_fcst_idx specifies output file forecast index 1088 ! cl4_fcst_len specifies forecast axis length 1089 ! cl4_match_len specifies number of unique matches per observation 1090 ! cl4_leadtime specifies the forecast axis lead time 1091 ! 1092 &namcl4 1093 cl4_match_len = 7 1094 cl4_fcst_len = 3 1095 cl4_fcst_idx = 1 2 3 1 2 3 1 1096 cl4_vars = "forecast" "forecast" "forecast" "persistence" "persistence" 1097 "persistence" "best_estimate" 1098 cl4_leadtime = 12 36 60 1099 cl4_prefix = "class4" 1100 cl4_date = "20130101" 1101 cl4_vn = "12.0" 1102 cl4_sys = "FOAM" 1103 cl4_cfg = "AMM7" 1104 cl4_contact = "example@example.com" 1105 cl4_inst = "UK Met Office" 1106 / 1107 \end{verbatim} 1108 \end{alltt} 1109 1110 \subsubsection{Climatology interpolation} 1111 1112 The climatological counterpart is generated at the start of the run by restarting 1113 the model from climatology through appropriate use of \textbf{namtsd}. To override 1114 the offline observation operator read routine and to take advantage of the restart 1115 settings, specify the first entry in \textbf{cl4\_vars} as "climatology". This will then 1116 pipe the restart from climatology into the output class 4 file. As in every other 1117 class 4 matchup the input file, input index and output index must be specified. 1118 These can be replaced with dummy data since they are not used but they must be 1119 present to cycle through the matchups correctly. 1120 1121 \subsection{Advanced usage} 1122 1123 In certain cases it may be desirable to include both multiple model fields per 1124 observation window with multiple match ups per observation. This can be achieved 1125 by specifying \textbf{nn\_off\_freq} as well as the class 4 settings. Care must 1126 be taken in generating the off\_files list such that the files are arranged into 1127 consecutive blocks of single match ups. For example, 2 forecast fields 1128 of 12 hourly data would result in 4 separate read operations but only 2 write 1129 operations, 1 per forecast. 1130 1131 \begin{alltt} 1132 \tiny 1133 \begin{verbatim} 1134 off_files = "F1.nc" "F1.nc" "F2.nc" "F2.nc" 1135 ... 1136 cl4_fcst_idx = 1 2 1137 \end{verbatim} 1138 \end{alltt} 1139 1140 The above notation reveals the internal split between match up iterators and file 1141 iterators. This technique has not been used before so experimentation is needed 1142 before results can be trusted. 1143 1144 1145 1146 1147 \newpage 1148 788 1149 \section{Observation Utilities} 789 1150 \label{OBS_obsutils}
Note: See TracChangeset
for help on using the changeset viewer.