New URL for NEMO forge!   http://forge.nemo-ocean.eu

Since March 2022 along with NEMO 4.2 release, the code development moved to a self-hosted GitLab.
This present forge is now archived and remained online for history.
Changeset 4245 for branches/2013/dev_LOCEAN_CMCC_INGV_MERC_UKMO_2013/DOC – NEMO

Ignore:
Timestamp:
2013-11-19T12:19:21+01:00 (11 years ago)
Author:
cetlod
Message:

dev_locean_cmcc_ingv_ukmo_merc : merge in the MERC_UKMO dev branch with trunk rev 4119

File:
1 edited

Legend:

Unmodified
Added
Removed
  • branches/2013/dev_LOCEAN_CMCC_INGV_MERC_UKMO_2013/DOC/TexFiles/Chapters/Chap_OBS.tex

    r4147 r4245  
    55\label{OBS} 
    66 
    7 Authors: D. Lea, M. Martin, K. Mogensen, A. Vidard, A. Weaver...   % do we keep that ? 
     7Authors: D. Lea, M. Martin, K. Mogensen, A. Vidard, A. Weaver, A. Ryan, ...   % do we keep that ? 
    88 
    99\minitoc 
     
    4242where to obtain data and how to setup the namelist. Section~\ref{OBS_details} introduces some 
    4343more technical details of the different observation types used and also shows a more complete 
    44 namelist. Section~\ref{OBS_theory} introduces some of the theoretical aspects of the 
    45 observation operator including interpolation methods and running on multiple processors. 
    46 Section~\ref{OBS_obsutils} introduces some utilities to help working with the files produced 
    47 by the OBS code. 
     44namelist. Section~\ref{OBS_theory} introduces some of the theoretical aspects of the observation 
     45operator including interpolation methods and running on multiple processors. 
     46Section~\ref{OBS_ooo} describes the offline observation operator code. 
     47Section~\ref{OBS_obsutils} introduces some utilities to help working with the files 
     48produced by the OBS code. 
    4849 
    4950% ================================================================ 
     
    787788\newpage 
    788789 
     790% ================================================================ 
     791% Offline observation operator documentation 
     792% ================================================================ 
     793 
     794%\usepackage{framed} 
     795 
     796\section{Offline observation operator} 
     797\label{OBS_ooo} 
     798 
     799\subsection{Concept} 
     800 
     801The obs oper maps model variables to observation space. It is possible to apply this mapping 
     802without running the model. The software which performs this functionality is known as the 
     803\textbf{offline obs oper}. The obs oper is divided into three stages. An initialisation phase, 
     804an interpolation phase and an output phase. The implementation of which is outlined in the 
     805previous sections. During the interpolation phase the offline obs oper populates the model 
     806arrays by reading saved model fields from disk. 
     807 
     808There are two ways of exploiting this offline capacity. The first is to mimic the behaviour of 
     809the online system by supplying model fields at regular intervals between the start and the end 
     810of the run. This approach results in a single model counterpart per observation. This kind of 
     811usage produces feedback files the same file format as the online obs oper.  
     812The second is to take advantage of the offline setting in which multiple model counterparts can 
     813be calculated per observation. In this case it is possible to consider all forecasts verifying 
     814at the same time. By forecast, I mean any method which produces an estimate of physical reality 
     815which is not an observed value. In the case of class 4 files this means forecasts, analyses, persisted 
     816analyses and climatological values verifying at the same time. Although the class 4 file format 
     817doesn't account for multiple ensemble members or multiple experiments per observation, it is possible 
     818to include these components in the same or multiple files. 
     819 
     820%-------------------------------------------------------------------------------------------------------- 
     821% offline_oper.exe  
     822%-------------------------------------------------------------------------------------------------------- 
     823 
     824\subsection{Using the offline observation operator} 
     825 
     826\subsubsection{Building} 
     827 
     828In addition to \emph{OPA\_SRC} the offline obs oper requires the inclusion 
     829of the \emph{OOO\_SRC} directory. \emph{OOO\_SRC} contains a replacement \textbf{nemo.f90} and 
     830\textbf{nemogcm.F90} which overwrites the resultant \textbf{nemo.exe}. This is the approach taken 
     831by \emph{SAS\_SRC} and \emph{OFF\_SRC}. 
     832 
     833%-------------------------------------------------------------------------------------------------------- 
     834% Running  
     835%-------------------------------------------------------------------------------------------------------- 
     836\subsubsection{Running} 
     837 
     838The simplest way to use the executable is to edit and append the \textbf{ooo.nml} namelist to 
     839a full NEMO namelist and then to run the executable as if it were nemo.exe.  
     840 
     841\subsubsection{Quick script} 
     842 
     843A useful Python utility to control the namelist options can be found in \textbf{OBSTOOLS/OOO}. The 
     844functions which locate model fields and observation files can be manually specified. The package 
     845can be installed by appropriate use of the included setup.py script. 
     846 
     847Documentation can be auto-generated by Sphinx by running \emph{make html} in the \textbf{doc} directory. 
     848 
     849%-------------------------------------------------------------------------------------------------------- 
     850% Configuration section 
     851%-------------------------------------------------------------------------------------------------------- 
     852\subsection{Configuring the offline observation operator} 
     853The observation files and settings understood by \textbf{namobs} have been outlined in the online 
     854obs oper section. In addition there are two further namelists wich control the operation of the offline 
     855obs oper. \textbf{namooo} which controls the input model fields and \textbf{namcl4} which controls the 
     856production of class 4 files.  
     857 
     858\subsubsection{Single field} 
     859 
     860In offline mode model arrays are populated at appropriate time steps via input files. 
     861At present, \textbf{tsn} and \textbf{sshn} are populated by the default read routines.  
     862These routines will be expanded upon in future versions to allow the specification of any 
     863model variable. As such, input files must be global versions of the model domain with 
     864\textbf{votemper}, \textbf{vosaline} and optionally \textbf{sshn} present. 
     865 
     866For each field read there must be an entry in the \textbf{namooo} namelist specifying the 
     867name of the file to read and the index along the \emph{time\_counter}. For example, to 
     868read the second time counter from a single file the namelist would be. 
     869 
     870\begin{alltt} 
     871\tiny 
     872\begin{verbatim}  
     873!---------------------------------------------------------------------- 
     874!       namooo Offline obs_oper namelist 
     875!---------------------------------------------------------------------- 
     876!   ooo_files    specifies the files containing the model counterpart 
     877!   nn_ooo_idx   specifies the time_counter index within the model file 
     878&namooo 
     879   ooo_files = "foo.nc" 
     880   nn_ooo_idx = 2 
     881/ 
     882\end{verbatim}  
     883\end{alltt} 
     884 
     885\subsubsection{Multiple fields per run} 
     886 
     887Model field iteration is controlled via \textbf{nn\_ooo\_freq} which specifies 
     888the number of model steps at which the next field gets read. For example, if 
     88912 hourly fields are to be interpolated in a setup where 288 steps equals 24 hours. 
     890 
     891\begin{alltt} 
     892\tiny 
     893\begin{verbatim}  
     894!---------------------------------------------------------------------- 
     895!       namooo Offline obs_oper namelist 
     896!---------------------------------------------------------------------- 
     897!   ooo_files    specifies the files containing the model counterpart 
     898!   nn_ooo_idx   specifies the time_counter index within the model file 
     899!   nn_ooo_freq  specifies number of time steps between read operations 
     900&namooo 
     901   ooo_files = "foo.nc" "foo.nc" 
     902   nn_ooo_idx = 1 2 
     903   nn_ooo_freq = 144 
     904/ 
     905\end{verbatim}  
     906\end{alltt} 
     907 
     908The above namelist will result in feedback files whose first 12 hours contain 
     909the first field of foo.nc and the second 12 hours contain the second field. 
     910 
     911%\begin{framed} 
     912\textbf{Note} Missing files can be denoted as "nofile". 
     913%\end{framed} 
     914 
     915It is easy to see how a collection of fields taken fron a number of files  
     916at different indices can be combined at a particular frequency in time to  
     917generate a pseudo model evolution. As long as all that is needed is a single 
     918model counterpart at a regular interval then namooo is all that needs to 
     919be edited. However, a far more interesting approach can be taken in which 
     920multiple forecasts, analyses, persisted analyses and climatologies are 
     921considered against the same set of observations. For this a slightly more 
     922complicated approach is needed. It is referred to as \emph{Class 4} since 
     923it is the fourth metric defined by the GODAE intercomparison project. 
     924 
     925%-------------------------------------------------------------------------------------------------------- 
     926% Class 4 file section 
     927%-------------------------------------------------------------------------------------------------------- 
     928\subsubsection{Multiple model counterparts per observation a.k.a Class 4} 
     929 
     930A generalisation of feedback files to allow multiple model components per observation. For a single 
     931observation, as well as previous forecasts verifying at the same time there are also analyses, persisted 
     932analyses and climatologies.  
     933 
     934 
     935The above namelist performs two basic functions. It organises the fields 
     936given in \textbf{namooo} into groups so that observations can be matched 
     937up multiple times. It also controls the metadata and the output variable 
     938of the class 4 file when a write routine is called. 
     939 
     940%\begin{framed} 
     941\textbf{Note: ln\_cl4} must be set to \emph{.TRUE.} in \textbf{namobs}  
     942to use class 4 outputs. 
     943%\end{framed} 
     944 
     945\subsubsection{Class 4 naming convention} 
     946 
     947The standard class 4 file naming convention is as follows. 
     948 
     949\noindent 
     950\linebreak 
     951\textbf{\$\{prefix\}\_\$\{yyyymmdd\}\_\$\{sys\}\_\$\{cfg\}\_\$\{vn\}\_\$\{kind\}\_\$\{nproc\}.nc} 
     952 
     953\noindent 
     954\linebreak 
     955Much of the namelist is devoted to specifying this convention. The 
     956following namelist settings control the elements of the output 
     957file names. Each should be specified as a single string of character data. 
     958 
     959\begin{description} 
     960\item[cl4\_prefix] 
     961Prefix for class 4 files e.g. class4 
     962\item[cl4\_date] 
     963YYYYMMDD validity date 
     964\item[cl4\_sys] 
     965The name of the class 4 model system e.g. FOAM 
     966\item[cl4\_cfg] 
     967The name of the class 4 model configuration e.g. orca025 
     968\item[cl4\_vn] 
     969The name of the class 4 model version e.g. 12.0 
     970\end{description} 
     971 
     972\noindent 
     973The kind is specified by the observation type internally to the obs oper. The processor 
     974number is specified internally in NEMO.  
     975 
     976\subsubsection{Class 4 file global attributes} 
     977 
     978Global attributes necessary to fulfill the class 4 file definition. These 
     979are also useful pieces of information when collaborating with external 
     980partners. 
     981 
     982\begin{description} 
     983\item[cl4\_contact] 
     984Contact email for class 4 files. 
     985\item[cl4\_inst] 
     986The name of the producers institution. 
     987\item[cl4\_cfg] 
     988The name of the class 4 model configuration e.g. orca025 
     989\item[cl4\_vn] 
     990The name of the class 4 model version e.g. 12.0 
     991\end{description} 
     992 
     993\noindent 
     994The obs\_type, 
     995creation date and validity time are specified internally to the obs oper. 
     996 
     997\subsubsection{Class 4 model counterpart configuration} 
     998 
     999As seen previously it is possible to perform a single sweep of the 
     1000obs oper and specify a collection of model fields equally spaced  
     1001along that sweep. In the class 4 case the single sweep is replaced 
     1002with multiple sweeps and a certain ammount of book keeping is 
     1003needed to ensure each model counterpart makes its way to the  
     1004correct piece of memory in the output files. 
     1005 
     1006\noindent 
     1007\linebreak 
     1008In terms of book keeping, the offline obs oper needs to know how many 
     1009full sweeps need to be performed. This is specified via the  
     1010\textbf{cl4\_match\_len} variable and is the total number of model 
     1011counterparts per observation. For example, a 3 forecasts plus 3 persistence 
     1012fields plus an analysis field would be 7 counterparts per observation. 
     1013 
     1014\begin{alltt} 
     1015\tiny 
     1016\begin{verbatim} 
     1017   cl4_match_len = 7 
     1018\end{verbatim} 
     1019\end{alltt} 
     1020 
     1021Then to correctly allocate a class 4 file the forecast axis must be defined. This 
     1022is controlled via \textbf{cl4\_fcst\_len}, which in out above example would be 3. 
     1023 
     1024\begin{alltt} 
     1025\tiny 
     1026\begin{verbatim} 
     1027   cl4_fcst_len = 3 
     1028\end{verbatim} 
     1029\end{alltt} 
     1030 
     1031Then for each model field it is necessary to designate what class 4 variable and 
     1032index along the forecast dimension the model counterpart should be stored in the 
     1033output file. As well as a value for that lead time in hours, this will be useful 
     1034when interpreting the data afterwards.  
     1035 
     1036\begin{alltt} 
     1037\tiny 
     1038\begin{verbatim} 
     1039   cl4_vars = "forecast" "forecast" "forecast" "persistence" "persistence" 
     1040              "persistence" "best_estimate" 
     1041   cl4_fcst_idx = 1 2 3 1 2 3 1 
     1042   cl4_leadtime = 12 36 60  
     1043\end{verbatim} 
     1044\end{alltt} 
     1045 
     1046In terms of files and indices of fields inside each file the class 4 approach 
     1047makes use of the \textbf{namooo} namelist. If our fields are in separate files 
     1048with a single field per file our example inputs will be specified. 
     1049 
     1050\begin{alltt} 
     1051\tiny 
     1052\begin{verbatim} 
     1053   ooo_files = "F.1.nc" "F.2.nc" "F.3.nc" "P.1.nc" "P.2.nc" "P.3.nc" "A.1.nc" 
     1054   nn_ooo_idx = 1 1 1 1 1 1 1 
     1055\end{verbatim} 
     1056\end{alltt} 
     1057 
     1058When we combine all of the naming conventions, global attributes and i/o instructions 
     1059the class 4 namelist becomes. 
     1060 
     1061\begin{alltt} 
     1062\tiny 
     1063\begin{verbatim} 
     1064!---------------------------------------------------------------------- 
     1065!       namooo Offline obs_oper namelist 
     1066!---------------------------------------------------------------------- 
     1067!   ooo_files    specifies the files containing the model counterpart 
     1068!   nn_ooo_idx   specifies the time_counter index within the model file 
     1069!   nn_ooo_freq  specifies number of time steps between read operations 
     1070&namooo 
     1071   ooo_files = "F.1.nc" "F.2.nc" "F.3.nc" "P.1.nc" "P.2.nc" "P.3.nc" "A.1.nc" 
     1072   nn_ooo_idx = 1 1 1 1 1 1 1 
     1073/ 
     1074!---------------------------------------------------------------------- 
     1075!       namcl4 Offline obs_oper class 4 namelist 
     1076!---------------------------------------------------------------------- 
     1077! 
     1078!  Naming convention 
     1079!  ----------------- 
     1080!  cl4_prefix    specifies the output file prefix 
     1081!  cl4_date      specifies the output file validity date 
     1082!  cl4_sys       specifies the model counterpart system 
     1083!  cl4_cfg       specifies the model counterpart configuration 
     1084!  cl4_vn        specifies the model counterpart version 
     1085!  cl4_inst      specifies the model counterpart institute 
     1086!  cl4_contact   specifies the file producers contact details 
     1087! 
     1088!  I/O specification 
     1089!  ----------------- 
     1090!  cl4_vars      specifies the names of the output file netcdf variable 
     1091!  cl4_fcst_idx  specifies output file forecast index 
     1092!  cl4_fcst_len  specifies forecast axis length 
     1093!  cl4_match_len specifies number of unique matches per observation 
     1094!  cl4_leadtime  specifies the forecast axis lead time  
     1095! 
     1096&namcl4 
     1097   cl4_match_len = 7 
     1098   cl4_fcst_len = 3 
     1099   cl4_fcst_idx = 1 2 3 1 2 3 1 
     1100   cl4_vars = "forecast" "forecast" "forecast" "persistence" "persistence" 
     1101              "persistence" "best_estimate" 
     1102   cl4_leadtime = 12 36 60 
     1103   cl4_prefix = "class4" 
     1104   cl4_date = "20130101" 
     1105   cl4_vn = "12.0" 
     1106   cl4_sys = "FOAM" 
     1107   cl4_cfg = "AMM7" 
     1108   cl4_contact = "example@example.com" 
     1109   cl4_inst = "UK Met Office" 
     1110/ 
     1111\end{verbatim} 
     1112\end{alltt} 
     1113 
     1114\subsubsection{Climatology interpolation} 
     1115 
     1116The climatological counterpart is generated at the start of the run by restarting  
     1117the model from climatology through appropriate use of \textbf{namtsd}. To override 
     1118the offline observation operator read routine and to take advantage of the restart 
     1119settings, specify the first entry in \textbf{cl4\_vars} as "climatology". This will then 
     1120pipe the restart from climatology into the output class 4 file. As in every other 
     1121class 4 matchup the input file, input index and output index must be specified. 
     1122These can be replaced with dummy data since they are not used but they must be 
     1123present to cycle through the matchups correctly.  
     1124 
     1125\subsection{Advanced usage} 
     1126 
     1127In certain cases it may be desirable to include both multiple model fields per 
     1128observation window with multiple match ups per observation. This can be achieved 
     1129by specifying \textbf{nn\_ooo\_freq} as well as the class 4 settings. Care must 
     1130be taken in generating the ooo\_files list such that the files are arranged into 
     1131consecutive blocks of single match ups. For example, 2 forecast fields  
     1132of 12 hourly data would result in 4 separate read operations but only 2 write 
     1133operations, 1 per forecast. 
     1134 
     1135\begin{alltt} 
     1136\tiny 
     1137\begin{verbatim} 
     1138   ooo_files = "F1.nc" "F1.nc" "F2.nc" "F2.nc" 
     1139... 
     1140   cl4_fcst_idx = 1 2 
     1141\end{verbatim} 
     1142\end{alltt} 
     1143 
     1144The above notation reveals the internal split between match up iterators and file 
     1145iterators. This technique has not been used before so experimentation is needed 
     1146before results can be trusted. 
     1147 
     1148 
     1149 
     1150 
     1151\newpage 
     1152 
    7891153\section{Observation Utilities} 
    7901154\label{OBS_obsutils} 
     
    8021166handling observation files and the feedback file output from the NEMO observation operator. 
    8031167The utilities are as follows 
     1168 
     1169\subsubsection{c4comb} 
     1170 
     1171The program c4comb combines multiple class 4 files produced by individual processors in an 
     1172MPI run of NEMO offline obs\_oper into a single class 4 file. The program is called in the following way: 
     1173 
     1174\begin{alltt} 
     1175\footnotesize 
     1176\begin{verbatim} 
     1177c4comb.exe outputfile inputfile1 inputfile2 ... 
     1178\end{verbatim} 
     1179\end{alltt} 
    8041180 
    8051181\subsubsection{corio2fb} 
Note: See TracChangeset for help on using the changeset viewer.