New URL for NEMO forge!

Since March 2022 along with NEMO 4.2 release, the code development moved to a self-hosted GitLab.
This present forge is now archived and remained online for history.
Changeset 4063 – NEMO

Changeset 4063

2013-10-15T18:35:30+02:00 (11 years ago)

Include offline observation operator documentation.

1 edited


  • branches/2013/dev_r3987_UKMO4_OBS/DOC/TexFiles/Chapters/Chap_OBS.tex

    r3294 r4063  
    7 Authors: D. Lea, M. Martin, K. Mogensen, A. Vidard, A. Weaver...   % do we keep that ? 
     7Authors: D. Lea, M. Martin, K. Mogensen, A. Vidard, A. Weaver, A. Ryan, ...   % do we keep that ? 
    4242where to obtain data and how to setup the namelist. Section~\ref{OBS_details} introduces some 
    4343more technical details of the different observation types used and also shows a more complete 
    44 namelist. Section~\ref{OBS_theory} introduces some of the theoretical aspects of the 
    45 observation operator including interpolation methods and running on multiple processors. 
    46 Section~\ref{OBS_obsutils} introduces some utilities to help working with the files produced 
    47 by the OBS code. 
     44namelist. Section~\ref{OBS_theory} introduces some of the theoretical aspects of the observation 
     45operator including interpolation methods and running on multiple processors. 
     46Section~\ref{OBS_ooo} describes the offline observation operator code. 
     47Section~\ref{OBS_obsutils} introduces some utilities to help working with the files 
     48produced by the OBS code. 
    4950% ================================================================ 
     789% ================================================================ 
     790% Offline observation operator documentation 
     791% ================================================================ 
     795\section{Offline observation operator} 
     800The obs oper maps model variables to observation space. It is possible to apply this mapping 
     801without running the model. The software which performs this functionality is known as the 
     802\textbf{offline obs oper}. The obs oper is divided into three stages. An initialisation phase, 
     803an interpolation phase and an output phase. The implementation of which is outlined in the 
     804previous sections. During the interpolation phase the offline obs oper populates the model 
     805arrays by reading saved model fields from disk. 
     807There are two ways of exploiting this offline capacity. The first is to mimic the behaviour of 
     808the online system by supplying model fields at regular intervals between the start and the end 
     809of the run. This approach results in a single model counterpart per observation. This kind of 
     810usage produces feedback files the same file format as the online obs oper.  
     811The second is to take advantage of the offline setting in which multiple model counterparts can 
     812be calculated per observation. In this case it is possible to consider all forecasts verifying 
     813at the same time. By forecast, I mean any method which produces an estimate of physical reality 
     814which is not an observed value. In the case of class 4 files this means forecasts, analyses, persisted 
     815analyses and climatological values verifying at the same time. Although the class 4 file format 
     816doesn't account for multiple ensemble members or multiple experiments per observation, it is possible 
     817to include these components in the same or multiple files. 
     820% offline_oper.exe  
     823\subsection{Using the offline observation operator} 
     827In addition to the code required to build nemo.exe the offline obs oper requires the inclusion 
     828of the \emph{OOO\_SRC directory} and \emph{offline\_oper.exe} to be specified as a target. 
     829Depending on your build system this can be achieved in several ways. But for users of 
     830the \textbf{makenemo} script, a flag has been added which adds OOO\_SRC to the  
     831\textbf{config.txt} and which also adds a secondary target to the \textbf{bld.cfg}. 
     834% Running  
     838The simplest way to use the executable is to edit and append the \textbf{ooo.nml} namelist to 
     839a full NEMO namelist and then to run the executable as if it were nemo.exe.  
     843%Not yet implemented. 
     846% Configuration section 
     848\subsection{Configuring the offline observation operator} 
     849The observation files and settings understood by \textbf{namobs} have been outlined in the online 
     850obs oper section. In addition there are two further namelists wich control the operation of the offline 
     851obs oper. \textbf{namoff} which controls the input model fields and \textbf{namcl4} which controls the 
     852production of class 4 files.  
     854\subsubsection{Single field} 
     856In offline mode model arrays are populated at appropriate time steps via input files. 
     857At present, \textbf{tsn} and \textbf{sshn} are populated by the default read routines.  
     858These routines will be expanded upon in future versions to allow the specification of any 
     859model variable. As such, input files must be global versions of the model domain with 
     860\textbf{votemper}, \textbf{vosaline} and optionally \textbf{sshn} present. 
     862For each field read there must be an entry in the \textbf{namoff} namelist specifying the 
     863name of the file to read and the index along the \emph{time\_counter}. For example, to 
     864read the second time counter from a single file the namelist would be. 
     870!       namoff Offline obs_oper namelist 
     872!   off_files    specifies the files containing the model counterpart 
     873!   nn_off_idx   specifies the time_counter index within the model file 
     875   off_files = "" 
     876   nn_off_idx = 2 
     881\subsubsection{Multiple fields per run} 
     883Model field iteration is controlled via \textbf{nn\_off\_freq} which specifies 
     884the number of model steps at which the next field gets read. For example, if 
     88512 hourly fields are to be interpolated in a setup where 288 steps equals 24 hours. 
     891!       namoff Offline obs_oper namelist 
     893!   off_files    specifies the files containing the model counterpart 
     894!   nn_off_idx   specifies the time_counter index within the model file 
     895!   nn_off_freq  specifies number of time steps between read operations 
     897   off_files = "" "" 
     898   nn_off_idx = 1 2 
     899   nn_off_freq = 144 
     904The above namelist will result in feedback files whose first 12 hours contain 
     905the first field of and the second 12 hours contain the second field. 
     908\textbf{Note} Missing files can be denoted as "nofile". 
     911It is easy to see how a collection of fields taken fron a number of files  
     912at different indices can be combined at a particular frequency in time to  
     913generate a pseudo model evolution. As long as all that is needed is a single 
     914model counterpart at a regular interval then namoff is all that needs to 
     915be edited. However, a far more interesting approach can be taken in which 
     916multiple forecasts, analyses, persisted analyses and climatologies are 
     917considered against the same set of observations. For this a slightly more 
     918complicated approach is needed. It is referred to as \emph{Class 4} since 
     919it is the fourth metric defined by the GODAE intercomparison project. 
     922% Class 4 file section 
     924\subsubsection{Multiple model counterparts per observation a.k.a Class 4} 
     926A generalisation of feedback files to allow multiple model components per observation. For a single 
     927observation, as well as previous forecasts verifying at the same time there are also analyses, persisted 
     928analyses and climatologies.  
     931The above namelist performs two basic functions. It organises the fields 
     932given in \textbf{namoff} into groups so that observations can be matched 
     933up multiple times. It also controls the metadata and the output variable 
     934of the class 4 file when a write routine is called. 
     937\textbf{Note: ln\_cl4} must be set to \emph{.TRUE.} in \textbf{namobs}  
     938to use class 4 outputs. 
     941\subsubsection{Class 4 naming convention} 
     943The standard class 4 file naming convention is as follows. 
     951Much of the namelist is devoted to specifying this convention. The 
     952following namelist settings control the elements of the output 
     953file names. Each should be specified as a single string of character data. 
     957Prefix for class 4 files e.g. class4 
     959YYYYMMDD validity date 
     961The name of the class 4 model system e.g. FOAM 
     963The name of the class 4 model configuration e.g. orca025 
     965The name of the class 4 model version e.g. 12.0 
     969The kind is specified by the observation type internally to the obs oper. The processor 
     970number is specified internally in NEMO.  
     972\subsubsection{Class 4 file global attributes} 
     974Global attributes necessary to fulfill the class 4 file definition. These 
     975are also useful pieces of information when collaborating with external 
     980Contact email for class 4 files. 
     982The name of the producers institution. 
     984The name of the class 4 model configuration e.g. orca025 
     986The name of the class 4 model version e.g. 12.0 
     990The obs\_type, 
     991creation date and validity time are specified internally to the obs oper. 
     993\subsubsection{Class 4 model counterpart configuration} 
     995As seen previously it is possible to perform a single sweep of the 
     996obs oper and specify a collection of model fields equally spaced  
     997along that sweep. In the class 4 case the single sweep is replaced 
     998with multiple sweeps and a certain ammount of book keeping is 
     999needed to ensure each model counterpart makes its way to the  
     1000correct piece of memory in the output files. 
     1004In terms of book keeping, the offline obs oper needs to know how many 
     1005full sweeps need to be performed. This is specified via the  
     1006\textbf{cl4\_match\_len} variable and is the total number of model 
     1007counterparts per observation. For example, a 3 forecasts plus 3 persistence 
     1008fields plus an analysis field would be 7 counterparts per observation. 
     1013   cl4_match_len = 7 
     1017Then to correctly allocate a class 4 file the forecast axis must be defined. This 
     1018is controlled via \textbf{cl4\_fcst\_len}, which in out above example would be 3. 
     1023   cl4_fcst_len = 3 
     1027Then for each model field it is necessary to designate what class 4 variable and 
     1028index along the forecast dimension the model counterpart should be stored in the 
     1029output file. As well as a value for that lead time in hours, this will be useful 
     1030when interpreting the data afterwards.  
     1035   cl4_vars = "forecast" "forecast" "forecast" "persistence" "persistence" 
     1036              "persistence" "best_estimate" 
     1037   cl4_fcst_idx = 1 2 3 1 2 3 1 
     1038   cl4_leadtime = 12 36 60  
     1042In terms of files and indices of fields inside each file the class 4 approach 
     1043makes use of the \textbf{namoff} namelist. If our fields are in separate files 
     1044with a single field per file our example inputs will be specified. 
     1049   off_files = "" "" "" "" "" "" "" 
     1050   nn_off_idx = 1 1 1 1 1 1 1 
     1054When we combine all of the naming conventions, global attributes and i/o instructions 
     1055the class 4 namelist becomes. 
     1061!       namoff Offline obs_oper namelist 
     1063!   off_files    specifies the files containing the model counterpart 
     1064!   nn_off_idx   specifies the time_counter index within the model file 
     1065!   nn_off_freq  specifies number of time steps between read operations 
     1067   off_files = "" "" "" "" "" "" "" 
     1068   nn_off_idx = 1 1 1 1 1 1 1 
     1071!       namcl4 Offline obs_oper class 4 namelist 
     1074!  Naming convention 
     1075!  ----------------- 
     1076!  cl4_prefix    specifies the output file prefix 
     1077!  cl4_date      specifies the output file validity date 
     1078!  cl4_sys       specifies the model counterpart system 
     1079!  cl4_cfg       specifies the model counterpart configuration 
     1080!  cl4_vn        specifies the model counterpart version 
     1081!  cl4_inst      specifies the model counterpart institute 
     1082!  cl4_contact   specifies the file producers contact details 
     1084!  I/O specification 
     1085!  ----------------- 
     1086!  cl4_vars      specifies the names of the output file netcdf variable 
     1087!  cl4_fcst_idx  specifies output file forecast index 
     1088!  cl4_fcst_len  specifies forecast axis length 
     1089!  cl4_match_len specifies number of unique matches per observation 
     1090!  cl4_leadtime  specifies the forecast axis lead time  
     1093   cl4_match_len = 7 
     1094   cl4_fcst_len = 3 
     1095   cl4_fcst_idx = 1 2 3 1 2 3 1 
     1096   cl4_vars = "forecast" "forecast" "forecast" "persistence" "persistence" 
     1097              "persistence" "best_estimate" 
     1098   cl4_leadtime = 12 36 60 
     1099   cl4_prefix = "class4" 
     1100   cl4_date = "20130101" 
     1101   cl4_vn = "12.0" 
     1102   cl4_sys = "FOAM" 
     1103   cl4_cfg = "AMM7" 
     1104   cl4_contact = "" 
     1105   cl4_inst = "UK Met Office" 
     1110\subsubsection{Climatology interpolation} 
     1112The climatological counterpart is generated at the start of the run by restarting  
     1113the model from climatology through appropriate use of \textbf{namtsd}. To override 
     1114the offline observation operator read routine and to take advantage of the restart 
     1115settings, specify the first entry in \textbf{cl4\_vars} as "climatology". This will then 
     1116pipe the restart from climatology into the output class 4 file. As in every other 
     1117class 4 matchup the input file, input index and output index must be specified. 
     1118These can be replaced with dummy data since they are not used but they must be 
     1119present to cycle through the matchups correctly.  
     1121\subsection{Advanced usage} 
     1123In certain cases it may be desirable to include both multiple model fields per 
     1124observation window with multiple match ups per observation. This can be achieved 
     1125by specifying \textbf{nn\_off\_freq} as well as the class 4 settings. Care must 
     1126be taken in generating the off\_files list such that the files are arranged into 
     1127consecutive blocks of single match ups. For example, 2 forecast fields  
     1128of 12 hourly data would result in 4 separate read operations but only 2 write 
     1129operations, 1 per forecast. 
     1134   off_files = "" "" "" "" 
     1136   cl4_fcst_idx = 1 2 
     1140The above notation reveals the internal split between match up iterators and file 
     1141iterators. This technique has not been used before so experimentation is needed 
     1142before results can be trusted. 
    7881149\section{Observation Utilities} 
Note: See TracChangeset for help on using the changeset viewer.