New URL for NEMO forge!

Since March 2022 along with NEMO 4.2 release, the code development moved to a self-hosted GitLab.
This present forge is now archived and remained online for history.
Changeset 4245 for branches/2013/dev_LOCEAN_CMCC_INGV_MERC_UKMO_2013/DOC/TexFiles/Chapters/Chap_OBS.tex – NEMO

2013-11-19T12:19:21+01:00 (10 years ago)

dev_locean_cmcc_ingv_ukmo_merc : merge in the MERC_UKMO dev branch with trunk rev 4119

1 edited


  • branches/2013/dev_LOCEAN_CMCC_INGV_MERC_UKMO_2013/DOC/TexFiles/Chapters/Chap_OBS.tex

    r4147 r4245  
    7 Authors: D. Lea, M. Martin, K. Mogensen, A. Vidard, A. Weaver...   % do we keep that ? 
     7Authors: D. Lea, M. Martin, K. Mogensen, A. Vidard, A. Weaver, A. Ryan, ...   % do we keep that ? 
    4242where to obtain data and how to setup the namelist. Section~\ref{OBS_details} introduces some 
    4343more technical details of the different observation types used and also shows a more complete 
    44 namelist. Section~\ref{OBS_theory} introduces some of the theoretical aspects of the 
    45 observation operator including interpolation methods and running on multiple processors. 
    46 Section~\ref{OBS_obsutils} introduces some utilities to help working with the files produced 
    47 by the OBS code. 
     44namelist. Section~\ref{OBS_theory} introduces some of the theoretical aspects of the observation 
     45operator including interpolation methods and running on multiple processors. 
     46Section~\ref{OBS_ooo} describes the offline observation operator code. 
     47Section~\ref{OBS_obsutils} introduces some utilities to help working with the files 
     48produced by the OBS code. 
    4950% ================================================================ 
     790% ================================================================ 
     791% Offline observation operator documentation 
     792% ================================================================ 
     796\section{Offline observation operator} 
     801The obs oper maps model variables to observation space. It is possible to apply this mapping 
     802without running the model. The software which performs this functionality is known as the 
     803\textbf{offline obs oper}. The obs oper is divided into three stages. An initialisation phase, 
     804an interpolation phase and an output phase. The implementation of which is outlined in the 
     805previous sections. During the interpolation phase the offline obs oper populates the model 
     806arrays by reading saved model fields from disk. 
     808There are two ways of exploiting this offline capacity. The first is to mimic the behaviour of 
     809the online system by supplying model fields at regular intervals between the start and the end 
     810of the run. This approach results in a single model counterpart per observation. This kind of 
     811usage produces feedback files the same file format as the online obs oper.  
     812The second is to take advantage of the offline setting in which multiple model counterparts can 
     813be calculated per observation. In this case it is possible to consider all forecasts verifying 
     814at the same time. By forecast, I mean any method which produces an estimate of physical reality 
     815which is not an observed value. In the case of class 4 files this means forecasts, analyses, persisted 
     816analyses and climatological values verifying at the same time. Although the class 4 file format 
     817doesn't account for multiple ensemble members or multiple experiments per observation, it is possible 
     818to include these components in the same or multiple files. 
     821% offline_oper.exe  
     824\subsection{Using the offline observation operator} 
     828In addition to \emph{OPA\_SRC} the offline obs oper requires the inclusion 
     829of the \emph{OOO\_SRC} directory. \emph{OOO\_SRC} contains a replacement \textbf{nemo.f90} and 
     830\textbf{nemogcm.F90} which overwrites the resultant \textbf{nemo.exe}. This is the approach taken 
     831by \emph{SAS\_SRC} and \emph{OFF\_SRC}. 
     834% Running  
     838The simplest way to use the executable is to edit and append the \textbf{ooo.nml} namelist to 
     839a full NEMO namelist and then to run the executable as if it were nemo.exe.  
     841\subsubsection{Quick script} 
     843A useful Python utility to control the namelist options can be found in \textbf{OBSTOOLS/OOO}. The 
     844functions which locate model fields and observation files can be manually specified. The package 
     845can be installed by appropriate use of the included script. 
     847Documentation can be auto-generated by Sphinx by running \emph{make html} in the \textbf{doc} directory. 
     850% Configuration section 
     852\subsection{Configuring the offline observation operator} 
     853The observation files and settings understood by \textbf{namobs} have been outlined in the online 
     854obs oper section. In addition there are two further namelists wich control the operation of the offline 
     855obs oper. \textbf{namooo} which controls the input model fields and \textbf{namcl4} which controls the 
     856production of class 4 files.  
     858\subsubsection{Single field} 
     860In offline mode model arrays are populated at appropriate time steps via input files. 
     861At present, \textbf{tsn} and \textbf{sshn} are populated by the default read routines.  
     862These routines will be expanded upon in future versions to allow the specification of any 
     863model variable. As such, input files must be global versions of the model domain with 
     864\textbf{votemper}, \textbf{vosaline} and optionally \textbf{sshn} present. 
     866For each field read there must be an entry in the \textbf{namooo} namelist specifying the 
     867name of the file to read and the index along the \emph{time\_counter}. For example, to 
     868read the second time counter from a single file the namelist would be. 
     874!       namooo Offline obs_oper namelist 
     876!   ooo_files    specifies the files containing the model counterpart 
     877!   nn_ooo_idx   specifies the time_counter index within the model file 
     879   ooo_files = "" 
     880   nn_ooo_idx = 2 
     885\subsubsection{Multiple fields per run} 
     887Model field iteration is controlled via \textbf{nn\_ooo\_freq} which specifies 
     888the number of model steps at which the next field gets read. For example, if 
     88912 hourly fields are to be interpolated in a setup where 288 steps equals 24 hours. 
     895!       namooo Offline obs_oper namelist 
     897!   ooo_files    specifies the files containing the model counterpart 
     898!   nn_ooo_idx   specifies the time_counter index within the model file 
     899!   nn_ooo_freq  specifies number of time steps between read operations 
     901   ooo_files = "" "" 
     902   nn_ooo_idx = 1 2 
     903   nn_ooo_freq = 144 
     908The above namelist will result in feedback files whose first 12 hours contain 
     909the first field of and the second 12 hours contain the second field. 
     912\textbf{Note} Missing files can be denoted as "nofile". 
     915It is easy to see how a collection of fields taken fron a number of files  
     916at different indices can be combined at a particular frequency in time to  
     917generate a pseudo model evolution. As long as all that is needed is a single 
     918model counterpart at a regular interval then namooo is all that needs to 
     919be edited. However, a far more interesting approach can be taken in which 
     920multiple forecasts, analyses, persisted analyses and climatologies are 
     921considered against the same set of observations. For this a slightly more 
     922complicated approach is needed. It is referred to as \emph{Class 4} since 
     923it is the fourth metric defined by the GODAE intercomparison project. 
     926% Class 4 file section 
     928\subsubsection{Multiple model counterparts per observation a.k.a Class 4} 
     930A generalisation of feedback files to allow multiple model components per observation. For a single 
     931observation, as well as previous forecasts verifying at the same time there are also analyses, persisted 
     932analyses and climatologies.  
     935The above namelist performs two basic functions. It organises the fields 
     936given in \textbf{namooo} into groups so that observations can be matched 
     937up multiple times. It also controls the metadata and the output variable 
     938of the class 4 file when a write routine is called. 
     941\textbf{Note: ln\_cl4} must be set to \emph{.TRUE.} in \textbf{namobs}  
     942to use class 4 outputs. 
     945\subsubsection{Class 4 naming convention} 
     947The standard class 4 file naming convention is as follows. 
     955Much of the namelist is devoted to specifying this convention. The 
     956following namelist settings control the elements of the output 
     957file names. Each should be specified as a single string of character data. 
     961Prefix for class 4 files e.g. class4 
     963YYYYMMDD validity date 
     965The name of the class 4 model system e.g. FOAM 
     967The name of the class 4 model configuration e.g. orca025 
     969The name of the class 4 model version e.g. 12.0 
     973The kind is specified by the observation type internally to the obs oper. The processor 
     974number is specified internally in NEMO.  
     976\subsubsection{Class 4 file global attributes} 
     978Global attributes necessary to fulfill the class 4 file definition. These 
     979are also useful pieces of information when collaborating with external 
     984Contact email for class 4 files. 
     986The name of the producers institution. 
     988The name of the class 4 model configuration e.g. orca025 
     990The name of the class 4 model version e.g. 12.0 
     994The obs\_type, 
     995creation date and validity time are specified internally to the obs oper. 
     997\subsubsection{Class 4 model counterpart configuration} 
     999As seen previously it is possible to perform a single sweep of the 
     1000obs oper and specify a collection of model fields equally spaced  
     1001along that sweep. In the class 4 case the single sweep is replaced 
     1002with multiple sweeps and a certain ammount of book keeping is 
     1003needed to ensure each model counterpart makes its way to the  
     1004correct piece of memory in the output files. 
     1008In terms of book keeping, the offline obs oper needs to know how many 
     1009full sweeps need to be performed. This is specified via the  
     1010\textbf{cl4\_match\_len} variable and is the total number of model 
     1011counterparts per observation. For example, a 3 forecasts plus 3 persistence 
     1012fields plus an analysis field would be 7 counterparts per observation. 
     1017   cl4_match_len = 7 
     1021Then to correctly allocate a class 4 file the forecast axis must be defined. This 
     1022is controlled via \textbf{cl4\_fcst\_len}, which in out above example would be 3. 
     1027   cl4_fcst_len = 3 
     1031Then for each model field it is necessary to designate what class 4 variable and 
     1032index along the forecast dimension the model counterpart should be stored in the 
     1033output file. As well as a value for that lead time in hours, this will be useful 
     1034when interpreting the data afterwards.  
     1039   cl4_vars = "forecast" "forecast" "forecast" "persistence" "persistence" 
     1040              "persistence" "best_estimate" 
     1041   cl4_fcst_idx = 1 2 3 1 2 3 1 
     1042   cl4_leadtime = 12 36 60  
     1046In terms of files and indices of fields inside each file the class 4 approach 
     1047makes use of the \textbf{namooo} namelist. If our fields are in separate files 
     1048with a single field per file our example inputs will be specified. 
     1053   ooo_files = "" "" "" "" "" "" "" 
     1054   nn_ooo_idx = 1 1 1 1 1 1 1 
     1058When we combine all of the naming conventions, global attributes and i/o instructions 
     1059the class 4 namelist becomes. 
     1065!       namooo Offline obs_oper namelist 
     1067!   ooo_files    specifies the files containing the model counterpart 
     1068!   nn_ooo_idx   specifies the time_counter index within the model file 
     1069!   nn_ooo_freq  specifies number of time steps between read operations 
     1071   ooo_files = "" "" "" "" "" "" "" 
     1072   nn_ooo_idx = 1 1 1 1 1 1 1 
     1075!       namcl4 Offline obs_oper class 4 namelist 
     1078!  Naming convention 
     1079!  ----------------- 
     1080!  cl4_prefix    specifies the output file prefix 
     1081!  cl4_date      specifies the output file validity date 
     1082!  cl4_sys       specifies the model counterpart system 
     1083!  cl4_cfg       specifies the model counterpart configuration 
     1084!  cl4_vn        specifies the model counterpart version 
     1085!  cl4_inst      specifies the model counterpart institute 
     1086!  cl4_contact   specifies the file producers contact details 
     1088!  I/O specification 
     1089!  ----------------- 
     1090!  cl4_vars      specifies the names of the output file netcdf variable 
     1091!  cl4_fcst_idx  specifies output file forecast index 
     1092!  cl4_fcst_len  specifies forecast axis length 
     1093!  cl4_match_len specifies number of unique matches per observation 
     1094!  cl4_leadtime  specifies the forecast axis lead time  
     1097   cl4_match_len = 7 
     1098   cl4_fcst_len = 3 
     1099   cl4_fcst_idx = 1 2 3 1 2 3 1 
     1100   cl4_vars = "forecast" "forecast" "forecast" "persistence" "persistence" 
     1101              "persistence" "best_estimate" 
     1102   cl4_leadtime = 12 36 60 
     1103   cl4_prefix = "class4" 
     1104   cl4_date = "20130101" 
     1105   cl4_vn = "12.0" 
     1106   cl4_sys = "FOAM" 
     1107   cl4_cfg = "AMM7" 
     1108   cl4_contact = "" 
     1109   cl4_inst = "UK Met Office" 
     1114\subsubsection{Climatology interpolation} 
     1116The climatological counterpart is generated at the start of the run by restarting  
     1117the model from climatology through appropriate use of \textbf{namtsd}. To override 
     1118the offline observation operator read routine and to take advantage of the restart 
     1119settings, specify the first entry in \textbf{cl4\_vars} as "climatology". This will then 
     1120pipe the restart from climatology into the output class 4 file. As in every other 
     1121class 4 matchup the input file, input index and output index must be specified. 
     1122These can be replaced with dummy data since they are not used but they must be 
     1123present to cycle through the matchups correctly.  
     1125\subsection{Advanced usage} 
     1127In certain cases it may be desirable to include both multiple model fields per 
     1128observation window with multiple match ups per observation. This can be achieved 
     1129by specifying \textbf{nn\_ooo\_freq} as well as the class 4 settings. Care must 
     1130be taken in generating the ooo\_files list such that the files are arranged into 
     1131consecutive blocks of single match ups. For example, 2 forecast fields  
     1132of 12 hourly data would result in 4 separate read operations but only 2 write 
     1133operations, 1 per forecast. 
     1138   ooo_files = "" "" "" "" 
     1140   cl4_fcst_idx = 1 2 
     1144The above notation reveals the internal split between match up iterators and file 
     1145iterators. This technique has not been used before so experimentation is needed 
     1146before results can be trusted. 
    7891153\section{Observation Utilities} 
    8021166handling observation files and the feedback file output from the NEMO observation operator. 
    8031167The utilities are as follows 
     1171The program c4comb combines multiple class 4 files produced by individual processors in an 
     1172MPI run of NEMO offline obs\_oper into a single class 4 file. The program is called in the following way: 
     1177c4comb.exe outputfile inputfile1 inputfile2 ... 
Note: See TracChangeset for help on using the changeset viewer.