ORCHIDEE trunk rev 2247 with LMDZ5 trunk rev 2076
This page is dedicated to technical validation of the model. For scientific validation, see more simulations here: testing/rev2247/simulations?
Parallelism
- ORCHIDEE gives the identical results for simulations with different number of cores MPI and/or OMP. It does not matter if the executable is compiled for pur MPI mode or for mixt MPI/OMP mode.
- At curie, compile options must be updated, see below
- Simulations coupled to LMDZ do not succeed in debug mode (and not dev at ada). This seems be due to following change on LMDZ : http://web.lmd.jussieu.fr/trac/changeset/2011/LMDZ5/trunk/libf/phylmd/coefcdrag.F90. The problem seems to be only in the first time-step.
- The executable compiled in pure MPI and in mixt MPI_OMP gives the same results if the same optimization mode is used (prod/dev/debug)
- Sequential mode is tested with gfortran : simulations are reproducible
ADA/IDRIS
- Compilation en mode prod, dev and debug are successful for pure mpi and mixt mpi_omp.
- Coupled mode in dev and debug mode: simulations stops
- Changing the number of cores MPI and/or OMP does not change the results. For example running on 23MPI=32MPI=16MPI.2OMP
- The executable compiled in pure MPI and in hybrid MPI_OMP gives the same results if the same optimization mode is used (prod/dev/debug)
- Using optimization mode prod does not give the same results as optimization mode dev or debug
- Using optimization mode dev or debug gives the same results
CURIE/TGCC
- Compilation en mode prod, dev and debug are successful for pure mpi and mixt mpi_omp.
- Offline mode: all simulations are identical : compile mode, different number of cores MPI, pure mpi or mixt mpi_omp (but only running on 1 omp)
- Coupled mode:
- Simulations are not identical while changing the number of cores MPI and or OMP also without ORCHIDEE using LMDZ bucket
- In debug mode: simulations stops (stops when ORCHIDEE is activated, ok with bucket)
- Coupled mode with modifications in compile options for LMDZ as in commit http://web.lmd.jussieu.fr/trac/changeset/2108:
- Simulations are identical while changing the number of cores MPI and or OMP
- Simulations are identical between prod, dev and debug compile options
- Simulations still stops in debug mode for LMDZOR simulations
local PC using gfortran
- Compiling with gfortran for sequential mode : prod, dev and debug is ok.
- Simulations are reproductive
- Tests are done with ORCHIDEE offline Choisnel and CWRR, for prod, dev and debug mode
- Tests are done in coupled mode for prod mode
Restartablilty
- Same simulation set up is reproducible (see above)
- 1+1 not = 2 in coupled and in offline mode
Performance
LMDZOR at curie
Following performances are observed using LMDZ r2076 modified by adding -fp-model precise in compile options, with AP(old physics) and NPv3.2(CMIP5 version of new physics) :
Resolution | LMDZ physics | Nb cores | Consumption per 10 year | Elapsed time per 10 year |
96x95x39 | AP | 32MPI*1OMP | 740hCpu | 23h |
96x95x39 | NPv3.2 | 32MPI*1OMP | 1220hCpu | 38h |
96x95x39 | AP | 32MPI*2OMP | 850hCpu | 13h |
96x95x39 | AP | 32MPI*4OMP | 1100hCpu | 8h30min |
Note:
- Using Choisnel or CWRR do not change the run time in coupled mode.
- No difference in time using executable MPI or MPI_OMP on the same number of cores MPI and 1OMP.
ORCHIDEE offline global at curie with CRU-NCEP
Resolution | Hydrology | Nb cores MPI | Consumption per 10 year | Elapsed time for 10 year | set up with spinup analytic |
1deg | Choisnel | 32MPI | 52h | 1.5h | no |
1deg | CWRR | 32MPI | 65h | 2h | no |
0.5deg | Choisnel | 32MPI | 180h | 5.5h | no |
0.5deg | CWRR | 32MPI | 270h | 8.5h | no |
0.5deg | Choisnel | 64MPI | 240h | 3.8h | no |
0.5deg | CWRR | 64MPI | 340h | 5.3h | no |
0.5deg | Choisnel | 64MPI | 310h | 4.8h | yes |
0.5deg | CWRR | 64MPI | 460h | 7.2h | yes |
Note:
- The first year, starting without restart files is more than 2 times longer for CWRR than the following years. For Choisnel it is about 1.5 times longer.
- The 2 last lines are using the experiment set up with spinup analytic. This set up also activates less output. The cpu consumption is higher for an unkknown reason. To be investigated.
Details of simulations
Default set up
LMDZ is always run with it's old physics package (called AP). ORCHIDEE is always run activating Stomate. Tests are done with both Choisnel and CWRR hydrology.
ADA/IDRIS
The model is installed by the trusting here : /workgpfs/rech/lmd/rlmd979/TRUSTING/ORCHIDEE/RUNDIR/trunk_rev2247_20140717T1820/modipsl
****** LMDZORChoi en prod OK (les test bigtest=n) *********************************************************************************************** results.out.cross:DIRECTORIES OK : prod_mpi/23mpi = prod_mpi_omp/16mpi_4omp.1 results.out.prod_mpi:DIRECTORIES OK : prod_mpi/23mpi = prod_mpi/32mpi results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/16mpi_4omp.1 = prod_mpi_omp/16mpi_4omp.2 results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/16mpi_4omp.1 = prod_mpi_omp/16mpi_4omp.3 results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/16mpi_4omp.1 = prod_mpi_omp/16mpi_8omp results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/16mpi_4omp.1 = prod_mpi_omp/32mpi_1omp results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/16mpi_4omp.1 = prod_mpi_omp/32mpi_2omp ****** LMDZORCWRR en prod tout OK *********************************************************************************************** results.out.cross:DIRECTORIES OK : prod_mpi/23mpi = prod_mpi_omp/16mpi_4omp.1 results.out.prod_mpi:DIRECTORIES OK : prod_mpi/23mpi = prod_mpi/32mpi results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/16mpi_4omp.1 = prod_mpi_omp/16mpi_4omp.2 results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/16mpi_4omp.1 = prod_mpi_omp/16mpi_4omp.3 results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/16mpi_4omp.1 = prod_mpi_omp/16mpi_8omp results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/16mpi_4omp.1 = prod_mpi_omp/32mpi_1omp results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/16mpi_4omp.1 = prod_mpi_omp/32mpi_2omp ****** LMDZORChoi en dev not OK *********************************************************************************************** results.out.dev_mpi:Following 0 DIRECTORIES finished: results.out.dev_mpi:Following 2 DIRECTORIES DID NOT FINISH: 23mpi 32mpi ****** ORChoi prod/dev/debug OK ****** Compilation debug=dev mais pas debug=prod ou dev=prod ****** *********************************************************************************************** results.out.cross:DIRECTORIES OK : debug_mpi/23mpi = debug_mpi_omp/10mpi_1omp.1 results.out.cross:DIRECTORIES OK : debug_mpi/23mpi = dev_mpi/23mpi results.out.cross:DIRECTORIES OK : debug_mpi/23mpi = dev_mpi_omp/10mpi_1omp.1 results.out.cross:DIRECTORIES DIFFER : debug_mpi/23mpi NOT = prod_mpi/23mpi results.out.cross:DIRECTORIES DIFFER : debug_mpi/23mpi NOT = prod_mpi_omp/10mpi_1omp.1 results.out.debug_mpi:DIRECTORIES OK : debug_mpi/23mpi = debug_mpi/32mpi results.out.debug_mpi_omp:DIRECTORIES OK : debug_mpi_omp/10mpi_1omp.1 = debug_mpi_omp/10mpi_1omp.2 results.out.debug_mpi_omp:DIRECTORIES OK : debug_mpi_omp/10mpi_1omp.1 = debug_mpi_omp/10mpi_1omp.3 results.out.debug_mpi_omp:DIRECTORIES OK : debug_mpi_omp/10mpi_1omp.1 = debug_mpi_omp/32mpi_1omp results.out.dev_mpi:DIRECTORIES OK : dev_mpi/23mpi = dev_mpi/32mpi results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/10mpi_1omp.1 = dev_mpi_omp/10mpi_1omp.2 results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/10mpi_1omp.1 = dev_mpi_omp/10mpi_1omp.3 results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/10mpi_1omp.1 = dev_mpi_omp/32mpi_1omp results.out.prod_mpi:DIRECTORIES OK : prod_mpi/23mpi = prod_mpi/32mpi results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/10mpi_1omp.1 = prod_mpi_omp/10mpi_1omp.2 results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/10mpi_1omp.1 = prod_mpi_omp/10mpi_1omp.3 results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/10mpi_1omp.1 = prod_mpi_omp/32mpi_1omp ****** ORCWRR OK *********************************************************************************************** results.out.cross:DIRECTORIES OK : debug_mpi/23mpi = debug_mpi_omp/10mpi_1omp.1 results.out.cross:DIRECTORIES OK : debug_mpi/23mpi = dev_mpi/23mpi results.out.cross:DIRECTORIES OK : debug_mpi/23mpi = dev_mpi_omp/10mpi_1omp.1 results.out.cross:DIRECTORIES DIFFER : debug_mpi/23mpi NOT = prod_mpi/23mpi results.out.cross:DIRECTORIES DIFFER : debug_mpi/23mpi NOT = prod_mpi_omp/10mpi_1omp.1 results.out.debug_mpi:DIRECTORIES OK : debug_mpi/23mpi = debug_mpi/32mpi results.out.debug_mpi_omp:DIRECTORIES OK : debug_mpi_omp/10mpi_1omp.1 = debug_mpi_omp/10mpi_1omp.2 results.out.debug_mpi_omp:DIRECTORIES OK : debug_mpi_omp/10mpi_1omp.1 = debug_mpi_omp/10mpi_1omp.3 results.out.debug_mpi_omp:DIRECTORIES OK : debug_mpi_omp/10mpi_1omp.1 = debug_mpi_omp/32mpi_1omp results.out.dev_mpi:DIRECTORIES OK : dev_mpi/23mpi = dev_mpi/32mpi results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/10mpi_1omp.1 = dev_mpi_omp/10mpi_1omp.2 results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/10mpi_1omp.1 = dev_mpi_omp/10mpi_1omp.3 results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/10mpi_1omp.1 = dev_mpi_omp/32mpi_1omp results.out.prod_mpi:DIRECTORIES OK : prod_mpi/23mpi = prod_mpi/32mpi results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/10mpi_1omp.1 = prod_mpi_omp/10mpi_1omp.2 results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/10mpi_1omp.1 = prod_mpi_omp/10mpi_1omp.3 results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/10mpi_1omp.1 = prod_mpi_omp/32mpi_1omp
CURIE/TGCC
The model is installed by the trusting here : /ccc/scratch/cont003/dsm/p86ghatt/TRUSTING/ORCHIDEE/RUNDIR/trunk_rev2247_20140717T1824/modipsl
****** LMDZORCWRR ****** debug : simulations did not finish ****** prod and dev : all simulations finished, results differs when changing the number of cores MPI/OMP. ****** Simulations are reproductable at same number of cores. ************************************************************************************************************* results.out.debug_mpi:Following 0 DIRECTORIES finished: results.out.debug_mpi:Following 2 DIRECTORIES DID NOT FINISH: 23mpi 32mpi results.out.debug_mpi_omp:Following 0 DIRECTORIES finished: results.out.debug_mpi_omp:Following 3 DIRECTORIES DID NOT FINISH: 16mpi_8omp 32mpi_1omp 32mpi_2omp results.out.dev_mpi:Following 2 DIRECTORIES finished: 23mpi 32mpi results.out.dev_mpi:Following 0 DIRECTORIES DID NOT FINISH: results.out.dev_mpi:DIRECTORIES DIFFER : dev_mpi/23mpi NOT = dev_mpi/32mpi results.out.dev_mpi_omp:Following 3 DIRECTORIES finished: 16mpi_8omp 32mpi_1omp 32mpi_2omp results.out.dev_mpi_omp:Following 0 DIRECTORIES DID NOT FINISH: results.out.dev_mpi_omp:DIRECTORIES DIFFER : dev_mpi_omp/16mpi_8omp NOT = dev_mpi_omp/32mpi_1omp results.out.dev_mpi_omp:DIRECTORIES DIFFER : dev_mpi_omp/16mpi_8omp NOT = dev_mpi_omp/32mpi_2omp results.out.prod_mpi:Following 2 DIRECTORIES finished: 23mpi 32mpi results.out.prod_mpi:Following 0 DIRECTORIES DID NOT FINISH: results.out.prod_mpi:DIRECTORIES DIFFER : prod_mpi/23mpi NOT = prod_mpi/32mpi results.out.prod_mpi_omp:Following 6 DIRECTORIES finished: 16mpi_4omp.1 16mpi_4omp.2 16mpi_4omp.3 16mpi_8omp 32mpi_1omp 32mpi_2omp results.out.prod_mpi_omp:Following 0 DIRECTORIES DID NOT FINISH: results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/16mpi_4omp.1 = prod_mpi_omp/16mpi_4omp.2 results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/16mpi_4omp.1 = prod_mpi_omp/16mpi_4omp.3 results.out.prod_mpi_omp:DIRECTORIES DIFFER : prod_mpi_omp/16mpi_4omp.1 NOT = prod_mpi_omp/16mpi_8omp results.out.prod_mpi_omp:DIRECTORIES DIFFER : prod_mpi_omp/16mpi_4omp.1 NOT = prod_mpi_omp/32mpi_1omp results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/16mpi_4omp.1 = prod_mpi_omp/32mpi_2omp ****** LMDZORChoi ****** debug: No simulations finished ****** prod and dev: simulations finished, results differs when changing the number of cores MPI/OMP. ****** Simulations are reprodusable at same number of cores. ************************************************************************************************************* results.out.cross:DIRECTORIES DIFFER : dev_mpi/23mpi NOT = dev_mpi_omp/16mpi_8omp results.out.cross:DIRECTORIES DIFFER : dev_mpi/23mpi NOT = prod_mpi/23mpi results.out.cross:DIRECTORIES DIFFER : dev_mpi/23mpi NOT = prod_mpi_omp/16mpi_1omp results.out.debug_mpi:Following 0 DIRECTORIES finished: results.out.debug_mpi:Following 2 DIRECTORIES DID NOT FINISH: 23mpi 32mpi results.out.debug_mpi_omp:Following 0 DIRECTORIES finished: results.out.debug_mpi_omp:Following 3 DIRECTORIES DID NOT FINISH: 16mpi_8omp 32mpi_1omp 32mpi_2omp results.out.dev_mpi:DIRECTORIES DIFFER : dev_mpi/23mpi NOT = dev_mpi/32mpi results.out.dev_mpi_omp:DIRECTORIES DIFFER : dev_mpi_omp/16mpi_8omp NOT = dev_mpi_omp/32mpi_1omp results.out.dev_mpi_omp:DIRECTORIES DIFFER : dev_mpi_omp/16mpi_8omp NOT = dev_mpi_omp/32mpi_2omp results.out.prod_mpi:Following 2 DIRECTORIES finished: 23mpi 32mpi results.out.prod_mpi:Following 0 DIRECTORIES DID NOT FINISH: results.out.prod_mpi:DIRECTORIES DIFFER : prod_mpi/23mpi NOT = prod_mpi/32mpi results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/16mpi_1omp = prod_mpi_omp/16mpi_2omp results.out.prod_mpi_omp:DIRECTORIES DIFFER : prod_mpi_omp/16mpi_1omp NOT = prod_mpi_omp/16mpi_4omp.1 results.out.prod_mpi_omp:DIRECTORIES DIFFER : prod_mpi_omp/16mpi_1omp NOT = prod_mpi_omp/16mpi_4omp.2 results.out.prod_mpi_omp:DIRECTORIES DIFFER : prod_mpi_omp/16mpi_1omp NOT = prod_mpi_omp/16mpi_4omp.3 results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/16mpi_4omp.1 = prod_mpi_omp/16mpi_4omp.2 results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/16mpi_4omp.1 = prod_mpi_omp/16mpi_4omp.3 results.out.prod_mpi_omp:DIRECTORIES DIFFER : prod_mpi_omp/16mpi_1omp NOT = prod_mpi_omp/16mpi_8omp results.out.prod_mpi_omp:DIRECTORIES DIFFER : prod_mpi_omp/16mpi_1omp NOT = prod_mpi_omp/1mpi_1omp results.out.prod_mpi_omp:DIRECTORIES DIFFER : prod_mpi_omp/16mpi_1omp NOT = prod_mpi_omp/32mpi_1omp results.out.prod_mpi_omp:DIRECTORIES DIFFER : prod_mpi_omp/16mpi_1omp NOT = prod_mpi_omp/32mpi_2omp results.out.prod_mpi_omp:DIRECTORIES DIFFER : prod_mpi_omp/16mpi_1omp NOT = prod_mpi_omp/32mpi_4omp results.out.prod_mpi_omp:DIRECTORIES DIFFER : prod_mpi_omp/16mpi_1omp NOT = prod_mpi_omp/32mpi_8omp results.out.prod_mpi_omp:DIRECTORIES DIFFER : prod_mpi_omp/16mpi_1omp NOT = prod_mpi_omp/4mpi_1omp results.out.prod_mpi_omp:DIRECTORIES DIFFER : prod_mpi_omp/16mpi_1omp NOT = prod_mpi_omp/8mpi_1omp ****** ORChoi ****** All simulations finished. All simulations are identic. ************************************************************************************************************* results.out.cross:DIRECTORIES OK : debug_mpi/23mpi = debug_mpi_omp/10mpi_1omp.1 results.out.cross:DIRECTORIES OK : debug_mpi/23mpi = dev_mpi/16mpi results.out.cross:DIRECTORIES OK : debug_mpi/23mpi = dev_mpi_omp/10mpi_1omp.1 results.out.cross:DIRECTORIES OK : debug_mpi/23mpi = prod_mpi/16mpi results.out.cross:DIRECTORIES OK : debug_mpi/23mpi = prod_mpi_omp/10mpi_1omp.1 results.out.debug_mpi:DIRECTORIES OK : debug_mpi/23mpi = debug_mpi/32mpi results.out.debug_mpi_omp:DIRECTORIES OK : debug_mpi_omp/10mpi_1omp.1 = debug_mpi_omp/10mpi_1omp.2 results.out.debug_mpi_omp:DIRECTORIES OK : debug_mpi_omp/10mpi_1omp.1 = debug_mpi_omp/10mpi_1omp.3 results.out.debug_mpi_omp:DIRECTORIES OK : debug_mpi_omp/10mpi_1omp.1 = debug_mpi_omp/32mpi_1omp results.out.dev_mpi:DIRECTORIES OK : dev_mpi/16mpi = dev_mpi/18mpi results.out.dev_mpi:DIRECTORIES OK : dev_mpi/16mpi = dev_mpi/1mpi results.out.dev_mpi:DIRECTORIES OK : dev_mpi/16mpi = dev_mpi/20mpi results.out.dev_mpi:DIRECTORIES OK : dev_mpi/16mpi = dev_mpi/23mpi results.out.dev_mpi:DIRECTORIES OK : dev_mpi/16mpi = dev_mpi/32mpi results.out.dev_mpi:DIRECTORIES OK : dev_mpi/16mpi = dev_mpi/4mpi results.out.dev_mpi:DIRECTORIES OK : dev_mpi/16mpi = dev_mpi/8mpi results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/10mpi_1omp.1 = dev_mpi_omp/10mpi_1omp.2 results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/10mpi_1omp.1 = dev_mpi_omp/10mpi_1omp.3 results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/10mpi_1omp.1 = dev_mpi_omp/16mpi_1omp results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/10mpi_1omp.1 = dev_mpi_omp/1mpi_1omp results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/10mpi_1omp.1 = dev_mpi_omp/2mpi_1omp results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/10mpi_1omp.1 = dev_mpi_omp/32mpi_1omp results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/10mpi_1omp.1 = dev_mpi_omp/4mpi_1omp results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/10mpi_1omp.1 = dev_mpi_omp/8mpi_1omp results.out.prod_mpi:DIRECTORIES OK : prod_mpi/16mpi = prod_mpi/18mpi results.out.prod_mpi:DIRECTORIES OK : prod_mpi/16mpi = prod_mpi/1mpi results.out.prod_mpi:DIRECTORIES OK : prod_mpi/16mpi = prod_mpi/20mpi results.out.prod_mpi:DIRECTORIES OK : prod_mpi/16mpi = prod_mpi/23mpi results.out.prod_mpi:DIRECTORIES OK : prod_mpi/16mpi = prod_mpi/32mpi results.out.prod_mpi:DIRECTORIES OK : prod_mpi/16mpi = prod_mpi/4mpi results.out.prod_mpi:DIRECTORIES OK : prod_mpi/16mpi = prod_mpi/8mpi results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/10mpi_1omp.1 = prod_mpi_omp/10mpi_1omp.2 results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/10mpi_1omp.1 = prod_mpi_omp/10mpi_1omp.3 results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/10mpi_1omp.1 = prod_mpi_omp/16mpi_1omp results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/10mpi_1omp.1 = prod_mpi_omp/1mpi_1omp results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/10mpi_1omp.1 = prod_mpi_omp/2mpi_1omp results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/10mpi_1omp.1 = prod_mpi_omp/32mpi_1omp results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/10mpi_1omp.1 = prod_mpi_omp/4mpi_1omp results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/10mpi_1omp.1 = prod_mpi_omp/8mpi_1omp ****** ORCWRR ****** All simulation that finished gives the same results : compile mode, number of cores MPI and or OMP ****** debug: Not all simulations finished. This is probably due to a cpu time exit. The option CHECK_CWRR is very time consuming. ****** prod: OK. All simulations finished. Same results changing the number of cores MPI/OMP. Pur mpi gives the same results as mixt mpi_omp ****** Simulations are reprodusable at same number of cores. ************************************************************************************************************* results.out.cross:DIRECTORIES OK : debug_mpi_omp/10mpi_1omp.1 = dev_mpi/23mpi results.out.cross:DIRECTORIES OK : debug_mpi_omp/10mpi_1omp.1 = dev_mpi_omp/10mpi_1omp.1 results.out.cross:DIRECTORIES OK : debug_mpi_omp/10mpi_1omp.1 = prod_mpi/23mpi results.out.cross:DIRECTORIES OK : debug_mpi_omp/10mpi_1omp.1 = prod_mpi_omp/10mpi_1omp.1 results.out.debug_mpi:Following 0 DIRECTORIES finished: results.out.debug_mpi:Following 2 DIRECTORIES DID NOT FINISH: 23mpi 32mpi results.out.debug_mpi_omp:Following 4 DIRECTORIES finished: 10mpi_1omp.1 10mpi_1omp.2 10mpi_1omp.3 32mpi_1omp results.out.debug_mpi_omp:Following 0 DIRECTORIES DID NOT FINISH: results.out.debug_mpi_omp:DIRECTORIES OK : debug_mpi_omp/10mpi_1omp.1 = debug_mpi_omp/10mpi_1omp.2 results.out.debug_mpi_omp:DIRECTORIES OK : debug_mpi_omp/10mpi_1omp.1 = debug_mpi_omp/10mpi_1omp.3 results.out.debug_mpi_omp:DIRECTORIES OK : debug_mpi_omp/10mpi_1omp.1 = debug_mpi_omp/32mpi_1omp results.out.dev_mpi:Following 2 DIRECTORIES finished: 23mpi 32mpi results.out.dev_mpi:Following 0 DIRECTORIES DID NOT FINISH: results.out.dev_mpi:DIRECTORIES OK : dev_mpi/23mpi = dev_mpi/32mpi results.out.dev_mpi_omp:Following 4 DIRECTORIES finished: 10mpi_1omp.1 10mpi_1omp.2 10mpi_1omp.3 32mpi_1omp results.out.dev_mpi_omp:Following 0 DIRECTORIES DID NOT FINISH: results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/10mpi_1omp.1 = dev_mpi_omp/10mpi_1omp.2 results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/10mpi_1omp.1 = dev_mpi_omp/10mpi_1omp.3 results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/10mpi_1omp.1 = dev_mpi_omp/32mpi_1omp results.out.prod_mpi:Following 2 DIRECTORIES finished: 23mpi 32mpi results.out.prod_mpi:Following 0 DIRECTORIES DID NOT FINISH: results.out.prod_mpi:DIRECTORIES OK : prod_mpi/23mpi = prod_mpi/32mpi results.out.prod_mpi_omp:Following 4 DIRECTORIES finished: 10mpi_1omp.1 10mpi_1omp.2 10mpi_1omp.3 32mpi_1omp results.out.prod_mpi_omp:Following 0 DIRECTORIES DID NOT FINISH: results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/10mpi_1omp.1 = prod_mpi_omp/10mpi_1omp.2 results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/10mpi_1omp.1 = prod_mpi_omp/10mpi_1omp.3 results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/10mpi_1omp.1 = prod_mpi_omp/32mpi_1omp ****** LMDZ with bucket (no ORCHIDEE) ****** All simulation finished ****** Simulations with same number of cores are identical ****** Simulations are not the same while changing the number of cores MPI and/or OMP ****** Debug simulations seems to be identical while changing the number of cores MPI and/or OMP ************************************************************************************************************* results.out.cross:DIRECTORIES OK : debug_mpi/23mpi = debug_mpi_omp/16mpi_4omp.1 results.out.cross:DIRECTORIES DIFFER : debug_mpi/23mpi NOT = dev_mpi/23mpi results.out.cross:DIRECTORIES DIFFER : debug_mpi/23mpi NOT = dev_mpi_omp/16mpi_4omp.1 results.out.cross:DIRECTORIES DIFFER : debug_mpi/23mpi NOT = prod_mpi/23mpi results.out.cross:DIRECTORIES DIFFER : debug_mpi/23mpi NOT = prod_mpi_omp/16mpi_4omp.1 results.out.cross:DIRECTORIES DIFFER : prod_mpi/23mpi NOT = prod_mpi_omp/16mpi_4omp.1 results.out.cross:DIRECTORIES DIFFER : dev_mpi/23mpi NOT = dev_mpi_omp/16mpi_4omp.1 results.out.cross:DIRECTORIES OK : debug_mpi/23mpi = debug_mpi_omp/16mpi_4omp.1 results.out.debug_mpi:DIRECTORIES OK : debug_mpi/23mpi = debug_mpi/32mpi results.out.debug_mpi_omp:DIRECTORIES OK : debug_mpi_omp/16mpi_4omp.1 = debug_mpi_omp/16mpi_4omp.2 results.out.debug_mpi_omp:DIRECTORIES OK : debug_mpi_omp/16mpi_4omp.1 = debug_mpi_omp/16mpi_4omp.3 results.out.debug_mpi_omp:DIRECTORIES OK : debug_mpi_omp/16mpi_4omp.1 = debug_mpi_omp/16mpi_8omp results.out.debug_mpi_omp:DIRECTORIES OK : debug_mpi_omp/16mpi_4omp.1 = debug_mpi_omp/32mpi_1omp results.out.debug_mpi_omp:DIRECTORIES OK : debug_mpi_omp/16mpi_4omp.1 = debug_mpi_omp/32mpi_2omp results.out.dev_mpi:DIRECTORIES DIFFER : dev_mpi/23mpi NOT = dev_mpi/32mpi results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/16mpi_4omp.1 = dev_mpi_omp/16mpi_4omp.2 results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/16mpi_4omp.1 = dev_mpi_omp/16mpi_4omp.3 results.out.dev_mpi_omp:DIRECTORIES DIFFER : dev_mpi_omp/16mpi_4omp.1 NOT = dev_mpi_omp/16mpi_8omp results.out.dev_mpi_omp:DIRECTORIES DIFFER : dev_mpi_omp/16mpi_4omp.1 NOT = dev_mpi_omp/32mpi_1omp results.out.dev_mpi_omp:DIRECTORIES DIFFER : dev_mpi_omp/16mpi_4omp.1 NOT = dev_mpi_omp/32mpi_2omp results.out.prod_mpi:DIRECTORIES DIFFER : prod_mpi/23mpi NOT = prod_mpi/32mpi results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/16mpi_4omp.1 = prod_mpi_omp/16mpi_4omp.2 results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/16mpi_4omp.1 = prod_mpi_omp/16mpi_4omp.3 results.out.prod_mpi_omp:DIRECTORIES DIFFER : prod_mpi_omp/16mpi_4omp.1 NOT = prod_mpi_omp/16mpi_8omp results.out.prod_mpi_omp:DIRECTORIES DIFFER : prod_mpi_omp/16mpi_4omp.1 NOT = prod_mpi_omp/32mpi_1omp results.out.prod_mpi_omp:DIRECTORIES DIFFER : prod_mpi_omp/16mpi_4omp.1 NOT = prod_mpi_omp/32mpi_2omp
gfortran/ private pclinux laptop
The model is installed here /home/jgipsl/TRUSTING/ORCHIDEE/RUNDIR/trunk_rev2247_20140812T1536/modipsl