Instability in results due to new compilation environment

[nicolasmartin]

Description

To resolve a crash issue on AMM12 configuration tested with the trusting on trunk, I have upgraded the Intel environment on Ada HPC from 2013.0 (ifort 13) to 2013.1 (ifort 14) for all trusting tests.
At the very next testing with the same version of the code, no more crash on AMM12 trunk but ORCA1LIM3 & AMM12 turned red on 3.6 branch. All outputs were widely modified, much more than an epsilon machine or what we could expect from this kind of upgrade.

Analysis

I have lead a comparison between the 2 releases of ifort with zero optimisation ('-O0') and then I recovered the identical results.
Maybe a part of the code is a little weak and can be alter by a newly default options either from the new compiler or either from a new optimization.

Recommendation

Not upgrading your computing environment unless you are stuck... ;-)
Several tests should be conducted in order to identify the option or optimization of the compiler and the routines implicated in this phenomenon.

[mchekki]

Analysis

AMM12 config crashes using intel version 13, for O2 optimization, adding -check bounds solves the problem.

Moving down to O0 optimizations works also but this should be avoided as it slows down the run.

Moving to intel 15 (and higher) , no need to add -check bounds with O2 optimization.So probably the crash is due to a bug in version 13...

Still we get differences with solver.stat even using -O0, moving from version 15 to 16 ..(13->15 OK, 16->17 OK)

Intel recommendation for reproducibility of results

This is valid for processors with the same architecture.

Starting intel version 17, use :

-fp-model consistent

Before version 17 :

-fp-model source -fimf-arch-consistency=true -no-fma

I compared the solver.stat , with version 13/15/16/17 using "-O2 -r8 -traceback -fp-model source -fimf-arch-consistency=true -no-fma" and no differences were observed