Sette test not catching failed REPRO runs

[ayoung]

Context

Sette test did not catch failed reproducibility run and reported the task as "passed".

Analysis

The sette script finds the two REPRO jobs to compare by grepping the folder for "REPRO", assuming that there will be two matches.

# check reproducibility
rep1=ls -1rt $vdir/$nam/$mach/$dorv/ | grep REPRO | tail -2l | head -1
rep2=ls -1rt $vdir/$nam/$mach/$dorv/ | grep REPRO | tail -1l

This is fine if both jobs run or if neither job runs. However, just one of the jobs fail, there will only be one "REPRO*" directory in the folder hence rep1 and rep2 will have the same value. The result is that sette will compare the single REPRO job against itself and reports the job as "passed".

Fix

Check that rep1 != rep2 before continuing.

[ayoung]

I propose the following change:

# check reproducibility
rep1=`ls -1rt $vdir/$nam/$mach/$dorv/ | grep REPRO | tail -2l | head -1 `
rep2=`ls -1rt $vdir/$nam/$mach/$dorv/ | grep REPRO | tail -1l`
+ if [ $rep1 == $rep2 ]; then
+    rep2=''
+ fi
f1o=$vdir/$nam/$mach/$dorv/$rep1/ocean.output
f1s=$vdir/$nam/$mach/$dorv/$rep1/run.stat
f1t=$vdir/$nam/$mach/$dorv/$rep1/tracer.stat
f2o=$vdir/$nam/$mach/$dorv/$rep2/ocean.output
f2s=$vdir/$nam/$mach/$dorv/$rep2/run.stat
f2t=$vdir/$nam/$mach/$dorv/$rep2/tracer.stat

i.e. if the rep1 and rep2 are pointing to the same REPRO directories, set the second one to an empty string.

[ayoung]

In 13785:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found