Opened 6 weeks ago

Closed 3 weeks ago

#2556 closed Bug (fixed)

Sette test not catching failed REPRO runs

Reported by: ayoung Owned by: ayoung
Priority: low Milestone:
Component: tools Version: trunk
Severity: minor Keywords:



Sette test did not catch failed reproducibility run and reported the task as "passed".


The sette script finds the two REPRO jobs to compare by grepping the folder for "REPRO", assuming that there will be two matches.

# check reproducibility
rep1=ls -1rt $vdir/$nam/$mach/$dorv/ | grep REPRO | tail -2l | head -1
rep2=ls -1rt $vdir/$nam/$mach/$dorv/ | grep REPRO | tail -1l

This is fine if both jobs run or if neither job runs. However, just one of the jobs fail, there will only be one "REPRO*" directory in the folder hence rep1 and rep2 will have the same value. The result is that sette will compare the single REPRO job against itself and reports the job as "passed".


Check that rep1 != rep2 before continuing.

Commit History (1)


Catching the scenario where one REPRO run is successful and the other fails. See ticket #2556

Change History (3)

comment:1 Changed 5 weeks ago by ayoung

I propose the following change:

# check reproducibility
rep1=`ls -1rt $vdir/$nam/$mach/$dorv/ | grep REPRO | tail -2l | head -1 `
rep2=`ls -1rt $vdir/$nam/$mach/$dorv/ | grep REPRO | tail -1l`
+ if [ $rep1 == $rep2 ]; then
+    rep2=''
+ fi

i.e. if the rep1 and rep2 are pointing to the same REPRO directories, set the second one to an empty string.

comment:2 Changed 3 weeks ago by ayoung

In 13785:

Catching the scenario where one REPRO run is successful and the other fails. See ticket #2556

comment:3 Changed 3 weeks ago by ayoung

  • Resolution set to fixed
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.