Back to the main page.

Bug 1420 - test_ft_freqanalysis because stored data differs from newly compute data

Reported 2012-04-12 16:47:00 +0200
Modified 2014-03-12 12:21:38 +0100
Product: FieldTrip
Component: core
Version: unspecified
Hardware: PC
Operating System: Windows
Importance: P3 normal
Assigned to: Jörn M. Horschig
Depends on:
See also:

Jörn M. Horschig - 2012-04-12 16:47:16 +0200

every single assertion fails (i.e. for all datatypes). the stored freq in \home\common\matlab\fieldtrip\data\test\latest\freq\ always differs from the data that is computed on-the-fly. sometimes, difference are big, sometimes small, sometimes for many samples, sometimes for few. but we are dealing with mathematics here, which should be exact, so what goes wrong? It is not revision 5106, which made detrending default behaviour. I don't feel like going back 3000 revisions, testing every single change that occured... I already went back to r4608 in Google code, and I cannot see what might have changed. the stored data is from last year (feb-apr 2011) Boris, do you have information from what revision onwards this test failed?

Boris Reuderink - 2012-04-25 14:07:47 +0200

Unfortunately, I don't have that information. We could try to bisect the history, but that would also take a considerable investment. I can help you with this, if needed --- if there is no other way.

Boris Reuderink - 2012-05-04 13:52:53 +0200

Okay, I just met Jörn, and we can't easily identify a revision in which the test script ran successfully. This might be caused by a change in the path for the reference results --- which is not versioned in the repo. The best approach is probably to re-save the latest results, and work backwards in time to so when it starts to fail again.

Jörn M. Horschig - 2013-03-20 15:37:21 +0100

this got fixed in one of the last bug binges related but not similar is bug 1778

Jörn M. Horschig - 2013-12-09 10:58:08 +0100

Reopened because test script fails it fails because of 'different size in array.grad.chanunit{:,1}' freq.grad.chanunit was 'unknown' datanew.grad.chanunit is 'T', now action: overwrite old data on disk

Robert Oostenveld - 2013-12-09 11:17:01 +0100

(In reply to comment #4) agreed.

Jörn M. Horschig - 2013-12-09 13:26:52 +0100

all done, if the test script passes, I'm gonna close this one again

Jörn M. Horschig - 2013-12-11 09:45:03 +0100

the testscript still fails, but now due to differences in powspctrm. I checked on a mentat and the test passes, so I guess it is again caused by differences in matlab versions. How to fix?

Robert Oostenveld - 2013-12-11 11:51:57 +0100

(In reply to comment #7) how large is the difference? if it is in the "eps" range, the test script should be made rubust to the difference. Besides MATLAB isequal, there is alsy my test/private/identical function that allows relative and absolute tolerances. I believe that the xunit toolbox also has such a isalmostequal function.

Jörn M. Horschig - 2013-12-11 14:44:03 +0100

the discrepancy happens at dataset 10 (CTF275 data) [ok,msg] = identical(freq, datanew,'reltol',eps*1e6); which is 2.2*10^-10, so around 0.0000000001 but the max-ratio in matlab 2012b is 1.00000014 what a bummer :) similar issue at dataset 12 (itab153), but there the difference is 2.29*10^-10 I guess a re-write is safe?

Robert Oostenveld - 2013-12-11 15:38:57 +0100

(In reply to comment #9) yes

Jörn M. Horschig - 2014-01-22 10:08:35 +0100

well, it still sometimes bounces from passed to failed, but currently it's status is: ooooxoooo! if it fails more often, we could increase the identical threshold. For the last fail, I increased memory requirements: svn ci test_ft_freqanalysis.m -m "increases memory requirements by 0.25gb"Sending test_ft_freqanalysis.m Transmitting file data . Committed revision 9122.