Back to the main page.
Bug 1429 - ft_componentanalysis returns slightly different results from the stored test results
| Status | ASSIGNED |
| Reported | 2012-04-17 12:02:00 +0200 |
| Modified | 2013-11-03 19:57:00 +0100 |
| Product: | FieldTrip |
| Component: | core |
| Version: | unspecified |
| Hardware: | PC |
| Operating System: | Windows |
| Importance: | P3 normal |
| Assigned to: | Eelke Spaak |
| URL: | |
| Tags: | |
| Depends on: | |
| Blocks: | |
| See also: |
Eelke Spaak - 2012-04-17 12:02:32 +0200
While checking out test_ft_componentanalysis, I tracked its failing down to line 30: assert(isequalwithequalnans(data, datanew)); It turns out that (at least) the .unmixing field is slightly different in the two versions. The difference is on the order of 10^-10, though, while the weights are in the order of 10^-1, so probably this is not a 'real' bug. I would not know how to fix it, though.
Johanna - 2012-04-17 12:11:57 +0200
Hi Eelke, for which 'k' does this occur? I just did svn-update and my previous version (from last week) of test_ft_componentanalysis went through all the cfg.method options...does this new version also? (it seems only 'pca' is now tested). I ask because I did make a change with the .unmixing specific to DSS method, about 2 months ago, but that shouldn't affect PCA.
Eelke Spaak - 2012-04-17 12:16:11 +0200
(In reply to comment #1) The one I'm using indeed only checks PCA (which might be an issue in itself, of course: why not test every method?). The assertion fails for k = 1,2,3 and probably also the rest (which I haven't checked).
Roemer van der Meij - 2012-05-16 14:01:09 +0200
my suspicion is that this is related to the scaling of the data, that has been introduced a while ago. Before, the data entered the bss-algorithms unscaled, i.e. it could have values of 10e-12, whereas they are scaled to 10e0. See whether temporarily commenting out the scaling line reproduces the problem. If it indeed is a numerical accuracy issue, probably it is not a real bug and the reference data on disk should be updated with the most recent version of ft_componentanalysis
Jan-Mathijs Schoffelen - 2012-10-24 13:29:31 +0200
this issue is related to test_ft_componentanalysis failing. The general discussion in FT-meeting (24-Oct-2012) went into the direction of questioning the usefulness of comparing newly computed data against data stored on disk. The issue also applies to test_ft_timelockanalysis and test_ft_freqanalysis. In general, users should be able to expect that the numerical output does not change (unless the earlier version had a bug in the algorithm); another discussion is whether the implementation of the algorithm is correct. In the case of componentanalysis, it was questioned whether the reference data should be used at all for the second purpose. Better to use 'generated' data in this case. The latter purpose suggests that many test-functions need to be extended as to also test the correctness of the implementation.