Back to the main page.
Bug 2585 - ICA for same data differ even using fixed random seed
| Status | CLOSED FIXED |
| Reported | 2014-05-20 17:45:00 +0200 |
| Modified | 2016-01-15 09:37:31 +0100 |
| Product: | FieldTrip |
| Component: | core |
| Version: | unspecified |
| Hardware: | PC |
| Operating System: | Linux |
| Importance: | P5 major |
| Assigned to: | Jan-Mathijs Schoffelen |
| URL: | |
| Tags: | |
| Depends on: | |
| Blocks: | 2891 |
| See also: |
Andreas Wollbrink - 2014-05-20 17:45:03 +0200
Created attachment 628 matlab function performing two repetative component analyses to the same data performing a component analysis (runica, PCA upstream) twice to one and the same data (see the attached script) results in different IC (different topographies and time series) even using the cfg.randomseed = comp.cfg.callinfo.randomseed functionallity
Robert Oostenveld - 2014-05-20 18:21:27 +0200
I added a test script that confirms the problem mac011> svn commit test_bug2585.m Adding test_bug2585.m Transmitting file data . Committed revision 9554. mac011> svn commit test_bug2585.m Sending test_bug2585.m Transmitting file data . Committed revision 9555. when I add the statement rand(1) just before ft_componentanalysis calls runica, it is clear that the random state is set by FT as expected. However, runic results in (slightly) different results on repeated calls. Hence the issue is with eeglab/runic, not FT itself. The problem is also visible with rng(1,'twister') [a1, w1] = runica(data.trial{1}); rng(1,'twister') [a2, w2] = runica(data.trial{1}); where rng is the random number generator in matlab2014a (see utilities/private/randomseed for the Mathworks mess) Did you search on the eeglab mailing list or documentation?
Andreas Wollbrink - 2014-05-21 11:38:37 +0200
I just had a closer look in the external/eeglab/runica.m file. In line 812 the random number generator state is set dependent on the system clock which is not very promisong for having the same state on repetative calls This setting seems to be totally independent from the cfg.randomseed setting one provides to the ft_componentanalysis function (In reply to Robert Oostenveld from comment #1)
Robert Oostenveld - 2014-05-21 12:56:37 +0200
(In reply to Andreas Wollbrink from comment #2) The comments in the function state: % Revision 1.20 2004/06/29 16:44:09 scott % randomize data shuffling by clock state It seems to be in there for a long time already. Also in https://code.google.com/p/fieldtrip/source/list?path=/trunk/external/eeglab/runica.m&start=9549 it seems that it has been part of the code forever. Johanna, do you perhaps know whether the randseed was ever expected to work with runica? I know that we use it for the HCP project in combination with fascia, and there it works fine.
Johanna - 2014-05-21 13:17:23 +0200
Hi Robert and Andreas, Oh dear, I'm sorry for not creating a more thorough test script. In bug 1205 comment 7, I state that I had tested it on fastica and it worked. It seems I never tested beyond that, as there is no test_bug1205 file. Thus, I can't claim that it ever worked with runica. I used fastica for my own work so I didn't notice the problem.
Robert Oostenveld - 2014-05-21 16:57:27 +0200
(In reply to Johanna from comment #4) thanks for the answer. I also think that it is fair that FieldTrip sets the random seed, but that we cannot take responsibility for external toolboxes. E.g. I would rather not change runica.m in the fieldtrip release, because that is the same as the code in eeglab. But... Andreas, you are of course free to change it in your copy of runica.m Let me also CC Arno, he might want to share his opinion with us
Johanna - 2015-02-04 17:42:42 +0100
At a minimum then, if the eeglab version won't change by eeglab developers, then the FT documentation should state that it doesn't work for 'runica'.
Arnaud Delorme - 2015-02-05 20:01:47 +0100
The reason that EEGLAB does not reset the random seed in runica.m is that some researcher like to boostrap the ICA decomposition to assess the stability of components. What could be done is add a new optional input to the function that would set the random seed so the decomposition is always the same. Arno
Arnaud Delorme - 2015-02-05 20:03:08 +0100
If someone does such a thing, we will update the runica.m function in EEGLAB as well (assuming the new option is not used by default). Thanks, Arno
Robert Oostenveld - 2015-02-06 15:58:03 +0100
(In reply to Arnaud Delorme from comment #8) We should ensure that we have the latest version of runica (ffrom eeglab svn) in fieldtrip/external/eeglab, and then make the change. I believe that I can svn commit to eeglab. ----- mac011> svn info Path: . Working Copy Root Path: /Users/roboos/matlab/eeglab URL: https://sccn.ucsd.edu/svn/software/eeglab Relative URL: ^/eeglab Repository Root: https://sccn.ucsd.edu/svn/software Repository UUID: 1e58238c-3a72-4c0f-aabe-36d030380f39 Revision: 10465 Node Kind: directory Schedule: normal Last Changed Author: arno Last Changed Rev: 10465 Last Changed Date: 2014-12-02 23:36:56 +0100 (Tue, 02 Dec 2014) ------- mac011> cp ./functions/sigprocfunc/runica.m ~/matlab/fieldtrip/external/eeglab/ mac011> cd ~/matlab/fieldtrip/external/eeglab/ mac011> svn status
Jan-Mathijs Schoffelen - 2015-05-08 16:10:08 +0200
[jansch@mentat002 eeglab]$ svn commit -m "enhancement - implemented optional resetting of the random number generator (default = true)" runica.m Sending runica.m Transmitting file data . Committed revision 10397.
Jan-Mathijs Schoffelen - 2015-07-20 10:09:18 +0200
Hi Arno, If you have some time at some point, could you look into this for a second? I don't know whether this has been followed up from the eeglab side, but the FT-version of runica now allows for an optional fixed random seed. It would be nice to keep the versions of runica in FT and Eeglab in sync. Best, JM
Jan-Mathijs Schoffelen - 2015-09-17 11:10:25 +0200
This is a gentle reminder for Arno, to look into it at the EEGlab side. Closing on the FT-end for now.
Arnaud Delorme - 2015-09-18 01:47:23 +0200
I have committed the changes to EEGLAB and also updated the documentation. https://bitbucket.org/sccn_eeglab/eeglab/commits/b22169c2bc6ded74a3baafc948f4d520cae5d734#chg-functions/sigprocfunc/runica.m