Back to the main page.

Bug 2861 - ft_sourceanalysis keeps data.filter leading to an error in later sourcestatistics

Status CLOSED FIXED
Reported 2015-03-03 18:46:00 +0100
Modified 2016-01-15 09:38:15 +0100
Product: FieldTrip
Component: core
Version: unspecified
Hardware: PC
Operating System: Windows
Importance: P5 normal
Assigned to: Jan-Mathijs Schoffelen
URL:
Tags:
Depends on:
Blocks:
See also: http://bugzilla.fcdonders.nl/show_bug.cgi?id=2867

Jens Klinzing - 2015-03-03 18:46:00 +0100

When using ft_sourceanalysis the filter is saved in the output structure (data.filter). When calculating single-trial estimates this field now exists additionally to data.trials(i).filter. I think this is the case since revision 10253. The field data.filter leads to an error in ft_datatype_source (line 180) when the source output is given to ft_sourcestatistics (see full error msg below). The error happens when the field 'filter' is processed. This line in ft_datatype_source is only reached if the data contains single trials estimates. Deleting the field data.filter solves the problem. I have only tested this only with DICS, cfg.rawtrials='yes', and a predefined filter. Here is my code: cfg = []; cfg.vol = vol; cfg.grid = grid; cfg.frequency = tfr.freq; cfg.method = 'dics'; cfg.dics.projectnoise = 'yes'; cfg.dics.lambda = '5%'; cfg.dics.keepfilter = 'yes'; cfg.dics.realfilter = 'yes'; cfg.dics.powmethod = 'lambda1'; cfg.keeptrials = 'yes'; cfg.grad = grad; cfg.grid.filter = filter; cfg.rawtrial = 'yes'; source_trials = ft_sourceanalysis(cfg, tfr); source_trials = freq: 1.3977e+01 cumtapcnt: [160x1 double] dim: [20 25 22] filter: {11000x1 cell} inside: [11000x1 logical] pos: [11000x3 double] method: 'rawtrial' trial: [1x160 struct] df: 160 trialinfo: [160x1 double] cfg: [1x1 struct] cfg = []; cfg.dim = data.dim; cfg.correctm = 'cluster'; cfg.method = 'montecarlo'; cfg.statistic = 'indepsamplesT'; cfg.clusteralpha = 0.01; cfg.clusterstatistic = 'maxsum'; cfg.clustertail = 0; cfg.minnbchan = 2; cfg.tail = 0; cfg.alpha = 0.025; cfg.numrandomization = 1000; cfg.design = design; cfg.ivar = 1; cfg.parameter = 'pow'; stats = ft_sourcestatistics(cfg, source_trials); Subscripted assignment dimension mismatch. Error in ft_datatype_source (line 180) val{indx(k)}(1,:,:,:) = dat{indx(k)}; Error in ft_checkdata (line 239) data = ft_datatype_source(data); Error in ft_sourcestatistics (line 97) varargin{i} = ft_checkdata(varargin{i}, 'datatype', 'source', 'feedback', 'no');


Jan-Mathijs Schoffelen - 2015-03-04 13:23:16 +0100

Thanks for the notification Jens. I am not sure whether it's due specifically to the 'filter'. I noticed that the conversion in ft_datatype_source was flaky for any subfield that is represented as a cell-array in source.trial(i) (so also for the csd and noisecd for example: why this did not give a crash in your case I do not know). I have updated ft_datatype_source for this purpose, and ft_datatype_source (on my end at least) does not choke on line 180 anymore (also for the filters). Could you give it a try on your end with the latest revision? Thanks, Jan-Mathijs [jansch@mentat002 fieldtrip]$ svn diff utilities/ft_datatype_source.m Index: utilities/ft_datatype_source.m =================================================================== --- utilities/ft_datatype_source.m (revision 10264) +++ utilities/ft_datatype_source.m (working copy) @@ -173,6 +173,7 @@ if iscell(dat) + datsiz(1) = nrpt; % swap the size of pos with the size of rpt val = cell(npos,1); indx = find(source.inside); for k=1:length(indx) [jansch@mentat002 fieldtrip]$ svn commit -m "bugfix - fixed faulty conversion from source data with trials" utilities/ft_datatype_source.m Sending utilities/ft_datatype_source.m Transmitting file data . Committed revision 10265.


Jens Klinzing - 2015-03-04 20:35:01 +0100

Hi Jan-Mathijs, thanks for your fast reply. Unfortunately, the same error still occurs (now the line is 181). If you like, here is one of the datasets (and the design) that I'm trying to calculate statistics on. https://www.dropbox.com/sh/6ivoz6hxyjkmv7a/AAAGSVONMuMn6lkUhelTC5_ma?dl=0


Jens Klinzing - 2015-03-05 00:42:10 +0100

The problem is that the filter has 3 values per channel. The line of error is: val{indx(k)}(1,:,:,:) = dat{indx(k)}; size(val{indx(k)}(1,:,:,:)) ans = 1 273 size(dat{indx(k)}) ans = 3 273


Jan-Mathijs Schoffelen - 2015-03-05 10:55:12 +0100

hi Jens, Would it be possible to upload in the Dropbox a structure that has only 2 trials? This should be sufficient to reproduce the problem, but wouldn't take that long to download :-). (you could do source.trial = source.trial(1:2)) and then upload...


Jens Klinzing - 2015-03-05 11:08:38 +0100

Hi Jan-Mathijs, yes that makes a lot of sense. Sorry for not thinking about it. You find the reduced dataset (~250MB) in the same dropbox folder. Best, Jens


Jan-Mathijs Schoffelen - 2015-03-05 11:30:35 +0100

Thanks. I see where it comes from. The bookkeeping code gets confused because there is indeed a filter in source.filter, as well as in source.trial.filter. We need to iron this out, but it is at least clear what's causing it. As such, there's no reason to keep the filter in the output when calling ft_sourceanalysis for the second time (when doing the cfg.rawtrial round), right? So, would you be OK with for the time being just specifying cfg.dics.keepfilter = 'no'; ?


Jens Klinzing - 2015-03-05 11:43:38 +0100

Yes, that would work. Thanks a lot for your help and to the whole fieldtrip team. Your work is highly appreciated over here. Greetings from Tübingen, Jens


Jan-Mathijs Schoffelen - 2015-08-11 13:41:31 +0200

=================================================================== --- ft_sourceanalysis.m (revision 10577) +++ ft_sourceanalysis.m (working copy) @@ -1037,7 +1037,7 @@ end if exist('grid', 'var') - source = copyfields(grid, source, {'pos', 'inside', 'leadfield', 'filter'}); + source = copyfields(grid, source, {'pos', 'inside', 'leadfield'});%, 'filter'}); end if exist('dip', 'var') [jansch@mentat002 fieldtrip]$ svn commit -m "enhancement - do not preallocate filter in output, to avoid double occurrences" ft_sourceanalysis.m Sending ft_sourceanalysis.m Transmitting file data . Committed revision 10579. I think that this should fix it.