Back to the main page.
Bug 3417 - Crashing during ft_freqstatistics
| Status | CLOSED FIXED |
| Reported | 2018-03-05 18:35:00 +0100 |
| Modified | 2019-08-10 12:37:08 +0200 |
| Product: | FieldTrip |
| Component: | core |
| Version: | unspecified |
| Hardware: | Macintosh |
| Operating System: | Mac OS |
| Importance: | P5 normal |
| Assigned to: | Robert Oostenveld |
| URL: | |
| Tags: | |
| Depends on: | |
| Blocks: | |
| See also: |
Sam Wass - 2018-03-05 18:35:07 +0100
Created attachment 860 Wrapper script attached. Sample data file that causes the crash can be downloaded here: https://drive.google.com/open?id=1secH_W-I-afX3CN6MCrpxmloNqYvTiGZ I am a new FieldTrip user. I am trying to run a script that a colleague has sent me, which is basically a wrapper script to call the ft_freqstatistics calculations. They work fine on his computer, which has a 2012 version of FieldTrip and Matlab 2013a. But when we try to run either a later version of Fieldtrip (e.g. 2015) or a later version of Matlab (e.g. 2017b), we get the following bug: Error using findcluster (line 50) invalid dimension of spatdimneighbstructmat Error in clusterstat (line 201) posclusobs = findcluster(reshape(postailobs, [cfg.dim,1]),channeighbstructmat,cfg.minnbchan); Error in ft_statistics_montecarlo (line 348) [stat, cfg] = clusterstat(cfg, statrand, statobs); Error in ft_freqstatistics (line 193) [stat, cfg] = statmethod(cfg, dat, design); The bug seems to occur irrespective of whether we are running the analyses on PC or Mac. I have just recreated the bug with a clean install on a Mac, with the latest version of Fieldtrip and Matlab 2016a. I attach the wrapper script and a sample dataset that causes the crash on our systems. Many thanks, Sam
Robert Oostenveld - 2018-03-07 18:17:57 +0100
*** Bug 3418 has been marked as a duplicate of this bug. ***
Robert Oostenveld - 2018-03-07 18:25:19 +0100
I have downloaded the data and made a personal branch with the test script. I have not tested the actual problem yet. ip-145-116-191-182> git push --set-upstream origin bug3417 Counting objects: 11, done. Delta compression using up to 4 threads. Compressing objects: 100% (11/11), done. Writing objects: 100% (11/11), 3.30 KiB | 0 bytes/s, done. Total 11 (delta 8), reused 0 (delta 0) remote: Resolving deltas: 100% (8/8), completed with 6 local objects. To github.com:robertoostenveld/fieldtrip.git * [new branch] bug3417 -> bug3417 Branch bug3417 set up to track remote branch bug3417 from origin.
Sam Wass - 2018-03-08 09:36:06 +0100
Dear Robert, Thanks! Let us know if there is anything else you need... Sam
Robert Oostenveld - 2018-03-14 11:12:44 +0100
I ran the test script with MATLAB 2013a and 2017b >> ver ------------------------------------------------------------------------------------------------------- MATLAB Version: 8.1.0.604 (R2013a) MATLAB License Number: 38957 Operating System: Linux 3.10.0-327.36.3.el7.x86_64 #1 SMP Mon Oct 24 16:09:20 UTC 2016 x86_64 Java Version: Java 1.6.0_17-b04 with Sun Microsystems Inc. Java HotSpot(TM) 64-Bit Server VM mixed mode ------------------------------------------------------------------------------------------------------- gives this Warning: adding /home/mrphys/roboos/matlab/fieldtrip/external/spm8 toolbox to your MATLAB path Error using findcluster (line 50) invalid dimension of spatdimneighbstructmat Error in clusterstat (line 201) posclusobs = findcluster(reshape(postailobs, [cfg.dim,1]),channeighbstructmat,cfg.minnbchan); Error in ft_statistics_montecarlo (line 352) [stat, cfg] = clusterstat(cfg, statrand, statobs); Error in ft_freqstatistics (line 193) [stat, cfg] = statmethod(cfg, dat, design); Error in test_bug3417 (line 60) stats = ft_freqstatistics(cfg,data1,data2) >> ver ----------------------------------------------------------------------------------------------------- MATLAB Version: 9.3.0.713579 (R2017b) MATLAB License Number: 38957 Operating System: Linux 3.10.0-327.36.3.el7.x86_64 #1 SMP Mon Oct 24 16:09:20 UTC 2016 x86_64 Java Version: Java 1.8.0_121-b13 with Oracle Corporation Java HotSpot(TM) 64-Bit Server VM mixed mode ----------------------------------------------------------------------------------------------------- gives this Warning: adding /home/mrphys/roboos/matlab/fieldtrip/external/spm8 toolbox to your MATLAB path Error using findcluster (line 50) invalid dimension of spatdimneighbstructmat Error in clusterstat (line 201) posclusobs = findcluster(reshape(postailobs, [cfg.dim,1]),channeighbstructmat,cfg.minnbchan); Error in ft_statistics_montecarlo (line 352) [stat, cfg] = clusterstat(cfg, statrand, statobs); Error in ft_freqstatistics (line 193) [stat, cfg] = statmethod(cfg, dat, design); Error in test_bug3417 (line 60) stats = ft_freqstatistics(cfg,data1,data2) To me it seems there is no MATLAB version dependency in the error.
Robert Oostenveld - 2018-03-14 11:20:21 +0100
The error has to do with a discrepancy in the number of channels (14) and the neighbor specification (3x3) Looking at the data used in the test script, I see data1 = struct with fields: globspctrm: [24×14×81 double] label: {'Positive'} dimord: 'rpt_freq_time' freq: [1 2 3 4 5 6 7 8 9 10 11 12 13 14] time: [1×81 double] which is not a valid FieldTrip data structure (see ft_datatype_freq). It surprises me that it runs with an old version of FieldTrip, and I would not vouch for the correctness of the results with the old version.
Valdas Noreika - 2018-03-14 14:59:22 +0100
Thank you Robert. Just wanted to clarify: these data come from a single EEG channel, and the dimensions of data are 24 subjects x 14 frequencies x 81 time samples. If the issue is due to incompatible neighbor specification (3x3), how should it be set for 14 frequency bins? We currently don't specify neighborhood, and probably a default (3x3) is used at some stage of clustering. Many thanks, Valdas
Robert Oostenveld - 2018-03-14 15:08:48 +0100
(In reply to Valdas Noreika from comment #6) The geometrical dimension (one channel) is also relevant, although it is a singleton-dimension. Clustering fails is implemented for space (channels or source positions), time and frequency. In your case there is no cluster over multiple channels, but time and freq are still interesting to cluster over. You probably should add cfg.neighbours=[] to indicate to ft_freqstatistics that you are not interested in spatial neighbors. I think that gloqspctrm should be 1x24×14×81 and that dimord should be chan_rpt_freq_time... although, more commonly (since compatible with TFR analysis with keeptrials=yes) would be rpt_chan_freq_time and hence 24×1x14×81. FieldTrip has gotten smarter over the years in terms of dealing with flexible dimensions in the data, so chan_rpt_freq_time might work, but rpt_chan_freq_time (with one channel) should work.
Robert Oostenveld - 2018-03-15 10:09:36 +0100
I had another look at the test script and realized that it was easy to fix. See https://github.com/fieldtrip/fieldtrip/pull/680 and https://github.com/fieldtrip/fieldtrip/blob/master/test/test_bug3417.m The relevant sections of code3 I added are % fix the invalid data structure, see ft_datatype_freq data1.dimord = 'rpt_chan_freq_time'; data2.dimord = 'rpt_chan_freq_time'; data1.globspctrm = reshape(data1.globspctrm, [nsubj 1 freqs times]); data2.globspctrm = reshape(data2.globspctrm, [nsubj 1 freqs times]); data1.label = {'TheSame'}; data2.label = {'TheSame'}; and cfg.neighbours = []; cfg.channel = 'all'; for the clustering. I also made some small cleanups to the FT code itself (formatting and documentation only).
Valdas Noreika - 2018-03-15 10:49:11 +0100
Fantastic, thank you so much!!!
Sam Wass - 2018-03-19 10:06:44 +0100
Yes - this works very well now. Many, many thanks for your help - much appreciated.