Back to the main page.
Bug 2978 - "Plot Trial" button on GUI for ft_rejectvisual (summary mode) plots average of all trials instead of the single trial requested
| Status | CLOSED FIXED |
| Reported | 2015-10-05 18:21:00 +0200 |
| Modified | 2019-08-10 12:40:45 +0200 |
| Product: | FieldTrip |
| Component: | core |
| Version: | unspecified |
| Hardware: | PC |
| Operating System: | Windows |
| Importance: | P5 normal |
| Assigned to: | Robert Oostenveld |
| URL: | |
| Tags: | |
| Depends on: | 30643128 |
| Blocks: | |
| See also: | http://bugzilla.fieldtriptoolbox.org/show_bug.cgi?id=3012http://bugzilla.fieldtriptoolbox.org/show_bug.cgi?id=3032http://bugzilla.fieldtriptoolbox.org/show_bug.cgi?id=3296 |
Teresa Madsen - 2015-10-05 18:21:17 +0200
This is another bug that I'm just encountering now that I've recently upgraded to Matlab R2015b. I run: cfg = []; cfg.layout = 'vertical'; cfg.method = 'summary'; data = ft_rejectvisual(cfg,data); And everything looks fine, but once I plot any single trial, only that trial continues to be replotted (with the label of the trial I asked for) regardless of what number I put in. Since there's no error, I'm not sure where to look in the code to narrow down the problem. Let me know if you're able to reproduce it.
Teresa Madsen - 2015-11-20 21:09:00 +0100
Actually, it turns out it's plotting an average of all the trials, and it happens in R2015a as well. Here's exactly what I'm doing: %% load continuous LFP data, all channels cfg = []; cfg.dataset = nexfile{r,p}; cfg.channel = 'AD*'; cfg.continuous = 'yes'; cfg.outputfile = outputfile1; data = ft_preprocessing(cfg); %% break data into 100 chunks for visualization % during artifact rejection process, will be % re-merged before filtering cfg = []; cfg.dataset = nexfile{r,p}; cfg.trialdef.triallength = floor(... (data.sampleinfo(2)-data.sampleinfo(1))... /100)/data.fsample; cfg.trialdef.ntrials = 100; cfg = ft_definetrial(cfg); data = ft_redefinetrial(cfg, data); Along with other details, the Command Window displays, "the input is raw data with 16 channels and 1 trials," which is true of the data going into ft_redefinetrial. %% reject bad channels based on extreme kurtosis cfg = []; cfg.layout = 'vertical'; cfg.method = 'summary'; data = ft_rejectvisual(cfg,data); Here, it says, "the input is raw data with 16 channels and 100 trials," which is correct for the data now going into ft_rejectvisual. But when I type, say, "70" into the "Plot trial" box and hit enter, the Command Window now displays: the call to "ft_selectdata" took 0 seconds the input is timelock data with 16 channels and 51775 timebins the call to "ft_selectdata" took 0 seconds averaging trials averaging trial 100 of 100 What appears in another figure is actually an average of all 100 trials, but it is labeled as Trial 70. Why has it changed to interpreting this as timelock data, and why is it averaging the trials?
Teresa Madsen - 2015-12-02 19:12:43 +0100
Now I'm encountering a new version of this bug - now with errors, so at least a place to start! Again, I run this: cfg = []; cfg.layout = 'vertical'; cfg.method = 'summary'; data = ft_rejectvisual(cfg,data); The GUI pops up just fine, and the command window displays this: the input is raw data with 12 channels and 200 trials Warning: correcting numerical inaccuracy in the time axes > In ft_warning (line 181) In ft_datatype_raw>fixtimeaxes (line 310) In ft_datatype_raw (line 262) In ft_checkdata (line 219) In ft_rejectvisual (line 136) showing a summary of the data for all channels and trials computing metric [---------------------------------------------------------] All the other functions of the GUI seem to work, but when I put a trial number into the "plot trial" box and hit "enter," I get this: Warning: correcting numerical inaccuracy in the time axes > In ft_warning (line 181) In ft_datatype_raw>fixtimeaxes (line 310) In ft_datatype_raw (line 262) In ft_checkdata (line 219) In ft_multiplotER (line 140) In rejectvisual_summary>display_trial (line 610) In uiwait (line 81) In rejectvisual_summary (line 136) In ft_rejectvisual (line 275) Warning: correcting numerical inaccuracy in the time axes > In ft_warning (line 181) In ft_datatype_raw>fixtimeaxes (line 310) In ft_datatype_raw (line 262) In ft_checkdata (line 219) In ft_checkdata>raw2timelock (line 1296) In ft_checkdata (line 400) In ft_multiplotER (line 140) In rejectvisual_summary>display_trial (line 610) In uiwait (line 81) In rejectvisual_summary (line 136) In ft_rejectvisual (line 275) Error using nan Requested 200x12x5911295 (105.7GB) array exceeds maximum array size preference. Creation of arrays greater than this limit may take a long time and cause MATLAB to become unresponsive. See array size limit or preference panel for more information. Error in ft_checkdata>raw2timelock (line 1318) tmptrial = nan(ntrial, nchan, length(time)); Error in ft_checkdata (line 400) data = raw2timelock(data); Error in ft_multiplotER (line 140) varargin{i} = ft_checkdata(varargin{i}, 'datatype', {'timelock', 'freq'}); Error in rejectvisual_summary>display_trial (line 610) ft_multiplotER(cfg_mp, info.data); 1318 tmptrial = nan(ntrial, nchan, length(time)); I think this error occurs because I've set the offset of all my trials to the start of the recording, so it's trying to create a matrix large enough to fit the whole continuous stream of data into each of the 200 trials. If I change them all to 0 offset, rejectvisual_summary>display_trial runs without errors, but displays an average of the first 2 seconds (my shortest trial length) of all trials, followed by a flat line for the rest of the length of the longest trial (~30.1s). But the real problem is that it's trying to convert the data to timelock again. Why? What does that mean? How do I get it to display just the one trial I requested?
Weiyong Xu - 2016-04-28 14:52:22 +0200
Hi, this bug still exists. Every time I try to plot one single trial, it always give the same averaged trials. It may be quite misleading if you didn't notice it is actually an averaged trial because it will look quite good and you will be tempted to keep this trial(which is not plotted).
Jan-Mathijs Schoffelen - 2016-05-11 16:48:13 +0200
(In reply to Weiyong Xu from comment #3) Since you did such a great job in the ft_combineplanar bug, do you feel lucky enough to tackle this one as well? :o)
Weiyong Xu - 2016-05-12 17:42:18 +0200
(In reply to Jan-Mathijs Schoffelen from comment #4) Hi, This bug is originated from ft_multiplotER? line 327 [cfg, varargin{:}] = rollback_provenance(cfg, varargin{:}); This lead to cfg.trials='all',because the data structure in varargin{:} which was passed from rejectvisual_summary does not have any trial plot index from the user(by typing in the textbox in GUI). This overwrite: line 158 cfg.trials = ft_getopt(cfg, 'trials', 'all', 1); which recieve the correct trial plot index from the user(by typing in the textbox in GUI).So before plotting ft_timelockanalysis averages all the trials for all the single trial plot. It maybe not a good idea to change code in ft_multiplotER, so I would suggest only pass a single trial data to ft_multiplotER when it is called in rejectvisual_summary to plot specific single trial.I made a few line code change in rejectvisual_summary and it seems to work well.
Robert Oostenveld - 2016-05-13 15:01:17 +0200
(In reply to Weiyong Xu from comment #5) Weiyong suggested this https://github.com/fieldtrip/fieldtrip/pull/156 to which I replied with this --- although the problem is visible in ft_rejectvisual, that is not where the problem originates, nor should be solved. I loaded a dataset (Subject01.ds) and with cfg = []; cfg.layout = 'CTF151.lay'; cfg.trials = 255; % this has a jump ft_multiplotER(cfg, data) I am also getting the average. So ft_multiplotER is not behaving as it should and not consistent with its own documentation. --- the core issue is that ft_multiplotER is not doing data handling correctly. It has many layers of data selection bolted onto each other (over the years) and they are not consistent in all cases. Most important is to switch to using ft_selectdata consistently. The use cases that I am aware of are raw data, averaged on the fly timelock data with keeptrials, averaged on the fly timelock data with avg, no averaging needed power spectrum (no time) with keep trials, averaged on the fly power spectrum (no time) already averaged cross-spectrum (no time) with keep trials, ref channel selection, no averaging needed cross-spectrum (no time), ref channel selection, no averaging needed The cross-spectrum will usually be averaged/normalized into a coherence estimate, but in principle can be single trial. It could also have leave-one-out jackknife estimates of coherence. I will make a test script that goes through these options and reimplement the data selection.
Robert Oostenveld - 2016-05-13 16:24:48 +0200
(In reply to Robert Oostenveld from comment #6) I rewrote considerable parts of the data handling in ft_multiplotER. Could someone check out https://github.com/fieldtrip/fieldtrip/pull/161 and report here whether it works for you? thanks Robert
Weiyong Xu - 2016-05-13 18:41:44 +0200
(In reply to Robert Oostenveld from comment #7) On my computer, there are still some error messages, one is in: line 368 ft_plot_lay(lay, 'box', boxflg, 'label', labelflg, 'outline', outlineflg, 'point', 'no', 'mask', 'no'); the variable lay is not defined after you changed the code. so I added this line of code back: lay = ft_prepare_layout(cfg, varargin{1}); then there is another error "Index exceeds matrix dimensions." in line 527 data = dat(intersect(seldat1, seldat2), :); I am not sure where this problem is from. Weiyong
Robert Oostenveld - 2016-05-17 17:00:55 +0200
(In reply to Weiyong Xu from comment #8) that was due to a small typo in the code, "dimtok" was not what it should be. I updated the code, can you once more pull my branch bug2978 and test it?
Weiyong Xu - 2016-05-18 09:55:14 +0200
(In reply to Robert Oostenveld from comment #9) Thanks for fixing the bug. I tested branch bug2978 with my own neuromag data. Now the ft_rejectvisual is plotting the single trials that the user has typed in. As I looked in the test_bug2978 script, it seemed that the ft_multiplotER handle the data from FT_PREPROCESSING correctly, for example, it plots the averaged trials defined in cfg.trials; But ft_multiplotER seemed to ignore the cfg.trials for ERFs produced by FT_TIMELOCKANALYSIS, and always plot the averaged trials even if the ERFs data were from FT_TIMELOCKANALYSIS with cfg.keeptrials = 'yes'. the codes are as follows: cfg = []; timelock1 = ft_timelockanalysis(cfg, test_trials); cfg.keeptrials = 'yes'; timelock2 = ft_timelockanalysis(cfg, test_trials); cfg=[]; cfg.layout='neuromag306all.lay'; figure; ft_multiplotER(cfg,timelock2) cfg=[]; cfg.layout='neuromag306all.lay'; cfg.trials=[3]; figure; ft_multiplotER(cfg,timelock2)
Robert Oostenveld - 2016-05-18 11:04:42 +0200
(In reply to Weiyong Xu from comment #10) thanks for testing. You are right, I did not yet sufficiently consider all inputs that the function might receive. I am also not sure about freq and tidmefreq data. I will extend the test script.
Stefan Wiens - 2017-05-14 12:26:05 +0200
I found this bug and am wondering if there is a solution now? At least, for me it still does not work. I downloaded a new version of ft last week (fieldtrip-20170511) with Matlab 2016B. I run these simple commands: cfg = []; cfg.layout = 'biosemi64.lay'; cfg.metric = 'range'; cfg.latency = [-.1 .6]; ft_rejectvisual(cfg, data); The plot seems to work, but when I want to plot a specific trial ("Plot trial", to the left of the quit button), ft does not plot the trial number I entered; instead, it plots the LAST trial (or the average?). The matlab output actually states that whereas the figure title states the trial number that I entered.
Robert Oostenveld - 2017-05-19 14:59:06 +0200
(In reply to Stefan Wiens from comment #12) Hi Stefan, trial selection and on-the-fly averaging needs to be reimplemented in ft_multiplotER and in ft_multiplotTFR. I made an attempt at it about a year ago, but that one did not result in a solution. I think that a local (non-global) solution just for reject visual would be easy to implement and at least get this out of the way. I will come with a suggestion.
Robert Oostenveld - 2017-05-22 09:18:01 +0200
(In reply to Robert Oostenveld from comment #13) I made an update of the code, see https://github.com/fieldtrip/fieldtrip/pull/442
Robert Oostenveld - 2017-05-22 09:20:01 +0200
I have merged https://github.com/fieldtrip/fieldtrip/pull/442 That means that it is fixed for ft_rejectvisual. The problem in ft_multiplotER remains, though. I will make a new bug report for that.
Robert Oostenveld - 2019-08-10 12:34:35 +0200
This closes a whole series of bugs that have been resolved (either FIXED/WONTFIX/INVALID) for quite some time. If you disagree, please file a new issue on https://github.com/fieldtrip/fieldtrip/issues.