Back to the main page.
Bug 79 - ft_redefinetrial with overlapping trials (e.g. after ft_appenddata)
| Status | CLOSED FIXED |
| Reported | 2010-05-07 14:27:00 +0200 |
| Modified | 2017-12-04 10:08:30 +0100 |
| Product: | FieldTrip |
| Component: | preproc |
| Version: | unspecified |
| Hardware: | PC |
| Operating System: | Linux |
| Importance: | P3 normal |
| Assigned to: | Jan-Mathijs Schoffelen |
| URL: | |
| Tags: | |
| Depends on: | |
| Blocks: | |
| See also: | http://bugzilla.fieldtriptoolbox.org/show_bug.cgi?id=3379 |
Arjen Stolk - 2010-05-07 14:27:52 +0200
Simulation of the bug using two datasets (1 contains zeros, 2 contains ones). The datasets are appended (e.g. cause system failure during recording) and then redefined (e.g. prior to the frequency analysis process). Redefinition has been attempted in two ways; 1) trl definition and 2) cfg.begsample & cfg.endsample specification. % stuff Fs = 1200; cfg = []; cfg.layout = 'CTF275.lay'; cfg.layout = prepare_layout(cfg); label = cfg.layout.label(1:273,1); % dataset 1; all ones dataset1.trial{1,1} = zeros(273,2400); dataset1.time{1,1} = (0:1/Fs:2-1/Fs); dataset1.label = label; dataset1.fsample = Fs; dataset1.cfg.previous.previous.trl(1,1) = 1; dataset1.cfg.previous.previous.trl(1,2) = 2400; dataset1.cfg.previous.previous.trl(1,3) = 0; % dataset2; all zeros dataset2.trial{1,1} = ones(273,2400); dataset2.time{1,1} = (0:1/Fs:2-1/Fs); dataset2.label = label; dataset2.fsample = Fs; dataset2.cfg.previous.previous.trl(1,1) = 1; dataset2.cfg.previous.previous.trl(1,2) = 2400; dataset2.cfg.previous.previous.trl(1,3) = 0; % append data cfg = []; dataset3 = ft_appenddata(cfg, dataset1, dataset2); % redefine trial with trl cfg = []; cfg.trl(1,1) = 1; cfg.trl(1,2) = 1200; cfg.trl(1,3) = 0; cfg.trl(2,1) = 1201; cfg.trl(2,2) = 2400; cfg.trl(2,3) = 0; dataset4 = ft_redefinetrial(cfg, dataset3); % result ??? Error using ==> fetch_data at 114 some of the requested samples occur twice in the data when commenting line 113 and 114 in fetch_data, and thus, ignoring the error; both trials in dataset4 solely contain 'ones' which means that dataset2 alone has been used for input % redefine trial with cfg.begsample and cfg.endsample cfg = []; cfg.begsample(1,1) = 1; cfg.endsample(1,1) = 1200; cfg.begsample(2,1) = 1201; cfg.endsample(2,1) = 2400; dataset5 = ft_redefinetrial(cfg, dataset3); % result trial 1 has zeros (originating from dataset1) trial 2 has ones (originating from dataset2) disadvantage of this method is that one cannot cut trials in pieces (and create more trials) but only select a part from each trial
Jan-Mathijs Schoffelen - 2010-05-17 09:23:34 +0200
The first question here is actually: What IS the bug. Arjen's problem is related to the fact that when calling ft_redefinetrial on an appended dataset (consisting of 2 or more different recordings), the appended trl-matrix contains 'overlapping' epochs. This is caught as an error in the low-level public/fetch_data. This feature of fetch_data also causes problems if a similar data structure is used in ft_databrowser (this is a separate bug assigned to Ingrid, but should be treated in this context as well). Robert pointed out to a 'philosophical' problem related to this issue. This concerns the fact that in this context fieldtrip uses the trl-matrix which is stored somewhere in the history of cfgs to a data-structure. This is against fieldtrip-philosophy, in that each data-structure should be self-explanatory. Having to dig into the past (by calling findcfg(data.cfg, 'trl') could lead to the extraction of ambiguous or ill-defined trl-matrices (as we have seen). Yet, trl-matrices have been used extensively in the past to code for trial specific parameters in the 4th until 100th columns, and by taking care of the trl-matrix in each analysis step allows us to keep track of these parameters consistently. For future development this has to be reconsidered and possibly added as a higher level field in the data-structure. (in line with Martin Vincks proposal to deal with some things in the ft_spikeXXX functions). For the time being I think a few changes need to be made to the existing code: ft_appenddata: check explicitly whether the original data structures come from the same recording. If so, append the trl-matrices. If not, append them, but replace the first 2 columns either by nans (don't know what this will be doing to the rest of the code), or by an internally consistent description (at least with respect to the trl-matrix). The second option may also have unwanted side-effects though. ft_redefinetrial: throw an error if the redefinition is based on a trl-matrix, and if the trl-matrix contained in the data is ambiguous/inconsistent/contains nans ft_databrowser: .... fetch_data: only do the overlapping samples check when this is needed Final point related to ft_redefinetrial: Arjan mentions that it may be a desired option to be able to cut out more than one epoch out of a longer one. As of yet, this functionality does not exist as far as I know, but investigate whether this is desirable, and if so, implement it.
Robert Oostenveld - 2010-05-19 13:57:28 +0200
add fields to data structure: trialdef (raw data only) trialinfo (4th column of original trl matrix and upwards) to ensure backward compatibility first build into checkdata the conversion from old-fashioned (containing cfg.previousxxx.trl) to new (containing .trialdef and .trialinfo) etc....