Back to the main page.
Bug 2220 - demeaning with a baseline and ft_preprocessing does not work
| Status | CLOSED FIXED |
| Reported | 2013-07-13 23:49:00 +0200 |
| Modified | 2014-03-12 12:20:37 +0100 |
| Product: | FieldTrip |
| Component: | preproc |
| Version: | unspecified |
| Hardware: | PC |
| Operating System: | Linux |
| Importance: | P3 critical |
| Assigned to: | Jörn M. Horschig |
| URL: | |
| Tags: | |
| Depends on: | |
| Blocks: | |
| See also: |
Flor Kusnir - 2013-07-13 23:49:58 +0200
Hi, I am performing MEG analysis with FieldTrip. I was performing a simple demeaning operation on a dataset using ft_preprocessing, with baseline [-0.5 0], when I realised that the sensor data in the actual baseline period did not have a mean of 0, as it should. By looking into the actual FieldTrip code, I figured out that in the function 'ft_preprocessing,' in line 358, in the following command, dataout.trial{i}, dataout.label, tmptime, cfg] = preproc(data.trial{i}(rawindx,:), data.label(rawindx), offset2time(0, data.fsample, size(data.trial{i},2)), cfg, begpadding, endpadding); instead of using the actual time points of the dataset, a set of NEW time points always starting at zero is created by the 3rd input argument of the above command (also shown below): offset2time(0, data.fsample, size(data.trial{i},2)) This simply means that if my desired baseline is [-0.5 0], or in general any negative time array (as it usually is), then the only time point that is actually considered as baseline is time point ZERO (because it's the nearest to both values of my baseline period). The data value at this time point is thus subtracted from the rest of the trial, so that all trials end up with zero as the first data value. So, by running cfg=[]; cfg.demean='yes'; cfg.baselinewindow=[-0.5 0]; datanew=ft_preprocessing(cfg,data); the result is that the first point in each trial and in each channel is 0, as this is the only point taken as baseline. The error occurs with the latest FieldTrip version as of now (8310) but it also occurs with the previous FieldTrip version I was using, which I downloaded through svn checkout on the 4th of July. I have attached a matlab data file called bugdatafile.mat containing a FieldTrip data structure called 'data.' The command I ran for demeaning is the one a few lines above. I hope I'm not missing something obvious, and if I'm not, could you please tell me the last FieldTrip version in which you know for sure that demeaning worked correctly (as this is critical for my previous analyses!)? Thanks very much (and fingers crossed I'm just missing something obvious)! Flor
Flor Kusnir - 2013-07-13 23:50:52 +0200
Created attachment 494 data file
Jan-Mathijs Schoffelen - 2013-07-14 14:59:44 +0200
bug confirmed: revision 8246 introduced it. I don't see a strong reason why data.time{i} has been replaced by offset2time, apart from my hunch that ft_preproc_padding does not return the padded time axis. @Jörn: could you confirm my hunch? If so, would it be an idea that ft_preproc_padding returns the updated time axis and fix this with high priority? Thanks
Jörn M. Horschig - 2013-07-15 08:37:39 +0200
Hi, indeed the reason why I introduced the temporary time-axis was because by padding, the number of sample changes and thereby the old, correct time-axis cannot be used. I never considered that someone wants to use a baseline whilst calling preprocessing... my bad. Imho it would be best to adjust the first argument of offset2time appropriately rather than fiddling around with ft_preproc_padding (as e.g. there is no fsample field in the lower-level function, leading to numerical inaccuracies and what have you). Also, a similar thing probably needs to be done further below in case of reading from disk, where offset2time is also used. I'm gonna fix this later this morning
Jörn M. Horschig - 2013-07-15 08:38:27 +0200
added JM (see last comment)
Jan-Mathijs Schoffelen - 2013-07-15 08:51:48 +0200
I agree. Upon second thought ft_preproc_padding should be low-level, so it will not be taking (nor outputting) a time-axis.
Jan-Mathijs Schoffelen - 2013-07-15 11:11:19 +0200
*** Bug 2204 has been marked as a duplicate of this bug. ***
Jörn M. Horschig - 2013-07-15 12:28:14 +0200
time-axes are not a good thing ;) Using offset2time might not be usable in case of in-memory-data because it could be resampled (i.e. trialinfo and sampleinfo become invalid) or not contain information about the offset somewhere (neither in sampleinfo nor in the time-axis). So actually, the only way to go here is work directly on the time-axis, i.e. apply ft_preproc_padding on the time-axis to interpolate it in some way. But what shall happen if the time-axis goes from -0.5 to 0.5 and the requested baseline-window is [-1 0] - should the padded data be used to compute the baseline? I assume not, thus I will add a 'nan' option to ft_preproc_padding which pads nans, thereby increasing the #samples of the time-axis but won't affect any numerical computation on the time-axis (as long as we are working with nan*-functions). svn ci ft_preprocessing.m test/test_bug2220.m -m "bugfix #2220 - baselining in preprocessing fixed" Sending ft_preprocessing.m Adding test/test_bug2220.m Transmitting file data .. Committed revision 8314. @Flor: I think this must work, given all the tests I ran. However your data is really small in magnitude in the order of 10^-11, and after baseline correction the baseline period is not exactly 0 but 10^-13. This gets really close to the numerical accuracy of any double-integer and I don't think there's a way to make it more accurate. thanks for reporting, and my sincere apologies for introducing this error :/ please check whether everything is according to your expectations and desires, let me know if not.
Jörn M. Horschig - 2013-07-15 12:29:27 +0200
svn ci preproc/ft_preproc_padding.m -m "added 'nan'-padding" Sending preproc/ft_preproc_padding.m Transmitting file data . Committed revision 8315. http://code.google.com/p/fieldtrip/source/detail?r=8314 http://code.google.com/p/fieldtrip/source/detail?r=8315