Back to the main page.
Bug 1318 - improve the filter padding options for the ft_preproc filter functions
| Status | CLOSED FIXED |
| Reported | 2012-02-08 13:25:00 +0100 |
| Modified | 2014-03-12 12:21:41 +0100 |
| Product: | FieldTrip |
| Component: | preproc |
| Version: | unspecified |
| Hardware: | PC |
| Operating System: | Mac OS |
| Importance: | P3 normal |
| Assigned to: | Jörn M. Horschig |
| URL: | |
| Tags: | |
| Depends on: | 1730 |
| Blocks: | |
| See also: | http://bugzilla.fcdonders.nl/show_bug.cgi?id=1561 |
Robert Oostenveld - 2012-02-08 13:25:12 +0100
Vladimir mentioned to me that it is desired for ft_specest_hilbert to have better filter padding. Since the filtering is done with ft_preproc functions, the improvement has to be implemented there. The idea is that mirroring the data at the edges might help. TODO - create a test case (named test_bugXXX.m with XXX the right number) that demonstrates a case with hand-crafted "random" data which would benefit from improved edge padding.
Jörn M. Horschig - 2012-06-27 16:24:06 +0200
Hey Vladimir, do you mind if I take this one up? It would be in line with bug 1561 on which Robert and we just worked a few minutes ago, so this one would be a quick fix now that I just started digging into the filtering stuff ;) Best, Jörn
Vladimir Litvak - 2012-06-27 16:29:39 +0200
Hi Jorn, I would be very happy if you take this. Actually I haven't realized till now that I was supposed to work on this at all. Good luck, Vladimir
Jörn M. Horschig - 2012-06-27 16:54:53 +0200
thanks ;)
Jörn M. Horschig - 2012-08-21 14:49:01 +0200
hm, two possibilities: either the high-level function (ft_preprocessing) will pad the mirrored data and remove it (my favourite) or the low-level function will do that (not sure how, some filthy trick like negative cfg.padding, or new option like cfg.mirrorpad). since I prefer the first solution, I thought about it such that mirror padding can only be done for in-memory data and will automatically be done if cfg.padding is used for in-memory data. anyone who wants to do mirror padding without going through the high-level function has to do that manually. any thoughts on this?
Jörn M. Horschig - 2012-08-21 14:50:42 +0200
uh, just saw that the unpadding is already done by the low-level function - so only the question whether the high- or low-level function should do the padding remains
Jörn M. Horschig - 2012-08-22 14:25:30 +0200
agreed upon in Ft meeting. Vladimir, is it okay for you if the padding is done in ft_preprocessing and not in the low-level functions? I just had a quick look at specest_hilbert and saw that the function is actually doing some cranky stuff which has absolutely nothing to do ft_preprocessing, so this solution does not help you. Is mirrorpadding always more favourable than zeropadding for specest_hilbert? I could then replace the zeropadding there
Vladimir Litvak - 2012-08-22 14:32:35 +0200
Hi Jorn, ft_preprocessing is not good as we don't use it. If it's in ft_preproc_... functions then it's OK. I took the mirror padding idea from the Cardiff group who actually use Hilbert for spectral estimation. I don't have much experience with it myself but my feeling is it's quite a good idea also for filtering in general. Vladimir
Robert Oostenveld - 2012-08-22 15:59:28 +0200
in the FT meeting one hour ago we discussed it, but there it was not clear that it was motivated by ft_spectest_hilbert. This changes the situation and necessitates that it is in either ft_preproc_bandpassfilter or in ft_spectest_hilbert itself. I suggest to change it like function [filt] = ft_preproc_bandpassfilter(dat, Fs, Fbp, N, type, dir) function [filt] = ft_preproc_bandpassfilter(dat, Fs, Fbp, N, type, dir, pad) where pad is a string, which can be zero, mirror, edge. The default (current) implementation is zero. mirror would be tmpdat = [fliplr(dat) dat dat]; and edge tmpdat = [repmat(dat(:,1),1,n) dat repmat(dat(:,end),1,n)]; I would keep the mean subtraction as it is, that will always be beneficial (or neutral). The same can be implemented in low/high and bandstop. The option should be passed somehow to the user in a cfg as cfg.bandpassfiltpad and to ft_specest_hilbert as filtpad (because that one only has one type of filter, whereas ft_preprocessing has multiple).
Roemer van der Meij - 2012-08-22 16:29:27 +0200
(In reply to comment #8) I that case we would also need two padding parameters I think, i.e. padtype and padlength: padtype = 'zero', 'edge', 'mirror' padlength = number, samples needed to be padded out to To keep things consistent, it might better to create a small function like: ft_preproc_padding(dat, padtype, padlength, mode) (mode being 'add', or 'remove') This can then be called from all filter functions, with the default being padtype = []; Trivial, but for mirroring it should be: tmpdat = [fliplr(dat) dat fliplr(dat)]; Or, for longer sizes ;), tmpdat = [fliplr(dat) dat fliplr(dat) dat fliplr(dat) dat fliplr(dat)];
Jörn M. Horschig - 2012-10-24 16:45:49 +0200
function [dat] = ft_preproc_padding(dat, padtype, padlength) % FT_PREPROC_PADDING performs padding on the data, i.e. adds or removes % samples to/from the data matrix. % % Use as % [dat] = ft_preproc_padding(dat, padtype, padlength) % where % dat data matrix (Nchan1 X Ntime) % padtype 'zero', 'edge', 'mirror' or 'remove' % padlength scalar, number of samples that will be padded 513 $ svn add ft_preproc_padding.m A ft_preproc_padding.m jorhor@mentat284:~/FieldTrip/trunk/preproc 514 $ svn ci ft_preproc_padding.m -m "enhancement-#1318- added new preproc_padding function that should (later) take over all padding apart from datapadding" Adding ft_preproc_padding.m Transmitting file data . Committed revision 6786. Neither did I test the function nor did is it used anywhere, yet. The mirror padding, however, turned out to be more complex if done properly: 1) also with mirror padding, only a certain number of samples to be padded is desired (imagine a 5minute piece of data would be mirrored... hello mr. memory problem!) 2) as Roemer pointed out, if the to-be-padded data is more than the # samples available, padding has to be reversed 3) we do not want the first (and last) sample each to be duplicated (i.e. we do not want [1 2 3 3 2 1] but rather [1 2 3 2 1]) Gonna replace all padding procedures by calls to this function next week. In the meanwhile, Vladimir, is is alright for you to have it this way?
Jörn M. Horschig - 2012-10-24 16:51:00 +0200
add mean-padding in line with ft_preproc_smooth
Jörn M. Horschig - 2012-11-16 14:31:21 +0100
Vladimir, ft_specest_hilbert could get new key-value pairs 'padtype' and 'padlength' and then call ft_preproc_padding. Default would be no padding - agree? The type of filtering can be set manual, different methods are implemented. On a more general note: the function 'preproc' currently supports different length for padding in the beginning and in the end of a data matrix. Should this be supported by the new function? Currently it's not.
Jörn M. Horschig - 2012-11-16 14:31:51 +0100
(In reply to comment #12) *the type of padding not filtering, of course...
Jörn M. Horschig - 2012-11-16 14:34:19 +0100
ahjaja, and I see default behaviour is zeropadding, that would be kept as it is of course...