Back to the main page.
Bug 1021 - sort out all the appenddata/time/freq/source and ft_selectdata bugs
| Status | CLOSED FIXED |
| Reported | 2011-10-09 12:02:00 +0200 |
| Modified | 2016-05-09 08:50:47 +0200 |
| Product: | FieldTrip |
| Component: | core |
| Version: | unspecified |
| Hardware: | PC |
| Operating System: | Mac OS |
| Importance: | P1 normal |
| Assigned to: | Jan-Mathijs Schoffelen |
| URL: | |
| Tags: | |
| Depends on: | 63774574682785192198710141146123215141985219121972224236323942444250825092538 |
| Blocks: | |
| See also: |
Robert Oostenveld - 2011-10-09 12:02:06 +0200
A whole list of bugs have been collected that either pertain to selectdata or to the append functions, i.e. #851, #921, #987, #637, #746, #1014 We should design a coherent solution and implement a consistent way to deal with this.
Stephen Whitmarsh - 2011-10-10 10:42:30 +0200
come by any time for some examples :)
Boris Reuderink - 2011-11-17 10:46:35 +0100
Changed the status of bugs without a specific owner to UNCONFIRMED. I'll try to replicate these bugs (potentially involving the submitter), and change confirmed bugs to NEW. Boris
Robert Oostenveld - 2012-01-11 17:50:58 +0100
I updated the "depends on" field of this bug. The list of related bugs that I was able to identify consists of bug 498, 637, 745, 746, 827, 851, 921, 987, 1014, 1146, 1232.
Robert Oostenveld - 2012-01-11 17:52:10 +0100
(In reply to comment #3) another way to get the list is by searching like this http://bugzilla.fcdonders.nl/buglist.cgi?quicksearch=ft_selectdata
Jan-Mathijs Schoffelen - 2013-02-20 13:35:14 +0100
Issues: ft_selectdata_old should be deprecated. In order for this to be possible: -ensure that the same (selecting) functionality is in ft_selectdata_new -ensure that the same (append) functionality is in ft_append... Further: -make inventory of high level functions that currently depend on ft_selectdata / ft_appendXXX -make test script
Jan-Mathijs Schoffelen - 2013-02-20 13:42:36 +0100
bash-3.2$ grep -l selectdata *.m */*.m ft_appendsource.m ft_channelnormalise.m ft_channelrepair.m ft_componentanalysis.m ft_connectivityanalysis.m ft_connectivityplot.m ft_denoise_cca.m ft_denoise_pca.m ft_denoise_synthetic.m ft_denoise_tsr.m ft_freqanalysis.m ft_freqdescriptives.m ft_freqstatistics.m ft_math.m ft_megplanar.m ft_megrealign.m ft_multiplotER.m ft_preprocessing.m ft_qualitycheck.m ft_redefinetrial.m ft_rejectvisual.m ft_resampledata.m ft_scalpcurrentdensity.m ft_singleplotER.m ft_sourcegrandaverage.m ft_sourcestatistics.m ft_timelockanalysis.m compat/ft_freqanalysis_old.m private/rejectvisual_channel.m private/selparam.m testing/test_bug97.m testing/test_ft_multiplotER.m testing/test_ft_multiplotTFR.m testing/test_ft_singleplotER.m testing/test_ft_topoplotER.m testing/test_selectdata_freqdata.m testing/test_selectdata_sourcedata.m test/test_bug1014.m test/test_bug1409.m test/test_bug1483.m test/test_bug1677.m test/test_bug1800.m test/test_bug798.m test/test_bug921.m test/test_bug932.m test/test_bug97.m test/test_example_preprocessing_eeg.m test/test_ft_selectdata.m test/test_ft_timelockanalysis_new.m test/test_ft_topoplotER.m test/test_tutorial_connectivity3.m utilities/ft_selectdata.m utilities/ft_selectdata_new.m utilities/ft_selectdata_old.m My guess is that most of the functionality is related to selecting trials.
Jan-Mathijs Schoffelen - 2013-02-20 13:48:17 +0100
bash-3.2$ grep -l ft_append *.m */*.m Contents.m ft_appenddata.m ft_appendfreq.m ft_appendsource.m ft_appendspike.m ft_appendtimelock.m ft_denoise_pca.m ft_denoise_tsr.m testing/test_appenddata_2datasets.m testing/test_trialdef.m test/test_bug1014.m test/test_bug1397.m test/test_bug1735.m test/test_bug798.m test/test_bug931.m test/test_bug932.m test/test_bug946.m test/test_example_preprocessing_eeg.m test/test_ft_appendfreq.m test/test_ft_selectdata.m test/test_tutorial_beamformer.m test/test_tutorial_coherence.m test/test_tutorial_connectivity3.m utilities/ft_selectdata_old.m
Jan-Mathijs Schoffelen - 2013-02-20 14:19:59 +0100
functionality of ft_selectdata used in those highlevel function depending on it. 'rpt': ft_channelnormalise.m ft_channelrepair.m ft_componentanalysis.m ft_denoise_pca.m ft_denoise_synthetic.m ft_denoise_tsr.m ft_freqanalysis.m ft_megplanar.m ft_megrealign.m ft_preprocessing.m ft_redefinetrial.m ft_rejectvisual.m ft_resampledata.m ft_scalpcurrentdensity.m ft_timelockanalysis.m 'param': ft_appendsource.m -> this is actually functionality to be implemented in the function itself, not calling ft_selectdata(_old) 'channel'; ft_connectivityplot.m ft_denoise_cca.m ft_freqanalysis.m private/rejectvisual_channel.m 'toilim': ft_qualitycheck.m ft_connectivityanalysis: 'jackknife', 'avgoverrpt', 'channel' ft_freqdescriptives: 'foilim', 'toilim', 'rpt', 'channel', 'jackknife' ft_multiplotER: 'channel', 'param', 'foilim', 'toilim', 'avgoverXXX' ft_singleplotER: 'channel', 'param', 'foilim', 'toilim', 'avgoverXXX' ft_math.m: I wonder whether this will work in the first place. ft_sourcegrandaverage: 'param', 'avgoverrpt' -> should call ft_appendsource
Jan-Mathijs Schoffelen - 2013-10-26 18:15:15 +0200
oops. adding a 'depend on', apparently removes all the other ones :-(
Jan-Mathijs Schoffelen - 2013-10-29 12:00:45 +0100
completed the functionality (in ft_selectdata_new, where the behavior should match the ft_selectdata_old) for freq-data (including testing it with the test function), that is: with respect to the selection part. The 'avgoverXXX' needs to be checked still. (Revision 8665)
Robert Oostenveld - 2014-04-10 18:01:49 +0200
I completely revamped the ft_selectdata_new function, resulting in ft_selectdata_newer mac011> svn commit Sending ft_selectdata.m Adding ft_selectdata_newer.m Adding private/getdimord.m Adding private/getdimsiz.m Transmitting file data .... Committed revision 9393. I have tested it with a large number of test scripts prior to committing, but don't want to enable it by default without running all test scripts. I enabled the new code for me only using the so called "jansch"-hack, i.e. I check for the username and if it is mine, it uses the newer implementation. Let's see what the dashboard reports...
Robert Oostenveld - 2014-04-11 08:48:49 +0200
(In reply to Robert Oostenveld from comment #11) > Let's see what the dashboard reports... It ran through this night, as visible from the log files that mention ft_selectdata_newer. There are no new problems, which means that none of the test scripts broke due to the newer implementation.
Robert Oostenveld - 2014-04-11 08:50:33 +0200
the selection of channels and trials in raw data is still dealt with in a different manner than all other selections. Also in relation to bug 1514, it is desirable to remove that exception. I will now work on that, and look into the other open bugs that pertain to data selection (as linked here).
Robert Oostenveld - 2014-04-11 13:51:14 +0200
mac011> svn commit Sending test/test_ft_selectdata.m Adding test/test_ft_selectdata_union.m Sending utilities/ft_selectdata_new.m Sending utilities/ft_selectdata_newer.m Transmitting file data .... Committed revision 9394. I made changes to the newest implementation pertaining to - use the same code for raw as for all others, i.e. with {rrpt}_chan_time - bug fixes for selmode=union
Robert Oostenveld - 2014-04-15 21:43:29 +0200
I incorporated the newer implementation of ft_selectdata in the main function. mac011> svn commit private/rollback_provenance.m utilities/ft_selectdata.m Sending private/rollback_provenance.m Sending utilities/ft_selectdata.m Transmitting file data .. Committed revision 9424.