Back to the main page.
Bug 1617 - check tutorials for use of *grandaverage with keepindividual='yes'
| Status | CLOSED FIXED |
| Reported | 2012-07-25 14:01:00 +0200 |
| Modified | 2018-03-10 21:33:27 +0100 |
| Product: | FieldTrip |
| Component: | documentation |
| Version: | unspecified |
| Hardware: | PC |
| Operating System: | Windows |
| Importance: | P3 normal |
| Assigned to: | |
| URL: | |
| Tags: | |
| Depends on: | 2578 |
| Blocks: | |
| See also: |
Boris Reuderink - 2012-07-25 14:01:35 +0200
and remove this obsolete step in the computation of statistics. Possibly also check other documentation.
Johanna - 2012-09-19 15:32:35 +0200
Boris, I'll take this unless you specifically want it.
Johanna - 2012-09-19 16:41:10 +0200
Sorry, I was too exuberant. I don't understand this bug well enough....
Roemer van der Meij - 2012-10-19 16:09:54 +0200
http://fieldtrip.fcdonders.nl/tutorial/cluster_permutation_timelock Shows this as well. What needs to be replaced: Change this general procedure: ***** cfg = []; cfg.keepindividual = 'yes'; gavg = ft_whatevergrandaverage(cfg,data1, data2, data3, dataN) cfg = []; cfg.... stats = ft_whateverstatistics(cfg,gavg) ***** Into this general procedure: ***** cfg = []; cfg.... stats = ft_whateverstatistics(cfg, data1, data2, data3, dataN) OR stats = ft_whateverstatistics(cfg, combineddata{:}) ***** This is important, because it forces people to realize that what they put into the statistics (and other) functions, is nothing more than a bunch of datasets after each other. This makes thinking about the design-matrix (or any option that uses multiple datasets) much easier and intuitive. The nastier thing is, is that some tutorials (like the above), don't explicitly generate the grand-averaged data-thing, but load it from a mat-file. So it would involve inserting some new code, or possibly changing some bits related to data-management.
Lilla Magyari - 2012-10-19 19:11:20 +0200
(In reply to comment #3) I made a for loop for the between trial comparison, but I haven't changed yet the between-subjects analysis.
Lilla Magyari - 2012-10-26 12:57:32 +0200
(In reply to comment #4) the old revision is restored on the ERF clusterstatistics site, because the change I made broke down the script at a later part.
Roemer van der Meij - 2012-10-29 11:10:42 +0100
Sorry if I caused any confusion. The thing that should be changed is not the computation of the average per subject, that is handled by ft_timelockanalysis of course. What should be changed is the usage of the output ft_grandaverageXXX with cfg.keepindividual=yes for statistics. I.e. using ft_grandaverageXXX as a 'gathering function' should be avoided in the documentation. If no grand-average is desired, then users should use a for loop and a cell-array to gather all the individual subjects, and use that cell-array in the call to ft_statisticsXXX. Alternatively users can also hardcode all the individual data-sets as input to the statistics function, like ft_statistics(cfg,data1,data2,dataN). That last option might be better as an example, but I don't remember whether the statistics in the tutorial is done on a lot of datasets. I.e. for 20 datasets the example will not provide a lot of clarity, and a for-loop+cell-array should be used. We could discuss in the meeting.
Lilla Magyari - 2012-10-29 14:07:35 +0100
(In reply to comment #6) hi Roemer, you did not cause any confusion. I misunderstood it (or did not pay enough attention when I was reading your comment.) Lilla
Lilla Magyari - 2013-07-23 22:23:18 +0200
The following sites contain keepindividual = 'yes': http://fieldtrip.fcdonders.nl/example/apply_clusterrandanalysis_on_tfrs_of_power_that_were_computed_with_besa?s[]=keepindividual http://fieldtrip.fcdonders.nl/tutorial/eventrelatedstatistics?s[]=keepindividual http://fieldtrip.fcdonders.nl/tutorial/cluster_permutation_timelock?s[]=keepindividual#within-subjects_experiments I re-assign this to joint development user. Lilla
Jan-Mathijs Schoffelen - 2013-11-29 11:17:26 +0100
Hi Nietzsche, could you give this one a try? I think Lilla already sums up where the documentation needs to be changed. We should then also change the corresponding test_tutorial_xxx functions. Let me know if things are unclear.
- 2013-12-02 12:17:46 +0100
I'm on it
- 2013-12-04 14:41:55 +0100
As discussed in today's FT meeting: - Remove all mentions in the documentation of <cfg.keepindividual = 'yes'> - We should remove this option from ft_XXgrandaverage - Show code to explain how each subject's data (from the same condition) should be placed into a cell array. One subject per cell alldata{k} = currsubjdat; - create function that will tease apart the concatenated matrix (Created by making cfg.keepindividual = 'yes') and place each subject's data into a separate cell - something about keeping previous cfg to keep the history? I will look into this.
Jan-Mathijs Schoffelen - 2013-12-04 15:01:29 +0100
(In reply to comment #11) To tease apart the GA.individual field: I don't know whether it's worth to write a separate function. It needs to be done only once. In other words, it does not have to be pretty code.
- 2014-01-08 16:13:41 +0100
busy restructuring the tutorial and improving the language for ease of understanding.
- 2014-01-08 19:20:34 +0100
Code: edited to fit the use of cell array as inarg for ft_timelockstatistics, instead of the grandaverage (with cfg.parameter = 'individuals'). text: edited to make tutorial more coherent and easily understandable. Previous version was a bit choppy. figures: these do not need to changed. I have tested to code to make sure that the plots still look the same. Dataset: I uploaded the new dataset to FTP but it's not working yet, I think it needs time to sync. I will check tomorrow
Jan-Mathijs Schoffelen - 2014-01-15 11:26:23 +0100
-don't forget to update the tutorial so that it uses the newly created file, rather than the old 'GA' file. -also, what about the other tutorials that use 'GA'-type of data stored on disk. -also, update the test_tutorial_xxx to reflect the code changes in the tutorial documentation
- 2014-01-15 17:36:11 +0100
I'm having some problems with the final exemplar code in this tutorial - where the within-subjects statistics is calculated and then ft_clusterplot is called. ft_clusterplot doesn't plot anything because there are no significant effects. I think this is because the neighbours that I have created differ from those in the original tutorial. The neighbours (sensors grouped on a triangular mesh) in the original tutorial are different from mine - please see the images and notes that I have made on the docx. document that I have attached to this comment. I'm wondering if I didn't call ft_prepare_neighbours properly, although I am very sure that the template and layout should be that of the CTF151 system.
- 2014-01-15 17:37:10 +0100
(In reply to n.lam from comment #16) Oh right, The original tutorial: http://fieldtrip.fcdonders.nl/tutorial/eventrelatedstatistics my updated version: http://fieldtrip.fcdonders.nl/development/eventrelatedstatistics_v2
Jan-Mathijs Schoffelen - 2014-01-16 08:52:24 +0100
where's the docx document?
- 2014-01-16 11:20:54 +0100
Created attachment 580 Figures of different output from ft_prepare_neighbours
Jörn M. Horschig - 2014-01-16 11:38:15 +0100
Two things to check: 1) I remember that the cluster in that part of the tutorial is awful, because it is marginally significant. On some runs, p is smaller than 0.05, in others it is bigger. I hope you checked multiple times :) Maybe it would be wise to add some sentences to the tutorial about this, and also point the reader to the "newly" added confidence interval for permutation tests (i.e. there is a warning if the CI includes 0.05) 2) We need to check whether the neighbours are actually different between the two method. They should be the same. The plotting obviously has to be different, because using a 2D layout there are no 3D sensor positions available. Let me have a look at this one and check what goes wrong where, because I know the prepare_neighbours code better :)
- 2014-01-16 12:22:13 +0100
Issue (1) Yes, I ran it 5 times and didn't get lucky. I can of course change the alpha level, but this could lead to rather different results.
- 2014-01-22 14:22:06 +0100
Feedback from FT meeting: - try a 1-tailed test - look at confidence interval output - limit latency (will require changing the text).
- 2014-01-31 16:46:58 +0100
1-tailed test with toi [-0.25 2] allows for significant time-channel clusters. *any shorter toi also gives significant clusters*. Using 1-tailed with toi [-0.5 2] gives NO sig. clusters. The current significant clusters are in a slightly longer time range (0.24 - 0.64) than the original (0.33 - 0.52). For the purpose of a tutorial I think this fine. Have now updated: text on the wiki to accommodate the above cfg changes, as well as updated the new figures from ft_clusterplot. @ JM - could you please take a look at the whole tutorial? I have rewritten various partso f the tutorial, updated the code, and provided cell arrays as input arguments to ft_XX_statistics. I've checked that everything works but it'd be great to have a second pair of eyes. If good, I will then update test_tutorial_eventrelatedstatistics. I would like to use this new tutorial at the Göttingen workshop next week. Thanks! N.
- 2014-02-03 14:10:41 +0100
using toi [-0.25 2] and cfg.tail = 1; still doesn't guarantee that each iteration will produce significant clusters. I will use toi [-0.25 1] and cfg.tail = 0; %2 tailed, as this has a higher likelihood of giving significant results each time (less likely to fail), and I don't need an adhoc explanation of 'teaching the user how to use cfg.tail'.
Jan-Mathijs Schoffelen - 2014-02-03 14:15:03 +0100
please do.
- 2014-02-03 14:18:21 +0100
done. test script also updated
- 2014-02-03 14:24:48 +0100
test script ran without error. Closing bug.
- 2014-02-03 14:26:01 +0100
Oops. Suppose to leave a comment and close bug in one step.
- 2014-02-03 14:26:57 +0100
Close bug (for the 3rd try).
Jörn M. Horschig - 2014-02-03 14:47:34 +0100
congratz on successfully closing :)
Roemer van der Meij - 2014-02-03 15:00:01 +0100
My congratulations as well! That totally deserves cake next wednesday?
- 2014-02-03 15:01:21 +0100
haha, sorry for spamming due to my inability to close :p
- 2014-02-03 15:51:42 +0100
For some reason, I totally didn't see this... tutorials (1) cluster_permutation_timelock (2) cluster_permutation_freq also have cfg.keepindividuals = 'yes'. I will update these soon.
Jan-Mathijs Schoffelen - 2014-05-14 13:33:16 +0200
We discussed at the FT meeting that last week at the toolkit this was (again) causing confusion. Is it clear to you how to proceed on this one?
Jan-Mathijs Schoffelen - 2014-05-14 13:34:29 +0200
(Robert here): also, ft_math should be used to compute the raw effect rather than using plain MATLAB.
Jörn M. Horschig - 2014-05-14 13:42:36 +0200
(In reply to Jan-Mathijs Schoffelen from comment #35) see also bug #2578
- 2014-06-18 16:21:41 +0200
finally, updated code and text for the tutorial on cluster_permutation_timelock. That code is quite old!
- 2014-06-18 17:28:11 +0200
for cluster_permutation_freq: I updated the GA_TFR_orig file, saved the older one (with cfg.keepindividual = 'yes') under this name "GA_TFR_orig_old_retainsindividualtrials", and edited some text.