Back to the main page.
Bug 2255 - cfg.previous is very large after source statistics slowing matlab workspace managing
| Status | CLOSED WONTFIX |
| Reported | 2013-08-19 15:05:00 +0200 |
| Modified | 2019-08-10 12:32:38 +0200 |
| Product: | FieldTrip |
| Component: | core |
| Version: | unspecified |
| Hardware: | PC |
| Operating System: | Linux |
| Importance: | P3 normal |
| Assigned to: | |
| URL: | |
| Tags: | |
| Depends on: | |
| Blocks: | 2101 |
| See also: |
Diego Lozano Soldevilla - 2013-08-19 15:05:22 +0200
After source statistics cfg.previous can be very large ~7GB slowing down matlab's workspace managing. Display the stat output variable can take a minute in some cases. Robert believes that the large number of .previous field can fragment matlab's memory, making variable manipulations highly unefficient. As cfg is very large, a cfg sample is saved in home/common/4robert folder
Robert Oostenveld - 2013-08-20 14:07:49 +0200
Thanks Diego, I copied the 16GB file to the appropriate location. roboos@mentat001> cd /home/common/temporary/4robert/ roboos@mentat001> ls -l total 16513984 -rw-r--r-- 1 dieloz electromag 16843980017 Aug 19 15:43 statcfg.mat roboos@mentat001> cp statcfg.mat /home/common/matlab/fieldtrip/data/test/bug2255.mat
Roemer van der Meij - 2013-08-21 15:36:45 +0200
Might be very related to bug 2101. Part of my last comment there: ******** For every subject many multiple copies of the grads, vols and grids are kept, many in nested usercfgs. Single large fields (like vol.tri) are removed because of checkconfig, but there are many other ones which are present very often (grad, grid, vol). Examples: vol.pnt 4000x3 double grad.chanori 302x3 double grad.chanpos 302x3 double grad.chantype 302x1 cell grad.coilori 595x3 double grad.coilpos 595x3 double grad.label 302x1 cell grad.balance.G1BR.labelorg 281x1 cell grad.balance.G1BR.labelnew 281x1 cell grad.balance.GX1BR... grid.inside 1x2998 double grid.inside 1x2782 double For all 24 subjects they are present at least several times, and some of the field are present inside themselves as well (e.g. grid.inside and grid.cfg.grid.inside). ********
Arjen Stolk - 2014-04-06 22:37:00 +0200
This is still an issue to date. E.g. group level statistics at the source level, of about 50 subjects: >> whos Name Size Bytes Class Attributes cfg_of_stat 1x1 1264681528 struct stat 1x1 1265170684 struct where the cfg field of the stat structure is large due to the previous field: previous_of_cfg_stat 1x49 1264557902 cell see Roemer's post for details on which fields are still represented.
Jan-Mathijs Schoffelen - 2016-03-01 17:16:01 +0100
It sounds that this is just a given, due to the provenance keeping. Are we going to do anything with this information? If not, I suggest to close.
Diego Lozano Soldevilla - 2016-03-01 17:19:49 +0100
(In reply to Jan-Mathijs Schoffelen from comment #4) you can close it
Roemer van der Meij - 2016-03-03 14:30:04 +0100
Let's close it. Commenting to add bug 2101 explicitly to this bug, which should also be closed.
Jan-Mathijs Schoffelen - 2016-03-03 14:50:26 +0100
change status