Back to the main page.
Bug 3063 - build support for Nihon Koden M00 data files
| Status | ASSIGNED |
| Reported | 2016-02-04 12:15:00 +0100 |
| Modified | 2016-08-31 11:58:36 +0200 |
| Product: | FieldTrip |
| Component: | core |
| Version: | unspecified |
| Hardware: | PC |
| Operating System: | Linux |
| Importance: | P5 normal |
| Assigned to: | Diego Lozano Soldevilla |
| URL: | |
| Tags: | |
| Depends on: | |
| Blocks: | |
| See also: |
Diego Lozano Soldevilla - 2016-02-04 12:15:42 +0100
I will create support for Nihon Koden *.M00 datafiles in ft_read_header and ft_read_event. There'll be a fileio/private/read_nihonkoden.m that will do the work.
Robert Oostenveld - 2016-02-04 12:46:13 +0100
(In reply to Diego Lozano Soldevilla from comment #0) please try to stay close to (or simply copy) the EEGLAB reader (that apparently exists) so that we can in the future easily share improvements back and forth between the projects. I realize that for EDF and Neuroscan have moved away from each other, which takes extra efforts in benefitting from each others improvements.
Robert Oostenveld - 2016-02-04 12:48:46 +0100
Oh, and please do it according to http://www.fieldtriptoolbox.org/development/git, where you make a fork (on github) of the fieldtrip/fieldtrip repo, and then a clone (on your computer) of your dieloz/fieldtrip fork.
Diego Lozano Soldevilla - 2016-02-04 14:15:42 +0100
(In reply to Robert Oostenveld from comment #1) Hi Robert, Yes I though the same but I found problems. I downloaded an external eeglab plugin (http://sccn.ucsd.edu/wiki/EEGLAB_Extensions ; Import Nihon Koden M00 files (beta)) but looking at the code I'd need more editing. Reading the *.set *.fdt using the eeglab2fieldtrip function gave error because there's no channel labels between others. At first glance, the plugin does not seem to get the channel labels and header info. This is the reason I decided to do work from scratch
Diego Lozano Soldevilla - 2016-02-04 14:23:34 +0100
(In reply to Robert Oostenveld from comment #2) I did it for the SMI implementation: https://github.com/fieldtrip/fieldtrip/pull/74 Was it wrong? I did my best with git but I'm still a rookie ;)
Robert Oostenveld - 2016-02-04 17:03:15 +0100
(In reply to Diego Lozano Soldevilla from comment #4) that was good!
Diego Lozano Soldevilla - 2016-07-26 12:11:28 +0200
(In reply to Robert Oostenveld from comment #5) Dear Chen and Lu, I'm implementing the reading functions for the Nihon Koden data. I've few simple questions: 1) I guess the trigger channel you're using is 'DC01' (number 196) Could you please confirm to me that? 2) In case channel 'DC01' is the trigger channel, I see many jumps in the TTL recordings (see attached figure). Do you consider a trigger x consecutive number of up/down states? For now I'm making events for anything is different from zero (positve and negative). Please let me know how do you deal with that.
Lu - 2016-07-26 13:34:39 +0200
Created attachment 804 transf_mark_m00
Diego Lozano Soldevilla - 2016-07-26 13:40:51 +0200
Created attachment 805 picture triggers
Diego Lozano Soldevilla - 2016-07-26 13:41:57 +0200
(In reply to Diego Lozano Soldevilla from comment #8) thank you Lu!
Lu - 2016-07-26 13:47:45 +0200
(In reply to Diego Lozano Soldevilla from comment #9) could you see my comment about the trigger channels? I submit it but I can't find my comment here!
Lu - 2016-07-26 13:55:07 +0200
(In reply to Diego Lozano Soldevilla from comment #6) Hi diego, (1) Actually,we send a 5V of pulse to the DC channels to make triggers.We record four trigger channels: DC09, DC10, DC11, DC12, which simulate the binary system. The correspondences between the signal on DC channels and event marks are as follows: DC09 --> 1, DC10 --> 2, DC11 --> 4, DC12 --> 8. Therefore, our event mark could be varied from 1 to 15. (2) Now we make events if the signal on DC channels is larger than 1V. Because the signal on DC channel is not stable sometimes, if the value was too small, it may recognize the wrong event. But there is no standard answer I think. I attatch our relevant script here which may give you more information.
Diego Lozano Soldevilla - 2016-07-26 14:43:29 +0200
(In reply to Lu from comment #11) Thanks a lot Lu: I'll take it into account!
Diego Lozano Soldevilla - 2016-07-26 14:53:53 +0200
(In reply to Diego Lozano Soldevilla from comment #12) Hi Lu, Just another clarification. In the file you sent me as example (chen_part1.m00) there aren't the channel triggers you mentioned in your last reply... I only have DC01 but no DC09, DC10, DC11, DC12. Am I have a file that it's the exception? I also have a channel named "Trigger" but It's plenty of zeros
Lu - 2016-07-26 15:20:32 +0200
(In reply to Diego Lozano Soldevilla from comment #13) sorry about that! I'm afraid I send you a defective data. Now I will send you another one. But it will take some time.
Diego Lozano Soldevilla - 2016-07-26 15:22:06 +0200
(In reply to Lu from comment #14) Thank you very much for all your help Lu! No worries: so far I can still work simulating the trigger channels and adapting your script. Let me know (by gmail/dropbox) how to access your new dataset and I'll work on it
Diego Lozano Soldevilla - 2016-07-27 16:26:42 +0200
(In reply to Diego Lozano Soldevilla from comment #15) Hi Lu, I made a draft and I found the following triggers in your dataset: 2 3 4 5 6 7 8 15. Does it sound ok to you?
Lu - 2016-07-27 16:36:42 +0200
(In reply to Diego Lozano Soldevilla from comment #16) yes, that's right.
Diego Lozano Soldevilla - 2016-07-27 16:42:38 +0200
(In reply to Lu from comment #17) Cool! Then it's working as expected ;)
Diego Lozano Soldevilla - 2016-08-30 14:50:46 +0200
(In reply to Diego Lozano Soldevilla from comment #18) After the workshop, people from guangzhou warned me that the trigger code was not correct. They provided me with scripts to get the proper triggering output. Remains to be improved the following: - Speed up trigger reading and data reading: ft_read_events and ft_read_header load all data and look for the 4 channel triggers (DC09, DC10, DC11, DC12). I took eyelink_asc strategy to speed up data reading (keeping all it in memory). However fileio/private/read_nihonkhoden_m00.m (data reading) and read_nihonkohden_hdr.m (header reading) need to be improved to speed up data reading using other function faster than textscan. Question: does somebody know an ascii matlab function to read fast only the proper channel trigger (sampled by columns) to speed up event/data reading? Nihon Khoden uses kind of DC channels to compute triggers from 1:15. Somebody could point me to a fileformat that solves these issues similarly!
Robert Oostenveld - 2016-08-31 11:38:38 +0200
(In reply to Diego Lozano Soldevilla from comment #19) Do we have a demo file somewhere? Could you send that to me (not attached here)? If it is ascii, it is hard to speed up the actual reading. It is not possible to jump to a well-known location in the file to a specific channel. The approach I use for other "slow" files is to read it once and keep it persistent in memory. This starts at line 203 in ft_read_header if isempty(cache) if any(strcmp(headerformat, {'bci2000_dat', 'eyelink_asc', 'gtec_mat', 'mega_neurone', 'smi_txt', 'biosig'})) cache = true; else cache = false; end end and continues in ft_read_data. If event reading and parsing is done with ft_read_event and private/read_trigger, then the caching would be used automatically (as that calls ft_read_data). Note that it seems to me (browsing the code) that caching is not 100% complete and consistent for all formats. But mega_neurone is a recent and complete example.
Robert Oostenveld - 2016-08-31 11:42:48 +0200
Some ideas, browsing through the previous comments - fixing "lab-specific" trigger inconsistencies should not be done in the low-level code, but in a trialfun. So for the trigger problems it should be checked whether the file is interpreted correctly and whether the local stimulus presentation PC was sending overlapping TTL pulses (which sometimes happens). - persistent reading of the files is best left to the "high" level ft_read_xxx code and not in the low-level code, as that makes it easier to maintain.