Back to the main page.

Bug 3063 - build support for Nihon Koden M00 data files

Reported 2016-02-04 12:15:00 +0100
Modified 2016-08-31 11:58:36 +0200
Product: FieldTrip
Component: core
Version: unspecified
Hardware: PC
Operating System: Linux
Importance: P5 normal
Assigned to: Diego Lozano Soldevilla
Depends on:
See also:

Diego Lozano Soldevilla - 2016-02-04 12:15:42 +0100

I will create support for Nihon Koden *.M00 datafiles in ft_read_header and ft_read_event. There'll be a fileio/private/read_nihonkoden.m that will do the work.

Robert Oostenveld - 2016-02-04 12:46:13 +0100

(In reply to Diego Lozano Soldevilla from comment #0) please try to stay close to (or simply copy) the EEGLAB reader (that apparently exists) so that we can in the future easily share improvements back and forth between the projects. I realize that for EDF and Neuroscan have moved away from each other, which takes extra efforts in benefitting from each others improvements.

Robert Oostenveld - 2016-02-04 12:48:46 +0100

Oh, and please do it according to, where you make a fork (on github) of the fieldtrip/fieldtrip repo, and then a clone (on your computer) of your dieloz/fieldtrip fork.

Diego Lozano Soldevilla - 2016-02-04 14:15:42 +0100

(In reply to Robert Oostenveld from comment #1) Hi Robert, Yes I though the same but I found problems. I downloaded an external eeglab plugin ( ; Import Nihon Koden M00 files (beta)) but looking at the code I'd need more editing. Reading the *.set *.fdt using the eeglab2fieldtrip function gave error because there's no channel labels between others. At first glance, the plugin does not seem to get the channel labels and header info. This is the reason I decided to do work from scratch

Diego Lozano Soldevilla - 2016-02-04 14:23:34 +0100

(In reply to Robert Oostenveld from comment #2) I did it for the SMI implementation: Was it wrong? I did my best with git but I'm still a rookie ;)

Robert Oostenveld - 2016-02-04 17:03:15 +0100

(In reply to Diego Lozano Soldevilla from comment #4) that was good!

Diego Lozano Soldevilla - 2016-07-26 12:11:28 +0200

(In reply to Robert Oostenveld from comment #5) Dear Chen and Lu, I'm implementing the reading functions for the Nihon Koden data. I've few simple questions: 1) I guess the trigger channel you're using is 'DC01' (number 196) Could you please confirm to me that? 2) In case channel 'DC01' is the trigger channel, I see many jumps in the TTL recordings (see attached figure). Do you consider a trigger x consecutive number of up/down states? For now I'm making events for anything is different from zero (positve and negative). Please let me know how do you deal with that.

Lu - 2016-07-26 13:34:39 +0200

Created attachment 804 transf_mark_m00

Diego Lozano Soldevilla - 2016-07-26 13:40:51 +0200

Created attachment 805 picture triggers

Diego Lozano Soldevilla - 2016-07-26 13:41:57 +0200

(In reply to Diego Lozano Soldevilla from comment #8) thank you Lu!

Lu - 2016-07-26 13:47:45 +0200

(In reply to Diego Lozano Soldevilla from comment #9) could you see my comment about the trigger channels? I submit it but I can't find my comment here!

Lu - 2016-07-26 13:55:07 +0200

(In reply to Diego Lozano Soldevilla from comment #6) Hi diego, (1) Actually,we send a 5V of pulse to the DC channels to make triggers.We record four trigger channels: DC09, DC10, DC11, DC12, which simulate the binary system. The correspondences between the signal on DC channels and event marks are as follows: DC09 --> 1, DC10 --> 2, DC11 --> 4, DC12 --> 8. Therefore, our event mark could be varied from 1 to 15. (2) Now we make events if the signal on DC channels is larger than 1V. Because the signal on DC channel is not stable sometimes, if the value was too small, it may recognize the wrong event. But there is no standard answer I think. I attatch our relevant script here which may give you more information.

Diego Lozano Soldevilla - 2016-07-26 14:43:29 +0200

(In reply to Lu from comment #11) Thanks a lot Lu: I'll take it into account!

Diego Lozano Soldevilla - 2016-07-26 14:53:53 +0200

(In reply to Diego Lozano Soldevilla from comment #12) Hi Lu, Just another clarification. In the file you sent me as example (chen_part1.m00) there aren't the channel triggers you mentioned in your last reply... I only have DC01 but no DC09, DC10, DC11, DC12. Am I have a file that it's the exception? I also have a channel named "Trigger" but It's plenty of zeros

Lu - 2016-07-26 15:20:32 +0200

(In reply to Diego Lozano Soldevilla from comment #13) sorry about that! I'm afraid I send you a defective data. Now I will send you another one. But it will take some time.

Diego Lozano Soldevilla - 2016-07-26 15:22:06 +0200

(In reply to Lu from comment #14) Thank you very much for all your help Lu! No worries: so far I can still work simulating the trigger channels and adapting your script. Let me know (by gmail/dropbox) how to access your new dataset and I'll work on it

Diego Lozano Soldevilla - 2016-07-27 16:26:42 +0200

(In reply to Diego Lozano Soldevilla from comment #15) Hi Lu, I made a draft and I found the following triggers in your dataset: 2 3 4 5 6 7 8 15. Does it sound ok to you?

Lu - 2016-07-27 16:36:42 +0200

(In reply to Diego Lozano Soldevilla from comment #16) yes, that's right.

Diego Lozano Soldevilla - 2016-07-27 16:42:38 +0200

(In reply to Lu from comment #17) Cool! Then it's working as expected ;)

Diego Lozano Soldevilla - 2016-08-30 14:50:46 +0200

(In reply to Diego Lozano Soldevilla from comment #18) After the workshop, people from guangzhou warned me that the trigger code was not correct. They provided me with scripts to get the proper triggering output. Remains to be improved the following: - Speed up trigger reading and data reading: ft_read_events and ft_read_header load all data and look for the 4 channel triggers (DC09, DC10, DC11, DC12). I took eyelink_asc strategy to speed up data reading (keeping all it in memory). However fileio/private/read_nihonkhoden_m00.m (data reading) and read_nihonkohden_hdr.m (header reading) need to be improved to speed up data reading using other function faster than textscan. Question: does somebody know an ascii matlab function to read fast only the proper channel trigger (sampled by columns) to speed up event/data reading? Nihon Khoden uses kind of DC channels to compute triggers from 1:15. Somebody could point me to a fileformat that solves these issues similarly!

Robert Oostenveld - 2016-08-31 11:38:38 +0200

(In reply to Diego Lozano Soldevilla from comment #19) Do we have a demo file somewhere? Could you send that to me (not attached here)? If it is ascii, it is hard to speed up the actual reading. It is not possible to jump to a well-known location in the file to a specific channel. The approach I use for other "slow" files is to read it once and keep it persistent in memory. This starts at line 203 in ft_read_header if isempty(cache) if any(strcmp(headerformat, {'bci2000_dat', 'eyelink_asc', 'gtec_mat', 'mega_neurone', 'smi_txt', 'biosig'})) cache = true; else cache = false; end end and continues in ft_read_data. If event reading and parsing is done with ft_read_event and private/read_trigger, then the caching would be used automatically (as that calls ft_read_data). Note that it seems to me (browsing the code) that caching is not 100% complete and consistent for all formats. But mega_neurone is a recent and complete example.

Robert Oostenveld - 2016-08-31 11:42:48 +0200

Some ideas, browsing through the previous comments - fixing "lab-specific" trigger inconsistencies should not be done in the low-level code, but in a trialfun. So for the trigger problems it should be checked whether the file is interpreted correctly and whether the local stimulus presentation PC was sending overlapping TTL pulses (which sometimes happens). - persistent reading of the files is best left to the "high" level ft_read_xxx code and not in the low-level code, as that makes it easier to maintain.

Robert Oostenveld - 2016-08-31 11:58:36 +0200

do you have the updated FT code in a git branch and could you make a pull request out of it? ... I looked it up myself, and yes: there is I suggest to merge the two reading functions, and give them it an interface like [hdr, dat] = read_nihonkoden_m00(filename, begsample, endsample) if nargout==1 % only read and return hdr else % read header and data, return both % here it would be optional whether begsample/endsample are specified and used % if not specified, read all data end That makes it easy to add the data to hdr.orig.dat at the level of ft_read_header and to reuse it in ft_read_data (where the requested subsection is made), preventing the ascii file from being read multiple times. The fast way (cached) would then be to read all data samples and channels in ft_read_header, store it in hdr.orig.dat and make subsections from that matrix in ft_read_data. Oh, I realize that earlier I confused the 'cache' option and adding the data to the hdr.orig field with the use of a persistent variable. It is the combination of both that makes the reading fast throughout.