J Korean Med Sci.  2017 Aug;32(8):1243-1250. 10.3346/jkms.2017.32.8.1243.

Decoding Saccadic Directions Using Epidural ECoG in Non-Human Primates

• Lee J
• Choi H
• Lee S
• Cho BH
• Ahn KH
• Kim IY
• Lee KM
• Jang DP

• KMID: 2439456
• DOI: http://doi.org/10.3346/jkms.2017.32.8.1243

Abstract

A brain-computer interface (BCI) can be used to restore some communication as an alternative interface for patients suffering from locked-in syndrome. However, most BCI systems are based on SSVEP, P300, or motor imagery, and a diversity of BCI protocols would be needed for various types of patients. In this paper, we trained the choice saccade (CS) task in 2 non-human primate monkeys and recorded the brain signal using an epidural electrocorticogram (eECoG) to predict eye movement direction. We successfully predicted the direction of the upcoming eye movement using a support vector machine (SVM) with the brain signals after the directional cue onset and before the saccade execution. The mean accuracies were 80% for 2 directions and 43% for 4 directions. We also quantified the spatial-spectro-temporal contribution ratio using SVM recursive feature elimination (RFE). The channels over the frontal eye field (FEF), supplementary eye field (SEF), and superior parietal lobule (SPL) area were dominantly used for classification. The α-band in the spectral domain and the time bins just after the directional cue onset and just before the saccadic execution were mainly useful for prediction. A saccade based BCI paradigm can be projected in the 2D space, and will hopefully provide an intuitive and convenient communication platform for users.

Keyword

Brain-Computer Interfaces; Saccade; Non-Human Primate; Epidural ECoG

MeSH Terms

Brain
Brain-Computer Interfaces
Classification
Cues
Eye Movements
Frontal Lobe
Haplorhini
Humans
Parietal Lobe
Primates*
Quadriplegia
Saccades
Support Vector Machine

Figure

• 

  Fig. 1 Electrode position and CS task. Two types of electrode patches were implanted into the monkey's cerebral cortex. (A) The rectangular type (4 by 8) of the electrode patches were implanted in the left hemisphere for monkey 14, and the circular type (32 channels) of electrode patches were inserted along the central line for monkey 5. Normal channels are shown as yellow circles, and bad channels are shown as gray circles. (B) The associations between color and spatial location shown were pre-trained before the inactivation experiments. (C) The visual events in the trial are schematically depicted along the time line: the appearance of the fixation target at the center, display of four alternative gray targets in the periphery, the onset of a color cue at the center, the color-cue turning-off signaling when to make the saccade, and the saccade to the color-matched target.CS = choice saccade.

• 

  Fig. 2 Decoding accuracy. The decoding accuracy was calculated in three conditions (left vs. right, top vs. bottom, and 4 directions), 3 sessions, and 2 epoch types (TG and SS). The decoding performance was significantly higher than the chance level (50% for 2 classes and 25% for 4 classes) for M14 (A, B, C) and M5 (D, E, F), respectively. The horizontal line means the chance level.TG = target on, SS = saccade start.

• 

  Fig. 3 Confusion matrix on the condition of the “TG” and “SS.” The confusion matrix of the actual 4 directions (45°, 135°, 225°, 315°) and the predicted directions was calculated for 2 epoch types for Monkey 14 (A, B) and Monkey 5 (C, D), respectively.TG = target on, SS = saccade start.

• 

  Fig. 4 Spatial-spectro-temporal contribution ratio. (A-D) The channel contribution weight is presented as a spatial map. (E-H) The frequency contribution was presented as the weight ratio of each frequency band: α, β, low γ, and high γ. (I-L) The time contribution weight ratio graph. Error bars indicates SEM.SEM = standard error of the mean.

Reference

1. Wolpaw JR, Birbaumer N, McFarland DJ, Pfurtscheller G, Vaughan TM. Brain-computer interfaces for communication and control. Clin Neurophysiol. 2002; 113:767–791. PMID: 12048038.

2. Lebedev MA, Nicolelis MA. Brain-machine interfaces: past, present and future. Trends Neurosci. 2006; 29:536–546. PMID: 16859758.

3. Waldert S, Pistohl T, Braun C, Ball T, Aertsen A, Mehring C. A review on directional information in neural signals for brain-machine interfaces. J Physiol Paris. 2009; 103:244–254. PMID: 19665554.

4. Mak JN, Arbel Y, Minett JW, McCane LM, Yuksel B, Ryan D, Thompson D, Bianchi L, Erdogmus D. Optimizing the P300-based brain-computer interface: current status, limitations and future directions. J Neural Eng. 2011; 8:025003. PMID: 21436525.

5. Santhanam G, Ryu SI, Yu BM, Afshar A, Shenoy KV. A high-performance brain-computer interface. Nature. 2006; 442:195–198. PMID: 16838020.

6. Velliste M, Perel S, Spalding MC, Whitford AS, Schwartz AB. Cortical control of a prosthetic arm for self-feeding. Nature. 2008; 453:1098–1101. PMID: 18509337.

7. Ifft PJ, Shokur S, Li Z, Lebedev MA, Nicolelis MA. A brain-machine interface enables bimanual arm movements in monkeys. Sci Transl Med. 2013; 5:210ra154.

8. Bjornsson CS, Oh SJ, Al-Kofahi YA, Lim YJ, Smith KL, Turner JN, De S, Roysam B, Shain W, Kim SJ. Effects of insertion conditions on tissue strain and vascular damage during neuroprosthetic device insertion. J Neural Eng. 2006; 3:196–207. PMID: 16921203.

9. Dickey AS, Suminski A, Amit Y, Hatsopoulos NG. Single-unit stability using chronically implanted multielectrode arrays. J Neurophysiol. 2009; 102:1331–1339. PMID: 19535480.

10. Schalk G, Miller KJ, Anderson NR, Wilson JA, Smyth MD, Ojemann JG, Moran DW, Wolpaw JR, Leuthardt EC. Two-dimensional movement control using electrocorticographic signals in humans. J Neural Eng. 2008; 5:75–84. PMID: 18310813.

11. Pistohl T, Schulze-Bonhage A, Aertsen A, Mehring C, Ball T. Decoding natural grasp types from human ECoG. Neuroimage. 2012; 59:248–260. PMID: 21763434.

12. Frisoli A, Loconsole C, Leonardis D, Banno F, Barsotti M, Chisari C, Bergamasco M. A new gaze-BCI-driven control of an upper limb exoskeleton for rehabilitation in real-world tasks. IEEE Trans Syst Man Cybern C Appl Rev. 2012; 42:1169–1179.

13. Zander TO, Kothe C, Jatzev S, Gaertner M. Enhancing human-computer interaction with input from active and passive brain-computer interfaces. In : Tan DS, Nijholt A, editors. Brain-computer Interfaces: Applying Our Minds to Human-computer Interaction. London: Springer;2010. p. 181–199.

14. Baek DH, Lee J, Byeon HJ, Choi H, Young Kim I, Lee KM, Jungho Pak J, Pyo Jang D, Lee SH. A thin film polyimide mesh microelectrode for chronic epidural electrocorticography recording with enhanced contactability. J Neural Eng. 2014; 11:046023. PMID: 25024292.

15. Lee KM, Ahn KH, Keller EL. Saccade generation by the frontal eye fields in rhesus monkeys is separable from visual detection and bottom-up attention shift. PLoS One. 2012; 7:e39886. PMID: 22761923.

16. Delorme A, Makeig S. EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J Neurosci Methods. 2004; 134:9–21. PMID: 15102499.

17. Nolan H, Whelan R, Reilly RB. FASTER: fully automated statistical thresholding for EEG artifact rejection. J Neurosci Methods. 2010; 192:152–162. PMID: 20654646.

18. Mitra S, Nizamie SH, Goyal N, Tikka SK. Evaluation of resting state gamma power as a response marker in schizophrenia. Psychiatry Clin Neurosci. 2015; 69:630–639. PMID: 25854748.

19. Seeber M, Scherer R, Wagner J, Solis-Escalante T, Müller-Putz GR. High and low gamma EEG oscillations in central sensorimotor areas are conversely modulated during the human gait cycle. Neuroimage. 2015; 112:318–326. PMID: 25818687.

20. Ray S, Crone NE, Niebur E, Franaszczuk PJ, Hsiao SS. Neural correlates of high-gamma oscillations (60–200 Hz) in macaque local field potentials and their potential implications in electrocorticography. J Neurosci. 2008; 28:11526–11536. PMID: 18987189.

21. Cortes C, Vapnik V. Support-vector networks. Mach Learn. 1995; 20:273–297.

22. Chang CC, Lin CJ. LIBSVM: a library for support vector machines. ACM Trans Intell Syst Technol. 2011; 2:27.

23. Cho BH, Yu H, Kim KW, Kim TH, Kim IY, Kim SI. Application of irregular and unbalanced data to predict diabetic nephropathy using visualization and feature selection methods. Artif Intell Med. 2008; 42:37–53. PMID: 17997291.

24. Cho BH, Yu H, Lee J, Chee YJ, Kim IY, Kim SI. Nonlinear support vector machine visualization for risk factor analysis using nomograms and localized radial basis function kernels. IEEE Trans Inf Technol Biomed. 2008; 12:247–256. PMID: 18348954.

25. Kübler A, Nijboer F, Mellinger J, Vaughan TM, Pawelzik H, Schalk G, McFarland DJ, Birbaumer N, Wolpaw JR. Patients with ALS can use sensorimotor rhythms to operate a brain-computer interface. Neurology. 2005; 64:1775–1777. PMID: 15911809.

26. Chao ZC, Nagasaka Y, Fujii N. Long-term asynchronous decoding of arm motion using electrocorticographic signals in monkeys. Front Neuroeng. 2010; 3:3. PMID: 20407639.

27. Goldberg ME, Segraves MA. The visual and frontal cortices. Rev Oculomot Res. 1989; 3:283–313. PMID: 2486326.

28. Hamm JP, Dyckman KA, Ethridge LE, McDowell JE, Clementz BA. Preparatory activations across a distributed cortical network determine production of express saccades in humans. J Neurosci. 2010; 30:7350–7357. PMID: 20505102.

29. Hamm JP, Sabatinelli D, Clementz BA. Alpha oscillations and the control of voluntary saccadic behavior. Exp Brain Res. 2012; 221:123–128. PMID: 22782481.

30. Romei V, Brodbeck V, Michel C, Amedi A, Pascual-Leone A, Thut G. Spontaneous fluctuations in posterior α-band EEG activity reflect variability in excitability of human visual areas. Cereb Cortex. 2008; 18:2010–2018. PMID: 18093905.

31. Mathewson KE. Pulsed Out of Awareness: EEG Alpha Oscillations Represent a Pulsed Inhibition of Ongoing Cortical Processing. Urbana, IL: University of Illinois at Urbana-Champaign;2011.

32. McDowell JE, Clementz BA. Ocular-motor delayed-response task performance among schizophrenia patients. Neuropsychobiology. 1996; 34:67–71. PMID: 8904734.

33. Reilly JL, Lencer R, Bishop JR, Keedy S, Sweeney JA. Pharmacological treatment effects on eye movement control. Brain Cogn. 2008; 68:415–435. PMID: 19028266.

34. McDowell JE, Clementz BA. Behavioral and brain imaging studies of saccadic performance in schizophrenia. Biol Psychol. 2001; 57:5–22. PMID: 11454432.

35. Hong LE, Summerfelt A, Mitchell BD, O’Donnell P, Thaker GK. A shared low-frequency oscillatory rhythm abnormality in resting and sensory gating in schizophrenia. Clin Neurophysiol. 2012; 123:285–292. PMID: 21862398.

36. Kelly SP, Lalor EC, Reilly RB, Foxe JJ. Visual spatial attention tracking using high-density SSVEP data for independent brain-computer communication. IEEE Trans Neural Syst Rehabil Eng. 2005; 13:172–178. PMID: 16003896.

37. van Gerven M, Jensen O. Attention modulations of posterior alpha as a control signal for two-dimensional brain-computer interfaces. J Neurosci Methods. 2009; 179:78–84. PMID: 19428515.

38. Brooks JL, List A. Searching for the role of the frontal eye fields in the visual attention network. J Neurosci. 2006; 26:2145–2146. PMID: 16495440.

39. Sheliga BM, Riggio L, Craighero L, Rizzolatti G. Spatial attention-determined modifications in saccade trajectories. Neuroreport. 1995; 6:585–588. PMID: 7766869.

40. Inhoff AW, Radach R, Starr M, Greenberg S. Allocation of visuo-spatial attention and saccade programming during reading. In : Kennedy A, Radach R, Heller D, Pynte J, editors. Reading as a Perceptual Process. Oxford: Elsevier;2000. p. 221–246.