Defining and Quantifying Users’ Mental Imagery-based BCI skills: a first step

OBJECTIVE: While promising for many applications, electroencephalography
(EEG)-based brain-computer interfaces (BCIs) are still scarcely used outside
laboratories, due to a poor reliability. It is thus necessary to study and fix
this reliability issue. Doing so requires the use of appropriate reliability
metrics to quantify both the classification algorithm and the BCI user’s
performances. So far, classification accuracy (CA) is the typical metric used for
both aspects. However, we argue in this paper that CA is a poor metric to study
BCI users’ skills. Here, we propose a definition and new metrics to quantify such
BCI skills for mental imagery (MI) BCIs, independently of any classification
algorithm.

APPROACH: We first show in this paper that CA is notably unspecific, discrete,
training data and classifier dependent, and as such may not always reflect
successful self-modulation of EEG patterns by the user. We then propose a
definition of MI-BCI skills that reflects how well the user can self-modulate EEG
patterns, and thus how well he could control an MI-BCI. Finally, we propose new
performance metrics, classDis, restDist and classStab that specifically measure
how distinct and stable the EEG patterns produced by the user are, independently
of any classifier.

MAIN RESULTS: By re-analyzing EEG data sets with such new metrics, we indeed
confirmed that CA may hide some increase in MI-BCI skills or hide the user
inability to self-modulate a given EEG pattern. On the other hand, our new
metrics could reveal such skill improvements as well as identify when a mental
task performed by a user was no different than rest EEG.

SIGNIFICANCE: Our results showed that when studying MI-BCI users’ skills, CA
should be used with care, and complemented with metrics such as the new ones
proposed. Our results also stressed the need to redefine BCI user training by
considering the different BCI subskills and their measures. To promote the
complementary use of our new metrics, we provide the Matlab code to compute them
for free and open-source.

 

Auteurs Bordeaux Neurocampus