Incremental fuzzy machine learning for online classification of emotions in games from EEG data streams

Emotions have an important role in human-computer interaction and decision-making processes. The identification of the human emotional state has become a need for more realistic and interactive machines and computer systems. The greatest challenge is the availability of high-performance online learning algorithms that can effectively handle individual differences and nonstationarities from physiological data streams, i.e., algorithms that self-customize to a new user with no subject-specific calibration data. We describe an evolving Gaussian fuzzy classification (eGFC) approach, which is supported by an online semi-supervised learning algorithm to recognize emotion patterns from electroencephalogram (EEG) data streams. We present a method to extract and select features from bands of the Fourier spectra of EEG data. The data are provided by 28 individuals playing the games Train Sim World, Unravel, Slender: The Arrival, and Goat Simulator for 20 minutes each-a public dataset. According to the arousal-valence system, different emotions prevail in each game (boredom, calmness, horror, and joy). We analyze 14 electrodes/brain regions individually, and the effect of time window lengths, bands, and dimensionality reduction on the accuracy of eGFC. The eGFC model is user-independent and learns its structure and parameters from scratch. We conclude that both brain hemispheres may assist classification, especially electrodes on the frontal (Af3-Af4) area, followed by the occipital (01-02) and temporal (T7-T8) areas. We observed thatpatterns may be eventually found in any frequency band; however, the alpha (8-13 Hz), delta (1-4 Hz), and theta (4-8 Hz) bands, in this order, are more monotonically correlated with the emotion classes. eGFC has been shown to be effective for real-time learning from a large volume of EEG data. It reaches a 72.20% accuracy using a compact 6-rule structure on average, 10-second time windows, and a 1.8 ms/sample average processing time on a highly stochastic time-varying 4-class classification problem.