EEG-Based Emotion Recognition via Efficient Convolutional Neural Network and Contrastive Learning

The progress of Human-Computer Interaction (HCI) and Neural Networks technology has facilitated the precise evaluation of human emotional states, which is crucial for understanding their impact on behavior and decision-making. Monitoring emotional states aids personal well-being, enhances emotional intelligence, and boosts the probability of individual career success.

The human emotional state can be deciphered by exploring two affective patterns: dominant affective expressions and recessive affective expressions.

Dominant expressions include explicit and observable bodily changes, such as facial expressions, eye movements, limb gestures, and linguistic cues. These observations do not accurately map themselves to actual emotional states.

In contrast, recessive expressions stemming from involuntary physiological signals such as electrooculography, electrocardiography (ECG), electromyography, and EEG offer valuable insights into emotional states, for they remain unaffected by behavioral or linguistic masking.

Electroencephalography (EEG) signals, with superior temporal resolution, emerge as the preferred choice for accurate emotion recognition tasks. Two primary models guiding emotion recognition tasks are the discrete and dimensional models.

The discrete model categorizes emotions into eight basic categories - fear, anger, acceptance, sadness, curiosity, surprise, joy, and disgust, whereas a more nuanced dimensional model operates within a 3-D emotion space, utilizing valence (the pleasantness of a stimulus), arousal (the intensity of emotion provoked by a stimulus), and dominance (the degree of control exerted by a stimulus) as axes for mapping emotions. This dimensional model represents real human emotions more, capturing subtle differences and offering a broader range of emotional expressions.

Traditional EEG-based emotion recognition consists of two steps - feature extraction and classifier construction. Despite promising results, these existing models based on convolutional neural networks (CNNs) pose computational and costing challenges.

This study introduces an end-to-end efficient CNN (ECNN) model for EEG-based emotion recognition. The ECNN model comprises a feature extraction model with adaptable block modules and a linear, fully connected layer classifier with two hidden units.

The ECNN model employs two emotion datasets:  DEAP records EEG signals from subjects watching music videos, while DREAMER includes EEG and ECG signals collected while viewing film segments. The ECNN model demonstrates exceptional performance, achieving recognition accuracies of 98.35%, 98.51%, and 98.55% on DEAP and 96.89%, 97.03%, and 97.04% on DREAMER for valence, arousal, and dominance axes, respectively.

To enhance accuracy, supervised contrastive loss is incorporated into the ECNN model, combining it with cross-entropy loss. This method improves discriminability between EEG-based emotion samples, fostering efficient emotional representation. This hybrid loss method, integrating cross-entropy and contrastive losses, outperforms conventional approaches. Furthermore, a projection model is introduced to diminish representation space, facilitating precise sample distance measurement.

The integration of CNNs and contrastive learning presents a significantly advanced and improved emotional recognition system with an average accuracy of 98.47% and 96.98% across the three axes when deployed on DEAP and DREAMERS datasets. Future applications of HCI-based emotional intelligence modules can help create better emotional cognition, detect and diagnose epilepsy, depression, and sleep stage classification, and improve mental health monitoring systems.