Overview
This study explores the applicability of memorization indicators to identify overfitting during the recalibration of deep learning models for surface electromyography (sEMG) decoders, particularly when confronted with low sample sizes. The investigation focuses on a recently proposed class of memorization indicators derived from the activation statistics of rectified linear units (ReLU) within deep neural networks.
The research suggests that these activation-based indicators, calculable directly from training data without requiring a separate validation set, can serve as an early warning system for unsuccessful learning in sEMG calibration scenarios characterized by limited data availability.
Research Context
Deep learning models applied to sEMG data can benefit from subject-specific (re-)calibration. This necessity arises because sufficiently large and diverse datasets for training fully generic decoders are currently unavailable. However, practical constraints on user acceptance severely limit the number of repetitions that can be collected during calibration.
This limitation increases the risk of overfitting, which can, in some cases, lead to a degradation of performance compared to the uncalibrated model. Traditional overfitting indicators, such as validation performance monitoring and regularization techniques like early stopping, present challenges in this low-sample regime. These methods typically necessitate additional held-out data, which is often not feasible in real-world calibration scenarios.
Approach
The study utilized a transfer learning experimental setup. A convolutional neural network (CNN) was initially pre-trained on an sEMG benchmark dataset using data from multiple subjects. Following this pre-training, the CNN was fine-tuned for individual users, employing only a small number of repetitions during the calibration phase.
During this calibration process, the researchers simultaneously monitored both the decoding performance of the CNN and the activation behavior of its last hidden layer. The focus was on the activation statistics of ReLU units to derive memorization indicators.
Findings
- Decreases in test accuracy observed during the fine-tuning phase were accompanied by characteristic changes in activation rates within the last hidden layer.
- This correlation suggests that activation-based memorization indicators can signal unsuccessful learning in sEMG calibration settings where limited sample data is used.
- The employed memorization indicators are based solely on the activation statistics of ReLU units and can be computed directly from training data, eliminating the need for an additional validation set.
Why This Matters
The findings provide initial evidence that activation-based memorization indicators offer a promising tool for the early detection of overfitting during the recalibration of sEMG decoders. This is particularly relevant in situations where data availability is severely restricted, and conventional overfitting indicators are difficult to apply. The ability to identify unsuccessful learning early could prevent performance degradation and improve the practical applicability of sEMG-based deep learning models.