A multi-modality fusion model based on dual-task measurement for the automatic detection of early-stage cognitive impairment

Objective: The aim of this research is to develop a simple automated screening approach for earlystage cognitive impairment with high performance. Method: Our approach is based on the dual-task paradigm, where individuals perform two tasks simultaneously, typically a physical task combined with a cognitive task. In this study, we developed a dual-task system capable of collecting gait data, cognitive scores, and EEG (electroencephalography) signals within a single five-minute measurement. EEG data were recorded while the individual performs both gait and cognitive tasks simultaneously. To improve overall accuracy, we introduced a novel fusion algorithm with an innovative loss function to integrate gait, cognitive scores, and EEG modalities. In our fusion model, we employ cross-attention mechanisms to capture implicit relationships among the three modalities. Furthermore, our proposed loss function minimizes the overlap between different modalities to maximize their complementary contributions. Results: The experimental results validate the effectiveness of both the proposed fusion model and the new loss function. The proposed approach outperforms existing methods in nearly all conditions, achieving an overall performance of AUC: 0.9845, sensitivity: 0.9659, and specificity: 0.9535. Furthermore, compared with other methods, our approach demonstrates a relatively superior cross fold stability and sensitivity-specificity balance. Conclusion: This study presents an effective multi-modality fusion framework for early-stage cognitive impairment screening. The experimental results confirm the validity and robustness of the proposed algorithm. Significance: The proposed algorithm, supported by comprehensive analysis, facilitates early detection of cognitive impairment, ultimately contributing to the prevention and treatment of dementia.