Investigating Static and Sequential Models for Intervention-Free Selection Using Multimodal Data of EEG and Eye Tracking
by , , , ,
Abstract:
Multimodal data is increasingly used in cognitive prediction models to better analyze and predict different user cognitive processes. Classifiers based on such data, however, have different performance characteristics. We discuss in this paper an intervention-free selection task using multimodal data of EEG and eye tracking in three different models. We show that a sequential model, LSTM, is more sensitive but less precise than a static model SVM. Moreover, we introduce a confidence-based Competition-Fusion model using both SVM and LSTM. The fusion model further improves the recall compared to either SVM or LSTM alone, without decreasing precision compared to LSTM. According to the results, we recommend SVM for interactive applications which require minimal false positives (high precision), and recommend LSTM and highly recommend Competition-Fusion Model for applications which handle intervention-free selection requests in an additional post-processing step, requiring higher recall than precision.
Reference:
Investigating Static and Sequential Models for Intervention-Free Selection Using Multimodal Data of EEG and Eye Tracking (Mazen Salous, Felix Putze, Tanja Schultz, Hild Jutta, Jürgen Beyerer), In ICMI 2018 – ICMI 2018 Workshop on Modeling Cognitive Processes from Multimodal Data, 2018.
Bibtex Entry:
@inproceedings{salous_putze_2018icmi_mcpmd,
  author={Salous, Mazen and Putze, Felix and Schultz, Tanja and Hild Jutta and Beyerer, J\"{u}rgen},
  title={Investigating Static and Sequential Models for Intervention-Free Selection Using Multimodal Data of EEG and Eye Tracking},
  booktitle={{ICMI} 2018 -- ICMI 2018 Workshop on Modeling Cognitive Processes from Multimodal Data},
  isbn = {978-1-4503-5692-3},
  numpages = {6},
  url={https://www.csl.uni-bremen.de/cms/images/documents/publications/salous_putze_2018icmi_mcpmd.pdf},
  address = {Boulder, Colorado, USA},
  year={2018},
  abstract={Multimodal data is increasingly used in cognitive prediction models to better analyze and predict different user cognitive processes. Classifiers based on such data, however, have different performance characteristics. We discuss in this paper an intervention-free selection task using multimodal data of EEG and eye tracking in three different models. We show that a sequential model, LSTM, is more sensitive but less precise than a static model SVM. Moreover, we introduce a confidence-based Competition-Fusion model using both SVM and LSTM. The fusion model further improves the recall compared to either SVM or LSTM alone, without decreasing precision compared to LSTM. According to the results, we recommend SVM for interactive applications which require minimal false positives (high precision), and recommend LSTM and highly recommend Competition-Fusion Model for applications which handle intervention-free selection requests in an additional post-processing step, requiring higher recall than precision.},
}