Lester Loschky¹ (), Prasanth Chandran¹, Brian Howatt¹, Yifeng Huang², Jeremy Munsell³, Brayden Wallace¹, Lindsey Wilson¹, Ally Dinkel¹, Sidney D'Mello, Minh Hoai², Sanjay Rebello³; ¹Kansas State University, ²SUNY Stonybrook, ³Purdue University

Computer-Assisted Instruction (CAI) is here to stay. However, given it’s unsupervised nature, a key problem for CAI is estimating students' attention. Nevertheless, CAI is an ideal context for investigating these issues, because it allows comprehensive measurements of learners' behavior during naturalistic learning activities. Here, we report on the initial results of a study of 100 students’ attentional states while studying a multimodal Physics module and their learning outcomes. Our study uses a 2x2 matrix to characterize online learners’ attentional and cognitive states (D’Mello, 2016). First, when learners are looking at the screen, we distinguish between when they are thinking about the content versus when they are not (i.e., on-task vs. mind-wandering). Second, when learners are looking elsewhere (i.e., off-screen), we similarly distinguish between when they are thinking about the content versus not. Method: To measure students’ attentional states, we combined information from a webcam, eye tracker, egocentric camera (showing what students looked at), and probes to measure mind-wandering. After learners had finished the module, we synchronized the multi-modal data and conducted a retrospective recall to ask students about their attentional and cognitive states whenever they had gazed away from the materials. To determine learning effects of the above attentional states, we used a 26-item pre-test, post-test, and 1-week retention test. Initial Results: Learners with less prior knowledge spent more time looking on-screen while thinking about the content, which was positively correlated with learning. However, learners with less prior knowledge also spent more time looking on-screen, but mind-wandering, which was negatively correlated with learning. Learners with greater prior knowledge spent more time looking off-screen, but thinking about the content, which was uncorrelated with learning. However, learners with greater prior knowledge also spent more time looking off-screen, but not thinking about the content, which was negatively correlated with learning.

Acknowledgements: Funding: National Science Foundation Grant 2100071 to LL, MH, SR