Paper Reading #4: Pay Attention! Designing Adaptive Agents that Monitor and Improve User Engagement

Pay Attention! Designing Adaptive Agents that Monitor and Improve User Engagement

Dan Szafir, Bilge Mutlu

Department of Computer Sciences, University of Wisconsin–Madison

1210 West Dayton Street, Madison, WI 53706, USA

fdszafir,bilgeg@cs.wisc.edu

Author Bios:

Dan Szafir

• Assistant professor of computer science, psychology, and industrial engineering at the University of Wisconsin–Madison.

• PhD from Carnegie Mellon University‘s Human-Computer Interaction Institute.

Bilge Mutlu

• In the Department of Computer Sciences at the University of Wisconsin-Madison.

Summary:

A robotic agent that tells a story and monitors the users behaviors, emotions and  mental states to best keep the user's attention on the story.  Recently, studies have been using electroencephalography(EEG) signals to measure alertness and attention. This new technology is commercially available making brain-computer interface(BCI) possible. The robotic agent uses this new advancement in the form of large headphones to monitor the FP1 region of the cortex which is thought to control learning, mental states and concentration.  When the attention of the user on the story begins to decrease, the robotic agent changes its tone, engages gestures and other techniques known to regain a user's attention.  These techniques have been thoroughly studied and many long standing educational theories have been made.

Related work not referenced in the paper:

• On the Use of Electrooculogram for Efficient Human Computer Interfaces
• Explains how the use of different BCI types can be used to help people physically disabled to interact with computers. It mainly described the use of electrooculogram(EOG) which follows eye movement, but explains how EEG would be a more reliable way of BCI interaction.
• A hybrid platform based on EOG and EEG signals to restore communication for patients afflicted with progressive motor neuron diseases
• The title is so long it pretty much speaks for itself. It focuses on the use of the BCI tools to help people impaired physically interact with specific devices. It does not consider their use to monitor a user's attention like the robotic agent.
• Brain–Computer Interfaces for Multimodal Interaction: A Survey and Principles
• This paper is very similar to Pay Attention! and talks about how EEG measurements can be used to help people suffering from attention deficit hyperactivity disorder or non-disabled people stay focused on a single task or a story in our case.  It does not actually implement this idea though and stays in the realm of potential uses for this new technology. No real experiments were done like in the paper I am writing about.
• Brainput: Enhancing Interactive Systems with StreamingfNIRS Brain Input
• This paper sees the use of BCI devices as a new way of interacting with computers and other devices to maximize a person's productivity by preventing them  from being mentally overwhelmed with tasks and not as a learning aid. 
• Adaptive Brain Interfaces
• Like many of the papers involving BCI devices, this paper also is about how it can be used for physically disabled persons interact as they could never have before.
• Towards passive brain–computer interfaces: applying brain–computer interface technology to human–machine systems in general
• Again, another title so long it describes its content too well. It explains the development of BCI and where the authors think it will be in the future. It gives a few examples of BCI devices, not any using it as a way of keeping a user's attention like the robotic agent.
• Electroencephalogram-Based Control of an Electric Wheelchair
• In the beginning of BCI research the focus was on aiding the disabled and this paper is one of them only it has focused on the use to control a wheel chair instead of a computer like the other papers I have been finding.  Like the other papers, it does not consider BCI devices as a way of keeping a user's attention either.
• Combining Eye Gaze Input With a Brain–Computer Interface for Touchless Human–Computer Interaction
• This paper again is focused on the idea of being able to control a computer pointer with nothing but your gaze and thoughts. There are many papers on BCI with this same idea, but none of them that I have found consider its use for teaching by keeping the user's attention.
• Using a Low-Cost Electroencephalograph for Task Classification in HCI Research
• They had three different tasks that the user would perform and the EEG device would differentiate which task they were performing from stimulation in the brain. It can measure a user's mind state, but does not measure  their attention or concentration.
•  Towards Ambulatory Brain-Computer Interfaces: A Pilot Study with P300 Signals
• When a user is in motion, the EEG signal is reduced significantly. This paper proposes a solution to this problem by measuring a specific brain signal called P300.  The author's idea of a potential use for this is mainly entertainment purposes and not learning which is the focus of the robotic agent in the paper I am reading. 

Evaluation:

There were two hypotheses the author's wanted to evaluate. The first was that the educational attention grabbers performed by the robotic agent when EEG measurements of the user's attention drops will raise their attention and improve their learning performance. The second states that the robotic agent's engagements with the user when their attention decreases will also help motivate them and increase their rapport with the agent. To test these hypotheses, participants are placed in a room with a robotic agent that beings with a lesson on the zodiac signs followed by a long story.  Then objective questions about the zodiac signs were asked and then questions about the story were asked. During the lesson on zodiac symbols no attention grabbing cues were used, so it was as if they were simply listening to an audio tape. This part was meant to be a buffer to draw the participant's attention from the real test of the long story.  There were three groups that each listened to the story in three distinct ways. the first was with no attention cues, the second was with randomly timed cues and the third was with the adaptive cues when the user was loosing focus.

The results from the questions backed their first hypothesis.  Subjective observations of the participants after their interaction with the robotic agent showed interesting results.  Females showed a higher sense of motivation and rapport with the agent confirming the second hypothesis, but the male participants had the exact opposite reactions not how the authors expected. The authors considered this as a possible result of the appearance of the robot being small and having a child's voice which could be easier for women to connect with than men.

Discussion:
The objective data gathered showed that the adaptive robotic agent improved the learning curve of the participants by a significant margin. The idea of using EEG measurements to track attention and concentration is not a completely novel idea, but I could not find any other paper where the authors use a human like robot to act as a teacher.  This was the only paper I found that had actual observed data of a BCI device positively impacting a user's ability to retain new subjects they have just learned.