Paper Reading #6: Using Rhythmic Patterns as an Input Method

Using Rhythmic Patterns as an Input Method

Emilien Ghomi     Guillaume Faure     Stéphane Huot     Olivier Chapuis     Michel Beaudouin-Lafon

ghomi@lri.fr         gfaure@lri.fr            huot@lri.fr           chapuis@lri.fr       mbl@lri.fr

Univ Paris-Sud (LRI)                        CNRS (LRI)                        INRIA

F-91405 Orsay, France                    F-91405 Orsay, France       F-91405 Orsay, France

Author Bios:

Emilien Ghomi   

• Ph.D Student at Université Paris-Sud

• Michel Beaudouin-Lafon and  Stéphane Huot are his advisors

Guillaume Faure

• Ph.D Student at Université Paris-Sud

• Michel Beaudouin-Lafon is his supervisor

Stéphane Huot  

• Associate Professor at Université Paris-Sud

• PhD from University of Nantes and the Computer Science Dpt. of the Ecole des Mines de Nantes

Olivier Chapuis

• Research Scientist at LRI (Paris-Sud).

• Ph.D. in Mathematics from University Paris VII Diderot

Michel Beaudouin-Lafon

• Senior member of Institut Universitaire de France

• Ph.D. thesis at Laboratoire de Recherche en Informatique, Univ. Paris-Sud

All are members of the InSitu research team (LRI & INRIA Saclay Ile-de-France).

Summary:

The authors of this paper are testing the use of short 2-6 beat rhythmic patterns as a replacement for interacting with computers, phones and other devices with the use of various motion or touch pad sensors.  The uses of rhythms as controls is endless, from turning a phone on vibrate or skipping a song just by tapping it in your pocket to replacing hot key shortcuts in computer programs.  They posed some questions that would test if implementing technology with rhythmic controls is worth developing: 

• Are people able to learn and memorize patterns? 
• Can they use them to trigger commands? 
• Which patterns make sense for interaction and how to design a vocabulary? 
• What is the best feedback helps executing and learning patterns? 
• How to design effective recognizers that do not require training?

Related work not referenced in the paper:

• Five-Key Text Input Using Rhythmic Mappings
• Uses multiple taps to represent a single character on a keypad that is much smaller than a normal full sized keyboard.  It does not use a beat but only a sequence of button presses.  
• Rhythmic Interaction with a Mobile Device
• The authors wanted to be able to measure the movement of a phone in a 3D space to interpret rhythmic gestures to provide spatiotemporal gesture classification.  This does not use rhythmic tapping with a beat to command a device.
• Music Wall: A Tangible User Interface Using Tapping as an Interactive Technique
• uses a sequence of taps to interface with a wall or table using embedded sensors for casual communication like knocking on a door rings the doorbell.  This is a similar idea only used on different types of devices for different purposes.
• RhythmLink: Securely Pairing I/O-Constrained Devices by Tapping
• The title is self explanatory. It uses a shared sequence of taps to securely link devices such as bluetooth headset to a phone.  The authors do not intend to use rhythms for controlling devices as intended in the paper I am evaluating.
• inTUIt – Simple Identification on Tangible User Interfaces
• Uses tapping as a way of interacting, but does not go further into making different types of tapping different commands. 
• Exploring Reinforcement Learning for Mobile Percussive Collaboration
• Aims to makes real-time, multi-user musical expression on mobile devices just as intuitive as their physical counterparts.
• Sonic gestures and rhythmic interaction between the human and the computer
• Explains many different ways of interacting with a device such as tapping on a table, but does not actually implement an experiment or way of measuring the tapping sequences.
• Movement Sonification: Effects on Perception and Action
• Translates movement, such as a person jumping, into a sound wave using qualities such as the force the person exerts on the ground over time.
• Gesture Authentication with Touch Input for Mobile Devices
• Intends rhythmic patterns as passwords and not basic user commands for devices.
• Temporal Interaction Between an Artificial Orchestra Conductor and Human Musicians
• Is intended to guide an orchestra by listening to the tempo and rhythm of the music and moving accordingly as a human musician would do.

Evaluation:

two experiments were done. The first was to evaluate whether a computer can register the rhythmic patterns of a novice user effectively. The second was to see if patterns can be memorized as efficiently as the shortcuts used today. The first experiment used 30 patterns the participants listened to and saw a visual representation before attempting to replicate the pattern for the recognizer to interpret. There were four groups of participants. no feedback when tapping the rhythm, and audio feedback, a visual feedback and a combination of visual and audio feedback.  Figure 6 show the quantitative data gathered.  The qualitative data consisted of how the participants felt about the different forms of feedback. Many only wanted one form of feedback if they had both and the quantitative data clearly shows that no feedback did not work as well.  The second experiment used hot keys as an example of a long used shortcut system that their rhythmic patterns may be able to replace.  To see if using rhythms is just as fast as using hot keys, they assigned 14 patterns and hot keys to the same object with no correlation between the patterns and hot keys to the object. The audio only feedback was used since it was the most effective from experiment 1.

The participants had the choice of using either patterns or hot keys as long as they tried both. Most used the patterns because they found it more fun.  The experiment was stretched over two days to get a better accuracy with how the participants memorized the shortcuts.  There were two groups, one where participants were told the shortcuts and then did a memory test and the other group was given help whenever they needed it.  The quantitative data are shown in the figures to the right.  In Figure 14, there is not much of a difference in the recall rate once the patterns and hot keys are learned.  In Figure 15, the help rates are also rather similar between the rhythmic patterns and hot keys showing that learning patterns is just as easy to learn as the conventional hot keys are.

Discussion:
They listed many ways that makes rhythmic patterns better than using hot keys, but I do not believe they will be able to replace them.  I see this as a great way to give commands to phones without having to take them out of your pocket or to skip songs as you jog and other thing like that.  They need to run an experiment testing how long it takes to do some simple tasks using patterns and hot keys.  I think that someone using the patterns will never be able to move as quickly as an experienced hot key user since pauses are an inherent part of giving a command rhythmically.