"Not crazy, just talking on the phone: Gestures and mobile phone conversations" by Carolyn Y. Wei is an intriguing paper I ran across recently. It basically focuses on a phenomena you may have certain notice: why and how mobile phone users engage in vivid nonverbal communication behaviors that do not benefit their communication partner (gesturing, smiling, and nodding their head). The most interesting part of the paper is about the design implications, such as creating mobile phones that can be sensitive to nonverbal communication behaviors (with paralinguistic social cues such as tone of voice, pitch, and volume). The "Jerk-o-Meter at MIT Medialab is based on this approach:
"The Jerk-O-Meter (or JerkoMeter) is a real-time speech feature analysis application that runs on your VOIP phone or cellphone that remedies precisely that experience. It uses speech features for activity and stress (and soon empathy) to measure if you are 'being a jerk' on the phone. The phone displays messages in case you are, and can also be setup to inform the person on the other end of the line that you're extremely busy."
(A courier in Seoul who make gestures when using his mobile phone)
But the most intriguing one is described in the original article:
"Mobile phone design can also respect existing research that suggests gestures are more meaningful to the speaker than the listener, and thus focus on innovations that aid the speaker. An example of this kind of design would be a mobile phone that senses gestures or other nonverbal behaviors and compares them with the words being spoken. If the words being spoken match the amount and nature of gesturing, then the phone might alert the user that she is performing well. Light could be used in such an interface: if the user is gesturing and speaking very animatedly, then a light on the phone might hold steady to indicate appropriate activity. If there seems to be a disconnect, for example, where the user is not saying anything but still gesturing, then the phone could alert the user with a pulsing light that she might appear odd to others. Similar feedback could be offered with paralinguistic features such as volume to notify speakers that they may be speaking overly loudly."
Why do I blog this? Observing how people gesture when talking on the phone is a situation that I have always conducted with curiosity and fascination. Especially because you can see it as an indicator how these gestures are nearly more important for the speaker than the listener. It's therefore intriguing to see how mobile phone design can benefit from this.