The Extraction of Nonverbal Behaviors: Using Video Images and Speech-Signal Analysis in Dyadic Conversation

This study examined the validity and possible utility of a new procedure for the extraction of nonverbal behaviors from dyadic conversation. Three methods were used to extract nonverbal behaviors (i.e., hand gestures, adaptors, and utterances). A novel automated method employing video images and speech-signal analysis software programs was compared to the more traditional coding and behavioral rating methods. The automated and coding methods provided an objective count of how many times a target behavior occurred, while behavioral ratings were based on more subjective impressions. Although there was no difference between the automated and coding methods for hand gestures, the coding method using an event recorder yielded marginally significantly more instances of adaptors and utterances as compared to the software programs. Measures of each nonverbal behavior were positively correlated across the different methods. In addition, interpersonal impressions of each speaker were rated by both observers and conversational partners. Although R ² was lower than for the coding/behavioral rating methods, nonverbal behaviors extracted using the software programs significantly predicted familiarity and activeness ratings from both observer and partner points of view. These results support the validity and possible utility of the software-based automated extraction procedure.