Kinesic Patterning in Deceptive and Truthful Interactions

A persistent question in the deception literature has been the extent to which nonverbal behaviors can reliably distinguish between truth and deception. It has been argued that deception instigates cognitive load and arousal that are betrayed through visible nonverbal indicators. Yet, empirical evidence has often failed to find statistically significant or strong relationships. Given that interpersonal message production is characterized by a high degree of simultaneous and serial patterning among multiple behaviors, it may be that patterns of behaviors are more diagnostic of veracity. Or it may be that the theorized linkage between internal states of arousal, cognitive taxation, and efforts to control behavior and nonverbal behaviors are wrong. The current investigation addressed these possibilities by applying a software program called THEME to analyze the patterns of kinesic movements (adaptor gestures, illustrator gestures, and speaker and listener head movements) rated by trained coders for participants in a mock crime experiment. Our multifaceted analysis revealed that the quantity and quality of patterns distinguish truths from untruths. Quantitative and qualitative analyses conducted by case and condition revealed high variability in the types and complexities of patterns that were produced and differences between truthful and deceptive respondents questioned about a theft. Patterns incorporating adaptors and illustrator gestures were correlated in counterintuitive ways with arousal, cognitive load, and behavioral control, and qualitative analyses produced unique insights into truthful and untruthful communication.     

Using Thin Slices for Behavioral Coding

This study investigated whether brief segments of nonverbal behavior or “thin slices” would be representative of behavior across a longer interaction. It was hypothesized that behavioral slices would correlate with behavior across a full 15-minute interaction. Five commonly investigated nonverbal behaviors were coded: gazing, gestures, nods, smiles, and self-touch. The findings showed moderate to high positive correlations between the thin slices (1-minute slice, total of two 1-minute slices, total of three 1-minute slices) and the full interaction for all five behaviors. In regards to behavioral coding, the thin-slice methodology proposed could save nonverbal researchers considerable time, energy, and money by decreasing resources dedicated to coding behavior in some types of interactions. Such results suggest that thin slices could represent an individual’s behavior across a longer length of time. Nora A. Murphy is now at the Department of Psychology, Brandeis University, Waltham, MA. The data was collected at Northeastern University and the research was supported by National Science Foundation grant SBR 97 09032 awarded to Judith A. Hall.     

Status Roles and Recall of Nonverbal Cues

In two experiments, interpersonal status was experimentally manipulated by assigning one dyad member to be the owner of a mock art gallery and the other to be the owner's assistant. Without forewarning, participants were asked immediately following the interaction to recall their partner's hand gestures, self-touch, gazing, smiling, and nodding. Accuracy of recall was determined by comparing these ratings to their partners' behavior as coded from the videotape. In both experiments, assistants were more accurate at recalling the amount of owners' self-touch than vice versa, but there was little evidence of an accuracy difference in recall of the other nonverbal cues. When accuracy was defined as the correlation between a participant's ratings of the partner's behaviors and the partner's actual behaviors, there was evidence that assistants were more accurate than owners when a combined p-value was calculated across both studies.     

Sex-Typed Body Movements as a Function of Severity of Gender Disturbance in Boys

Specific sex-typed body movements and gestures were recorded in a group of 23 gender-disturbed boys, aged 7 to 17 years, using three standardized procedures- behavioral observation of nine expressive gestures during a 10-minute test period, ratings of sitting, standing and walking on the Barlow Gender-Specific Motor Behavior Form, and ratings of book-carrying behavior. The study found significantly higher frequencies of hand clasp, hyperextension, and limp wrist gestures in gender-disturbed boys as compared to a nonclinical control group. There were significant correlations between severity of gender disturbance and both the frequency of hyperextension gestures and the percentage of feminine motor behaviors recorded on the Barlow Gender-Specific Motor Behavior Form. However, sex-typed carrying behavior did not discriminate between normal and gender-disturbed boys.     

Co-Speech Gesture Mimicry in the Process of Collaborative Referring During Face-to-Face Dialogue

Mimicry has been observed regarding a range of nonverbal behaviors, but only recently have researchers started to investigate mimicry in co-speech gestures. These gestures are considered to be crucially different from other aspects of nonverbal behavior due to their tight link with speech. This study provides evidence of mimicry in co-speech gestures in face-to-face dialogue, the most common forum of everyday talk. In addition, it offers an analysis of the functions that mimicked co-speech gestures fulfill in the collaborative process of creating a mutually shared understanding of referring expressions. The implications bear on theories of gesture production, research on grounding, and the mechanisms underlying behavioral mimicry.     

Patterns of Nonverbal Behavior Associated with Truth and Deception: Illustrations from Three Experiments

The digital age has brought with it new and powerful computer-based methods of analyzing heretofore elusive patterns of nonverbal behavior. C-BAS (Meservy 2010) is a computer-assisted behavioral observation tool for identifying and tracking nonverbal behaviors from video. THEME (Magnusson, The hidden structure of interaction: from neurons to culture patterns, IOS Press, Amsterdam, pp 4–22, 2005) is a software program that discovers patterns among discrete events in time-ordered data. Together, these tools enable more precise measurement and analysis of nonverbal behavioral dynamics. Applications to three corpora derived from interpersonal deception experiments reveal unique nonverbal patterns that distinguish deceptive from nondeceptive interactions. The first and second experiments produced serial, hierarchically related patterns of behaviors that differed in length and complexity between truthful and deceptive participants during interviews about a theft and cheating, respectively. The third experiment produced differential patterns by and among group members completing a task. Deceivers were inclined toward strategic initiations and interactional control, whereas suspicious group members adopted a more passive, possibly watchful stance. Discovery of these patterns challenges the prevailing view that nonverbal behaviors are too faint and inconsistent to identify deceptive communication. Results have numerous implications regarding the following: the development of new measurement tools locating significant effects of nonverbal behaviors, support for theory that coherent and repetitive relationships exist within and among interactants’ communication, demonstration of the role of nonverbal behaviors in deceptive communication and the dynamic and strategic nature of deception.     

Infant Smiling Dynamics and Perceived Positive Emotion

To better understand early positive emotional expression, automated software measurements of facial action were supplemented with anatomically based manual coding. These convergent measurements were used to describe the dynamics of infant smiling and predict perceived positive emotional intensity. Over the course of infant smiles, degree of smile strength varied with degree of eye constriction (cheek raising, the Duchenne marker), which varied with degree of mouth opening. In a series of three rating studies, automated measurements of smile strength and mouth opening predicted naïve (undergraduate) observers’ continuous ratings of video clips of smile sequences, as well as naïve and experienced (parent) ratings of positive emotion in still images from the sequences. An a priori measure of smile intensity combining anatomically based manual coding of both smile strength and mouth opening predicted positive emotion ratings of the still images. The findings indicate the potential of automated and fine-grained manual measurements of facial actions to describe the course of emotional expressions over time and to predict perceptions of emotional intensity.     

Multimodal Integration of Dynamic Audio–Visual Cues in the Communication of Agreement and Disagreement

Recent research has stressed the importance of using multimodal and dynamic features to investigate the role of nonverbal behavior in social perception. This paper examines the influence of low-level visual and auditory cues on the communication of agreement, disagreement, and the ability to convince others. In addition, we investigate whether the judgment of these attitudes depends on ratings of socio-emotional dimensions such as dominance, arousal, and valence. The material we used consisted of audio–video excerpts that represent statements of agreement and disagreement, as well as neutral utterances taken from political discussions. Each excerpt was rated on a number of dimensions: agreement, disagreement, dominance, valence, arousal, and convincing power in three rating conditions: audio-only, video-only, and audio–video. We extracted low-level dynamic visual features using optical flow. Auditory features consisted of pitch measurements, vocal intensity, and articulation rate. Results show that judges were able to distinguish statements of disagreement from agreement and neutral utterances on the basis of nonverbal cues alone, in particular, when both auditory and visual information were present. Visual features were more influential when presented along with auditory features. Perceivers mainly used changes in pitch and the maximum speed of vertical movements to infer agreement and disagreement, and targets appeared more convincing when they showed consistent and rapid movements on the vertical plane. The effect of nonverbal features on ratings of agreement and disagreement was completely mediated by ratings of dominance, valence, and arousal, indicating that the impact of low-level audio–visual features on the perception of agreement and disagreement depends on the perception of fundamental socio-emotional dimensions.     

Recent Improvements in Portraying Acts in Symbolic Interaction Behavioral Specimens

Behavioral specimens are slices of ongoing interaction that take particular behavioral observations from beginning to end and report the temporal sequence of the behaviors under investigation. This article takes the position that behavioral specimens as a symbolic interaction tool preserve the context of natural interaction in interpersonal relationships. Whereas this technique of data collection was first used by Barker and Wright in 1954, over 30 years ago, the format of the specimen presented here is new, because nonverbal behaviors are now recorded in a fashion that reflects the timing of the behaviors in relation to verbalizations. The intent of the new specimen format is to enhance the ecological validity of recordings through a more accurate representation of the contemporaneous nature of nonverbal behaviors and to convey a working knowledge of the actors under analysis.     

Specification of nonverbal behaviors for clinical assessment

Nine categories of nonverbal behavior (extremity movements, self-manipulations, facial expression, posture, orienting, gestures, voice quality/tone, speech rate/pressure, and sense of timing) were tested in a standardized role play situation of social skills. Each category was judged using a new midi-level system of assessment which permitted specification of component behaviors but allowed observers to make single ratings at the ends of videotaped episodes. The midi-level measurements were as reliable and practical as more traditional global measures of social skill and social anxiety. Midis were superior to globals (i.e., single overall ratings of skill and anxiety) in terms of predicting physiological indices of social anxiety. Voice quality/tone and sense of timing appeared to be the best predictors of criterion social skill measures and self-manipulations, extremity movements, and gestures had the highest weights in predicting criterion measures of social anxiety.     

Nonverbal relational control in family communication

A nonverbal component for an observational coding system of verbal relational control, the Family Relational Communication Control Coding System (FRCCCS; Friedlander & Heatherington, 1989) was developed and validated for research with families or other groups. In so doing, it was demonstrated that nonverbal relational control behaviors can be reliably identified and that these behaviors enhance communicative meaning in predictable ways. First, a pool of nonverbal behaviors with relational control meaning (e.g., head nod, raised eyebrow) was identified from consultation with family therapy researchers and from the literature. Second, the behaviors that were retained from the results of a content validity test were perceived by three independent samples as commonly understood, discrete attempts to either gain control (one-up) or relinquish control (one down) of a social relationship. Still other behaviors were eliminated based on results of a cluster analysis, the interpretation of which was the basis for composing operational definitions for four classes of nonverbal behaviors. Finally, assessments of the interjudge reliability and criterion validity of the nonverbal coding scheme supported its psychometric adequacy. Comparison of effect sizes showed that these discrete nonverbal behaviors accounted for over twice the variance in observers' perceptions as the verbal behaviors.     

The interpretation of nonverbal behavior in a role-defined interaction sequence: The police-citizen encounter

Videotapes of staged police-citizen interactions were shown to subjects. Nonverbal behaviors of the citizen were programmed to reflect empirically derived reactions to spatial intrusion versus nonintrusion conditions by the police officer. In the first study subjects made attributional ratings of the citizen based on the nonverbal behaviors only with the physical distance between the interactants held constant. Citizens exhibiting spatial intrusion reaction behaviors were judged to be significantly more deceptive, guilty, anxious, and generally more suspicious and unfavorable than citizens not exhibiting such behaviors. In the second study physical distance was explicitly manipulated to create a close versus distant condition. The behavioral cue manipulation from the first study was reproduced in each of these proximity conditions. Many replications of results from the first study were obtained indicating that the preponderant cause of impression formation of the citizen was based upon spatial intrusion behavioral cues, regardless of the perceived physical distance between the interactants. However, significant interaction effects between proximity and behavioral cue conditions were obtained and confirmed an attribution theory prediction regarding degree of confidence in impression-formation ratings made of the citizen.     

Choosing between micro and macro nonverbal measurement: Application to selected vocalic and kinesic indices

Selecting the appropriate size of measurement unit in nonverbal research requires empirical as well as conceptual analysis. A definition is offered to distinguish macroscopic from microscopic measurement. Merits of each measurement approach are reviewed briefly in terms of cost, efficiency, precision, and analysis flexibility, and four additional criteria are advocated as central considerations in choosing measurement units: (1) isomorphism between the form of measurement and the phenomenological experience of interactants, (2) reliability, (3) concurrent validity, and (4) predictive validity. These criteria are applied to the analysis of 20 vocalic and kinesic nonverbal behaviors measured microscopically and macroscopically. Reliability and validity results demonstrate that either approach may be acceptable for many behaviors, but that relatively molecular coding may be preferable for highly dynamic and objective behaviors, while relatively molar coding may be preferable for static behaviors and perceptual judgments.     

Nonverbal Social Sensing in Action: Unobtrusive Recording and Extracting of Nonverbal Behavior in Social Interactions Illustrated with a Research Example

Nonverbal behavior coding is typically conducted by “hand”. To remedy this time and resource intensive undertaking, we illustrate how nonverbal social sensing, defined as the automated recording and extracting of nonverbal behavior via ubiquitous social sensing platforms, can be achieved. More precisely, we show how and what kind of nonverbal cues can be extracted and to what extent automated extracted nonverbal cues can be validly obtained with an illustrative research example. In a job interview, the applicant’s vocal and visual nonverbal immediacy behavior was automatically sensed and extracted. Results show that the applicant’s nonverbal behavior can be validly extracted. Moreover, both visual and vocal applicant nonverbal behavior predict recruiter hiring decision, which is in line with previous findings on manually coded applicant nonverbal behavior. Finally, applicant average turn duration, tempo variation, and gazing best predict recruiter hiring decision. Results and implications of such a nonverbal social sensing for future research are discussed.     

Nonverbal behavior in families with adolescents

In fifty-three families, mothers, fathers and one adolescent were videotaped discussing two issues relevant to the adolescent's behavior: the aspects of their behavior they could change, and rules appropriate to life in the family. Family members later watched these videotapes and rated themselves and each other every 15 seconds on levels of perceived anxiety, involvement, strength and friend-liness. From these videotapes, each family member's behavior was also coded for the use of smiles, various face and head movements and gestures. Results revealed that adolescents looked frequently at both parents, but parents looked even more at each other and the adolescent. Adolescents smiled more than their parents, while mothers smiled and nodded their heads more than fathers. More nonverbal expresivity occurred in conversations between parents and daughters than between parents and sons, although these results were mainly due to the behavior of fathers with daughters. Ratings of involvement by parents were related to the adolescent's nonverbal behaviors, but other ratings were not. Mothers rated adolescents as more involved when they used more head up movements and nods. Fathers rated adolescents as more involved when they gazed and smiled more.     

The face of repressive coping: Social context and the display of hostile expressions and social smiles

The present study examined the nonverbal correlates of repressive coping, extending previous research in two ways: (1) participants' nonverbal behaviors were observed in either of two conditions that differed with respect to the salience of public identity; (2) an anatomically-based facial coding system was used to assess participants' emotion expressions and symbolic communication behaviors. Sixty female undergraduates, classified as repressive, low-anxious, or high-anxious, were videotaped during the preparation and delivery of a self-disclosing speech. During both the preparation and delivery, the salience of participants' public identities was either minimized (low-salience condition) or maximized (high-salience condition). Repressors and nonrepressors exhibited similar frequencies of hostile facial expressions. Repressors differed from nonrepressors by their frequent expressions of social smiles and conversational illustrators when their public selves were most salient. These findings suggest that certain symbolic communication behaviors may be nonverbal analogues of cognitive coping processes, and they support the utility of including expressive behaviors in conceptualizations of emotion-focused coping.     

Making a (False) Impression: The Role of Business Experience in First Impressions of CEO Leadership Ability

Recent studies have demonstrated that judgments of business leaders’ faces predict their organizations’ financial performance. To date, these predictions have been derived exclusively from the impressions of naïve perceivers. Here, we tested how perceivers’ knowledge and experience in business might relate to their judgments of CEOs’ leadership ability from nonverbal facial cues. In Study 1, business students performed similarly to non-business students when rating faces for leadership ability. Business professionals with many years of experience exhibited significantly lower accuracy than professionals without business experience in Study 2, however. Following previous research demonstrating that experience in a particular domain can ironically reduce the accuracy of individuals’ judgments, our findings suggest that perceivers’ experience in executive business management positions may inhibit them from accurately judging leadership ability from nonverbal information. Domain-specific knowledge may therefore impair the accuracy of first impressions.     

Nonverbal response patterns in physician-patient interactions: A functional analysis

In this paper, we examined physician-patient interactions in terms of the communicative functions accomplished during these encounters. Specifically, the nonverbal communicative exchanges of 38 physician-patient interactions in a family practice clinic were investigated. Two distinctive communicative patterns characterized these interactions. First, physicians nonverbally exerted greater dominance and control by employing longer speaking turns, more social touch, and more pauses while speaking than did the patients. Secondly, physicians and patients tended to reciprocate nonverbal behaviors signalling affiliation including gaze, response latencies, body orientations, and gestures. Consistent with previous research, there was a strong positive relationship between patients' satisfaction with health care and perceptions of the physicians' affiliativeness. Although there were few significant effects, physicians perceived less affiliative tended to be more vocally and gesturally active relative to the patients' nonverbal behavior styles. Finally, there were positive relationships between patients' perceptions of physicians' dominance and the physicians' use of task touch, and the extent to which physicians produced more interruptions and maintained more indirect body orientations relative to the patients' performances of these behaviors. The results are discussed in terms of the communicative functions of nonverbal behavior in physician-patient interactions and of patients' preferences for physicians' nonverbal expressiveness.The authors wish to thank Joe Cappella, Howard Waitzkin, Martha Davis, and William B. Stiles for their helpful comments on previous drafts of this paper.     

Three levels of measurement of social skill and social anxiety

This study investigates the development and assessment of micro-level behavioral measures of social anxiety and social skill and compares these measures to both midi and global level assessments as well as to a measure of physiological arousal. All behavioral ratings were made on similar Likert scales, thus potentially enhancing their clinical utility. Intraclass correlations exceeded .80 for the judges' ratings. Results of correlational analyses indicate several significant correlates of global anxiety ratings and of physiological arousal. Constituent micro-level behaviors were correlated with five of nine midi-level behaviors. Correlation between any midi-level behavior and its highest correlated constituent micro-level behavior was similar to the multiple correlation between the midi-level behavior and all its constituent behaviors. Implications for the clinical utility of the different levels of measurement studies were discussed.This research was supported by a grant from the Veterans Administration.The authors are indebted to Donna Bicki, Pamela Downing, Kenneth Lander, and Mary Ann Paxson for their assistance in completing this study.