Multimodal representational gestures in the transition to multi-word productions

The aim of this study was to analyze the use of representational gestures from a multimodal point of view in the transition from one-word to multi-word constructions. Twenty-one Spanish-speaking children were observed longitudinally at 18, 21, 24, and 30 months of age. We analyzed the production of deictic, symbolic, and conventional gestures and their coordination with different verbal elements. Moreover, we explored the relationship between gestural multimodal and unimodal productions and independent measures of language development. Results showed that gesture production remains stable in the period studied. Whereas deictic gestures are frequent and mostly multimodal from the beginning, conventional gestures are rare and mainly unimodal. Symbolic gestures are initially unimodal, but between 24 and 30 months of age, this pattern reverses, with more multimodal symbolic gestures than unimodal. In addition, the frequency of multimodal representational gestures at specific ages seems to be positively related to independent measures of vocabulary and morphosyntax development. By contrast, the production of unimodal representational gestures appears negatively related to these measures. Our results suggest that multimodal representational gestures could have a facilitating role in the process of learning to combine meanings for communicative goals.     

The context of infants' giving gestures in mother-infant dyads: Typical giving gestures and those contingent on exploration and play

This study aimed to focus on a niche that has not yet been investigated in infants' gesture studies that is the effect of the prior context of one specific gestural behavior (gives) on maternal behavior. For this purpose, we recruited 23 infants at 11 and 13 months of age yielded 246 giving gesture bouts that were performed in three contexts: typical when the object was offered immediately, contingent on exploration, and contingent on play. The analysis revealed that maternal responses to infants' giving gestures varied and were affected by their age and gesture context. Hence, mothers amended their responses according to the background that generated each gesture. The number of verbal responses to infants' giving gestures decreased as the infants aged, whereas the number of pretense responses increased. For infants aged 11 months, mothers generally provided motor responses to typical gestures. However, for infants aged 13 months, this trend declined and was replaced by a strong positive correlation between giving gestures contingent on play and verbal responses. We concluded that the type of activity with objects prior to employing giving gestures could enhance infants' symbolic skills because caregivers monitor the contingent act that yields the gesture that shapes their response.     

The Relation Between Gesture and Speech in Congenitally Blind and Sighted Language-Learners

Gesture is widely regarded to play an important role in communication, both in conjunction with and independent of speech. Indeed, gesture is known to develop even before the onset of spoken words. However, little is known about the communicative conditions under which gesture emerges. The aim of this study was to explore the role of vision in early gesturing. We examined gesture development in 5 congenitally blind and 5 sighted toddlers videotaped longitudinally between the ages of 14 and 28 months in their homes while engaging in free play with a parent or experimenter. All of the blind children were found to produce at least some gestures during the one-word stage of language development. However, gesture production was relatively low among the blind children relative to their sighted peers. Moreover, although blind and sighted children produced the same overall set of gesture types, the distribution of gesture types across categories differed. In addition, blind children used gestures primarily to communicate about objects that were nearby, while sighted children used them for nearby as well as distally located objects. These findings suggest that gesture may play different roles in the language-learning process for sighted and blind children. Nevertheless, they also make it clear that gesture is a robust phenomenon of early communicative development, emerging even in the absence of experience with a visual model.     

The Role of Gesture in Children's Comprehension of Spoken Language:Now They Need It, Now They Don't

Two experiments investigated gesture as a form of external support for spoken language comprehension. In both experiments, children selected blocks according to a set of videotaped instructions. Across trials, the instructions were given using no gesture, gestures that reinforced speech, and gestures that conflicted with speech. Experiment 1 used spoken messages that were complex for preschool children but not for kindergarten children. Reinforcing gestures facilitated speech comprehension for preschool children but not for kindergarten children, and conflicting gestures hindered comprehension for kindergarten children but not for preschool children. Experiment 2 tested preschool children with simpler spoken messages. Unlike Experiment 1, preschool children's comprehension was not facilitated by reinforcing gestures. However, children's comprehension also was not hindered by conflicting gestures. Thus, the effects of gesture on speech comprehension depend both on the relation of gesture to speech, and on the complexity of the spoken message.     

Mental Imagery and Production of Hand Gestures While Speaking in Younger and Older Adults

Speech and conversational hand gestures were recorded during interviews of 23 younger (M = 21 years) and 19 older adults (M = 70 years). Three kinds of questions were used in order to activate either visual images, motor images, or no mental image (abstract topics). On average, the rate of gesture production did not differ in younger and older adults, but it was significantly influenced by imagery conditions. Gesture production was higher in the motor than in the visual imagery condition, and lowest in the abstract condition. A significant interaction between age and imagery conditions influenced the proportion of representational gestures, which were relatively less frequent in older adults, especially in the visual imagery condition. Content analysis of verbal responses showed that imagery values did not differ in younger and older adults, but that concrete words were less frequent in responses to abstract questions than in the two other conditions. The implications of these results concerning the mechanisms of gesture production and the age-related changes in conversational behavior are discussed.     

Representational Gesture,Pointing Gesture,and Memory Recall of Preschool Children

This study investigated the effects of two different types of hand gestures on memory recall of preschool children. Experiment 1 found that children who were instructed to use representational gestures while retelling an unfamiliar story retrieved more information about the story than children who were asked to hold their hands still. In addition, children who engaged in some forms of bodily movements other than hand gestures also recalled better. Experiment 2 showed that a simpler and more basic form of gesture, the pointing gesture, had a similar effect on recollecting and retelling the details of a story. The findings provide evidence for the beneficial effects of hand gestures, both representational gestures and pointing gestures, on cognitive processes such as memory retrieval and verbal communication for preschool aged children.     

Co-Speech Gesture Mimicry in the Process of Collaborative Referring During Face-to-Face Dialogue

Mimicry has been observed regarding a range of nonverbal behaviors, but only recently have researchers started to investigate mimicry in co-speech gestures. These gestures are considered to be crucially different from other aspects of nonverbal behavior due to their tight link with speech. This study provides evidence of mimicry in co-speech gestures in face-to-face dialogue, the most common forum of everyday talk. In addition, it offers an analysis of the functions that mimicked co-speech gestures fulfill in the collaborative process of creating a mutually shared understanding of referring expressions. The implications bear on theories of gesture production, research on grounding, and the mechanisms underlying behavioral mimicry.     

A parent gesture intervention as a means to increase parent declarative pointing and child vocabulary

This study examined whether a brief parent gesture training resulted in a change in the communicative intent of pointing gestures used by parents of infants from age 10–12 months and whether specific types of points (declarative vs. imperative) were more or less likely to predict later child language skill at 18 months. Compared to parents who were randomized to the control group, parents in the intervention group produced significantly more declarative pointing gestures as a result of the intervention. Moreover, parents’ use of declarative points at 12 months was predictive of later child vocabulary comprehension at 18 months. These findings suggest that a short-term parent training can have important effects on the communicative intentions conveyed through gesture which predict vocabulary development.     

The Organization of Words and Symbolic Gestures in 18‐Month‐Olds’ Lexicons: Evidence From a Disambiguation Task

Infants’ early communicative repertoires include both words and symbolic gestures. The current study examined the extent to which infants organize words and gestures in a single unified lexicon. As a window into lexical organization, eighteen‐month‐olds’ (N = 32) avoidance of word–gesture overlap was examined and compared with avoidance of word–word overlap. The current study revealed that when presented with novel words, infants avoided lexical overlap, mapping novel words onto novel objects. In contrast, when presented with novel gestures, infants sought overlap, mapping novel gestures onto familiar objects. The results suggest that infants do not treat words and gestures as equivalent lexical items and that during a period of development when word and symbolic gesture processing share many similarities, important differences also exist between these two symbolic forms.     

A Comparative Study on Representational Gestures in Italian and Japanese Children

This study compares words and gestures produced in a controlled experimental setting by children raised in different linguistic/cultural environments to examine the robustness of gesture use at an early stage of lexical development. Twenty-two Italian and twenty-two Japanese toddlers (age range 25?C37?months) performed the same picture-naming task. Italians produced more spoken correct labels than Japanese but a similar amount of representational gestures temporally matched with words. However, Japanese gestures reproduced more closely the action represented in the picture. Results confirm that gestures are linked to motor actions similarly for all children, suggesting a common developmental stage, only minimally influenced by culture.     

How Do We Communicate About Pain? A Systematic Analysis of the Semantic Contribution of Co-speech Gestures in Pain-focused Conversations

The purpose of the present study was to investigate co-speech gesture use during communication about pain. Speakers described a recent pain experience and the data were analyzed using a ‘semantic feature approach’ to determine the distribution of information across gesture and speech. This analysis revealed that a considerable proportion of pain-focused talk was accompanied by gestures, and that these gestures often contained more information about pain than speech itself. Further, some gestures represented information that was hardly represented in speech at all. Overall, these results suggest that gestures are integral to the communication of pain and need to be attended to if recipients are to obtain a fuller understanding of the pain experience and provide help and support to pain sufferers.     

Children's facial and gestural decoding and encoding: Relations between skills and with popularity

This study examined preschool children's decoding and encoding of facial emotions and gestures, interrelationships between these skills, and the relationship between these skills and children's popularity. Subjects were 34 preschoolers (eighteen 4-year-olds, sixteen 5-year-olds), with an equal number of boys and girls. Children's nonverbal skill was measured on four tasks: decoding emotions, decoding gestures, encoding facial emotions, and encoding gestures. Children's popularity was measured by teacher ratings. Analyses revealed the following major findings: (a) There were no age or gender effects on performance on any of the tasks. (b) Children performed better on decoding than encoding tasks, suggesting that nonverbal comprehension precedes production. Also, children appeared better at facial emotion skills than gesture skills. There were significant correlations between decoding and encoding gestures, and between encoding gestures and encoding emotions. (c) Multiple regression analyses indicated that encoding emotions and decoding gestures were marginally predictive of popularity. In addition, when children's scores on the four tasks were combined via z-score transformations, children's aggregate nonverbal skill correlated significantly with peer popularity.Portions of this paper were presented at the meeting of the American Psychological Society, San Diego, CA, June, 1992. We thank the Child Study Center of Wellesley College, Janine Jarrell, Jennifer Mascola, and David Mills for their cooperation, and Carlene Nelson, Mark Runco, and Ed Stearns for statistical support. We also appreciate the valuable suggestions from Robin Akert, Annick Mansfield, the anonymous reviewers, and especially the guest editor.     

Unspoken Diversity: Cultural Differences in Gestures

This paper describes the use of video to explore cultural differences in gestures. Video recordings were used to capture a large sample of international gestures, and these are edited into a documentary video, A World of Gestures: Culture and Nonverbal Communication. This paper describes the approach and methodology used. A number of specific questions are examined: Are there universally understood hand gestures?; Are there universal categories of gestures—i.e., universal messages with unique instances in each society?; Can the exact same gesture have opposite meanings in two cultures?; Can individuals articulate and explain the gestures common in their culture?; How can video methods provide “visual replication” of nuanced behaviors such as gestures?; Are there gender differences in knowing or performing gestures?; and finally, Is global diversity collapsing toward Western gestural forms under the onslaught of cultural imperialism? The research findings suggest that there are both cultural “differences” and also cultural “meta-differences”—more profound differences involving deeply embedded categories of meaning that make cultures unique.     

Lending a Hand to Imagery? The Impact of Visuospatial Working Memory Interference Upon Iconic Gesture Production in a Narrative Task

Visuospatial working memory is thought to be responsible for imagery generation (Cattaneo et al. in Imagery and spatial cognition: methods, models and cognitive assessment, John Benjamins Publishing Company, Amsterdam, 2006). This memory system was manipulated by varying visual perceptual input (see Baddeley and Andrade in Theories of memory, Psychology Press, Hove, 1998) in a narrative task in order to investigate the association between iconic gesture production and visuospatial working memory interference. Participants watched two short cartoon clips and were asked to relay a narrative about what they had seen in the videos to an experimenter. Participants were randomly assigned to relay their narrative while wearing video glasses with either a simple or complex moving image (unrelated to the cartoons) projected onto the lenses. It was hypothesized that if gesture production plays a role in facilitating visuospatial working memory resource activation, then participants in the complex visual distractor condition would display a higher rate of gesture production. Participants in the complex visual distractor condition gestured significantly more than participants in the simple visual distractor condition. These results are interpreted as lending support to the argument that iconic gestures may play a functional role in activating visuospatial working memory resources during a narrative task. Since visuospatial working memory is thought to support imagery, these results also suggest that gesture production may facilitate imagery generation.     

Temporal Closeness Promotes Imitation of Meaningful Gestures in Face-to-Face Communication

The present research investigated how temporal distance affects imitation of gestures during a conversation. According to construal level theory, psychological closeness (vs. distance) leads to preference for more pictorial and embodied (vs. symbolic and amodal) forms of communication as well as to greater context sensitivity. We thus hypothesized that people are more sensitive to the contextual meaning of gestures and show more imitation of meaningful (vs. meaningless) gestures when primed with temporal closeness (compared to distance). Participants (N = 91) were assigned to the role of a job interviewer for an internship that would begin in 1 week (vs. in 1 year) and interacted with an ostensible applicant who displayed various meaningful and meaningless gestures. When the internship was to start in 1 week, participants imitated meaningful gestures more often than meaningless gestures. When the internship was to start in 1 year, participants imitated both gesture types equally frequently. This finding indicates that people are more likely to embody verbal statements of an interaction partner when feeling psychologically close to the subject of the conversation.     

Hand over Heart Primes Moral Judgments and Behavior

Morality is a prominent guide of both action and perception. We argue that non-emotional gestures can prime the abstract concept of honesty. Four studies demonstrated that the emblematic gesture associated with honesty (putting a hand on one’s heart) increased the level of honesty perceived by others, and increased the honesty shown in one’s own behavior. Target persons performing this gesture were described in terms associated with honesty, and appeared more trustworthy to others than when the same targets were photographed with a control gesture. Persons performing the hand-over-heart gesture provided more honest assessments of others’ attractiveness, and refrained from cheating, as compared to persons performing neutral gestures. These findings suggest that bodily experience associated with abstract concepts can influence both one’s perceptions of others, and one’s own complex actions. Further, our findings suggest that this influence is not mediated by changes in affective states.     

Attention to Speech-Accompanying Gestures: Eye Movements and Information Uptake

There is growing evidence that addressees in interaction integrate the semantic information conveyed by speakers’ gestures. Little is known, however, about whether and how addressees’ attention to gestures and the integration of gestural information can be modulated. This study examines the influence of a social factor (speakers’ gaze to their own gestures), and two physical factors (the gesture’s location in gesture space and gestural holds) on addressees’ overt visual attention to gestures (direct fixations of gestures) and their uptake of gestural information. It also examines the relationship between gaze and uptake. The results indicate that addressees’ overt visual attention to gestures is affected both by speakers’ gaze and holds but for different reasons, whereas location in space plays no role. Addressees’ uptake of gesture information is only influenced by speakers’ gaze. There is little evidence of a direct relationship between addressees’ direct fixations of gestures and their uptake.