The Effect of Pointing Gesture on Spatial Memory

Abstract

Pointing gestures are hand movements that identify references in space. These gestures are either directed to concrete locations where the entities situate (e.g., index finger points to the library) or to virtual locations that represent the entities (e.g., index finger points to an empty space on the left that is associated with the library). Pointing gestures to concrete locations direct listeners? attention to the target objects while pointing gestures to virtual locations help listeners to simulate an image of the spatial layout of the objects. This research aimed to examine whether encoding these two types of pointing gestures enhanced spatial memory in three experiments. Listeners seldom process pointing gestures alone. There are other sources of spatial information, such as maps and verbal descriptions. Hence, the way pointing gestures influence spatial memory might interact with other spatial cues. Study 1 examined the effect of pointing gestures to concrete locations on spatial memory and explored how such effect interacted with types of speech (spatial or non-spatial). The participants watched the narrator reciting spatial or non-spatial statements about fictitious countries while pointing to their locations on the maps. The findings showed that, when the maps were present, pointing gestures did not aid spatial recall. However, there was a significant interaction between the type of speech and the presence of pointing gesture, which highlights the importance of examining speech content that accompanies pointing. Study 2 manipulated the visibility of maps and examined whether pointing gestures aided spatial memory when the maps were hardly perceived. Pointing gestures aided spatial recall when the map was visually ambiguous, but the effect was marginally significant. Study 3 removed the maps entirely. The narrator pointed to the virtual locations that represented the countries. The results showed that pointing gestures enhanced location recall regardless of the types of co-occurring speech. As a result, the effect of pointing gestures on spatial memory interacts with the presence of maps and types of co-occurring speech. Pointing gestures do not always facilitate spatial memory. When the map is clear, pointing gestures appear to be redundant, probably due to the presence of other visual cues that were sufficient for efficient encoding of spatial location. However, pointing gestures are not redundant when directed to an unclear map or to a virtual location. They could serve to clarify reference that is present but unclear. In addition, pointing to a virtual location facilitates spatial memory regardless of the content of the accompanying speech. When the accompanying speech is spatial, pointing to a virtual location provides an alternative source of spatial information that could strengthen memory trace. When the accompanying speech is non-spatial, pointing to a virtual location provides an indispensible, only source of spatial information. These findings have especially relevant implications for classroom use of pointing gestures.