Exploring multimodal prominence in Slicing gesture sequences

Researchers involved: Silva Ladewig (StabiGest, University of Göttingen), Frank Kügler and Alina Gregori (MultIS, Goethe University Frankfurt), Pilar Prieto and Paula Ginesa Sánchez Ramón (MultIS, UPF Barcelona)

The envisioned cooperation between ViCom projects “StabiGest” and “MultIS” is based on the observation that recurrent gestures frequently appear in extended sequences involving various recurrent gestures or different versions of a single recurrent gesture. This collaboration focuses on the prosodic analysis of sequences of recurrent gestures in the StabiGest corpus and an analysis of gestural prominence of these recurrent gestures. The interesting insights that could be gained from this collaboration include the following: a) assessing the rhythm of recurrent gestures and whether that rhythm complies to the prosodic rhythm; b) if components of the recurrent gestures align with prosody, which are the prosodic constituents that they align with?; c) are recurrent gestures prominent per se, or do we find differences in the perceived visual prominence of these gestures, and if so, how do the individual parts of the recurrent gestures and their alignment with prosodic events like pitch accents contribute to the prominence assessment?; d) how are recurrent gestures temporally aligned with higher-level prosodic structure and discursive units to functionally mark discourse cohesion? Addressing these issues, we will add prosodic and gestural prominence annotation to the StabiGest corpus.

We will focus on the “slicing” gesture, known for its simple and clear handshape, often observed in larger recurring sequences. The handshape involves a slightly spread flat hand, facing the speaker’s body or central gesture space along the sagittal plane. It is commonly executed with the hand’s edge facing downwards or positioned within the speaker’s gesture space. This gesture can be performed with one or both hands simultaneously, maintaining the same handshape. Variations in handedness and palm orientation correspond to variations in meaning.