Researchers involved: Paula Sánchez-Ramón, Alina Gregori, Pilar Prieto, Frank Kügler (MultIS), Marion Bonnet (Georg-August-University Göttingen), Kurt Fassl Erbach (Goethe University Frankfurt / Saarland University)
The goal of this project is to investigate if gesture can disambiguate information structure as prosody does in disjunctive questions (1).
(1) Is Marcia allergic to dairy or soy? (Pruitt and Roelofsen 2013, 632)
This goal will be pursued with experimental research that aims to further our understanding of the type of contributions that gesture can make to the interpretation of utterances. Typically, prosody is crucial to determining meaning in disjunctive questions: those with a final fall in pitch give rise to an interpretation as an alternative question (2a) and a final rise in pitch gives rise to an interpretation as a yes/no question (2b) (Pruitt & Roelofsen 2013: 632).
(2)
a. Is Marcia allergic to dairy or is she allergic to soy?
b. Is Marcia allergic to either dairy or soy?
The interpretation between the two prosodic patterns has been analyzed in various ways: the final fall has been said to introduce a restriction on the space of possible answers either as a presupposition (e.g. Biezma & Rawlins 2012) or an update imposed on the common ground (e.g. Roelofsen & Pruitt 2011). In either case, the semantic contribution is said to focus on whether one of the disjuncts is true (e.g. Roelofsen & Pruitt 2011) or on the implication that the at-issue set of possibilities is exhausted by the given alternatives (e.g. Biezma & Rawlins 2012). Alternatively, an overall difference in focus structure has been said to be the source of the distinct meanings (e.g. Beck and Kim 2006 i.a.). As consensus on which analysis best captures this phenomenon is far from being reached, investigating the extent to which gesture can interact with such utterances can potentially add to the debate.
As it stands, gesture semantics (e.g. Ebert 2024, i.a.) assumes a similarity predicate between an entity identified in an utterance and one introduced with a co-speech gesture—e.g. if a round shape is gestured while simultaneously uttering window (3), then the similarity predicate is interpreted as the window being similar to the gesture and the discourse referent, z, introduced by the gesture (4)—i.e. that both are round. Crucially, the similarity predicate is not-at-issue content and therefore marked p⁎.
(3) ⟦windowROUND⟧ = [x] ∧ windowp⁎(x) ∧ [z] ∧ z = Ig ∧ SIMp⁎(x, z)
(4) ⟦ROUND⟧ = [z] ∧ z = Ig, where Ig is a rigid designator to the gesture g (cf. Ebert 2024: 180)
What has not yet been examined is if co-speech gestures are limited to the introduction of referents and similarity predicates as in (4), or whether they might also be able to give rise to the sorts of differences in meaning seen with prosody in disjunctive questions.
Should co-speech gesture be able to disambiguate disjunctive questions like (1), then major additions to formal gesture theory would be needed. Given (3) represents the state of the art of gesture semantics, additional theory capturing the contribution to disjunctive questions would be needed. Moreover, recall that some assume that a final prosodic fall gives rise to the implication that the at-issue set of possibilities is exhausted by the given alternatives (e.g. Biezma & Rawlins 2012). What is unclear is how not-at-issue gestures should give rise to implications regarding at-issue content. This might undermine accounts like Biezma & Rawlins (2012) unless one argues that either the gestured content is at-issue or their not-at-issue content can restrict the meaning of the at-issue content.
To test whether gesture can behave as intonation does in disjunctive questions the following experiment is proposed. An experiment with 3×3 Latin-square design will test for prosody–gesture interaction. The factor PROSODY will have the levels FALL, RISE, and UNCLEAR, where the unclear prosody is interrupted in some way—e.g. via static or low volume. The factor GESTURE will have the levels FALL, RISE, and SUBJECT.
The FALL condition will be realized with a co-speech gesture on each disjunct—starting from hands at the speaker’s side, one hand raises into a palm-up with the first disjunct and is held until the final syllable of the second disjunct at which point the hand falls back to the speaker’s side (5a). The RISE condition will be realized with a co-speech palm-up gesture on the sentence’s final syllable (5b), and the SUBJECT condition will be realized with a co-speech palm-up gesture on the subject of the sentence (5c).
(5)
a. Is Marcia allergic to dairyPALM-UP or to soyPALM-FALL?
b. Is Marcia allergic to dairy or soyPALM-UP?
c. Is MarciaPALM-UP allergic to dairy or soy?
As in Pruitt and Roelofsen (2013), participants will observe the stimuli and respond to a forced choice task with options like the items in (2) along with a third option, “other” for manual entry of an alternative. The stimuli will be presented as a series of videos distributed across nine lists so that no condition based on the same sentence (e.g. 5a–5c) is seen by a single individual. We will use the six base sentences of Pruitt and Roelofsen (2013), yielding 36 items in the current design. The same 14 filler items will be used for each list. Using the power tables in Brysbaert & Stevens (2018), to have >80% power with 36 items, we will recruit 99 participants, 11 per list.
The gestures in the GESTURE-SUBJECT items are not expected to contribute any relevant meaning to this experiment, so PROSODY-FALL–GESTURE-SUBJECT items are expected to elicit alternative question responses, PROSODY-RISE–GESTURE-SUBJECTitems are expected to elicit yes/no question responses, and PROSODY-UNCLEAR–GESTURE-SUBJECT items are expected to elicit random selection between the two responses.
The gestures in the GESTURE-FALL items are expected to be interpreted as realization of the semantic contribution of the final-fall prosody, so both PROSODY-FALL–GESTURE-FALL and PROSODY-UNCLEAR–GESTURE-FALL items are expected to elicit alternative question responses. The gestures in the GESTURE-RISE items are expected to be interpreted as realizations of the semantic contribution of the final-rise prosody, so both PROSODY-RISE–GESTURE-RISE and PROSODY-UNCLEAR–GESTURE-RISEitems are expected to elicit yes/no question responses.
Because prosody is presumably the more common indicator of reading for disjunctive questions, PROSODY-RISE–GESTURE-FALLitems are expected to elicit primarily yes/no question responses and PROSODY-FALL–GESTURE-RISE items are expected to elicit alternative question responses.