Visual and Non-visual Means of Perspective Taking in Language

Project Description

Perspective is an important part of the information conveyed by linguistic utterances, and there are many ways in which languages signal whether an utterance is to be understood as expressing the speaker’s or narrator’s perspective or the perspective of some discourse referent. Direct Discourse, Indirect Discourse, Free Indirect Discourse, Protagonist Projection and Viewpoint Shifting are ways of conveying information from a discourse referent’s perspective whose syntactic, semantic and pragmatic properties have been investigated extensively (see, e.g, Schlenker 2004, Sharvit 2008, Stokke 2013, Eckardt 2014, Maier 2015, Hinterwimmer 2017, Hinterwimmer 2019, Abrus.n 2020), and it is well known that there are ways to signal the prominence of the speaker’s or narrator’s perspective such as the use of deictic expressions, speech act particle and certain modal verbs (see Zeman 2019 for a recent overview) and the use of demonstrative pronouns to refer to topical protagonists (Hinterwimmer, Brocher and Patil 2020, Hinterwimmer 2020). At the same time, it is well-known that perspective taking can also be expressed at the level of co-speech gestures, i.e. gestures that speakers produce while uttering sentences (see Ebert & Ebert 2014, Schlenker 2018, and Ebert, Ebert & Hörnig 2020 for recent analyses of co-speech gestures in the formal semantic realm). In particular, there are two kinds of iconic gestures that are often used by speakers when they describe scenes or events to their interlocuters and that clearly reveal a perspective: character viewpoint gestures (CVG), on the one hand, and observer viewpoint gestures (OVG), on the other (McNeill 1992, Parrill 2010, 2012, Stec 2012, 2016). When performing the former, the speaker impersonates an individual participating in the event described by the sentence and enacts the event from that person’s point of view by using her entire body in combination with facial expressions. When performing the latter, in contrast, the speaker depicts the event described by the sentence as if it was observed from a distance, usually by using the hands exclusively which then represent a participant, with the hand’s trajectory representing that participant’s path, for instance.

What has not been investigated systematically, however, is the interaction of linguistic and gestural perspective taking (but see Hinterwimmer, Patil and Ebert 2020 and Ebert and Hinterwimmer to appear for first steps). Our goal in this project is therefore to arrive at a better understanding of how the two types of perspective taking interact. In particular, we want to address the following questions via experimental methods such as acceptability rating and forced-choice studies: Is there a preference for the perspectives expressed on the linguistic and the gestural level to be aligned? If so, how strong is this preference and under what conditions can it be overwritten? Is there a difference between the various kinds of perspective taking in this regard? On the basis of the experimental evidence we will develop a theoretical model of how the perspectival information conveyed on the two levels is integrated in such a way that it conveys a single coherent message. This will not only have important ramifications for our understanding of perspective taking but for the interaction of speech and co-speech gestures more generally.