Gestures or Signs? Comparing Manual and Non-manual Constructions Sharing the Same Form in Co-speech Gesture and Sign Language: A Corpus-driven Approach. (GeSi)

Project Participants

Anastasia Bauer

X

Dr. Anastasia Bauer
(Principal Investigator)

University of Cologne
anastasia.bauer@uni-koeln.de

Anastasia Bauer is interested in different anguage modalities and in contact between them. Her current work is focused on comparing constructions sharing a similar form in co-speech gesture and sign language. As a PhD fellow in the project „Village Sign“ within the collaborative research programme (EUROCORES) EuroBABEL „Better Analyses Based on Endangered Languages“, she worked on an endangered sign language in Aboriginal Australia. Her dissertation explores the spatial grammar in Yolŋu sign language. As a postdoc, she worked at the universities of Hamburg and Cologne. Her postdoctoral project focuses on two cross-modal language contact phenomena in Russian Sign Language, i.e. mouthings and fingerspellings. She is currently the Principal Investigator of the ViCom project “Gestures or signs? Comparing manual and non-manual constructions sharing the same form in co-speech gesture and sign language: a corpus-driven approach. (GeSi)”.

Selected publications

  • Mohr, Susanne & Anastasia Bauer (2022). Gesture, sign languages and multimodality. Svenja Völkel & Nico Nassenstein (Eds.). Approaches to Language and Culture, De Gruyter Mouton, 159-196. https://doi.org/10.1515/9783110726626-007
  • Bauer, Anastasia & Maria Kyuseva. (2022) New insights on mouthings in sign languages: evidence from corpus and elicitation studies of Russian sign language. Frontiers in Psychology. https://doi.org/10.3389/fpsyg.2021.779958
  • Bauer, Anastasia. 2020. Das Konzept der multimodalen Sprache am Beispiel von der Russischen Gebärdensprache. Bulletin der deutschen Slavistik 26, Berlin: Frank & Timme Verlag, 131-139.
  • Bauer, Anastasia. 2019. „When words meet signs: A corpus-based study on variation of mouthing in Russian Sign Language“. In: Anastasia Bauer & Bunčić, Daniel (Eds.). Linguistische Beiträge zur Slavistik, Specimina philologiae Slavicae 198, pp. 9-35.
  • Bauer, Anastasia. 2014. The Use of Signing Space in a Shared Sign Language of Australia. Berlin: Mouton de Gruyter. https://doi.org/10.1515/9781614515470

Anna Kuder

X

Dr. Anna Kuder
(Postdoc Researcher)

University of Cologne
akuder@uni-koeln.de

Anna Kuder is interested in nonmanual elements of sign languages and in the use of discourse particles across different language modalities. She obtained her PhD in 2020 at the University of Warsaw, after defending her thesis titled “Negation markers in Polish Sign Language (PJM)”. While doing her PhD she served as an Investigator or Principal Investigator in numerous projects (both research and educational) concerning PJM. She was involved in all stages of building the PJM Corpus and participated in the project of creating the first on-line Corpus-based Dictionary of PJM. As a postdoc, she is currently working at the University of Cologne, where she is conducting a comparative corpus-based study of gestural elements in Polish Sign Language, German Sign Language, and Russian Sign Language. She is also currently joining the ViCom project “Gestures or signs? Comparing manual and non-manual constructions sharing the same form in co-speech gesture and sign language: a corpus-driven approach. (GeSi)” as a postdoc.

Selected publications

  • S. Gabarró López & A. Kuder (2022), A corpus-based study of the “Away gestures” across four signed languages: DGS, LSC, LSFB and PJM, Belgian Journal of Linguistics, 36, in print.
  • A. Kuder (2022), Making Sign Language Corpora Comparable: A study of Palm-Up and Throw-Away in Polish Sign Language, German Sign Language, and Russian Sign Language. In: Workshop Proceedings. 10th Workshop on the Representation and Processing of Sign Languages: Multilingual Sign Language Resources. Language Resources and Evaluation Conference} (LREC), Marseille, France, 25 June 2022, ELRA, 110–117.
  • A. Kuder, J. Wójcicka, P. Mostowski & P. Rutkowski (2022), Open Repository of the Polish Sign Language Corpus: Publication Project of the Polish Sign Language Corpus. In: Workshop Proceedings. 10th Workshop on the Representation and Processing of Sign Languages: Multilingual Sign Language Resources. Language Resources and Evaluation Conference (LREC), Marseille, France, 25 June 2022, ELRA, 118–123.
  • A. Kuder (2021), Negation markers in Polish Sign Language (PJM), Sign Language and Linguistics, 24(1), pp. 118-131.
  • A. Kuder, J. Filipczak, P. Mostowski, P. Rutkowski & T. Johnston (2018), What Corpus-based Research on Negation in Auslan and PJM Tells Us about Building and Using Sign Language Corpora. In: M. Bono et al. (eds.), Workshop Proceedings. 8th Workshop on the Representation and Processing of Sign Languages: Involving the Language Community. Language Resources and Evaluation Conference (LREC), Miyazaki, Japan, 12 May 2018, ELRA, pp. 101-106.

Roman Poryadin

X

Roman Poryadin

Project Description

There is a number of manual and non-manual constructions which are not constrained to signed language only, they are also observed in co-speech gesture. The following forms will be systematically treated in this project: palm-up, throw away, pointing, list buoys, eyebrow raise and sidewards body leans. While some of these selected constructions have been researched, other forms remain un(der)studied and there is no fine-graded comparison of all these constructions between signers and speakers. This project fills this gap. It aims at providing a detailed corpus-based analysis of the selected constructions in both co-speech gesture and sign. The goal is to determine how – and to what degree – these constructions in sign language differ from comparable forms in gesture on both functional and formation grounds. As these constructions share the same modality, they will be given the same theoretical treatment and will be investigated along a number of various dimensions blurring the strict gesture-sign binary and backing the understanding of these manual and non-manual activities as forming a cross-modal continuum along which functional conventionalization and lexicalization takes place. Distinguishing signs and gesture, the two prime examples of visual communication, this project also provides further insights into the interaction of different channels and the grammatical system(s) underlying this interaction contributing to a new modality-free comprehensive theoretical model of language and communication.

Project Activities

Publications

Bauer A, Kuder A, Schulder M, Schepens J (2025) Correction: Phonetic differences between affirmative and feedback head nods in German Sign Language (DGS): A pose estimation study. PLOS ONE 20(3): e0321229. https://doi.org/10.1371/journal.pone.0321229

Bauer, A., Trettenbrein, P. C., Amici, F., Ćwiek, A., Fuchs, S., Krause, L., Kuder, A., Ladewig, S., Schulder, M., Schumacher, P., Spruijt, D., Zulberti, C. & Schulte-Rüther, M. (2025). Data Collection in Multimodal Language and Communication Research: A Flexible Decision Framework. https://doi.org/10.31234/osf.io/42tud_v1 

Henlein, A., A. Bauer, R. Bhattacharjee, A. Ćwiek, A. Gregori, F.Kügler, J. Lemanski, A. Lücking, A. Mehler, P. Prieto, P. G. Sánchez-Ramón, J. Schepens, M. Schulte-Rüther, S. R.Schweinberger & C. I. von Eiff. 2024. An Outlook for AI Innovation in MultimodalCommunication Research. Lecture Notes in Computer Science, Springer Cham.

Kuder, Anna & Anastasia Bauer. (submitted). When do you smile back: Exploring the alignment of smiling behavior in spontaneous dyadic signed interactions. To be published in Expressing Emotions in Sign Languages, Eds. Sarah Schwarzenberg, Simon Kollien, Nina-Kristin Meister, Thomas Finkbeiner, Annika Herrmann. Sign Languages and Deaf Communities, de Gruyter Mouton & Ishara Press.

Events
DateEventOrganization
09. – 10.10. 2024PALM UP in Gesture and Sign Symposium: Theoretical, Typological and Methodological Perspectives for Future ResearchSandra Debreslioska, Anastasia Bauer, Anna Kuder & and Pamela Perniss
Short-term Collaborations

Comparing a recurrent non-manual movement in spoken and signed languages (2024) – Anastasia Bauer & Silva Ladewig

Examining mouthings with Virtual Reality (VR) glasses (Meta Quest Pro) (2024) –
Anastasia Bauer, Alexander Mehler, Alexander Henlein & Andy Lücking

Studying head nods with Computer Vision tool OpenPose (2024) – Anastasia Bauer, Anna Kuder & Marc Schulder

The Analysis of Mouthings in the Online DGS corpus (2024) –
Anastasia Bauer, Nina K. Meister, Patrick C. Trettenbrein & Liona Paulus