Participants
(Participants are listed in alphabetical order by their last name.)
A – D
Dr. Federica Amici
(Principal Investigator)
Project:
Compositional structures
in chimpanzee gestural communication
University of Leipzig
amici@eva.mpg.de
Federica Amici obtained her PhD at the Liverpool John Moores University in the UK. She then moved to Germany, where she is currently working in Katja Liebal’s group as a post-doc. Her main research interests lie in the evolutionary processes shaping the distribution of behavior and cognition in animals, with a special focus on primates and ungulates. Together with Katja Liebal, I am currently working on the complexity and ontogeny of primate communication, and I maintain a secret but intense love for psycholinguistics.
Selected publications
- Amici, F., Lembeck, M., Holodynski, M., & Liebal, K. 2022. Face to face interactions in chimpanzee and human mother-infant-dyads. Philosophical Transactions B.
- Amici, F., & Liebal, K. 2022. Primates unleashed. Behavioral and Brain Sciences.
- Amici, F., & Liebal, K. 2022. The social dynamics of communication in great and lesser apes (Pan troglodytes, Pongo pygmaeus abelii, Symphalangus syndactylus). Philosophical Transactions B, 377, 20210299.
- Amici, F., Oña, L., & Liebal, K. 2022. Compositionality in primate gestural communication and multicomponent signal displays. International Journal of Primatology. https://doi.org/10.1007/s10764-022-00316-9.
- Amici, F., Sanchez-Amaro, A., & Cacchione, T. 2022. Branching and working memory: a cross-linguistic approach. In: The Cambridge handbook of working memory and language (Eds. Schwiete, J.W., & Wen, Z.). Cambridge University Press.
Dr. Mailin Ines Antomo
(Principal Investigator)
Project:
Lying, deceiving, misleading:
are we committed to our gestures?
University of Göttingen
mailin.antomo@phil.uni-goettingen.de
Mailin Antomo’s research investigates phenomena at the interface of syntax, semantics, and pragmatics with a focus on German. She has worked, both theoretically and empirically, on embedded root phenomena, (not-)at-issueness, implicatures, presuppositions, lying and deceiving, and the acquisition of these phenomena. Furthermore, she is interested in educational linguistics. She obtained her PhD from the University of Göttingen in 2015, after being part of the DFG-funded research training group “Sentence types: Variation and Interpretation” at the university of Frankfurt. She is currently a permanent lecturer for German linguistics at the University of Göttingen and principal investigator of the project “Lying, deceiving, misleading: are we committed to our gestures?”.
Selected publications
- Antomo, Mailin (to appear): Lying with visual means. To appear in: Wiegmann, Alex (ed.): Lying, Fake News, and Bullshit. Bloomsbury, Advances in Experimental Philosophy.
- Chen, Yuqiu/Thalmann, Maik/Antomo, Mailin (2022): Presuppositions and at-issueness: Insights from language acquisition into the soft-hard distinction. In: Journal of Pragmatics 199, 21-46.
- Thalmann, Maik/Chen, Yuqiu/Müller, Susanne/Paluch, Markus/Antomo, Mailin (2021): Against PCI-GCI uniformity: Evidence from deceptive language in German and Chinese. In: Linguistische Berichte 267, 355-385.
- Antomo, Mailin/Müller, Susanne/Paluch, Markus/Paul, Katharina/Thalmann, Maik (2018): When children aren’t more logical than adults: An empirical investigation of lying by falsely implicating. In: Journal of Pragmatics 138, 135-148.
- Antomo, Mailin (2012): Projective Meaning and the Licensing of Embedded Root Phenomena. In: Boone, Enrico/Linke, Kathrin/Schulpen, Maartje (ed.): Proceedings of ConSOLE XIX, Leiden, 1-23.
Dr. Nadine Bade
(Principal Investigator)
Project:
Pragmatic reasoning with
(non-)visual alternatives
University of Potsdam
nadine.bade@uni-potsdam.de
(A short introduction and selected publications are to be updated.)
Dr. Anastasia Bauer
(Principal Investigator)
University of Cologne
anastasia.bauer@uni-koeln.de
Anastasia Bauer is interested in different anguage modalities and in contact between them. Her current work is focused on comparing constructions sharing a similar form in co-speech gesture and sign language. As a PhD fellow in the project „Village Sign“ within the collaborative research programme (EUROCORES) EuroBABEL „Better Analyses Based on Endangered Languages“, she worked on an endangered sign language in Aboriginal Australia. Her dissertation explores the spatial grammar in Yolŋu sign language. As a postdoc, she worked at the universities of Hamburg and Cologne. Her postdoctoral project focuses on two cross-modal language contact phenomena in Russian Sign Language, i.e. mouthings and fingerspellings. She is currently the Principal Investigator of the ViCom project “Gestures or signs? Comparing manual and non-manual constructions sharing the same form in co-speech gesture and sign language: a corpus-driven approach. (GeSi)”.
Selected publications
- Mohr, Susanne & Anastasia Bauer (2022). Gesture, sign languages and multimodality. Svenja Völkel & Nico Nassenstein (Eds.). Approaches to Language and Culture, De Gruyter Mouton, 159-196. https://doi.org/10.1515/9783110726626-007
- Bauer, Anastasia & Maria Kyuseva. (2022) New insights on mouthings in sign languages: evidence from corpus and elicitation studies of Russian sign language. Frontiers in Psychology. https://doi.org/10.3389/fpsyg.2021.779958
- Bauer, Anastasia. 2020. Das Konzept der multimodalen Sprache am Beispiel von der Russischen Gebärdensprache. Bulletin der deutschen Slavistik 26, Berlin: Frank & Timme Verlag, 131-139.
- Bauer, Anastasia. 2019. „When words meet signs: A corpus-based study on variation of mouthing in Russian Sign Language“. In: Anastasia Bauer & Bunčić, Daniel (Eds.). Linguistische Beiträge zur Slavistik, Specimina philologiae Slavicae 198, pp. 9-35.
- Bauer, Anastasia. 2014. The Use of Signing Space in a Shared Sign Language of Australia. Berlin: Mouton de Gruyter. https://doi.org/10.1515/9781614515470
Dr. Reetu Bhattacharjee
(Postdoc Researcher)
Project:
Gestures and Diagrams in Visual-Spatial Communication: Methodological Tools and Applications in Mathematics and Logic
University of Münster
reetu.bhattacharjee@uni-muenster.de
(Photo, a short introduction, and selected publications are to be updated.)
Dr. Aleksandra Ćwiek
(ViCom Data Network Coordinator, PI)
Leibniz-Zentrum Allgemeine Sprachwissenschaft Berlin
cwiek@leibniz-zas.de
Aleksandra Ćwiek is a researcher at the Leibniz-Centre General Linguistics (ZAS) in Berlin and a PI of the project “On the FLExibility and Stability of gesture-speecH coordination (FLESH): Evidence from production, comprehension, and imitation”. She received her Ph.D. from the Humboldt University of Berlin in 2022. In her doctoral thesis, she studied the diversity of linguistic iconicity and its role in language evolution and in language nowadays. She examined aspects such as iconic prosody, sound symbolism, and German ideophones. Her further interests include acoustic phonetics, speech-gesture link, cognition, and cross-linguistic research.
Selected publications
- Ćwiek, A. (2022). Iconicity in Language and Speech [PhD Thesis, Humboldt University of Berlin].
https://doi.org/10.18452/24544 - Ćwiek, A., Fuchs, S., Draxler, C., Asu, E. L., Dediu, D., … Perlman, M. (2021). Novel vocalizations are understood across cultures. Scientific Reports, 11(1), 10108. https://doi.org/10.1038/s41598-021-89445-4
- Ćwiek, A., Fuchs, S., Draxler, C., Asu, E. L., Dediu, D., … Winter, B. (2022). The bouba/kiki effect is robust across cultures and writing systems. Philosophical Transactions of the Royal Society B: Biological Sciences, 377(1841), 20200390.
- Fuchs, S., & Ćwiek, A. (2022). Sounds Full of Meaning and the Evolution of Language. Acoustics Today, 18(1), 43–51. https://doi.org/10.1121/AT.2022.18.2.43
- Wagner, P., Ćwiek, A., & Samlowski, B. (2019). Exploiting the speech-gesture link to capture fine-grained prosodic prominence impressions and listening strategies. Journal of Phonetics, 76, 100911.
https://doi.org/10.1016/j.wocn.2019.07.001
Prof. Dr. Christian Dobel
(Principal Investigator)
Project:
Audiovisual Perception of Emotion and Speech
in Hearing Individuals and Cochlear Implant Users
Jena University Hospital
christian.dobel@med.uni-jena.de
(Photo, a short introduction, and selected publications are to be updated.)
E – G
Prof. Dr. Cornelia Ebert
(Coordinator, Steering Committee Member, PI)
Project:
Visual and Non-visual Means of
Perspective Taking in Language
Goethe University Frankfurt
ebert@lingua.uni-frankfurt.de
Cornelia Ebert is a professor of linguistics/semantics at Goethe University in Frankfurt am Main. She received her Ph.D. in lingusitics at Potsdam University in 2006. Her thesis was titled »Quantificational topics. A scopal treatment of exceptional wide scope phenomena«. She was a lecturer and a researcher at Osnabrück University, Stuttgart University and the Leibniz-Zentrum Allgemeine Sprachwissenschaft in Berlin. Since 2021 she carries out a British-German cooperative research project entitled »Interactions between Dynamic Effects and Alternative-Based Inferences in the Study of Meaning« (IDEALISM). And since 2022 she is one of the coordinators of the DFG Priority Programme 2392 Visual Communication. Theoretical, Empirical, and Applied Perspectives (ViCom). 2020-24 she is Goethe Fellow at the Forschungskolleg Humanwissenschaften.
Selected publications
- Barnes, K. R., Ebert, C., Hörnig, R., & Stender, T. (2022). The at-issue status of ideophones in German: An experimental approach. Glossa: A Journal of General Linguistics, 7(1). https://doi.org/10.16995/glossa.5827
- Ebert, C., Ebert, C., & Hörnig, R. (2020). Demonstratives as dimension shifters. Proceedings of Sinn Und Bedeutung, 161-178 Pages. https://doi.org/10.18148/SUB/2020.V24I1.859
- Ebert, C. (2018). A comparison of sign language with speech plus gesture. Theoretical Linguistics, 44(3–4), 239–249. https://doi.org/10.1515/tl-2018-0016
- Ebert, C., Ebert, C., & Hinterwimmer, S. (2014). A unified analysis of conditionals as topics. Linguistics and Philosophy, 37(5), 353–408. https://doi.org/10.1007/s10988-014-9158-4
- Ebert, C. (2009). Quantificational topics: A scopal treatment of exceptional wide scope phenomena (Vol. 86). Springer Science & Business Media.
Prof. Dr. Natalia Filatkina
(Principal Investigator)
University of Hamburg
natalia.filatkina@uni-hamburg.de
Natalia Filatkina is a professor for German linguistics at Universität Hamburg with a focus on digital historical linguistics. Her research focuses on historic linguistics of German, language change, and multilingualism, amongst others. She received her PhD from Otto Friedrich Universität Bamberg in 2003. After being a scientific assistant (C1) at the Universität Trier, she was Director of the junior researcher group “Historische Formelhafte Sprache und Traditionen des Formulierens (HiFoS)” at the Universität Trier. Since then she has been a deputy professor at the Heinrich Heine-Universität Düsseldorf, academy professor at the Universität Trier and at the same time at the Akademie der Wissenschaften und der Literatur in Mainz. She obtained her habilitation at the Universität Trier. Since 2021, she is president of the Europhras-Gesellschaft.
Selected publications
- Filatkina, N. 2018. Historische formelhafte Sprache. Theoretische Grundlagen und Methoden ihrer Erforschung [Habilitationsschrift]. Berlin & Boston: De Gruyter Mouton.
- Filatkina, N., C. Moulin, I. Gurevych & R. Eckart de Castilho. 2015. Analyzing Formulaic Patterns in Historical Corpora. In Gippert, J. & R. Gehrke (eds.), Historical Corpora: Challenges and perspectives. Tübingen: Narr, 51–63.
- Filatkina, N. 2014. Constructionalization, Konstruktionswandel und figurative Sprache (sprach)historisch betrachtet. In Dalmas, M. & E. Piirainen (eds.), Figurative Sprache – Figurative Language – Langage figuré. Festgabe für Dmitrij O. Dobrovol’skij. Tübingen: Stauffenburg Verlag, 41–57.
- Filatkina, N. 2010. Phraseologie der germanischen Sprachen kontrastiv: Geschichte, Ergebnisse und Perspektiven. In Dammel, A., S. Kürschner & D. Nübling (eds.), Kontrastive Germanische Linguistik. Teilband 1 [Germanistische Linguistik, 206–209]. Hildesheim: Olms Weidmann, 275–309.
- Filatkina, N. 2005. Multi-methodologische Korpuserstellung als empirische Basis für phraseologische und phraseographische Untersuchungen. Am Beispiel des Lëtzebuergeschen. In Eckhard, E., J. E. Schmidt & D. Stellmacher (eds.), Moderne Dialektologie – Neue Dialektologie. Akten des 1. Kongresses der Internationalen Gesellschaft für Dialektologie des Deutschen (IGDD) am Forschungsinstitut für deutsche Sprache „Deutscher Sprachatlas“ der Philipps-Universität Marburg vom 5.-8. März 2003 [Zeitschrift für Dialektologie und Linguistik – Beihefte, vol. 130]. München: Franz Steiner, 555–572.
Thomas Finkbeiner
(Principal Investigator)
Project:
Parts of Speech and Iconicity in
German Sign Language (DGS)
University of Göttingen
thomas.finkbeiner@uni-goettingen.de
Thomas Albert Finkbeiner is deaf and grew up with German Sign Language (DGS) as his native language. He is a certified social worker / certified social pedagogue (FH), state-certified sign language lecturer and state-certified sign language interpreter for DGS and International Sign. Since 2017, he has been working at the Seminar for German Philology at the Georg-August-University Göttingen as a lecturer for DGS and Deaf Studies. He is involved in various research on sign languages as a researcher, consultant and translator. He is editing the first bimodal-bilingual book series in German Sign Language (DGS) and written German ‘Deutsche Gebärdensprache und Deaf Communities’ (Buske Verlag, with Nina-Kristin Meister) and the ‘German Sign Language Calendar’ (Buske Verlag, with Nina-Kristin Meister).
Selected publications
- Thomas Finkbeiner, Nina-Kristin Meister and Alexander Silbersdorff (in prep.): Gemeinsam auf dem Weg zur barrierefreien digitalen Hochschullehre. Mathematische Lehrvideos mit Deutscher Gebärdensprache (DGS) (in prep.). In: Neues Handbuch Hochschullehre. Berlin: DUZ Verlags- und Medienhaus.
- Thomas Finkbeiner and Nina-Kristin Meister (in prep.): Sprachkalender Deutsche Gebärdensprache 2024. Hamburg: Buske.
- Thomas Finkbeiner, Liona Paulus and Nina-Kristin Meister (in prep.): 100 Fragen und Antworten rund um die Deutsche Gebärdensprache (DGS). Hamburg: Buske.
- Nina-Kristin Pendzich, Jens-Michael Cramer, Thomas Finkbeiner, Annika Herrmann and Markus Steinbach (2022): How do Signers Mark Conditionals in German Sign Language? Insights from a Sentence Reproduction Task on the Use of Nonmanual and Manual Markers. In: Croatian Review of Rehabilitation Research 58 (Special Issue: Sign Language, Deaf Culture, and Bilingual Education), 206-226.
- Thomas Finkbeiner and Nina-Kristin Pendzich (2022): Sprachkalender Deutsche Gebärdensprache 2023. Hamburg: Buske.
Lea Fricke
(Postdoc Researcher)
Project:
Semantics and Pragmatics of Emojis in
Digital Communication (EmDiCom)
University of Bochum
Lea.Fricke@ruhr-uni-bochum.de
Lea Fricke’s research interests lie in the area of semantics, pragmatics and empirical linguistics. She has worked on embedded questions, the German prefield-es, scalar implicatures and experimental methodology. In November 2022, Lea joined the Ruhr-University Bochum to work in the ViCom project “Semantics and Pragmatics of Emojis in Digital Communication”. Before that, she worked as a university assistant at the University of Graz and was part of the XPrag.de project “Exhaustiveness in embedded questions across languages“.
Selected publications
- Cremers, Alexandre, Lea Fricke, Edgar Onea. Accepted. The importance of being earnest: How truth and evidence affect participants’ judgments. Glossa Psycholinguistics.
- Cortez Espinoza, Maya, Lea Fricke. To appear. On the interpretation of German einige. The effect of tense and cardinality. Proceedings of ELM 2.
- Zimmermann, Malte, Lea Fricke, Edgar Onea. 2022. Embedded questions are exhaustive alright, but… In A. Özgün, Y. Zinova (eds.) Proceedings of TbiLLC 2019, 173–194. Cham: Springer.
- Fricke, Lea. 2020. A southern German use of prefield-es: Evidence from the corpus and an experimental study. Zeitschrift für Sprachwissenschaft 39(1), 41–77.
- Tönnis, Swantje, Lea M. Fricke, Alexander Schreiber. 2018. Methodological Considerations on Testing Argument Asymmetry in German Cleft Sentences. In E. Fuß, M. Konopka, B. Trawiński, U. H. Waßner (eds.). Grammar and Corpora 2016, 231–240. Heidelberg: Heidelberg University Publishing.
Dr. Susanne Fuchs
(ViCom Data Network Coordinator, Steering Committee Member, PI)
Leibniz-Zentrum Allgemeine Sprachwissenschaft Berlin
fuchs@leibniz-zas.de
Susanne Fuchs (ZAS Berlin) investigates the biopsychosocial foundations of human interaction and focuses specifically on physiological processes, such as breathing and motor control. Her main areas of interests are:
1) The interplay between motion, breathing and cognition,
2) Speech preparation and pauses,
3) Multimodality and iconicity,
4) Biological and social aspects shaping individual behaviour in speech production and perception.
She uses manifold techniques, among them optitrack, inductance plethysmography, electropalatography and intraoral pressure sensors.
Selected publications
- Pouw, W., & Fuchs, S. (2022). Origins of vocal-entangled gesture. Neuroscience & Biobehavioral Reviews, 104836.
- Fuchs, S., & Rochet-Capellan, A. (2021). The respiratory foundations of spoken language. Annual Review of Linguistics, 7(1), 13-30.
- Ćwiek, A., Fuchs, S., Draxler, C., Asu, E. L., Dediu, D., Hiovain, K., … & Perlman, M. (2021). Novel vocalizations are understood across cultures. Scientific Reports, 11(1), 1-12.
- Floegel, M., Fuchs, S., & Kell, C. A. (2020). Differential contributions of the two cerebral hemispheres to temporal and spectral speech feedback control. Nature communications, 11(1), 1-12.
- Fuchs, S. (2019). Vocal tract variations affect vowel sounds. Nature Human Behaviour, 3(10), 1043-1044.
- Żygis, M., & Fuchs, S. (2023). Communicative constraints affect oro-facial gestures and acoustics: Whispered vs normal speech. The Journal of the Acoustical Society of America, 153(1), 613–626. https://doi.org/10.1121/10.0015251
Prof. Dr. Volker Gast
(Principal Investigator)
Project:
Audiovisual Perception of Emotion and Speech
in Hearing Individuals and Cochlear Implant Users
University of Jena
volker.gast@uni-jena.de
(Photo, a short introduction, and selected publications are to be updated.)
Alina Gregori
(Steering Committee Member, PhD Candidate, PhD Contact Person)
Project:
Co-speech Gestures and Prosody as
Multimodal Markers of Information Structure
Goethe University Frankfurt
gregori@lingua.uni-frankfurt.de
Alina Gregori received her Bachelors (2021) and Masters (2022) degree in theoretical linguistics at Goethe University Frankfurt with a focus on Phonology. She started working as a PhD student in the project MultIS in October 2022. Within the project, she investigates the prosody-gesture link in communication and the impact of information structure (focus, topic, givenness) on the synchronization of gestures and prosodic entities in German. The bigger picture of the project includes a comparative analysis of German and Catalan with regard to the multimodal marking of information structure. A central aim in MultIS is the empirical approach to previously established prosody-gesture models, considering experimental as well as spontaneous speech utterances. Alina Gregoris MA thesis (title: “Co-speech Gestures, Information Structure and Prosody: A Corpus Study on Prominence Peak Alignment”) served as a pilot and preparation for MultIS.
Selected publications
- Gregori, Alina & Kügler, Frank (2022). An Empirical Investigation on the Perceptual Similarity of Prosodic Language Types. In Proc. 1st International Conference on Tone and Intonation (TAI), 209-213, doi: 10.21437/TAI.2021-43.
Prof. Dr. Patrick Georg Grosz
(Principal Investigator)
Project:
Semantics and Pragmatics of Emojis in
Digital Communication (EmDiCom)
University of Oslo
p.g.grosz@iln.uio.no
Patrick Georg Grosz is Professor of Linguistics at the University of Oslo. He obtained a Mag.phil. in Linguistics from the University of Vienna (2005) and a PhD in Linguistics from MIT (2011). His interests include semantics, syntax, pragmatics, and their interfaces; he has worked on topics such as optatives, imperatives, discourse particles, pronouns, agreement, and the diachrony of ‘dunno’ indefinites. In his current research, he is focusing on the application of linguistic methodology beyond natural language, to objects such as emojis and gestures, with a particular emphasis on face emojis.
Selected publications
- Grosz, Patrick G. (submitted): Expressivity and Emojis. Submitted for The Oxford Handbook of Expressivity edited by Daniel Gutzmann.
- Grosz, Patrick G., Gabriel Greenberg, Christian De Leon, and Elsi Kaiser (accepted): A semantics of face emoji in discourse. To appear in Linguistics and Philosophy. [https://ling.auf.net/lingbuzz/005981]
- Grosz, Patrick G. (2022): Emojis and conditionals: Exploring the super linguistic interplay of pictorial modifiers and conditional meaning. Linguistics Vanguard. [https://doi.org/10.1515/lingvan-2021-0123]
- Grosz, Patrick G., Elsi Kaiser, and Francesco Pierini (2021): Discourse anaphoricity and first-person indexicality in emoji resolution. In Proceedings of Sinn und Bedeutung 25, 340-357. [https://doi.org/10.18148/sub/2021.v25i0.941]
- Kaiser, Elsi, and Patrick G. Grosz (2021): Anaphoricity in emoji: An experimental investigation of face and non-face emoji. Proceedings of the Linguistic Society of America 6, 1009-1023. [https://doi.org/10.3765/plsa.v6i1.5067]
H – K
Jonas Hartke
(PhD Candidate)
Project:
Lying, deceiving, misleading:
are we committed to our gestures?
University of Göttingen
jonasfelixhelmut.hartke@uni-goettingen.de
Jonas Hartke holds a B.A. in German Philology and Protestant Religion and a M.A. in German Philology, both from the Georg-August-University of Göttingen. His main research interests are the semantics-pragmatics interface, gestures and pragmatic phenomena like irony and lies. In his PhD thesis, he investigates whether it is possible to lie with gestures.
Dr. Alexander Henlein
(Postdoc Researcher)
Project:
Virtual Reality Sustained Multimodal Distributional Semantics for Gestures in Dialogue (GeMDiS)
Goethe University Frankfurt
henlein@em.uni-frankfurt.de
Alexander Henlein is a PostDoc at the Text Technology Lab (TTLab) of the Professorship for Computational Humanities / Text Technology of Prof. Dr. Alexander Mehler at the Goethe University Frankfurt. His doctoral research focused on the analysis of spatial semantics in language models, the extraction of object habitats from images, and the development of a VR-based Text2Scene system. Based on this work experience, he would like to develop VR-assisted communication systems within the scope of this project and use the data thus generated to create novel multimodal models.
Selected publications
- A. Henlein, A. Gopinath, N. Krishnaswamy, A. Mehler, J. Pustejovsky, “Grounding Human-Object Interaction to Affordance Behavior in Multimodal Datasets”, in Frontiers in Artificial Intelligence-Language and Computation, 2023 (accepted).
- A. Henlein and A. Mehler, “What do Toothbrushes do in the Kitchen? How Transformers Think our World is Structured,” in Proceedings of the 2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2022), 2022
- A. Henlein, G. Abrami, A. Kett, C. Spiekermann, and A. Mehler, “Digital Learning, Teaching and Collaboration in an Era of ubiquitous Quarantine,” in Remote Learning in Times of Pandemic – Issues, Implications and Best Practice, L. Daniela and A. Visvizin, Eds., Thames, Oxfordshire, England, UK: Routledge, 2021.
- A. Henlein, G. Abrami, A. Kett, and A. Mehler, “Transfer of ISOSpace into a 3D Environment for Annotations and Applications,” in Proceedings of the 16th Joint ACL – ISO Workshop on Interoperable Semantic Annotation, Marseille, 2020, pp. 32-35.
- A. Henlein and A. Mehler, “On the Influence of Coreference Resolution on Word Embeddings in Lexical-semantic Evaluation Tasks,” in Proceedings of The 12th Language Resources and Evaluation Conference, Marseille, France, 2020, pp. 27-33.
PD. Dr. Oliver Herbort
(Principal Investigator)
Project:
Representation of Pointing Uncertainty
for the Integration of Pointing Gestures and Speech
University of Würzburg
oliver.herbort@psychologie.uni-wuerzburg.de
Oliver Herbort is interested in human motor control, embodied choices, and the production and perception of pointing gestures. Specifically, he tries to understand the relationship between the body postures assumed during pointing, the objects or locations that are perceived to be indicated, and various situational factors. His methods include motion capture, virtual reality setups, and formal models. Oliver Herbort received his PhD from the University of Würzburg, was a post-doc in Würzburg and Tübingen, and is now an academic counsel at the University of Würzburg and PI of the VICOM project “representation of pointing uncertainty for the integration of pointing gestures and speech”.
Selected publications
- Herbort, O., Krause, L.-M., & Kunde, W. (2021). Perspective determines the production and interpretation of pointing gestures. Psychonomic Bulletin & Review, 28(2), 641-648. https://doi.org/10.3758/s13423-020-01823-7
- Herbort, O. & Kunde, W. (2016). Spatial (mis-)interpretation of pointing gestures to distal referents. Journal of Experimental Psychology: Human Perception and Performance, 42(1), 78-89. https://doi.org/10.1037/xhp0000126
- Herbort, O., & Kunde, W. (2018). How to point and to interpret pointing gestures? Instructions can reduce pointer-observer misunderstandings. Psychological Research, 82(2), 395-406. https://doi.org/10.1007/s00426-016-0824-8
- Krause, L.-M., & Herbort, O. (2021). The observer’s perspective determines which cues are used when interpreting pointing gestures. Journal of Experimental Psychology: Human Perception and Performance, 47(9), 1209-1225. https://doi.org/10.1037/xhp0000937
Prof. Dr. Annika Herrmann
(Principal Investigator)
University of Hamburg
annika.herrmann@uni-hamburg.de
Annika Herrmann is a professor for Sign Languages and Sign Language Interpreting at the Institute of German Sign Language and Communication of the Deaf (IDGS) at Universität Hamburg. Her research interests are theoretical and empirical sign language research, as well as experimental psycholinguistics. She received her PhD from University of Frankfurt am Main in 2010. After receiving her PhD, she was the Director of the Experimental Sign Language Lab at the Georg-August-University of Göttingen, has worked as a substitute professor for linguistics at the University of Cologne, and as a research assistant in an EEG-project at the University of Mainz. Annika Herrmann is co-director of the DGS-Korpus project and also co-editor of the series ‘Sign Languages and Deaf Communities’.
Selected publications
- Herrmann, A. & M. Steinbach. 2019. Expressive Gesten – expressive Bedeutungen. Expressivität in gebärdensprachlichen Erzählungen. In d’Avis, F. & R. Finkbeiner (eds.), Expressivität im Deutschen. Berlin: de Gruyter, 313–337.
- Herrmann, A. & N.-K. Pendzich. 2018. Between narrator and protagonist in fables of German Sign Language. In Hübl, A. & M. Steinbach (eds.), Linguistic Foundations of Narration in Spoken and Sign Languages. Amsterdam: Benjamins, 275–308.
- Herrmann, A. & N.-K. Pendzich. 2014. Nonmanual gestures in sign languages. In Müller, C., A. Cienki, E. Fricke, S. H. Ladewig, D. McNeill & J. Bressem (eds.), Handbook Body – Language – Communication. Berlin, Boston: De Gruyter Mouton, 2147–2160.
- Quer, J., R. Pfau & A. Herrmann. 2021. The Routledge Handbook of theoretical and experimental sign language research. London New York: Routledge, Taylor & Francis Group.
- Wienholz, A., D. Nuhbalaoglu, M. Steinbach, A. Herrmann & N. Mani. 2021. Phonological priming in German Sign Language: An eye tracking study using the visual world paradigm. Sign Language & Linguistics 24(1). 4– 35.
Prof. Dr. Stefan Hinterwimmer
(Principal Investigator)
Project:
Visual and Non-visual Means of
Perspective Taking in Language
University of Wuppertal
hinterwimmer@uni-wuppertal.de
Stefan Hinterwimmer’s research interests lie in the realms of semantics, pragmatics, the syntax-semantics interface, information structure and text linguistics. He has worked on adverbial quantification, free relative clauses, conditionals, topicality, specificity, pronoun resolution, the contrast between definite and demonstrative DPs and perspective taking. He received his PhD in 2006 from Humboldt-Universität zu Berlin. He is currently Akademischer Rat and Außerplanmäßiger Professor at the Department of German language and literature at the University of Wuppertal.
Selected publications
- Hinterwimmer, Stefan & Umesh Patil. 2022. The Interpretative options of anaphoric complex demonstratives. Glossa: a journal of general linguistics 7(1), 1–30.
- Hinterwimmer, Stefan, Umesh Patil & Cornelia Ebert. 2021. On the interaction of gestural and linguistic perspective taking. Frontiers in Communication 6. 625757.
- Hinterwimmer, Stefan. 2017. Two kinds of perspective taking in narrative texts. In D. Burgdorf, J. Collard, S. Maspong & B. Stefánsdóttir (eds.), Proceedings of Semantics and Linguistic Theory (SALT) 27, 282-301.
- Hinterwimmer, Stefan & David Schueler. 2015. Requantification, underquantification and partial focus in indefinites. Journal of Semantics 32, 749-797.
- Ebert, Christian, Cornelia Ebert & Stefan Hinterwimmer. 2014. A unified analysis of conditionals as topics. Linguistics and Philosophy 37(5). 353–408.
Šárka Kadavá
(PhD Candidate, PhD Contact Person)
Leibniz-Zentrum Allgemeine Sprachwissenschaft Berlin
kadava@leibniz-zas.de
Šárka Kadavá is a doctoral researcher in the DFG project On the FLExibility and Stability of gesture-speecH coordination (FLESH): Evidence from production, comprehension, and imitation. She is currently based in Leibniz-Zentrum Allgemeine Sprachwissenschaft (ZAS) in Berlin. Her research focuses on multimodality, on how different modalities contribute to the sharing of meaning and how they are coordinated with each other, and on how language-like structural features emerge in non-normative communication systems. She has also participated in a project developing Czech adaptation of MB-CDI at the Institute of Psychology, Czech Academy of Sciences.
Lisa-Marie Krause
(PhD Candidate)
Project:
Representation of Pointing Uncertainty
for the Integration of Pointing Gestures and Speech
University of Würzburg
lisa-marie.krause@uni-wuerzburg.de
Lisa-Marie started her PhD in 2019 at the University of Würzburg. In her research, she is mainly interested in the spatial interpretation of pointing gestures, especially how different factors, e.g., perspective or arm posture, are affecting this interpretation. Within ViCom, she will extend this focus by gesture-speech integration and pointing production. She aims to examine how spatial information that are expected to be communicated by pointers and the spatial information observers expected to be implied by pointing gestures match. Methodically, Lisa-Marie conducts her experiments using eye tracking in virtual reality as well as in real-life dyadic interactions.
Selected Publications
- Krause, L.-M., & Herbort, O. (2021). The observer’s perspective determines which cues are used when interpreting pointing gestures. Journal of Experimental Psychology: Human Perception and Performance, 47(9), 1209-1225. http://dx.doi.org/10.1037/xhp0000937
- Herbort, O., Krause, L.-M., & Kunde, W. (2021). Perspective determines the production and interpretation of pointing gestures. Psychonomic Bulletin & Review, 28(2), 641-648. https://doi.org/10.3758/s13423-020-01823-7
Dr. Anna Kuder
(Postdoc Researcher)
University of Cologne
akuder@uni-koeln.de
Anna Kuder is interested in nonmanual elements of sign languages and in the use of discourse particles across different language modalities. She obtained her PhD in 2020 at the University of Warsaw, after defending her thesis titled “Negation markers in Polish Sign Language (PJM)”. While doing her PhD she served as an Investigator or Principal Investigator in numerous projects (both research and educational) concerning PJM. She was involved in all stages of building the PJM Corpus and participated in the project of creating the first on-line Corpus-based Dictionary of PJM. As a postdoc, she is currently working at the University of Cologne, where she is conducting a comparative corpus-based study of gestural elements in Polish Sign Language, German Sign Language, and Russian Sign Language. She is also currently joining the ViCom project “Gestures or signs? Comparing manual and non-manual constructions sharing the same form in co-speech gesture and sign language: a corpus-driven approach. (GeSi)” as a postdoc.
Selected publications
- S. Gabarró López & A. Kuder (2022), A corpus-based study of the “Away gestures” across four signed languages: DGS, LSC, LSFB and PJM, Belgian Journal of Linguistics, 36, in print.
- A. Kuder (2022), Making Sign Language Corpora Comparable: A study of Palm-Up and Throw-Away in Polish Sign Language, German Sign Language, and Russian Sign Language. In: Workshop Proceedings. 10th Workshop on the Representation and Processing of Sign Languages: Multilingual Sign Language Resources. Language Resources and Evaluation Conference} (LREC), Marseille, France, 25 June 2022, ELRA, 110–117.
- A. Kuder, J. Wójcicka, P. Mostowski & P. Rutkowski (2022), Open Repository of the Polish Sign Language Corpus: Publication Project of the Polish Sign Language Corpus. In: Workshop Proceedings. 10th Workshop on the Representation and Processing of Sign Languages: Multilingual Sign Language Resources. Language Resources and Evaluation Conference (LREC), Marseille, France, 25 June 2022, ELRA, 118–123.
- A. Kuder (2021), Negation markers in Polish Sign Language (PJM), Sign Language and Linguistics, 24(1), pp. 118-131.
- A. Kuder, J. Filipczak, P. Mostowski, P. Rutkowski & T. Johnston (2018), What Corpus-based Research on Negation in Auslan and PJM Tells Us about Building and Using Sign Language Corpora. In: M. Bono et al. (eds.), Workshop Proceedings. 8th Workshop on the Representation and Processing of Sign Languages: Involving the Language Community. Language Resources and Evaluation Conference (LREC), Miyazaki, Japan, 12 May 2018, ELRA, pp. 101-106.
Prof. Dr. Frank Kügler
(Principal Investigator)
Project:
Co-speech Gestures and Prosody as
Multimodal Markers of Information Structure
Goethe University Frankfurt
kuegler@em.uni-frankfurt.de
Frank Kügler is Professor of Linguistics at Goethe University Frankfurt. His research interests are in cross-linguistic prosody from various different perspectives such as the expression and modelling of information structure, prominence and sentence mode in typologically unrelated languages, the interaction of tone and intonation, and prosodic phrasing and recursive prosodic constituents at the prosody-syntax interface. His recent interests include also the gestural marking of prosody. He has worked on the prosody of a number of typologically diverse languages including Mandarin, Hindi, Akan, Tswana, Ghanaian English, Yucatec Maya, Hungarian, and German among others.
Selected publications
- Grice, Martine & Kügler, Frank (2021) Prosodic Prominence – A Cross-Linguistic Perspective. Language and Speech, 64, 2, 253–260. https://doi.org/10.1177/00238309211015768
- Kügler, Frank (2016) Tone and intonation in Akan. In: Downing, Laura J. & Rialland, Annie (eds) Intonation in African Tone Languages. Berlin: Mouton de Gruyter, 89-129.
- Kügler, Frank, Baumann, Stefan & Röhr, Christine T. (2022) Deutsche Intonation, Modellierung und Annotation (DIMA) – Richtlinien zur prosodischen Annotation des Deutschen. In Schwarze, Cordula & Grawunder, Sven (eds.) Transkription und Annotation gesprochener Sprache und multimodaler Interaktion: Konzepte, Probleme, Lösungen. Tübingen: Narr, 23-54.
- Kügler, Frank & Sasha Calhoun. 2020. Prosodic Encoding of Information Structure: A typological perspective. In Carlos Gussenhoven & Aoju Chen (eds.), The Oxford Handbook of Language Prosody, 453–467. Oxford: OUP.
- Kügler, Frank & Caroline Féry. 2017. Postfocal downstep in German. Language and Speech 60(2). 260–288.
L – M
Dr. Silva Ladewig
(Principal Investigator)
Project:
Processes of Stabilization in Gestures.
A Media-specific and Cross-modal Approach
University of Göttingen
silva.ladewig@uni-goettingen.de
Silva Ladewig is interested in the embodied nature of language. For this she studies multimodality from a linguistic perspective in the fields of Pragmatics, Cognitive Grammar, Cognitive Semantics and Dynamic Multimodal Communication. Of special interest are stabilization processes in gestures and the interface between gesture and sign which she explores now at the Sign Language Lab at the University of Göttingen. Silva received her Ph.D. in Linguistics at the European University Viadrina in 2012 where she has worked in many interdisciplinary projects on the integration of gesture and speech, the linguistic potential of gestures, and the dynamics of multimodal metaphors.
Selected publications
- Harrison, Simon, Silva H. Ladewig & Jana Bressem (eds.) (2021). The diversity of recurrency: Recurrent gesture cross-linguistically, Gesture 20:2 (Special Issue).
- Harrison, Simon & Silva H. Ladewig (2021). Recurrent gestures throughout bodies, languages, and cultural practices. In: Gesture 20:2, 153–179.
- Ladewig, Silva H. (to appear). Recurrent gestures. In: Alan Cienki (Hg.), The Cambridge Handbook of Gesture Studies. Cambridge.
- Ladewig, Silva H. & Lena Hotze (2021). The Slapping movement as an embodied practice of dislike: Inter-affectivity in interactions among children. In: Gesture 20:2, 285–312.
- Ladewig, Silva H. (2020). Integrating Gestures. The Dimension of Multimodality in Cognitive Grammar. Berlin/ Amsterdam: De Gruyter Mouton
PD Dr. Jens Lemanski
(Principal Investigator)
Project:
Gestures and Diagrams in Visual-Spatial Communication: Methodological Tools and Applications in Mathematics and Logic
University of Münster / University of Hagen
jenslemanski@uni-muenster.de
PD Dr. Jens Lemanski is currently PI of the “History of Logic Diagrams in Kantianism” project at the University in Muenster (Thyssen-Stiftung) and Privatdozent for Philosophy at the FernUniversität in Hagen, Germany. He holds a cotutela-Ph.D in philosophy from the JGU Mainz and the Università del Salento (Lecce). He has been a research fellow at the WWU Muenster and the RU Bochum and he has published on the history and philosophy of logic, science, metaphysics, and the foundations of mathematics.
Selected publications
- J. Lemanski: World and Logic. London: College Publications, 2021.
- J. Lemanski (ed.): Language, Logic, and Mathematics in Schopenhauer. Cham: Birkhäuser, 2020.
- J. Lemanski, L. Demey: Schopenhauer’s Partition Diagrams and Logical Geometry. In A. Basu, G. Stapleton, S. Linker, C. Legg, E. Manalo & P. Viana (eds.), Diagrams 2021: Diagrammatic Representation and Inference (Lecture Notes in Artificial Intelligence 12909). pp. 149-165. 2021.
- J. Lemanski: Euler-type Diagrams and the Quantification of the Predicate, in: Journal of Philosophical Logic 49 (2): 401-416. 2020.
- J. Lemanski: Logic Diagrams, Sacred Geometry and Neural Networks, in: Logica Universalis 13 (4): 495-513. 2019.
Prof. Dr. Katja Liebal
(Principal Investigator)
Project:
Compositional structures
in chimpanzee gestural communication
University of Leipzig
katja.liebal@uni-leipzig.de
Katja Liebal obtained her PhD at the Max Planck Institute for Evolutionary Anthropology in Leipzig. She was Professor of Comparative Developmental Psychology at the Freie Universität Berlin, and is currently leading the group Human Biology and Primate Cognition at the University of Leipzig. Her main research interests include the communicative, emotional and social-cognitive abilities of non-human primates and human children from different cultural contexts, with a special focus on how developmental trajectories vary between and within humans and apes. For her research, she uses a cross-species, cross-cultural approach, combining observational and non-invasive experimental methods.
Selected publications
- Amici, F., Lembeck, M., Holodynski, M., & Liebal, K. 2022. Face to face interactions in chimpanzee and human mother-infant-dyads. Philosophical Transactions B.
- Amici, F., Oña, L., & Liebal, K. 2022. Compositionality in primate gestural communication and multicomponent signal displays. International Journal of Primatology. https://doi.org/10.1007/s10764-022-00316-9.
- Arbib, MA, Liebal, K, & Pika, S. 2008. Primate vocalization, gesture, and the evolution of human language. Current Anthropology, 49, 1053-1076.
- Liebal, K, Waller, BM, Burrows, AM, & Slocombe, KE. 2013. Primate communication: a multimodal approach. Cambridge University Press.
- Oña, L. S., Sandler, W., & Liebal, K. (2019). A stepping stone to compositionality in chimpanzee communication. PeerJ, 7, e7623. doi:10.7717/peerj.7623
Dr. Cornelia Loos
(Principal Investigator)
Project:
Exploring the Limits of Simultaneity: Encoding Caused Change-of-state Events with Classifier Constructions
in German Sign Language (DGS)
University of Hamburg
cornelia.loos@uni-hamburg.de
Cornelia Loos received her Ph.D. in Linguistics from the University of Texas at Austin in 2017. After completing a two-year postdoc project on aspects of the syntax and semantics of German Sign Language (DGS) in Göttingen as well as a second post-doc at the university of Hamburg working in the DGS-corpus project, she is currently a visiting professor at the Institute of German Sign Language and Communication of the Deaf in Hamburg. Her research interests cluster around the syntax-semantics interface, focusing on lexical semantics as well as experimental semantics and pragmatics. She works predominantly on signed languages and has investigated topics such as word class and sentencehood in American and German Sign Language (DGS), the syntax and semantics of resultative constructions in these two languages, the influence of iconicity on the semantics of taboo language in DGS, and, most recently, on response elements and NPIs in DGS.
Selected publications
- Loos, C. 2022. Sizing up adjectives: Delimiting the adjective class in American Sign Language. Sign Language & Linguistics.
- Loos, C. and D.J. Napoli. 2021. Expanding Echo: Coordinated Head Articulations as Nonmanual Enhancements in Sign Language Phonology. Cognitive Science 45(5). doi.org/10.1111/cogs.12958.
- Loos, C. 2020. Quite a mouthful: Comparing speech act verbs in Nederlandse Gebarentaal and American Sign Language. Linguistische Berichte.
- Loos, C., M. Steinbach and S. Repp. 2020. Affirming and rejecting assertions in German Sign Language. Sinn und Bedeutung 24 Proceedings.
- Loos, C., Cramer, J.M., and D.J. Napoli. 2019. Taboo terms in German Sign Language: Exploiting phonology. Cognitive Linguistics. Advance online publication. https://doi.org/10.1515/cog-2018-0077
- Loos, C., German, A., & Meier, R. P. (2022). Simultaneous structures in sign languages: Acquisition and emergence. Frontiers in Psychology, 7232.https://doi.org/10.3389/fpsyg.2022.992589
Dr. Andy Lücking
(Principal Investigator)
Project:
Virtual Reality Sustained Multimodal Distributional Semantics for Gestures in Dialogue (GeMDiS)
Goethe University Frankfurt
luecking@em.uni-frankfurt.de
Andy Lücking’s research contributes to a linguistic theory of human communication, that is, face-to-face interaction beyond single sentences. This involves the adaptation of dynamic dialogue semantics, the development of multimodal grammar extensions, occasionally the revision of traditional linguistic theories (e.g., quantification or pointing), the use of corpora and computational methods (as in the ViCom project GeMDiS), and taking an overarching cognitive perspective. Andy Lücking received a PhD in linguistics (Dr. phil.) in 2011 at Bielefeld University on iconicity and iconic gestures. He defended his habilitation in 2022 on “Aspects of multimodal communication” at the Laboratoire de Linguistique Formelle (LLF) at the Université Paris Cité.
Selected publications
- On the broader picture of multimodal communication:
Andy Lücking and Jonathan Ginzburg. “Leading voices: Dialogue semantics, cognitive science, and the polyphonic structure of multimodal interaction”. Language and Cognition , Volume 15 , Issue 1 , January 2023 , pp. 148–172, DOI: https://doi.org/10.1017/langcog.2022.30 - A dialogue- and gesture-friendly theory of quantification:
Andy Lücking and Jonathan Ginzburg. “Referential transparency as the proper treatment of quantification”. In: Semantics and Pragmatics 15, 4 (2022). doi: 10.3765/sp.15.4. (Early access: https://semprag.org/index.php/sp/article/view/sp.15.4) - A condensed, computational piece on how to assign meanings to (some kinds of) co-verbal gestures and integrate them in grammar:
Andy Lücking. “Modeling Co-verbal Gesture Perception in Type Theory with Records”. In: Proceedings of the 2016 Federated Conference on Computer Science and Information Systems. Hrsg. von Maria Ganzha, Leszek Maciaszek und Marcin Paprzycki. Vol. 8. Annals of Computer Science and Information Systems. IEEE, Sep. 2016, pp. 383–392. doi: 10.15439/2016F83. (Available at: https://annals-csis.org/proceedings/2016/drp/83.html) - Pointing gestures as search instructions:
Andy Lücking. “Witness-loaded and Witness-free Demonstratives”. In: Atypical Demonstratives. Syntax, Semantics and Pragmatics. Hrsg. von Marco Coniglio, Andrew Murphy, Eva Schlachter und Tonjes Veenstra. Linguistische Arbeiten 568. Berlin und Boston: De Gruyter, 2018, pp. 255–284. ISBN: 978-3-11-056029-9.(Preprint: https://www.researchgate.net/publication/303667514_Witness-loaded_and_Witness-free_Demonstratives)
Prof. Dr. Carina Lüke
(Principal Investigator)
Project:
Modality-Specific Effects on Language Processing
in Children with Developmental Language Disorder
University of Würzburg
carina.lueke@uni-wuerzburg.de
Carina Lüke’s research interests are in the area of communication and language acquisition in mono- and multilingual children, the identification and intervention in children with Developmental Language Disorders (DLD) and especially, the multimodal use of gestures and speech in typically an atypically developing children. She received her education in Rehabilitation Sciences (TU Dortmund University) and Clinical Linguistics (Bielefeld University), completed her Ph.D. 2015 at TU Dortmund University on the predictive value of early pointing gestures for later language skills in children. Since 2020 she is a full professor for Special Education and Therapy in Language and Communication Disorders at the University of Würzburg, were she also heads the Lab for Communication and Language (FoKuS). Within ViCom she is one of the PI of the project Modality-Specific Effects on Language Processing in Children with Developmental Language Disorder.
Selected publications
- Lüke, C., Liszkowski, U., & Ritterfeld, U. (2022). In bilinguals’ hands: Identification of bilingual, preverbal infants at risk for language delay. Frontiers in Pediatrics, 10:878163.
- Lüke, C., Ritterfeld, U., Grimminger, A., Rohlfing, K. J. & Liszkowski, U. (2020). Integrated communication system: Gesture and language acquisition in typically developing children and children with LD and DLD. Frontiers in Psychology, 11:118.
- Lüke, C., Ritterfeld, U., Grimminger, A., Liszkowski, U., & Rohlfing, K. J. (2017). Development of pointing gestures in children with typical and delayed language acquisition. Journal of Speech, Language, and Hearing Research, 60, 3185-3197.
- Lüke, C., Grimminger, A., Rohlfing, K. J., Liszkowski, U. & Ritterfeld, U. (2017). In infants’ hands: Identification of preverbal infants at risk for primary language delay. Child Development, 88, 484-492.
- Lüke, C. & Ritterfeld, U. (2014). The influence of iconic and arbitrary gestures on novel word learning in children with and without SLI. Gesture, 14, 204-225.
Prof. Dr. Alexander Mehler
(Principal Investigator)
Project:
Virtual Reality Sustained Multimodal Distributional Semantics for Gestures in Dialogue (GeMDiS)
Goethe University Frankfurt
mehler@em.uni-frankfurt.de
Alexander Mehler’s research interests include automatic analysis and synthesis of language and multimodal data in spoken and written communication. To this end, he studies multimodal and multiplex networks derived from social and communication networks using models of language evolution, machine learning, and complex network theory. He is particularly interested in measurement models that help overcome end-to-end learning and its issues. This involves models of multimodal semantics that focus on the explicit modelling of sign structures. Alexander Mehler develops and tests quantitative methods and machine learning models that merge with virtual and augmented reality. The goal of this research is to ground multimodal semantics based on human behaviour in virtual worlds.
Selected publications
- Mehler, A., Gleim, R., Gaitsch, R., Uslu, T. & Hemati, W. (2020, online). From Topic Networks to Distributed Cognitive Maps: Zipfian Topic Universes in the Area of Volunteered Geographic Information. Complexity, vol. 4, pp. 1-47. DOI: 10.1155/2020/4607025
- Mehler, A., Hemati, W., Welke, P., Konca, M. & Uslu, T. (2020, online). Multiple Texts as a Limiting Factor in Online Learning: Quantifying Dissimilarities of Knowledge Networks across Languages. Frontiers in Education (Digital Education), pp. 1-31. DOI: 10.3389/feduc.2020.562670
- Mehler A., Gleim, R., Lücking, A., Uslu, T., & Stegbauer, Ch. (2018). „On the Self-similarity of Wikipedia Talks: a Combined Discourse-analytical and Quantitative Approach“. In: Glottometrics 40, pp. 1–45
- Mehler, A., Lücking. A., & Abrami, G. (2014). „WikiNect: Image Schemata as a Basis of Gestural Writing for Kinetic Museum Wikis“. In: Universal Access in the Information Society, pp. 1–17. DOI: 10.1007/s10209-014-0386-8
- Mehler, A., Lücking, A. & Menke, P. (2012). „Assessing Cognitive Alignment in Different Types of Dialog by means of a Network Model“. In: Neural Networks 32, S. 159–164. DOI: 10.1016/j.neunet.2012.02.013
Dr. Nina-Kristin Meister
(Principal Investigator)
Project:
Parts of Speech and Iconicity in
German Sign Language (DGS)
University of Göttingen
ninakristin.meister@uni-goettingen.de
Nina-Kristin Meister (née Pendzich) received her PhD in German Linguistics from the Georg-August-University of Göttingen with a dissertation entitled “Lexical Nonmanuals in German Sign Language (DGS). An Empirical and Theoretical Investigation” (2018: Wilhelm von Humboldt-Preis German Linguistic Society (DGfS), 2017: Christian-Gottlob-Heyne-Preis Göttingen Graduate School of Humanities). Since 2017 she is the director of the Sign Language Lab at the Department of German Philology at the University of Göttingen and is involved in various research projects. Her research interests are in the field of theoretical and experimental sign language linguistics. She investigates the interfaces between gesture, emotion, and sign language, grammatical and lexical nonmanuals, narrative structures, various sentence types such as conditional sentences, and iconicity in sign languages. She has applied a variety of empirical methodologies and works with the Facial Action Coding System (FACS, Ekman et al. 2002, certified FACS-Coder). She is editing the first bimodal-bilingual book series in German Sign Language (DGS) and German ‘Deutsche Gebärdensprache und Deaf Communities’ (Buske Verlag, with Thomas Finkbeiner) and the journal ‘Linguistische Berichte’ (Buske Verlag, with Markus Steinbach).
Selected publications
- Nina-Kristin Pendzich, Jens-Michael Cramer, Thomas Finkbeiner, Annika Herrmann and Markus Steinbach (2022): How do Signers Mark Conditionals in German Sign Language? Insights from a Sentence Reproduction Task on the Use of Nonmanual and Manual Markers. In: Croatian Review of Rehabilitation Research 58 (Special Issue: Sign Language, Deaf Culture, and Bilingual Education), 206-226.
- Patrick C. Trettenbrein, Nina-Kristin Pendzich, Jens-Michael Cramer, Markus Steinbach and Emiliano Zaccarella (2021): Psycholinguistic Norms for more than 300 Lexical Signs in German Sign Language. In: Behavior Research Methods 53, 1817-1832. https://doi.org/10.3758/s13428-020-01524-y
- Nina-Kristin Pendzich (2020): Lexical Nonmanuals in German Sign Language (DGS). Empirical Studies and Theoretical Implications. Berlin: Mouton de Gruyter.
- Annika Herrmann and Nina-Kristin Pendzich (2018): Between Narrator and Protagonist in Fables of German Sign Language. In: Annika Hübl and Markus Steinbach (eds.), Linguistic Foundations of Narration in Spoken and Sign Languages. Amsterdam: Benjamins, 275-308.
- Annika Herrmann and Nina-Kristin Pendzich (2014): Nonmanual Gestures in Sign Languages. In: Cornelia Müller, Alan Cienki, Ellen Fricke, Silva H. Ladewig, David McNeill and Jana Bressem (eds.), Body – Language – Communication: An International Handbook on Multimodality in Human Interaction. Volume 2. Berlin: de Gruyter Mouton, 2149-2162.
O – R
Dr. Linda Oña
(Principal Investigator)
Project:
Compositional structures
in chimpanzee gestural communication
MPI for Human Development Berlin
ona@mpib-berlin.mpg.de
Linda obtained her PhD in biology at the Max Planck Institute for Evolutionary Anthropology and Freie Universität Berlin. Currently, she holds a Post Doc position at the Max Planck Institute for Human Development in Berlin. Linda’s research interests is social cognition, with a focus on communication, in various species, including humans, dogs and non-human primates. Her research methods are based on a comparative approach using experimental and observational study designs.
Selected publications
- Oña LS, Sandler W, & Liebal K (2019). A stepping stone to compositionality in chimpanzee communication. PeerJ, 7, e7623.
- Amici F, Oña LS & Liebal K (2022). Compositionality in Primate Gestural Communication and Multicomponent Signal Displays. Int J Primatol. https://doi.org/10.1007/s10764-022-00316-9.
- Bohn M, Liebal K, Oña LS, Tessler MH (2022). Great ape communication as contextual social inference: a computational modelling perspective. Phil. Trans. R. Soc. B 377: 20210096. https://doi.org/10.1098/rstb.2021.0096.
- Liebal K, Oña LS (2018) Mind the gap – intentional gestures and emotional facial expressions of nonhuman primates? Interaction Studies, 19:1/2, pp. 121–135.
- Liebal K, Oña LS (2018) Different Approaches to Meaning in Primate Gestural and Vocal Communication. Front. Psychol. 9:478. doi: 10.3389/fpsyg.2018.00478.
Prof. Dr. Pamela Perniss
(Principal Investigator)
Project:
The Gesture-to-Sign Trajectory: Phonological Parameters
in Production and Real-Time Comprehension
University of Cologne
pperniss@uni-koeln.de
Pamela Perniss’s research takes a multimodal approach to language and has focused in particular on the role of the visual modality and iconicity in shaping language structure and processing. Following her PhD at the Max Planck Institute for Psycholinguistics (2007), she held postdoctoral positions at the Deafness, Cognition and Language (DCAL) Research Centre (University College London) and at Radboud University Nijmegen before joining the Linguistics Department at the University of Brighton. Since 2019, she is a professor in the Faculty of Human Sciences at the University of Cologne and chair of the sign language interpreting (DGS-German) program.
Selected publications
- Perniss, P., Vinson, D., & Vigliocco, G. (2020). Making sense of the hands and mouth: the role of “secondary” cues to meaning in British Sign Language and English. Cognitive Science 44(7), e12868.
- Perniss, P., Lu, J., Morgan, G., & Vigliocco, G. (2017). Mapping language to the world: The role of iconicity in the sign language input. Developmental Science, e12551, 1-23.
- Perniss, P., Özyürek, A., & Morgan, G. (2015). The influence of the visual modality on language structure and language conventionalization: Insights from sign language and gesture. Topics in Cognitive Science,7(1), 2-11.
- Perniss, P., Thompson, R.L., & Vigliocco, G. (2010). Iconicity as a General Property of Language: Evidence from Spoken and Signed Languages. Frontiers in Psychology 1, 227.
Dr. Wim Pouw
(ViCom Data Network Coordinator, PI)
Donders Institute &
MPI for Psycholinguistics Nijmegen
w.pouw@donders.ru.nl
(A short introduction and selected publications are to be updated.)
Prof. Dr. Pilar Prieto
(Principal Investigator)
Project:
Co-speech Gestures and Prosody as
Multimodal Markers of Information Structure
Universitat Pompeu Fabra (UPF), Barcelona
pilar.prieto@upf.edu
Pilar Prieto is an ICREA Research Professor at the Department of Translation and Language Sciences at Universitat Pompeu Fabra, Barcelona, Catalunya. Her research focuses on the communicative role of prosody and gesture in language, as well as their significance in language development and second language learning. In the last years, she has worked on the multimodal marking of Information Structure, in both adult and children’s discourse. She currently serves as associate editor of the journals Language and Speech and Frontiers in Communication, and is coediting a special issue of the “Language and Cognition” journal on Multimodal Prosody.
Selected publications
- Brown, Lucien & Pilar Prieto. 2021. Gesture and prosody in multimodal communication. In Michael Haugh, Dániel Z. Kádár & Marina Terkourafi (eds.), The Cambridge Handbook of Sociopragmatics, 430–453. Cambridge: CUP.
- Esteve-Gibert, Núria & Pilar Prieto. 2013. Prosodic structure shapes the temporal realization of intonation and manual gesture movements. Journal of Speech, Language, and Hearing Research 56(3). 850-864.
- Prieto, Pilar, C. Puglesi, J. Borràs-Comes, E. Arroyo & J. Blat. 2015. Exploring the contribution of prosody and gesture to the perception of focus using an animated agent. Journal of Phonetics 49(1). 41-54.
- Rohrer, P., Florit-Pons, M., Vilà-Giménez, I., Prieto, P. (2022). “Children use non-referential gestures in narrative speech to mark discourse elements which update common ground.” Frontiers in Psychology 12:661339. doi: 10.3389/fpsyg.2021.661339
- Vanrell, M.M., Stella, A., Gili-Fivela, B., & Prieto, P. (2013). Prosodic manifestations of the Effort Code in Catalan, Italian, and Spanish contrastive focus. Journal of the International Phonetics Association 43(2), pp.195-220.
Dr. Andrea Reichenberger
(Principal Investigator)
Project:
Gestures and Diagrams in Visual-Spatial Communication: Methodological Tools and Applications in Mathematics and Logic
University of Siegen
andrea.reichenberger@uni-siegen.de
Andrea Reichenberger is currently Research Group Leader at the Department of Mathematics, University of Siegen. Her research activities focus on women’s contributions to logic, mathematics and computer science. Previously, she worked in various research projects at German universities, among them at the University of Hagen, at the Center for the History of Women Philosophers and Scientists at Paderborn University and in the DFG research project “Thought Experiment, Metaphor, Model” at the Institute for Philosophy I at the Ruhr University Bochum. Her doctoral dissertation about Émilie du Châtelet was published by Springer in 2016.
Selected publications
- Anger, Claudia, Theodor Berwe, Alfred Olszok, Andrea Reichenberger, Jens Lemanski. 2022. “Five dogmas of logic diagrams and how to escape them.” Language & Communication 87 (1): 258-270.
- Reichenberger, Andrea. 2023. “Gender Equality and Diversity. Challenges and Perspectives for the Historiography of Science.” In: M. L. Condé and M. Salomon, eds. Handbook of the Historiography of Science. Cham: Springer (forthcoming)
- Reichenberger, Andrea. 2022. “Rózsa Péter on the Philosophy and Foundations of Mathematics: A Reappraisal.” In: J. Peijnenburg and S. Verhaegh, eds.: Women in the History of Analytic Philosophy. Cham: Springer. (forthcoming)
- Reichenberger, Andrea. 2021 “Émilie Du Châtelet on Space and Time.” In: A-L. Rey, ed.: L’épistémologie et à la philosophie des sciences d’Emilie du Châtelet. Revue d’histoire des sciences 74(2), 331–355.
- Reichenberger, Andrea. 2018 “How to Teach History of Philosophy and Science: A Digital Based Case Study.” In: R. Pisano, ed.: Methods and Cognitive Modelling in the History and Philosophy of Science–&–Education. Special Issue on HPS–&–Education in Transversal. International Journal for the Historiography of Science 5, 84–99.
S
Paula G. Sánchez Ramón
(PhD Candidate)
Project:
Co-speech Gestures and Prosody as
Multimodal Markers of Information Structure
Universitat Pompeu Fabra (UPF), Barcelona
paulaginesa.sanchez@upf.edu
Paula G. Sánchez-Ramón is a first-year PhD student developing her research under an international joint cotutelle between Universitat Pompeu Fabra (Barcelona, Catalonia) and Goethe Universität (Frankfurt am Main, Germany). She holds a B.A. in Primary Education and Teaching (Universitat Jaume I) with a specialization in Teaching English as a Foreign Language (Universidad Internacional de La Rioja), and an M.A. in Learning Difficulties and Language Disorders with a concentration in Speech and Language (Universitat Oberta de Catalunya). As a member of the project MultIS, her research focuses on the prosodic and gestural marking of information structure in Catalan-speaking adults.
Dr. Simone Schäffner
(Principal Investigator)
Project:
Modality-Specific Effects on Language Processing
in Children with Developmental Language Disorder
University of Würzburg
simone.schaeffner@uni-wuerzburg.de
Simone Schäffner is interested in the multimodal aspects of language. Her work is focused on modality switching and the role of input-output modality compatibility while language processing. She received her PhD in Cognitive and Experimental Psychology from the RWTH Aachen University in 2018 with a dissertation entitled Modality-Specific Effects in Linguistic Multitasking. Afterwards she spent two years as a postdoc at the Cognition & Development Lab at the University of Koblenz-Landau. Since January 2021, she is working at the University of Würzburg at the department of Special Education and Therapy in Language and Communication Disorders. She is currently one of the Principal Investigators of the project Modality-Specific Effects on Language Processing in Children with Developmental Language Disorder.
Selected publications
- Schaeffner, S., Koch, I., & Philipp, A.M. (2018). Sensory-motor modality compatibility in multitasking: The influence of processing codes. Acta Psychologica, 191, 210-218. https://doi.org/10.1016/j.actpsy.2018.09.012
- Schaeffner, S., Koch, I., & Philipp, A.M. (2018). The role of learning in sensory-motor modality switching. Psychological Research, 82, 955–969. https://doi.org/10.1007/s00426-017-0872-8
- Schaeffner, S., Fibla, L., & Philipp, A.M. (2017). Bimodal language switching: New insights from signing and typing. Journal of Memory and Language, 94, 1-14. https://doi.org/10.1016/j.jml.2016.11.002
- Schaeffner, S., Koch, I., & Philipp, A.M. (2016). Semantic effects on sensory-motor modality switching. Journal of Cognitive Psychology, 28, 726-742. https://doi.org/10.1080/20445911.2016.1181636
- Schaeffner, S., Koch, I., & Philipp, A.M. (2016). The role of sensory-motor modality compatibility in language processing. Psychological Research, 80, 212-223. https://doi.org/10.1007/s00426-015-0661-1
Jun.-Prof. Dr. Tatjana Scheffler
(Principal Investigator)
Project:
Semantics and Pragmatics of Emojis in
Digital Communication (EmDiCom)
University of Bochum
tatjana.scheffler@rub.de
Tatjana Scheffler analyses communication practices in digital media, using corpus linguistic and computational methods. Current research topics include both theoretical analyses of phenomena in informal digital language (such as intensifiers, question tags, or emojis), as well as applied issues like the detection of hate speech and disinformation. She is a PI on several externally funded research projects on topics covering the variability of language in social media, the computational analysis of metaphors in online forums, disinformation detection, and the semantics and pragmatics of emojis. Tatjana received her PhD in 2008 from the University of Pennsylvania (USA), before working at the German Research Center for Artificial Intelligence (DFKI) in Berlin, as well as at the universities of Potsdam and Konstanz. In 2020, Tatjana Scheffler was appointed Assistant Professor (TT) of Digital Forensic Linguistics at Ruhr-Universität Bochum.
Selected publications
- Tatjana Scheffler, Hannah Seemann, Lesley-Ann Kern. The medium is not the message: Individual level register variation in blogs vs. tweets. Register Studies, Special Issue: Register and Social Media. 2022. https://doi.org/10.1075/rs.22009.sch
- Tatjana Scheffler, Lasse Brandt, Marie de la Fuente, and Ivan Nenchev. The processing of emoji-word substitutions: A self-paced-reading study. Computers and Human Behavior 127. 2022. doi: https://doi.org/10.1016/j.chb.2021.107076 Repository location: https://osf.io/d34y5/
- Tatjana Scheffler, Veronika Solopova, and Mihaela Popa-Wyatt. The Telegram chronicles of online harm. Journal of Open Humanities Data, 7, p.8. 2021. doi: http://doi.org/10.5334/johd.31
- Tatjana Scheffler. Conversations on Twitter. In: D. Fišer/M. Beißwenger (eds.), Investigating Computer-Mediated Communication: Corpus-Based Approaches To Language In The Digital World, Ljubljana: University Press. ISBN 978-961-237-950-6. 2017
- Scheffler, Tatjana. Two-dimensional Semantics. Clausal Adjuncts and Complements. Berlin, Boston: De Gruyter Mouton, 2013.
Dr. Martin Schulte-Rüther
(ViCom Data Network Coordinator, PI)
Project:
Multimodal Assessment of Dyadic Interaction
in Disorders of Social Interaction
University Medical Center Göttingen
martin.schulte-ruether@med.uni-goettingen.de
Martin’s research is focused on social interaction and emotional processing in typical development as well as in psychiatric and neurodevelopmental disorders, in particular Autism. He employs a broad spectrum of neuroscientific and behavioral methods, including MRI, fNIRS, physiological recordings, eye-tracking, and video-based behavioral analysis. Martin studied Psychology at the Ruhr-University Bochum and received his PhD at the University of Bielefeld working in cooperation with the research Center Jülich on the cognitive neuroscience of empathy. He continued as a Post-Doc at the Child and Adolescent Psychiatry of the RWTH Aachen, focusing on neuroimaging methods in children and adolescents with autism. Next, he headed a research group for Translational Brain Research at the RWTH Aachen, Jülich-Aachen Research Alliance. Currently, he is based at the University Medical Center Göttingen, and is a senior researcher and group leader of the Social Interaction and Developmental Neuroscience Lab at the Child and Adolescent Psychiatry Göttingen.
Selected publications
- Schulte-Rüther M, Kulvicius T, Stroth S, Wolff N, Roessner V, Marschik PB, Kamp-Becker I, Poustka L (2022). Using machine learning to improve diagnostic assessment of ASD in the light of specific differential and co-occurring diagnoses. Journal of Child Psychology and Psychiatry. https://doi.org/10.1111/jcpp.13650
- Hartz A, Guth B, Jording M, Vogeley K, Schulte-Rüther M (2021). Temporal Behavioral Parameters of on-going Gaze Encounters in a Virtual Environment. Frontiers in Psychology, doi:10.3389/fpsyg.2021.673982
- Kruppa J, Reindl V, Gerloff C, Oberwelland E, Prinz J, Herpertz-Dahlmann B, Konrad K, Schulte-Rüther M. (2020). Brain-to-Brain synchrony in children and adolescents with autism spectrum disorder in parent-child-dyads. Social Cognitive and Affective Neuroscience, doi:10.1093/scan/nsaa092
- Kruppa JA, Gossen A, Großheinrich N, Oberwelland E, Cholemkery H, Freitag C, Kohls G, Fink GR, Herpertz-Dahlmann B, Konrad K, Schulte-Rüther M (2019). Social Reinforcement Learning and its Neural Modulation by Oxytocin in Autism Spectrum Disorder. Neuropsychopharmacology, 44:749-756. doi:10.1038/s41386-018-0258-7
- Burkhardt, Petra. (2006). Inferential Bridging Relations Reveal Distinct Neural Mechanisms: Evidence from Event-Related Brain Potentials. Brain and Language, 98, 2, 159-168.Oberwelland E, Schilbach L, Barisic I, Krall SC, Vogeley K, Fink GR, Herpertz-Dahlmann B, Konrad K, Schulte-Rüther M(2017). Young adolescents with autism show abnormal joint attention network: A gaze contingent fMRI study. NeuroImage Clinical, 14:112-121, doi: 10.1016/j.nicl.2017.01.006
Prof. Dr. Petra B. Schumacher
(Steering Committee Member, PI)
Project:
The Gesture-to-Sign Trajectory: Phonological Parameters
in Production and Real-Time Comprehension
University of Cologne
petra.schumacher@uni-koeln.de
Petra B. Schumacher’s research focuses on discourse processing and interface phenomena, including anaphora resolution, information structure and experimental pragmatics. In her psycho- and neurolinguistic research, she has applied a wide variety of experimental methodologies and worked with different populations. She obtained her PhD from Yale University in 2004 and held positions at the Max Planck Institutes in Leipzig and Nijmegen as well as at the University of Marburg and Mainz. She is currently professor of empirical linguistics at the University of Cologne.
Selected publications
- Patterson, Clare & Petra B. Schumacher. (2021). Interpretation Preferences in Contexts with three Antecedents: Examining the Role of Prominence in German Pronouns. Applied Psycholinguistics, 1-35.
- Röhr, Christine T., Ingmar Brilmayer, Stefan Baumann, Martine Grice & Petra B. Schumacher. (2020). Signal-Driven and Expectation-Driven Processing of Accent Types. Language, Cognition and Neuroscience, 36,1, 33-59.
- Schumacher, Petra B. (2013). When Combinatorial Processing Results in Reconceptualization: Towards a New Approach of Compositionality. Frontiers in Psychology, 4, 677.
- Schumacher, Petra B. & Yu-Chen Hung. (2012). Positional Influences on Information Packaging: Insights from Topological Fields in German. Journal of Memory and Language, 67, 2, 295-310.
- Burkhardt, Petra. (2006). Inferential Bridging Relations Reveal Distinct Neural Mechanisms: Evidence from Event-Related Brain Potentials. Brain and Language, 98, 2, 159-168.
Sarah Schwarzenberg
(PhD Candidate)
University of Hamburg
sarah.schwarzenberg@uni-hamburg.de
Sarah Schwarzenberg is a scientific researcher at the Institute of German Sign Language and Communication of the Deaf (IDGS), Universität Hamburg. Her research interests are multimodality and sign language acquisition (both L1 and L2). In her master’s thesis at Universität Hamburg, she investigated the use of finger counting and number signs and their modality-specific features. In 2021, she started her doctoral thesis on metaphors in German Sign Language (L1) and written German (L2), focusing on the identification of metaphors with a multi-methodological approach, using corpus data and experimental methods.
Selected publications
- Schwarzenberg, S. 2022. A first approach to identifying metaphors of German Sign Language in the domain of cognition. Video presented at TISLR14, 27.-30.09.2022, Osaka, Japan.
- Bauer, S. 2022. Know your terminology: On the relevance of definitions. Presentation at STaPs19, 25.- 26.03.22, München, Mainz, Köln.
- Bauer, S. 2022. Looking at place of articulation as a first approach to identifying metaphors of German Sign Language in the domain of ‘cognition’. Presentation at the DGfS44, 23.-25.02.2022, University of Tübingen.
Prof. Dr. Stefan R. Schweinberger
(Steering Committee Member, PI)
Project:
Audiovisual Perception of Emotion and Speech
in Hearing Individuals and Cochlear Implant Users
University of Jena
stefan.schweinberger@uni-jena.de
Stefan Schweinberger is interested in cognition as well as cognitive and social neuroscience. Areas of his research include person perception and human interaction, with a particular focus on communication via the face and the voice. He received his PhD from the University of Konstanz in 1991, and worked as a professor at the Universities of Glasgow (2000-2005) and Jena (2005-present). He and his team use research methods linking brain and cognition/emotion, such as event-related brain potentials (ERP), eyetracking, or investigations of patients with focal brain lesions. His research also covers individual differences and constraints to person perception or communication, whether of sensory (e.g., hearing loss) or central origin (e.g., autism, prosopagnosia).
Selected publications
- Frühholz, S., & Schweinberger, S.R. (2021). Nonverbal auditory communication – Evidence for Integrated Neural Systems for Voice Signal Production and Perception. Progress in Neurobiology, 199, 101948. doi: 10.1016/j.pneurobio.2020.101948.
- Frühholz, S., & Schweinberger, S.R. (2021). Nonverbal auditory communication – Evidence for Integrated Neural Systems for Voice Signal Production and Perception. Progress in Neurobiology, 199, 101948. doi: 10.1016/j.pneurobio.2020.101948.
- Schweinberger, S.R., & von Eiff, C.I. (2022). Enhancing Socio-emotional Communication and QoL in Young CI Recipients: Perspectives from Parameter-specific Morphing and Caricaturing. Frontiers in Neuroscience, 16:956917. doi: 10.3389/fnins.2022.956917.
- Skuk, V.G., Kirchen, L., Oberhoffner, T., Guntinas-Lichius, O., Dobel, C., & Schweinberger, S.R. (2020). Parameter-specific Morphing Reveals Contributions of Timbre and F0 Cues to the Perception of Voice Gender and Age in Cochlear Implant Users. Journal of Speech, Language, and Hearing Research, 63(9), 3155-3175. doi: 10.1044/2020_JSLHR-20-00026
- Skuk, V.G., Kirchen, L., Oberhoffner, T., Guntinas-Lichius, O., Dobel, C., & Schweinberger, S.R. (2020). Parameter-specific Morphing Reveals Contributions of Timbre and F0 Cues to the Perception of Voice Gender and Age in Cochlear Implant Users. Journal of Speech, Language, and Hearing Research, 63(9), 3155-3175. doi: 10.1044/2020_JSLHR-20-00026
Door Spruijt
(PhD Candidate)
Project:
The Gesture-to-Sign Trajectory: Phonological Parameters
in Production and Real-Time Comprehension
University of Cologne
dspruijt@uni-koeln.de
Door Spruijt’s research has focused on sign languages since her BA and MA in sign linguistics at the University of Amsterdam. She has worked extensively with both corpus and elicited data from Sign Language of the Netherlands (NGT) and the rural Balinese sign language Kata Kolok. She is now a PhD student on the Gesture-to-Sign Trajectory Project, which investigates the facilitating and inhibiting effects of iconicity and the individual phonological parameters on acquiring the lexicon in second language learners of a sign language (L2M2-learners). Understanding these effects will help improve curricula for L2M2-learners, which she, as a certified teacher of NGT, has great affinity with.
Prof. Dr. Markus Steinbach
(Coordinator, Steering Committee Member, PI)
Project:
Parts of Speech and Iconicity in
German Sign Language (DGS)
University of Göttingen
markus.steinbach@phil.uni-goettingen.de
Markus Steinbach is professor of Linguistics at the German Department of the Universität of Göttingen. He was educated at the University of Frankfurt/Main and obtained a PhD in Linguistics at the Humboldt-University of Berlin in 1997. In 2009, he became habilitated with a study on interface phenomena in German and German Sign Language (DGS) at the University of Mainz. His research is concerned with the influence of language modality (spoken or sign languages) on language structure, development, and processing. The main focus of his research is on the relation between form and meaning, experimental linguistics, grammaticalization, and the interaction between (sign) language and gesture. He has been a principal investigator at the Göttingen Research Center ‘Text Structures’, the Research Training Group ‘Understanding Social Relationships’ and the Research Training Group ‘Form-meaning Mismatches’. He is editing the introductory series ‘Kurze Einführungen in die Germanistische Linguistik (KEGLI)’ (Winter Verlag, with Jörg Meibauer), the sign language series ‘Sign Languages and Deaf Communities (SLDC)’ (Mouton de Gruyter and Ishara Press, with Annika Herrmann) and the journal ‘Linguistische Berichte’ (Buske Verlag, with Nina-Kristin Meister). In 2010, he established the experimental sign language linguistics research group at the Universität of Göttingen (Sign Lab Göttingen), which is running (collaborative) projects on theoretical, cognitive, historical and cultural aspects of sign languages and Deaf communities and since 2022 he has been coordinating the Priority Program ‘Visual Communication (ViCom)’ together with Cornelia Ebert.
Selected publications
- Pfau, Roland, Martin Salzmann and Markus Steinbach (2018): The Syntax of Sign Language Agreement. Common Ingredients but Unusual Recipe. Glossa: A Journal of General Linguistics 3(1), 107, 1-46.
- Steinbach, Markus (2021): Role Shift – Theoretical Perspectives. In Josep Quer, Roland Pfau and Annika Herrmann (eds.), Theoretical and Experimental Sign Language Research. London: Routledge, 351-377.
- Steinbach, Markus (2022): Differential Object Marking in Sign Languages? Restrictions on (Object) Agreement in German Sign Language. In Andrew Nevins, Anita Peti-Stantic, Mark de Vos and Jana Willer-Gold (eds.), Angles of Object Agreement. Oxford: Oxford University Press, 209-240.
- Steinbach, Markus and Edgar Onea (2016): A DRT-Analysis of Discourse Referents and Anaphora Resolution in Sign Language. Journal of Semantics 33, 409-448.
- Wienholz, Anne, Derya Nuhbalaoglu, Nivedita Mani, Annika Herrmann, Edgar Onea and Markus Steinbach (2018): Pointing to the Right Side? An ERP Study on Anaphora Resolution in German Sign Language. In: PLOS ONE 13(9), 1-19.
T – Z
Patrick Trettenbrein
(Steering Committee Member, Postdoc Researcher)
Project:
Parts of Speech and Iconicity in
German Sign Language (DGS)
MPI for Human Cognitive & Brain Sciences Leipzig
trettenbrein@cbs.mpg.de
Patrick studied cognitive sciences (linguistics, philosophy, and psychology) in Graz and London, before moving to the Max Planck Institute for Human Cognitive & Brain Sciences in Leipzig for his PhD project. His main research interest is the neurobiology of language, focusing on the modality (in-)dependence of linguistic computations and representations in the brain. In other words, in his research he doesn’t ask, “How come (only) humans can speak?”—Instead, he investigates human language as a species-specific mode of cognition independent of the modality of language use (spoken, written, or signed).
Selected publications
- Trettenbrein, P. C., Papitto, G., Friederici, A. D., & Zaccarella, E. (2021). Functional neuroanatomy of language without speech: An ALE meta‐analysis of sign language. Human Brain Mapping, 42(3), 699–712. https://doi.org/10.1002/hbm.25254
- Zaccarella, E., & Trettenbrein, P. C. (2021). Neuroscience and syntax. In N. Allott, T. Lohndal, & G. Rey (Eds.), A Companion to Chomsky (pp. 325–347). Wiley-Blackwell.
- Trettenbrein, P. C., Pendzich, N.-K., Cramer, J.-M., Steinbach, M., & Zaccarella, E. (2021). Psycholinguistic norms for more than 300 lexical signs in German Sign Language (DGS). Behavior Research Methods, 53, 1817–1832. https://doi.org/10.3758/s13428-020-01524-y
- Trettenbrein, P. C. (2016). The demise of the synapse as the locus of memory: A looming paradigm shift? Frontiers in Systems Neuroscience, 10(88). https://doi.org/10.3389/fnsys.2016.00088
Celina Isabelle von Eiff
(PhD Candidate)
Project:
Audiovisual Perception of Emotion and Speech
in Hearing Individuals and Cochlear Implant Users
University of Jena
celina.isabelle.von.eiff@uni-jena.de
Celina I. von Eiff is interested in cognitive and social neuroscience, particularly in voice and face perception in individuals with hearing prostheses (i.e., cochlear implants). She uses behavioural measures and EEG to investigate how the human brain integrates sensory information after cochlear implantation. She currently works at the Department of Psychology of the Friedrich Schiller University Jena.
Selected publications
- von Eiff, C. I., Frühholz, S., Korth, D., Guntinas-Lichius, O., & Schweinberger, S. R. (2022). Crossmodal Benefits to Vocal Emotion Perception in Cochlear Implant Users. iScience.
- Schweinberger, S. R. & von Eiff, C. I. (2022). Enhancing socio-emotional communication and quality of life in young cochlear implant recipients: Perspectives from parameter-specific morphing and caricaturing. Frontiers in Neuroscience, 16.
- von Eiff, C. I., Skuk, V. G., Zäske, R., Nussbaum, C., Frühholz, S., Feuer, U., Guntinas-Lichius, O., & Schweinberger, S. R. (2022). Parameter-Specific Morphing Reveals Contributions of Timbre to the Perception of Vocal Emotions in Cochlear Implant Users. Ear and Hearing, 43(4), 1178-1188.
- Nussbaum, C., von Eiff, C. I., Skuk, V. G., & Schweinberger, S. R. (2022). Vocal emotion adaptation aftereffects within and across speaker genders: Roles of timbre and fundamental frequency. Cognition, 219.
- Schweinberger, S. R., von Eiff, C. I., Kirchen, L., Oberhoffner, T., Guntinas-Lichius, O., Dobel, C., Nussbaum, C., Zäske, R., & Skuk, V. G. (2020). The Role of Stimulus Type and Social Signal for Voice Perception in Cochlear Implant Users: Response to the Letter by Meister et al. Journal of Speech, Language, and Hearing Research.
Sebastian Walter
(PhD Candidate)
Project:
Visual and Non-visual Means of
Perspective Taking in Language
University of Wuppertal
s.walter@em.uni-frankfurt.de
Sebastian Walter completed his MA in linguistics at Goethe-University Frankfurt in September 2022. In October 2022 he joined the project Visual and Non-Visual Means of Perspective Taking in Language as a PhD student. His PhD thesis will investigate the interactions of perspective taking in the spoken and the gestural modality. Besides his research interests on the semantic contribution of speech-accompanying gestures and the expression of perspective in language, he is also interested in at-issueness, modals, conditionals, and verbal mood.
Selected publications
- Ebert, Cornelia, Giovanna Pirillo & Sebastian Walter. 2022. The role of gesture-speech alignment for gesture interpretation. In Sam Featherston, Robin Hörnig, Andreas Konietzko & Sophie von Wietersheim (eds.), Proceedings of Linguistic Evidence 2020, 65–77. Tübingen, Germany: University of Tübingen.
Vera Wolfrum
(PhD Candidate)
Project:
Modality-Specific Effects on Language Processing
in Children with Developmental Language Disorder
University of Würzburg
vera.wolfrum@uni-wuerzburg.de
Vera Wolfrum has completed her B.Sc. in Academic Speech Therapy/Logopedics (University of Würzburg) and M.Sc. in Teaching and Research Logopedics (RWTH Aachen University). From November 2022, she works as a PhD student within the project Modality-Specific Effects on Language Processing in Children with Developmental Language Disorder at the department of Special Education and Therapy in Language and Communication Disorders at the University of Würzburg.
Dr. Emiliano Zaccarella
(Principal Investigator)
Project:
Parts of Speech and Iconicity in
German Sign Language (DGS)
MPI for Human Cognitive & Brain Sciences Leipzig
zaccarella@cbs.mpg.de
Emiliano Zaccarella is an expert of visual, auditory and signed language processing. In his work, he uses brain imaging techniques (e.g., magnetic resonance imaging, electroencephalography, diffusion tensor imaging), behavioural methods and computational modelling to understand the general organizational principles of linguistic combinatorial abstraction in the human brain, linking ontogeny and phylogeny. He obtained his PhD in Cognitive Sciences at the University of Potsdam, in collaboration with the Max Planck Institute for Human Cognitive and Brain Sciences Leipzig and the doctoral program of the Berlin School of Mind and Brain of the Humboldt University of Berlin. He is currently group leader in the department of Neuropsychology of the Max Planck Institute for Human Cognitive and Brain Sciences Leipzig.
Selected publications
- Girard-Buttoz, C., Zaccarella, E., Bortolato, T., Friederici, A. D., Wittig R. M. & Crockford, C. (2022) Chimpanzees use numerous flexible vocal sequences with more than two vocal units: A step towards language? Communications Biology 5, 410. https://doi.org/10.1038/s42003-022-03350-8
- Trettenbrein, P. C., Papitto, G. Friederici, A. D. & Zaccarella, E. (2021) Functional neuroanatomy of language without speech, Human Brain Mapping 42 (3), 699-712. https://doi.org/10.1002/hbm.25254
- Trettenbrein, P. C., Pendzich, N. K., Cramer, J. M., Steinbach, M. & Zaccarella, E. (2021) Psycholinguistic norms for more than 300 lexical signs in German Sign Language (DGS), Behavior Research Methods 53, 1817–1832. https://doi.org/10.3758/s13428-020-01524-y
- Trettenbrein, P. C. & Zaccarella, E. (2021) Controlling Video Stimuli in Sign Language and Gesture Research: The OpenPoseR Package for Analyzing OpenPose Motion-Tracking Data in R, Frontiers in Psychology 12, 411. https://doi.org/10.3389/fpsyg.2021.628728
- Zaccarella, E., Papitto, G. & Friederici, A. D. (2020) Language and action in Broca’s area: Computational differentiation and cortical segregation, Brain and Cognition 147, 105651. https://doi.org/10.1016/j.bandc.2020.105651
Chiara Zulberti
(PhD Candidate)
Project:
Compositional structures
in chimpanzee gestural communication
University of Leipzig
chiara.zulberti@uni-leipzig.de
Chiara’s research focuses on the communicative system of one of our closest living relative: the chimpanzee. After graduating in Biology at the University of Studies of Milan, she expanded her knowledge touching topics of evolutionary anthropology, linguistics and neurosciences with her MSc in Animal Behaviour at the University of Zurich – where she collaborated with Prof. Dr. Simon Townsend and his research group. Do humans and chimpanzees share communicative abilities? What are the cognitive mechanisms enabling communication? Which selective pressures lead to the evolution of language? These are some of the broader questions involved in her research. For her PhD in the Human Biology & Primate Cognition group lead by Prof. Dr. Katja Liebal at the University of Leipzig, she is currently probing chimpanzee multimodal signals for potential combinatorial structures. Given that combinatoriality is a core feature of human language, the investigation of comparable structures in other primate species is crucial to uncover the evolutionary origins of this trait.
Mercator Fellows
Prof. Dr. Philippe Schlenker
(Mercator Fellow 2023)
CNRS (Institut Jean-Nicod, Paris)
& New York University
philippe.schlenker@gmail.com
Philippe Schlenker is a senior researcher at CNRS (Institut Jean-Nicod, Paris) and a Global Distinguished Professor at New York University. He was educated at École Normale Supérieure (Paris), and obtained a Ph.D. in Linguistics from MIT, and a Ph.D. in Philosophy from EHESS (Paris). He has taught at École Normale Supérieure, Paris, at the University of Southern California, at UCLA, and, since 2008, at NYU. P. Schlenker’s early interests included semantics, pragmatics, the philosophy of language and philosophical logic. He has conducted research on indexicals and indirect discourse, intensional semantics, anaphora, presuppositions, as well as semantic paradoxes. In recent work, he has advocated a program of ‘Super Semantics’ that seeks to expand the traditional frontiers of the field. He has investigated the semantics of sign languages, with special attention both to their logical structure and to the rich iconic means that interact with it. In order to have a point of comparison for these iconic phenomena, P. Schlenker has also investigated the logic and typology of gestures in spoken language. In collaborative work with primatologists and psycholinguists, he has laid the groundwork for a ‘primate semantics’ that seeks to apply the general methods of formal linguistics to primate vocalizations. And in recent research, he has advocated the development of a detailed semantics for music, albeit one that is very different from linguistic semantics.
Selected publications
- Signs: Schlenker and Lamberton: 2022, Meaningful Blurs: the Sources of Repetition-based Plurals in ASL. Linguistics & Philosophy 45: 201–264
- Gestures: Schlenker: 2020, Gestural Grammar. Nat. Language & Ling. Theory 38:887–936
- Signs and gestures: Schlenker; Bonnet, Marion; Lamberton, Jonathan; Lamberton, Jason; Chemla, Emmanuel; Santoro, Mirko; Geraci, Carlo: accepted with minor revisions, Iconic Syntax: Sign Language Classifier Predicates and Gesture Sequences. Linguistics & Philosophy
- Primates: Schlenker, Chemla, et al.: 2016, Formal Monkey Linguistics. Theoret. Linguistics 42,1-2:1–90
- Music: Schlenker: 2022, Musical Meaning within Super Semantics. Ling. & Philosophy 45:795–872.
Consultants
Dr. Liona Paulus
(Accessibility Consultant)
University of Cologne
lpaulus3@uni-koeln.de
(Photo, a short introduction, and selected publications are to be updated.)
Dr. Marc Schulder
(Open Data Consultant)
University of Hamburg
marc.schulder@uni-hamburg.de
Marc is a computational linguist working to further the open and ethical creation and use of language data, especially data for signed languages. After completing his PhD on the automatic creation of lexical resources for negation-causing content words in English and German, he joined the DGS-Korpus project, creators of the largest available discourse corpus of German Sign Language (DGS), in early 2019. As part of the project he investigates machine-assisted methods to support corpus linguistic research, possible implementations of open science principles and data standardisation for signed language data, and the creation of cross-lingual sign language resources. Marc is also co-creator of the Sign Language Dataset Compendium, a website collecting information on corpora and lexical resources for sign languages from around the globe, and of the workshop series archive sign-lang@LREC Anthology.
Selected publications
- Marc Schulder and Thomas Hanke. 2022. How to be FAIR when you CARE: The DGS Corpus as a Case Study of Open Science Resources for Minority Languages. In Proceedings of the 13th International Conference on Language Resources and Evaluation (LREC 2022), pages 164–173, Marseille, France. European Language Resources Association. URL: https://www.sign-lang.uni-hamburg.de/lrec/pub/2022.lrec-1.18.html
- Maria Kopf, Marc Schulder, and Thomas Hanke. 2022. The Sign Language Dataset Compendium: Creating an Overview of Digital Linguistic Resources. In Proceedings of the 10th Workshop on the Representation and Processing of Sign Languages(sign-lang@LREC 2022), pages 102–109, Marseille, France. European Language Resources Association. URL: https://www.sign-lang.uni-hamburg.de/lrec/pub/22025.html
- Gabriele Langer and Marc Schulder. 2020. Collocations in Sign Language Lexicography: Towards Semantic Abstractions for Word Sense Discrimination. In Proceedings of the 9th Workshop on the Representation and Processing of Sign Languages (sign-lang@LREC 2020), pages 127–134, Marseille, France. European Language Resources Association. URL: https://www.sign-lang.uni-hamburg.de/lrec/pub/20017.html
- Thomas Hanke, Marc Schulder, Reiner Konrad, and Elena Jahn. 2020. Extending the Public DGS Corpus in Size and Depth. In Proceedings of the 9th Workshop on the Representation and Processing of Sign Languages(sign-lang@LREC 2020), pages 75–82, Marseille, France. European Language Resources Association. URL: https://www.sign-lang.uni-hamburg.de/lrec/pub/20016.html
- Marc Schulder, Michael Wiegand, and Josef Ruppenhofer. 2021. Automatic generation of lexica for sentiment polarity shifters. Natural Language Engineering, 27(2):153–179. URL: https://doi.org/10.1017/S135132492000039X
PhD Contact Persons
Alina Gregori
(Steering Committee Member, PhD Candidate, PhD Contact Person)
Project:
Co-speech Gestures and Prosody as
Multimodal Markers of Information Structure
Goethe University Frankfurt
gregori@lingua.uni-frankfurt.de
Alina Gregori received her Bachelors (2021) and Masters (2022) degree in theoretical linguistics at Goethe University Frankfurt with a focus on Phonology. She started working as a PhD student in the project MultIS in October 2022. Within the project, she investigates the prosody-gesture link in communication and the impact of information structure (focus, topic, givenness) on the synchronization of gestures and prosodic entities in German. The bigger picture of the project includes a comparative analysis of German and Catalan with regard to the multimodal marking of information structure. A central aim in MultIS is the empirical approach to previously established prosody-gesture models, considering experimental as well as spontaneous speech utterances. Alina Gregoris MA thesis (title: “Co-speech Gestures, Information Structure and Prosody: A Corpus Study on Prominence Peak Alignment”) served as a pilot and preparation for MultIS.
Selected publications
- Gregori, Alina & Kügler, Frank (2022). An Empirical Investigation on the Perceptual Similarity of Prosodic Language Types. In Proc. 1st International Conference on Tone and Intonation (TAI), 209-213, doi: 10.21437/TAI.2021-43.
Šárka Kadavá
(PhD Candidate, PhD Contact Person)
Leibniz-Zentrum Allgemeine Sprachwissenschaft Berlin
kadava@leibniz-zas.de
Šárka Kadavá is a doctoral researcher in the DFG project On the FLExibility and Stability of gesture-speecH coordination (FLESH): Evidence from production, comprehension, and imitation. She is currently based in Leibniz-Zentrum Allgemeine Sprachwissenschaft (ZAS) in Berlin. Her research focuses on multimodality, on how different modalities contribute to the sharing of meaning and how they are coordinated with each other, and on how language-like structural features emerge in non-normative communication systems. She has also participated in a project developing Czech adaptation of MB-CDI at the Institute of Psychology, Czech Academy of Sciences.
Associate Members
Kathryn Barnes
(Associate Member)
Goethe University Frankfurt
barnes@lingua.uni-frankfurt.de
Kathryn Barnes is a doctoral researcher in formal semantics at Goethe University. She completed her BA in French and German Studies at the University of Warwick in 2016 and her MA in Linguistics at The University of Manchester in 2019, where her thesis was based on fieldwork on the semantics of modal verbs in Malay. Her research has since turned to iconicity, with a particular focus on ideophones, including experimentally investigating the meaning and at-issue status of ideophones crosslinguistically, as well as developing a semantic account for the phenomena. She is also interested in exploring how ideophones interact with other iconic enrichments such as gestures and understanding ideophone-like constructions in other modalities, for example classifier constructions and idiomatic signs in sign language.
Selected publications
- Barnes, K. R., Ebert, C., Hörnig, R., & Stender, T. (2022). The at-issue status of ideophones in German: An experimental approach. Glossa: A Journal of General Linguistics, 7(1). https://doi.org/10.16995/glossa.5827