Coordination
Prof. Dr. Cornelia Ebert
(Coordinator, Steering Committee Member, PI)
Project:
Visual and Non-visual Means of
Perspective Taking in Language
Goethe University Frankfurt
ebert@lingua.uni-frankfurt.de
Cornelia Ebert is a professor of linguistics/semantics at Goethe University in Frankfurt am Main. She received her Ph.D. in linguistics at Potsdam University in 2006. Her thesis was titled »Quantificational topics. A scopal treatment of exceptional wide scope phenomena«. She was a lecturer and a researcher at Osnabrück University, Stuttgart University and the Leibniz-Zentrum Allgemeine Sprachwissenschaft in Berlin. Since 2021 she carries out a British-German cooperative research project entitled »Interactions between Dynamic Effects and Alternative-Based Inferences in the Study of Meaning« (IDEALISM). And since 2022 she is one of the coordinators of the DFG Priority Programme 2392 Visual Communication. Theoretical, Empirical, and Applied Perspectives (ViCom). 2020-24 she is Goethe Fellow at the Forschungskolleg Humanwissenschaften.
Selected publications
- Barnes, K. R., Ebert, C., Hörnig, R., & Stender, T. (2022). The at-issue status of ideophones in German: An experimental approach. Glossa: A Journal of General Linguistics, 7(1). https://doi.org/10.16995/glossa.5827
- Ebert, C., Ebert, C., & Hörnig, R. (2020). Demonstratives as dimension shifters. Proceedings of Sinn Und Bedeutung, 161-178 Pages. https://doi.org/10.18148/SUB/2020.V24I1.859
- Ebert, C. (2018). A comparison of sign language with speech plus gesture. Theoretical Linguistics, 44(3–4), 239–249. https://doi.org/10.1515/tl-2018-0016
- Ebert, C., Ebert, C., & Hinterwimmer, S. (2014). A unified analysis of conditionals as topics. Linguistics and Philosophy, 37(5), 353–408. https://doi.org/10.1007/s10988-014-9158-4
- Ebert, C. (2009). Quantificational topics: A scopal treatment of exceptional wide scope phenomena (Vol. 86). Springer Science & Business Media.
Prof. Dr. Markus Steinbach
(Coordinator, Steering Committee Member, PI)
Project:
Parts of Speech and Iconicity in
German Sign Language (DGS)
University of Göttingen
markus.steinbach@phil.uni-goettingen.de
Markus Steinbach is professor of Linguistics at the German Department of the Universität of Göttingen. He was educated at the University of Frankfurt/Main and obtained a PhD in Linguistics at the Humboldt-University of Berlin in 1997. In 2009, he became habilitated with a study on interface phenomena in German and German Sign Language (DGS) at the University of Mainz. His research is concerned with the influence of language modality (spoken or sign languages) on language structure, development, and processing. The main focus of his research is on the relation between form and meaning, experimental linguistics, grammaticalization, and the interaction between (sign) language and gesture. He has been a principal investigator at the Göttingen Research Center ‘Text Structures’, the Research Training Group ‘Understanding Social Relationships’ and the Research Training Group ‘Form-meaning Mismatches’. He is editing the introductory series ‘Kurze Einführungen in die Germanistische Linguistik (KEGLI)’ (Winter Verlag, with Jörg Meibauer), the sign language series ‘Sign Languages and Deaf Communities (SLDC)’ (Mouton de Gruyter and Ishara Press, with Annika Herrmann) and the journal ‘Linguistische Berichte’ (Buske Verlag, with Nina-Kristin Meister). In 2010, he established the experimental sign language linguistics research group at the Universität of Göttingen (Sign Lab Göttingen), which is running (collaborative) projects on theoretical, cognitive, historical and cultural aspects of sign languages and Deaf communities and since 2022 he has been coordinating the Priority Program ‘Visual Communication (ViCom)’ together with Cornelia Ebert.
Selected publications
- Pfau, Roland, Martin Salzmann and Markus Steinbach (2018): The Syntax of Sign Language Agreement. Common Ingredients but Unusual Recipe. Glossa: A Journal of General Linguistics 3(1), 107, 1-46.
- Steinbach, Markus (2021): Role Shift – Theoretical Perspectives. In Josep Quer, Roland Pfau and Annika Herrmann (eds.), Theoretical and Experimental Sign Language Research. London: Routledge, 351-377.
- Steinbach, Markus (2022): Differential Object Marking in Sign Languages? Restrictions on (Object) Agreement in German Sign Language. In Andrew Nevins, Anita Peti-Stantic, Mark de Vos and Jana Willer-Gold (eds.), Angles of Object Agreement. Oxford: Oxford University Press, 209-240.
- Steinbach, Markus and Edgar Onea (2016): A DRT-Analysis of Discourse Referents and Anaphora Resolution in Sign Language. Journal of Semantics 33, 409-448.
- Wienholz, Anne, Derya Nuhbalaoglu, Nivedita Mani, Annika Herrmann, Edgar Onea and Markus Steinbach (2018): Pointing to the Right Side? An ERP Study on Anaphora Resolution in German Sign Language. In: PLOS ONE 13(9), 1-19.
Theresa Stender
(Program Manager, Steering Committee Member)
Goethe University Frankfurt
stender@lingua.uni-frankfurt.de
Theresa Stender completed her Bachelor’s degree in Germanistic Linguistics at the Humboldt University of Berlin. Her research interests encompass the study of iconicity in languages, particularly sound symbolism, ideophones, dimensions of meaning and multimodality, as well as natural language processing. Prior to joining ViCom, she contributed as a student research assistant to the project PSIMS: The Pragmatic Status of Iconic Meaning in Spoken Communication: Gestures, Ideophones, Prosodic Modulations.
Selected publications
- Barnes, K. R., Ebert, C., Hörnig, R., Stender, T., & Barnes, K. (2022). The at-issue status of ideophones in German: An experimental approach. Glossa: a journal of general linguistics, 7(1).
Elke Höhe-Kupfer
(Secretary)
Goethe University Frankfurt
hoehe-kupfer@lingua.uni-frankfurt.de
Lennart Fritzsche
(Student Assistant)
Goethe University Frankfurt
fritzsche@em.uni-frankfurt.de
Lennart is currently pursuing a Master’s degree in General Linguistics at Goethe-University Frankfurt, specializing in formal semantics. In addition to his studies, he works as a student assistant at both the Department of Psycholinguistics and the Department of Semantics, where he primarily contributes to the ViCom project. Lennart’s interests lie in theoretical and experimental semantics, particularly in exploring the semantics of gesture, the gesture-sign interface, and iconicity.
Natascha Schuldes
(Social Media Manager)
Goethe University Frankfurt
schuldes@lingua.uni-frankfurt.de
Magnus Poppe
(Student Assistant)
Goethe University Frankfurt
magnus.poppe@web.de
ViCom Data Network
Dr. Susanne Fuchs
(Data Network Coordinating Member, Steering Committee Member, PI)
Leibniz-Zentrum Allgemeine Sprachwissenschaft Berlin
fuchs@leibniz-zas.de
Susanne Fuchs (ZAS Berlin) investigates the biopsychosocial foundations of human interaction and focuses specifically on physiological processes, such as breathing and motor control. Her main areas of interests are:
1) The interplay between motion, breathing and cognition,
2) Speech preparation and pauses,
3) Multimodality and iconicity,
4) Biological and social aspects shaping individual behaviour in speech production and perception.
She uses manifold techniques, among them optitrack, inductance plethysmography, electropalatography and intraoral pressure sensors.
Selected publications
- Pouw, W., & Fuchs, S. (2022). Origins of vocal-entangled gesture. Neuroscience & Biobehavioral Reviews, 104836.
- Fuchs, S., & Rochet-Capellan, A. (2021). The respiratory foundations of spoken language. Annual Review of Linguistics, 7(1), 13-30.
- Ćwiek, A., Fuchs, S., Draxler, C., Asu, E. L., Dediu, D., Hiovain, K., … & Perlman, M. (2021). Novel vocalizations are understood across cultures. Scientific Reports, 11(1), 1-12.
- Floegel, M., Fuchs, S., & Kell, C. A. (2020). Differential contributions of the two cerebral hemispheres to temporal and spectral speech feedback control. Nature communications, 11(1), 1-12.
- Fuchs, S. (2019). Vocal tract variations affect vowel sounds. Nature Human Behaviour, 3(10), 1043-1044.
Dr. Aleksandra Ćwiek
(Data Network Coordinating Member, PI)
Leibniz-Zentrum Allgemeine Sprachwissenschaft Berlin
cwiek@leibniz-zas.de
Aleksandra Ćwiek is a researcher at the Leibniz-Centre General Linguistics (ZAS) in Berlin and a PI of the project “On the FLExibility and Stability of gesture-speecH coordination (FLESH): Evidence from production, comprehension, and imitation”. She received her Ph.D. from the Humboldt University of Berlin in 2022. In her doctoral thesis, she studied the diversity of linguistic iconicity and its role in language evolution and in language nowadays. She examined aspects such as iconic prosody, sound symbolism, and German ideophones. Her further interests include acoustic phonetics, speech-gesture link, cognition, and cross-linguistic research.
Selected publications
- Ćwiek, A. (2022). Iconicity in Language and Speech [PhD Thesis, Humboldt University of Berlin].
https://doi.org/10.18452/24544 - Ćwiek, A., Fuchs, S., Draxler, C., Asu, E. L., Dediu, D., … Perlman, M. (2021). Novel vocalizations are understood across cultures. Scientific Reports, 11(1), 10108. https://doi.org/10.1038/s41598-021-89445-4
- Ćwiek, A., Fuchs, S., Draxler, C., Asu, E. L., Dediu, D., … Winter, B. (2022). The bouba/kiki effect is robust across cultures and writing systems. Philosophical Transactions of the Royal Society B: Biological Sciences, 377(1841), 20200390.
- Fuchs, S., & Ćwiek, A. (2022). Sounds Full of Meaning and the Evolution of Language. Acoustics Today, 18(1), 43–51. https://doi.org/10.1121/AT.2022.18.2.43
- Wagner, P., Ćwiek, A., & Samlowski, B. (2019). Exploiting the speech-gesture link to capture fine-grained prosodic prominence impressions and listening strategies. Journal of Phonetics, 76, 100911.
https://doi.org/10.1016/j.wocn.2019.07.001
Dr. Wim Pouw
(Data Network Coordinating Member, PI)
Donders Institute &
MPI for Psycholinguistics Nijmegen
w.pouw@donders.ru.nl
(A short introduction and selected publications are to be updated.)
Kim Tien Nguyen
(Data Manager)
Goethe University Frankfurt
kimtien.nguyen@em.uni-frankfurt.de
Dr. Martin Schulte-Rüther
(Data Network Coordinating Member, PI)
Project:
Multimodal Assessment of Dyadic Interaction
in Disorders of Social Interaction
University Medical Center Göttingen
martin.schulte-ruether@med.uni-goettingen.de
Martin’s research is focused on social interaction and emotional processing in typical development as well as in psychiatric and neurodevelopmental disorders, in particular Autism. He employs a broad spectrum of neuroscientific and behavioral methods, including MRI, fNIRS, physiological recordings, eye-tracking, and video-based behavioral analysis. Martin studied Psychology at the Ruhr-University Bochum and received his PhD at the University of Bielefeld working in cooperation with the research Center Jülich on the cognitive neuroscience of empathy. He continued as a Post-Doc at the Child and Adolescent Psychiatry of the RWTH Aachen, focusing on neuroimaging methods in children and adolescents with autism. Next, he headed a research group for Translational Brain Research at the RWTH Aachen, Jülich-Aachen Research Alliance. Currently, he is based at the University Medical Center Göttingen, and is a senior researcher and group leader of the Social Interaction and Developmental Neuroscience Lab at the Child and Adolescent Psychiatry Göttingen.
Selected publications
- Schulte-Rüther M, Kulvicius T, Stroth S, Wolff N, Roessner V, Marschik PB, Kamp-Becker I, Poustka L (2022). Using machine learning to improve diagnostic assessment of ASD in the light of specific differential and co-occurring diagnoses. Journal of Child Psychology and Psychiatry. https://doi.org/10.1111/jcpp.13650
- Hartz A, Guth B, Jording M, Vogeley K, Schulte-Rüther M (2021). Temporal Behavioral Parameters of on-going Gaze Encounters in a Virtual Environment. Frontiers in Psychology, doi:10.3389/fpsyg.2021.673982
- Kruppa J, Reindl V, Gerloff C, Oberwelland E, Prinz J, Herpertz-Dahlmann B, Konrad K, Schulte-Rüther M. (2020). Brain-to-Brain synchrony in children and adolescents with autism spectrum disorder in parent-child-dyads. Social Cognitive and Affective Neuroscience, doi:10.1093/scan/nsaa092
- Kruppa JA, Gossen A, Großheinrich N, Oberwelland E, Cholemkery H, Freitag C, Kohls G, Fink GR, Herpertz-Dahlmann B, Konrad K, Schulte-Rüther M (2019). Social Reinforcement Learning and its Neural Modulation by Oxytocin in Autism Spectrum Disorder. Neuropsychopharmacology, 44:749-756. doi:10.1038/s41386-018-0258-7
- Burkhardt, Petra. (2006). Inferential Bridging Relations Reveal Distinct Neural Mechanisms: Evidence from Event-Related Brain Potentials. Brain and Language, 98, 2, 159-168.Oberwelland E, Schilbach L, Barisic I, Krall SC, Vogeley K, Fink GR, Herpertz-Dahlmann B, Konrad K, Schulte-Rüther M(2017). Young adolescents with autism show abnormal joint attention network: A gaze contingent fMRI study. NeuroImage Clinical, 14:112-121, doi: 10.1016/j.nicl.2017.01.006
Door Spruijt
(Data Network Coordinating Member, PhD Candidate)
Project:
The Gesture-to-Sign Trajectory: Phonological Parameters in Production and Real-Time Comprehension
University of Cologne
dspruijt@uni-koeln.de
Door Spruijt’s research has focused on sign languages since her BA and MA in sign linguistics at the University of Amsterdam. She has worked extensively with both corpus and elicited data from Sign Language of the Netherlands (NGT) and the rural Balinese sign language Kata Kolok. She is now a PhD student on the Gesture-to-Sign Trajectory Project, which investigates the facilitating and inhibiting effects of iconicity and the individual phonological parameters on acquiring the lexicon in second language learners of a sign language (L2M2-learners). Understanding these effects will help improve curricula for L2M2-learners, which she, as a certified teacher of NGT, has great affinity with.
Dr. Alexander Henlein
(Data Network Coordinating Member, Postdoc Researcher)
Project:
Virtual Reality Sustained Multimodal Distributional Semantics for Gestures in Dialogue (GeMDiS)
Goethe University Frankfurt
henlein@em.uni-frankfurt.de
Alexander Henlein is a PostDoc at the Text Technology Lab (TTLab) of the Professorship for Computational Humanities / Text Technology of Prof. Dr. Alexander Mehler at the Goethe University Frankfurt. His doctoral research focused on the analysis of spatial semantics in language models, the extraction of object habitats from images, and the development of a VR-based Text2Scene system. Based on this work experience, he would like to develop VR-assisted communication systems within the scope of this project and use the data thus generated to create novel multimodal models.
Selected publications
- A. Henlein, A. Gopinath, N. Krishnaswamy, A. Mehler, J. Pustejovsky, “Grounding Human-Object Interaction to Affordance Behavior in Multimodal Datasets”, in Frontiers in Artificial Intelligence-Language and Computation, 2023 (accepted).
- A. Henlein and A. Mehler, “What do Toothbrushes do in the Kitchen? How Transformers Think our World is Structured,” in Proceedings of the 2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2022), 2022
- A. Henlein, G. Abrami, A. Kett, C. Spiekermann, and A. Mehler, “Digital Learning, Teaching and Collaboration in an Era of ubiquitous Quarantine,” in Remote Learning in Times of Pandemic – Issues, Implications and Best Practice, L. Daniela and A. Visvizin, Eds., Thames, Oxfordshire, England, UK: Routledge, 2021.
- A. Henlein, G. Abrami, A. Kett, and A. Mehler, “Transfer of ISOSpace into a 3D Environment for Annotations and Applications,” in Proceedings of the 16th Joint ACL – ISO Workshop on Interoperable Semantic Annotation, Marseille, 2020, pp. 32-35.
- A. Henlein and A. Mehler, “On the Influence of Coreference Resolution on Word Embeddings in Lexical-semantic Evaluation Tasks,” in Proceedings of The 12th Language Resources and Evaluation Conference, Marseille, France, 2020, pp. 27-33.
Steering Committee
Prof. Dr. Cornelia Ebert
(Coordinator, Steering Committee Member, PI)
Project:
Visual and Non-visual Means of
Perspective Taking in Language
Goethe University Frankfurt
ebert@lingua.uni-frankfurt.de
Cornelia Ebert is a professor of linguistics/semantics at Goethe University in Frankfurt am Main. She received her Ph.D. in lingusitics at Potsdam University in 2006. Her thesis was titled »Quantificational topics. A scopal treatment of exceptional wide scope phenomena«. She was a lecturer and a researcher at Osnabrück University, Stuttgart University and the Leibniz-Zentrum Allgemeine Sprachwissenschaft in Berlin. Since 2021 she carries out a British-German cooperative research project entitled »Interactions between Dynamic Effects and Alternative-Based Inferences in the Study of Meaning« (IDEALISM). And since 2022 she is one of the coordinators of the DFG Priority Programme 2392 Visual Communication. Theoretical, Empirical, and Applied Perspectives (ViCom). 2020-24 she is Goethe Fellow at the Forschungskolleg Humanwissenschaften.
Selected publications
- Barnes, Kathryn Rose, Cornelia Ebert, Robin Hörnig & Theresa Stender. ‘The at-issue status of ideophones in German: An experimental approach’. Glossa: a journal of general linguistics 7 (1). 2022. doi: 10.16995/glossa.5827.
- (with Christian Ebert and Robin Hörnig) »Demonstratives as dimension shifters«, in: Proceedings of Sinn Und Bedeutung, 24, 2020.
- »A comparison of sign language with speech plus gesture: Commentary on Philippe Schlenker’s Target article ›Visible Meaning: Sign Language and the Foundations of Semantics‹«, in: Theoretical Linguistics, Vol. 44:3-4, pp.239–249, 2018
- Ebert, Christian & Cornelia Ebert & Stefan Hinterwimmer: A Unified Analysis of Conditionals as Topics, Linguistics & Philosophy 37(5), 353-408, 2014
- »Quantificational Topics: A Scopal Treatment of Exceptional Wide Scope Phenomena«, in: Studies in Linguistics and Philosophy, 86, Springer Netherlands, 2009. [https://doi.org/10.1007/978-90-481-2303-2]
Prof. Dr. Markus Steinbach
(Coordinator, Steering Committee Member, PI)
Project:
Parts of Speech and Iconicity in
German Sign Language (DGS)
University of Göttingen
markus.steinbach@phil.uni-goettingen.de
Markus Steinbach is professor for Linguistics at the German Department of the Universität of Göttingen. He was educated at the University of Frankfurt/Main and obtained a PhD in Linguistics at the Humboldt-University of Berlin in 1997. In 2009, he became habilitated with a study on interface phenomena in German and German Sign Language (DGS) at the University of Mainz. His research is concerned with the influence of language modality (spoken or sign languages) on language structure, development, and processing. The main focus of his research is on the relation between form and meaning, experimental linguistics, grammaticalization, and the interaction between (sign) language and gesture. He has been a principal investigator at the Göttingen Research Center ‘Text Structures’, the Research Training Group ‘Understanding Social Relationships’ and the Research Training Group ‘Form-meaning Mismatches’. He is editing the introductory series ‘Kurze Einführungen in die Germanistische Linguistik (KEGLI)’ (Winter Verlag, with Jörg Meibauer), the sign language series ‘Sign Languages and Deaf Communities (SLDC)’ (Mouton de Gruyter and Ishara Press, with Annika Herrmann) and the journal ‘Linguistische Berichte’ (Buske Verlag, with Nina-Kristin Meister). In 2010, he established the experimental sign language linguistics research group at the Universität of Göttingen (Sign Lab Göttingen), which is running (collaborative) projects on theoretical, cognitive, historical and cultural aspects of sign languages and Deaf communities and since 2022 he has been coordinating the Priority Program ‘Visual Communication (ViCom)’ together with Cornelia Ebert.
Selected publications
- Pfau, Roland, Martin Salzmann and Markus Steinbach (2018): The Syntax of Sign Language Agreement. Common Ingredients but Unusual Recipe. Glossa: A Journal of General Linguistics 3(1), 107, 1-46.
- Steinbach, Markus (2021): Role Shift – Theoretical Perspectives. In Josep Quer, Roland Pfau and Annika Herrmann (eds.), Theoretical and Experimental Sign Language Research. London: Routledge, 351-377.
- Steinbach, Markus (2022): Differential Object Marking in Sign Languages? Restrictions on (Object) Agreement in German Sign Language. In Andrew Nevins, Anita Peti-Stantic, Mark de Vos and Jana Willer-Gold (eds.), Angles of Object Agreement. Oxford: Oxford University Press, 209-240.
- Steinbach, Markus and Edgar Onea (2016): A DRT-Analysis of Discourse Referents and Anaphora Resolution in Sign Language. Journal of Semantics 33, 409-448.
- Wienholz, Anne, Derya Nuhbalaoglu, Nivedita Mani, Annika Herrmann, Edgar Onea and Markus Steinbach (2018): Pointing to the Right Side? An ERP Study on Anaphora Resolution in German Sign Language. In: PLOS ONE 13(9), 1-19.
Theresa Stender
(Program Manager, Steering Committee Member)
Goethe University Frankfurt
coordination@vicom.info
Theresa Stender completed her Bachelor’s degree in Germanistic Linguistics at the Humboldt University of Berlin. Her research interests encompass the study of iconicity in languages, particularly sound symbolism, ideophones, dimensions of meaning and multimodality, as well as natural language processing. Prior to joining ViCom, she contributed as a student research assistant to the project PSIMS: The Pragmatic Status of Iconic Meaning in Spoken Communication: Gestures, Ideophones, Prosodic Modulations.
Selected publications
- Barnes, K. R., Ebert, C., Hörnig, R., Stender, T., & Barnes, K. (2022). The at-issue status of ideophones in German: An experimental approach. Glossa: a journal of general linguistics, 7(1).
Prof. Dr. Petra B. Schumacher
(Steering Committee Member, PI)
Project:
The Gesture-to-Sign Trajectory: Phonological Parameters
in Production and Real-Time Comprehension
University of Cologne
petra.schumacher@uni-koeln.de
Petra B. Schumacher’s research focuses on discourse processing and interface phenomena, including anaphora resolution, information structure and experimental pragmatics. In her psycho- and neurolinguistic research, she has applied a wide variety of experimental methodologies and worked with different populations. She obtained her PhD from Yale University in 2004 and held positions at the Max Planck Institutes in Leipzig and Nijmegen as well as at the University of Marburg and Mainz. She is currently professor of empirical linguistics at the University of Cologne.
Selected publications
- Patterson, Clare & Petra B. Schumacher. (2021). Interpretation Preferences in Contexts with three Antecedents: Examining the Role of Prominence in German Pronouns. Applied Psycholinguistics, 1-35.
- Röhr, Christine T., Ingmar Brilmayer, Stefan Baumann, Martine Grice & Petra B. Schumacher. (2020). Signal-Driven and Expectation-Driven Processing of Accent Types. Language, Cognition and Neuroscience, 36,1, 33-59.
- Schumacher, Petra B. (2013). When Combinatorial Processing Results in Reconceptualization: Towards a New Approach of Compositionality. Frontiers in Psychology, 4, 677.
- Schumacher, Petra B. & Yu-Chen Hung. (2012). Positional Influences on Information Packaging: Insights from Topological Fields in German. Journal of Memory and Language, 67, 2, 295-310.
- Burkhardt, Petra. (2006). Inferential Bridging Relations Reveal Distinct Neural Mechanisms: Evidence from Event-Related Brain Potentials. Brain and Language, 98, 2, 159-168.
Prof. Dr. Stefan R. Schweinberger
(Steering Committee Member, PI)
Project:
Audiovisual Perception of Emotion and Speech
in Hearing Individuals and Cochlear Implant Users
University of Jena
stefan.schweinberger@uni-jena.de
Stefan Schweinberger is interested in cognition as well as cognitive and social neuroscience. Areas of his research include person perception and human interaction, with a particular focus on communication via the face and the voice. He received his PhD from the University of Konstanz in 1991, and worked as a professor at the Universities of Glasgow (2000-2005) and Jena (2005-present). He and his team use research methods linking brain and cognition/emotion, such as event-related brain potentials (ERP), eyetracking, or investigations of patients with focal brain lesions. His research also covers individual differences and constraints to person perception or communication, whether of sensory (e.g., hearing loss) or central origin (e.g., autism, prosopagnosia).
Selected publications
- Frühholz, S., & Schweinberger, S.R. (2021). Nonverbal auditory communication – Evidence for Integrated Neural Systems for Voice Signal Production and Perception. Progress in Neurobiology, 199, 101948. doi: 10.1016/j.pneurobio.2020.101948.
- Frühholz, S., & Schweinberger, S.R. (2021). Nonverbal auditory communication – Evidence for Integrated Neural Systems for Voice Signal Production and Perception. Progress in Neurobiology, 199, 101948. doi: 10.1016/j.pneurobio.2020.101948.
- Schweinberger, S.R., & von Eiff, C.I. (2022). Enhancing Socio-emotional Communication and QoL in Young CI Recipients: Perspectives from Parameter-specific Morphing and Caricaturing. Frontiers in Neuroscience, 16:956917. doi: 10.3389/fnins.2022.956917.
- Skuk, V.G., Kirchen, L., Oberhoffner, T., Guntinas-Lichius, O., Dobel, C., & Schweinberger, S.R. (2020). Parameter-specific Morphing Reveals Contributions of Timbre and F0 Cues to the Perception of Voice Gender and Age in Cochlear Implant Users. Journal of Speech, Language, and Hearing Research, 63(9), 3155-3175. doi: 10.1044/2020_JSLHR-20-00026
- Skuk, V.G., Kirchen, L., Oberhoffner, T., Guntinas-Lichius, O., Dobel, C., & Schweinberger, S.R. (2020). Parameter-specific Morphing Reveals Contributions of Timbre and F0 Cues to the Perception of Voice Gender and Age in Cochlear Implant Users. Journal of Speech, Language, and Hearing Research, 63(9), 3155-3175. doi: 10.1044/2020_JSLHR-20-00026
Dr. Susanne Fuchs
(ViCom Data Network Coordinator, Steering Committee Member, PI)
Leibniz-Zentrum Allgemeine Sprachwissenschaft Berlin
fuchs@leibniz-zas.de
Susanne Fuchs (ZAS Berlin) investigates the biopsychosocial foundations of human interaction and focuses specifically on physiological processes, such as breathing and motor control. Her main areas of interests are:
1) The interplay between motion, breathing and cognition,
2) Speech preparation and pauses,
3) Multimodality and iconicity,
4) Biological and social aspects shaping individual behaviour in speech production and perception.
She uses manifold techniques, among them optitrack, inductance plethysmography, electropalatography and intraoral pressure sensors.
Selected publications
- Pouw, W., & Fuchs, S. (2022). Origins of vocal-entangled gesture. Neuroscience & Biobehavioral Reviews, 104836.
- Fuchs, S., & Rochet-Capellan, A. (2021). The respiratory foundations of spoken language. Annual Review of Linguistics, 7(1), 13-30.
- Ćwiek, A., Fuchs, S., Draxler, C., Asu, E. L., Dediu, D., Hiovain, K., … & Perlman, M. (2021). Novel vocalizations are understood across cultures. Scientific Reports, 11(1), 1-12.
- Floegel, M., Fuchs, S., & Kell, C. A. (2020). Differential contributions of the two cerebral hemispheres to temporal and spectral speech feedback control. Nature communications, 11(1), 1-12.
- Fuchs, S. (2019). Vocal tract variations affect vowel sounds. Nature Human Behaviour, 3(10), 1043-1044.
Patrick Trettenbrein
(Steering Committee Member, Postdoc Researcher)
Project:
Parts of Speech and Iconicity in
German Sign Language (DGS)
MPI for Human Cognitive & Brain Sciences Leipzig
trettenbrein@cbs.mpg.de
Patrick studied cognitive sciences (linguistics, philosophy, and psychology) in Graz and London, before moving to the Max Planck Institute for Human Cognitive & Brain Sciences in Leipzig for his PhD project. His main research interest is the neurobiology of language, focusing on the modality (in-)dependence of linguistic computations and representations in the brain. In other words, in his research he doesn’t ask, “How come (only) humans can speak?”—Instead, he investigates human language as a species-specific mode of cognition independent of the modality of language use (spoken, written, or signed).
Selected publications
- Trettenbrein, P. C., Papitto, G., Friederici, A. D., & Zaccarella, E. (2021). Functional neuroanatomy of language without speech: An ALE meta‐analysis of sign language. Human Brain Mapping, 42(3), 699–712. https://doi.org/10.1002/hbm.25254
- Zaccarella, E., & Trettenbrein, P. C. (2021). Neuroscience and syntax. In N. Allott, T. Lohndal, & G. Rey (Eds.), A Companion to Chomsky (pp. 325–347). Wiley-Blackwell.
- Trettenbrein, P. C., Pendzich, N.-K., Cramer, J.-M., Steinbach, M., & Zaccarella, E. (2021). Psycholinguistic norms for more than 300 lexical signs in German Sign Language (DGS). Behavior Research Methods, 53, 1817–1832. https://doi.org/10.3758/s13428-020-01524-y
- Trettenbrein, P. C. (2016). The demise of the synapse as the locus of memory: A looming paradigm shift? Frontiers in Systems Neuroscience, 10(88). https://doi.org/10.3389/fnsys.2016.00088
Alina Gregori
(Steering Committee Member, PhD Candidate, PhD Contact Person)
Project:
Co-speech Gestures and Prosody as
Multimodal Markers of Information Structure
Goethe University Frankfurt
gregori@lingua.uni-frankfurt.de
Alina Gregori received her Bachelors (2021) and Masters (2022) degree in theoretical linguistics at Goethe University Frankfurt with a focus on Phonology. She started working as a PhD student in the project MultIS in October 2022. Within the project, she investigates the prosody-gesture link in communication and the impact of information structure (focus, topic, givenness) on the synchronization of gestures and prosodic entities in German. The bigger picture of the project includes a comparative analysis of German and Catalan with regard to the multimodal marking of information structure. A central aim in MultIS is the empirical approach to previously established prosody-gesture models, considering experimental as well as spontaneous speech utterances. Alina Gregoris MA thesis (title: “Co-speech Gestures, Information Structure and Prosody: A Corpus Study on Prominence Peak Alignment”) served as a pilot and preparation for MultIS.
Selected publications
- Gregori, Alina & Kügler, Frank (2022). An Empirical Investigation on the Perceptual Similarity of Prosodic Language Types. In Proc. 1st International Conference on Tone and Intonation (TAI), 209-213, doi: 10.21437/TAI.2021-43.
Priority Program ViCom (SPP 2392)
ViCom investigates the special features and linguistic significance of visual communication. This comprises sign languages as fully developed natural languages which exclusively rely on the visual channel for communication, but also visual means that enhance spoken language such as gestures. It aims at disclosing the specific characteristics of the visual modality as a communication channel and its interaction with other channels (especially the auditory channel) to develop a comprehensive theoretical linguistic model of human communication and its cognitive foundations.
ViCom focuses on the investigation of sign languages and gestures as prime examples of visual human communication. Additionally, the research agenda will comprise the investigation of gestures in didactic and therapeutic contexts, gestural aspects of vocal communication and in the written modality, pictorial narratives as well as visual communication strategies of non-linguistic species and in multimodal human-computer interaction systems. In theoretical linguistic research, both sign language and gestures, which have been recognised as fruitful research topic only very recently, are currently analysed by application of the linguistic vocabulary established for spoken languages. Only little attention has been paid to the modelling of the specific properties of the visual transmission channels. And it is now becoming evident that the formal linguistic repertoire needs to be extended to meet the specific requirements of visual communicative means.
However, although gesture research is a new and emerging topic in theoretical linguistics, it has been a central topic for a long time in other disciplines, such as cognitive linguistics, semiotics, psychology, neurosciences, animal communication, cultural studies, or conversation analysis. Likewise, cognitive oriented analyses of sign languages have highlighted the modality-specific (visual) properties of sign languages for a long time. From an applied perspective, the need for natural multimodal human-computer interfaces has been recognised for some time already. And finally, gestures are important for didactic and therapeutic uses due to their well-established role in language acquisition and learning.
So, while there already exists a substantial body of empirical data and a considerable amount of insights on the form and functions of visual communication, this knowledge is distributed across disciplines and has not been collected and unified in a comprehensive way yet. ViCom as a highly interdisciplinary priority programme brings together the above-mentioned different disciplines and research traditions under a theoretical, empirical, and applied perspective and will use these new insights to develop a new comprehensive theoretical model of language and communication that is able to account for all modalities. It will also help to advance linguistic methodology and technological, therapeutic, and didactic environments.