Project Participants
Prof. Dr. Pamela Perniss
(Principal Investigator)
University of Cologne
pperniss@uni-koeln.de
Pamela Perniss’s research takes a multimodal approach to language and has focused in particular on the role of the visual modality and iconicity in shaping language structure and processing. Following her PhD at the Max Planck Institute for Psycholinguistics (2007), she held postdoctoral positions at the Deafness, Cognition and Language (DCAL) Research Centre (University College London) and at Radboud University Nijmegen before joining the Linguistics Department at the University of Brighton. Since 2019, she is a professor in the Faculty of Human Sciences at the University of Cologne and chair of the sign language interpreting (DGS-German) program.
Selected publications
- Perniss, P., Vinson, D., & Vigliocco, G. (2020). Making sense of the hands and mouth: the role of “secondary” cues to meaning in British Sign Language and English. Cognitive Science 44(7), e12868.
- Perniss, P., Lu, J., Morgan, G., & Vigliocco, G. (2017). Mapping language to the world: The role of iconicity in the sign language input. Developmental Science, e12551, 1-23.
- Perniss, P., Özyürek, A., & Morgan, G. (2015). The influence of the visual modality on language structure and language conventionalization: Insights from sign language and gesture. Topics in Cognitive Science,7(1), 2-11.
- Perniss, P., Thompson, R.L., & Vigliocco, G. (2010). Iconicity as a General Property of Language: Evidence from Spoken and Signed Languages. Frontiers in Psychology 1, 227.
Prof. Dr. Petra B. Schumacher
(Principal Investigator)
University of Cologne
petra.schumacher@uni-koeln.de
Petra B. Schumacher’s research focuses on discourse processing and interface phenomena, including anaphora resolution, information structure and experimental pragmatics. In her psycho- and neurolinguistic research, she has applied a wide variety of experimental methodologies and worked with different populations. She obtained her PhD from Yale University in 2004 and held positions at the Max Planck Institutes in Leipzig and Nijmegen as well as at the University of Marburg and Mainz. She is currently professor of empirical linguistics at the University of Cologne.
Selected publications
- Patterson, Clare & Petra B. Schumacher. (2021). Interpretation Preferences in Contexts with three Antecedents: Examining the Role of Prominence in German Pronouns. Applied Psycholinguistics, 1-35.
- Röhr, Christine T., Ingmar Brilmayer, Stefan Baumann, Martine Grice & Petra B. Schumacher. (2020). Signal-Driven and Expectation-Driven Processing of Accent Types. Language, Cognition and Neuroscience, 36,1, 33-59.
- Schumacher, Petra B. (2013). When Combinatorial Processing Results in Reconceptualization: Towards a New Approach of Compositionality. Frontiers in Psychology, 4, 677.
- Schumacher, Petra B. & Yu-Chen Hung. (2012). Positional Influences on Information Packaging: Insights from Topological Fields in German. Journal of Memory and Language, 67, 2, 295-310.
- Burkhardt, Petra. (2006). Inferential Bridging Relations Reveal Distinct Neural Mechanisms: Evidence from Event-Related Brain Potentials. Brain and Language, 98, 2, 159-168.
Door Spruijt
(PhD Candidate)
University of Cologne
dspruijt@uni-koeln.de
Door Spruijt’s research has focused on sign languages since her BA and MA in sign linguistics at the University of Amsterdam. She has worked extensively with both corpus and elicited data from Sign Language of the Netherlands (NGT) and the rural Balinese sign language Kata Kolok. She is now a PhD student on the Gesture-to-Sign Trajectory Project, which investigates the facilitating and inhibiting effects of iconicity and the individual phonological parameters on acquiring the lexicon in second language learners of a sign language (L2M2-learners). Understanding these effects will help improve curricula for L2M2-learners, which she, as a certified teacher of NGT, has great affinity with.
Selected publications
- Spruijt, D., P. Pamela, S. Petra B. (2023) The Contribution of Individual Parameters to Perceived Iconicity and Transparency in Gesture-Sign Pairs. FEAST. Formal and Experimental Advances in Sign Language Theory, Vol. 5, December 2023, p. 194-209, doi:10.31009/FEAST.i5.16
Project Description
The visual modality is well-known for its high potential for iconic representation, in particular, of information related to size and shape, manual action, location and motion. This affordance is exploited in both sign language and gesture and accounts to a significant degree for the similarity of expression across these domains in these modes of communicative expression. In the present project, we investigate this similarity with respect to the transition from gesture to sign in L2M2 (second language, second modality) sign language acquisition. The similarities between sign and gesture have important implications for L2M2 acquisition of a sign language, as learners’ gestural repertoire can support but also interfere with learning the phonologically specified forms in the established lexicon of sign language. In the proposed project, we investigate learners’ trajectories of transition from gesture to sign as well as their perception by proficient/native signers. To this end, we employ behavioral measures, a sign repetition task in a longitudinal study and electrophysiological measures. The project is integral to the Priority Program ViCom because it contributes to the subject by investigating the gesture-to-sign trajectory from a psycho- and neurolinguistic perspective, thereby offering a cognitively grounded approach to visual communication.
Project Activities
Publications
Bauer, A., Trettenbrein, P. C., Amici, F., Ćwiek, A., Fuchs, S., Krause, L., Kuder, A., Ladewig, S., Schulder, M., Schumacher, P., Spruijt, D., Zulberti, C. & Schulte-Rüther, M. (2025). Data Collection in Multimodal Language and Communication Research: A Flexible Decision Framework. https://doi.org/10.31234/osf.io/42tud_v1
Gregori, A., Amici, F., Brilmayer, I., Ćwiek, A., Fritzsche, L., Fuchs, S., Henlein, A., Herbort, O., Kügler, F., Lemanski, J., Liebal, K., Lücking, A., Mehler, A., Tien Nguyen, K., Pouw, W., Prieto, P., Rohrer, P., Sanchez-Ramon, P. G., Schulte-Rüther, M., Schumacher P. B., Schweinberger, S., Struckmeier, V., Trettenbrein P. C., & von Eiff, C. (2023). A roadmap for technological innovation in multimodal communication research. In: Vincent G. Duffy (Ed.) Digital Human Modeling and Applications in Health, Safety, Ergonomics and Risk Management. HCII 2023. Lecture Notes in Computer Science, Vol. 14029, 402-438. Heidelberg: Springer. https://doi.org/10.1007/978-3-031-35748-0_30.
Price, A., & Spruijt, D. (2023). FEAST 2023 Tagungsbericht. Das Zeichen, 37(121), 188–193.
Spruijt, D., Perniss, P., & Schumacher, P. B. (2023). The contribution of individual parameters to perceived iconicity and transparency in gesture-sign pairs. FEAST. Formal and Experimental Advances in Sign Language Theory, 5, 194–209. https://doi.org/10.31009/FEAST.i5.16
Spruijt, D., Schumacher, P. B., & Perniss, P. (in prep). A database for lexical signs of DGS (German Sign Language) and German predominant gestures: transparency scores, extensive iconicity ratings and formal gesture-sign overlap.
Trettenbrein, P. C., Bümmerstede, J., Finkbeiner, T. A., Gehlbach, P., Meister, N.-K., Schiefner, A., Schumacher, P., Spruijt, D., Steinbach, M., & Perniss, P. (2024). Towards a “DGS-LEX”: A Roadmap for the Collaborative Creation of a Psycholinguistic Database for German Sign Language (DGS). Linguistische Berichte, 281, 87-96.
Conference contributions
Perniss, P., & Schumacher, P. B. (2022). The Gesture-to-Sign Trajectory: Phonological Parameter in Production and Real-Time Comprehension [Short presentation]. ViCom Kickoff Meeting 2022, Bad Homburg, Germany.
Spruijt, D., Perniss, P., & Schumacher, P. B. (2023). Phonological features in the abstraction from gesture to sign [Poster]. From Icon to Abstraction, Birmingham, United Kingdom.
Spruijt, D., Perniss, P., & Schumacher, P. B. (2023). Building a database of Gesture-Sign Pairs for L2M2 phonological acquisition studies [Short presentation]. ViCom Annual Meeting 2023, Bad Homburg, Germany.
Spruijt, D., Perniss, P., & Schumacher, P. B. (2024). Inventorying L2M2 learners’ phonetic deviations (… before testing their phonological status) [Short presentation]. ViCom Annual Meeting 2024, Bad Homburg, Germany.
Kadavá, Š., & Spruijt, D. (2024). Gesture to Sign Trajectory: FLESH ft. GtST [Short presentation]. ViCom Annual Meeting 2024, Bad Homburg, Germany.
Spruijt, D., & Kadavá, Š. (2025). Obvious, but only in context: An exploration of meaning making in the visual modality [Stage presentation]. Dimensions of Iconicity in the Visual Modality, Göttingen, Germany
Workshop presentations
Perniss, P., & Schumacher, P. B. (2022). The Gesture-to-Sign Trajectory: Phonological Parameter in Production and Real-Time Comprehension [Short presentation]. ViCom Kickoff Meeting 2022, Bad Homburg, Germany.
Spruijt, D., Perniss, P., & Schumacher, P. B. (2023). Phonological features in the abstraction from gesture to sign [Poster]. From Icon to Abstraction, Birmingham, United Kingdom.
Spruijt, D., Perniss, P., & Schumacher, P. B. (2023). Building a database of Gesture-Sign Pairs for L2M2 phonological acquisition studies [Short presentation]. ViCom Annual Meeting 2023, Bad Homburg, Germany.
Spruijt, D., Perniss, P., & Schumacher, P. B. (2024). Inventorying L2M2 learners’ phonetic deviations (… before testing their phonological status) [Short presentation]. ViCom Annual Meeting 2024, Bad Homburg, Germany.
Kadavá, Š., & Spruijt, D. (2024). Gesture to Sign Trajectory: FLESH ft. GtST [Short presentation]. ViCom Annual Meeting 2024, Bad Homburg, Germany.
Spruijt, D., & Kadavá, Š. (2025). Obvious, but only in context: An exploration of meaning making in the visual modality [Stage presentation]. Dimensions of Iconicity in the Visual Modality, Göttingen, Germany
Events
| Date | Event | Organization |
| 09. – 10.10. 2024 | PALM UP in Gesture and Sign Symposium: Theoretical, Typological and Methodological Perspectives for Future Research | Sandra Debreslioska, Anastasia Bauer, Anna Kuder & and Pamela Perniss |
Short-term Collaborations
Exploration and comparison of mobile high-density electroencephalographic recordings during interactive communication in virtual reality and real-world settings. Alexander Mehler, Alexander Henlein, Andy Lücking, Petra B. Schumacher & Ingmar Brilmayer (2024)
The Gesture-to-Sign Trajectory. Šárka Kadavá & Door Spruijt (2024-2025)