Emotion perception is multisensory and involves decoding and integrating input from both visual and auditory modalities. This dissertation investigates the influence of cultural and linguistic experience on multisensory perception of emotion. Experiment 1 examines the effect of cultural background on modality bias (i.e., the amount of influence a modality exerts when it is not the target modality) when perceiving multisensory emotions in one’s own culture. Participants from China showed a greater bias to the auditory modality, while participants from the U.S. showed a larger bias to the visual modality. While Experiment 1 demonstrates cultural differences in modality bias across East Asian and Western American cultures, Experiment 2 compares the same Chinese and American groups when they received emotional input associated with the two cultures. When the emotional input changed from their familiar culture to a less familiar culture, the Chinese group showed a reduced bias to the auditory channel, and both Chinese and American groups showed an increased bias to the visual channel. To examine how exposure to a second culture and language might influence modality bias during multisensory perception of emotion, Experiments 3, 4, and 5 test three groups of participants with various cultural and linguistic experience. In Experiment 3, Chinese-English bilinguals in the U.S. showed a reduced bias to the visual modality compared to native speakers in China when perceiving the input in the American culture. Furthermore, Chinese-English bilinguals demonstrated similar patterns of modality bias in Chinese and American cultures. In Experiment 4, Chinese heritage speakers who grew up in the U.S. demonstrated very similar modality bias patterns across their mainstream and heritage cultures. Compared to native English speakers, heritage speakers showed a reduced bias to the visual modality in their heritage culture. In Experiment 5, bicultural bilinguals who were born and raised in Singapore demonstrated a similar visual modality bias as native English speakers from American cultures, but they also showed an increased bias to the auditory modality under Chinese cultural context. This series of experiments reveal that the amount and specific type of cultural and linguistic exposure, and the informativeness of the emotional input interact together to influence the pattern of modality bias during multisensory perception of emotion. When the exposure to a new culture is limited and the emotional input is unfamiliar, the influence of culture on multisensory perception of emotion is minimal, and the more reliable visual modality shows a larger influence on emotion processing. As individuals increase their proficiency in a culture and language, their bias to the visual modality decreases. Overall, individuals maintain a consistent modality bias under different cultural and linguistic environments, and only show sensitivity to the culture of the input when their cultural and linguistic background is mixed and relatively balanced. Together, the five experiments demonstrate that multisensory emotion processing is influenced by cultural and linguistic experience, revealing an interactive perceptual system which adapts when the informativeness of the input changes.

Creator

• Chen, Peiyao

DOI

• https://doi.org/10.21985/n2-gnat-xz57

Subject

• Cognitive psychology

Language

• en

Alternate Identifier

• http://dissertations.umi.com/northwestern:14633
• etdadmin_upload_662275

Keyword

• cultures
• modality bias
• multisensory integration
• emotion perception

Date created

• 2019-01-01

Resource type

• Dissertation

Rights statement

• In Copyright