Skip to Main Content
 

Global Search Box

 
 
 
 

Files

File List


A study on audio-visual interaction.pdf (2.38 MB)

ETD Abstract Container

Abstract Header

A study on audio-visual interaction: How visual temporal cues influence grouping of auditory events

Fan, Yu
http://orcid.org/0000-0001-6512-8943
http://rave.ohiolink.edu/etdc/view?acc_num=osu1544732438004088

Abstract Details

2019, Doctor of Philosophy, Ohio State University, Music.
Music provides us a complex, rapidly changing acoustic spectrum, and when presented with such stimuli, the human auditory system spontaneously groups acoustic elements together. Real-world experience of rhythm is often multisensory, involving both auditory and visual systems, such as when we observe drumming. However, the manner in which auditory grouping behavior is influenced by concurrent temporal cues from visual stimuli is not fully known yet. This study investigates audio-visual interaction in rhythm perception. The central questions addressed are whether visual rhythm influences human’s rhythmic grouping of auditory event sequences, and whether musical training intensifies or weakens this influence. Two experiments were conducted. The preliminary experiment was developed to identify the visual stimuli, natural versus artificial, that would be appropriate to the present inquiry. Participants were recruited for a synchronized tapping test to natural visual stimuli and artificial visual stimuli. The stability of the synchronization and the rate of re-synchronization were measured in a synchronization task. The results show that participants’ synchronization performance of temporal perception was better with natural visual stimuli, which justified the use of a natural visual stimulus as the rhythmic stimulus in the main study on temporal perception. The main study testified the audio-visual interaction through a grouping task. Trained musicians and non-musicians were recruited. In the experiment, participants were asked to perform a grouping task through tapping to a cyclical 12-tone drumbeat sequence with different stress patterns that indicated different grouping structure tendencies (either duplets or triplets). While listening, they were asked to simultaneously watch visual stimuli 1) with duple cueing gestures; 2) with triple cueing gestures; or 3) without any cue. The results show that subjects’ duple/triple grouping choice is significantly related to the concurrent visual stimuli with duple/triple cues, and that visual rhythm had a much greater influence on musicians’ auditory grouping behavior than on that of non-musicians. The result of the first experiment, which incorporated novel and realistic visual stimuli in a rhythm perception task, bolsters the rationale of using closer-to-reality experimental designs. The result of the main study demonstrated and highlighted the role of visual sense in multisensory rhythm perception. Moreover, the current study also found that the audio-visual interaction in rhythmic grouping is correlated to long-term music training, and this finding contributes to the body of knowledge on the process of rhythmic perception.
Udo Will (Advisor)
Arved Ashby (Committee Member)
Marjorie K. M. Chan (Committee Member)
177 p.
Audiology; Behavioral Sciences; Cognitive Psychology; Experiments; Music

Recommended Citations

Citations

  • Fan, Y. (2019). A study on audio-visual interaction: How visual temporal cues influence grouping of auditory events [Doctoral dissertation, Ohio State University]. OhioLINK Electronic Theses and Dissertations Center. http://rave.ohiolink.edu/etdc/view?acc_num=osu1544732438004088

    APA Style (7th edition)

  • Fan, Yu. A study on audio-visual interaction: How visual temporal cues influence grouping of auditory events. 2019. Ohio State University, Doctoral dissertation. OhioLINK Electronic Theses and Dissertations Center, http://rave.ohiolink.edu/etdc/view?acc_num=osu1544732438004088.

    MLA Style (8th edition)

  • Fan, Yu. "A study on audio-visual interaction: How visual temporal cues influence grouping of auditory events." Doctoral dissertation, Ohio State University, 2019. http://rave.ohiolink.edu/etdc/view?acc_num=osu1544732438004088

    Chicago Manual of Style (17th edition)

osu1544732438004088
923
© 2019, all rights reserved.
This open access ETD is published by The Ohio State University and OhioLINK.