Cookies Policy
X

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies.

I accept this policy

Find out more here

Full Access Predictive power in models of audiovisual integration of speech

No metrics data to plot.
The attempt to load metrics for this article has failed.
The attempt to plot a graph for these metrics has failed.
The full text of this article is not currently available.

Brill’s MyBook program is exclusively available on BrillOnline Books and Journals. Students and scholars affiliated with an institution that has purchased a Brill E-Book on the BrillOnline platform automatically have access to the MyBook option for the title(s) acquired by the Library. Brill MyBook is a print-on-demand paperback copy which is sold at a favorably uniform low price.

Predictive power in models of audiovisual integration of speech

  • HTML
  • PDF
Add to Favorites
You must be logged in to use this functionality

image of Seeing and Perceiving
For more content, see Multisensory Research and Spatial Vision.

Seeing the talking face can influence the phoneme perceived from the voice. This facilitates speech perception in the natural case where the face and voice are congruent and can cause the McGurk illusion when they are not. The classical example of the McGurk illusion is when acoustic /aba/ is perceived as /ada/ when dubbed onto a face articulating /aga/. In order to fully understand the underlying process of integrating information across the senses we need a computational account with predictive power. The Fuzzy Logical Model of Perception is one computational account of audiovisual integration in speech perception. Here we describe alternative accounts in which integration is based on an early continuous internal representation on which the phonetic classes fall. We show that these alternative accounts can provide just as good a fit when corrected for the number of free parameters. We also show, using cross-validation, that they have greater, but not great, predictive power. Finally, we show that introducing a regularization term can amend the lack of predictive power. With regularization, models based on continuous representations have the highest predictive power.

Affiliations: 1: Technical University of Denmark, DK

Seeing the talking face can influence the phoneme perceived from the voice. This facilitates speech perception in the natural case where the face and voice are congruent and can cause the McGurk illusion when they are not. The classical example of the McGurk illusion is when acoustic /aba/ is perceived as /ada/ when dubbed onto a face articulating /aga/. In order to fully understand the underlying process of integrating information across the senses we need a computational account with predictive power. The Fuzzy Logical Model of Perception is one computational account of audiovisual integration in speech perception. Here we describe alternative accounts in which integration is based on an early continuous internal representation on which the phonetic classes fall. We show that these alternative accounts can provide just as good a fit when corrected for the number of free parameters. We also show, using cross-validation, that they have greater, but not great, predictive power. Finally, we show that introducing a regularization term can amend the lack of predictive power. With regularization, models based on continuous representations have the highest predictive power.

Loading

Full text loading...

/deliver/18784763/25/0/18784763_025_00_S098_text.html;jsessionid=PcGb2NAQ2HONQ_Xqbou1d0JG.x-brill-live-03?itemId=/content/journals/10.1163/187847612x647379&mimeType=html&fmt=ahah
/content/journals/10.1163/187847612x647379
Loading

Data & Media loading...

http://brill.metastore.ingenta.com/content/journals/10.1163/187847612x647379
Loading
Loading

Article metrics loading...

/content/journals/10.1163/187847612x647379
2012-01-01
2016-12-07

Sign-in

Can't access your account?
  • Key

  • Full access
  • Open Access
  • Partial/No accessInformation