Cookies Policy
X

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies.

I accept this policy

Find out more here

Full Access Regularization improves models of audiovisual integration in speech perception

No metrics data to plot.
The attempt to load metrics for this article has failed.
The attempt to plot a graph for these metrics has failed.
The full text of this article is not currently available.

Brill’s MyBook program is exclusively available on BrillOnline Books and Journals. Students and scholars affiliated with an institution that has purchased a Brill E-Book on the BrillOnline platform automatically have access to the MyBook option for the title(s) acquired by the Library. Brill MyBook is a print-on-demand paperback copy which is sold at a favorably uniform low price.

Regularization improves models of audiovisual integration in speech perception

  • PDF
  • HTML
Add to Favorites
You must be logged in to use this functionality

image of Multisensory Research
For more content, see Seeing and Perceiving and Spatial Vision.

Visual speech, the speech information mediated by the sight of articulatory mouth movements, can influence the auditory phonetic speech percept. This is demonstrated by the McGurk illusion, where an acoustic speech signal (e.g., /ba/) is perceived differently (as /da/) when perceived audio-visually, dubbed onto the video of an incongruent talking face (articulating /ga/). A computational account of the integration of information across the senses underlying the McGurk illusion has long been sought. One account, the Fuzzy Logical Model of Perception (FLMP, Massaro, 1998) posits that integration is based on fuzzy truth-values. Here we present an alternative accounts in which integration is based on continuous feature values. We show that such models can provide a better fit to observed data than the FLMP. In order to take model-flexibility into account, we cross-validate model fits and show that although feature based models have more predictive power than the FLMP, both types of models perform rather poorly. Finally, we show that the predictive power of both types of models improve when the models are regularized by Bayesian priors and that after regularization, feature based models have significantly better predictive power than the FLMP.

Affiliations: 1: Technical University of Denmark, Denmark

Visual speech, the speech information mediated by the sight of articulatory mouth movements, can influence the auditory phonetic speech percept. This is demonstrated by the McGurk illusion, where an acoustic speech signal (e.g., /ba/) is perceived differently (as /da/) when perceived audio-visually, dubbed onto the video of an incongruent talking face (articulating /ga/). A computational account of the integration of information across the senses underlying the McGurk illusion has long been sought. One account, the Fuzzy Logical Model of Perception (FLMP, Massaro, 1998) posits that integration is based on fuzzy truth-values. Here we present an alternative accounts in which integration is based on continuous feature values. We show that such models can provide a better fit to observed data than the FLMP. In order to take model-flexibility into account, we cross-validate model fits and show that although feature based models have more predictive power than the FLMP, both types of models perform rather poorly. Finally, we show that the predictive power of both types of models improve when the models are regularized by Bayesian priors and that after regularization, feature based models have significantly better predictive power than the FLMP.

Loading

Full text loading...

/deliver/22134808/26/10/22134808_026_00_S39_text.html;jsessionid=Eiph6j2NiaDveD12IS0BUoyk.x-brill-live-03?itemId=/content/journals/10.1163/22134808-000s0039&mimeType=html&fmt=ahah
/content/journals/10.1163/22134808-000s0039
Loading

Data & Media loading...

http://brill.metastore.ingenta.com/content/journals/10.1163/22134808-000s0039
Loading
Loading

Article metrics loading...

/content/journals/10.1163/22134808-000s0039
2013-05-16
2016-12-07

Sign-in

Can't access your account?
  • Key

  • Full access
  • Open Access
  • Partial/No accessInformation