Cookies Policy
X

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies.

I accept this policy

Find out more here

Full Access Visual and auditory spatial signals in naturalistic environments: A computationally-based analysis of functional imaging data

No metrics data to plot.
The attempt to load metrics for this article has failed.
The attempt to plot a graph for these metrics has failed.
The full text of this article is not currently available.

Brill’s MyBook program is exclusively available on BrillOnline Books and Journals. Students and scholars affiliated with an institution that has purchased a Brill E-Book on the BrillOnline platform automatically have access to the MyBook option for the title(s) acquired by the Library. Brill MyBook is a print-on-demand paperback copy which is sold at a favorably uniform low price.

Visual and auditory spatial signals in naturalistic environments: A computationally-based analysis of functional imaging data

  • HTML
  • PDF
Add to Favorites
You must be logged in to use this functionality

image of Multisensory Research
For more content, see Seeing and Perceiving and Spatial Vision.

A major challenge for fMRI studies is to use realistic stimuli, which is relevant for the understanding of multisensory interactions in the real world. Computational models (Itti et al., 1998; Kayser et al., 2005) have been used to track brain activity during viewing of complex audio-visual stimuli (Bartels et al., 2007; Bordier et al., 2013). We extended this approach to investigate activity associated with high-order spatial aspects in both vision and audition. We utilized a 3D-surround movie that included visual disparity cues and multiple sound sources (centre, front left/right, back left/right). For each visual frame, we computed a disparity map (Liu et al., 2011) and indexed absolute disparity (sum over the entire map) and gradient disparity (local contrast, Bordier et al., 2013). For audition, we indexed sound-spatiality by computing the correlation between each of the 5 external channels and a sixth channel that was delivered over headphones during fMRI (this contained primarily the ‘centre’ sound). We also indexed the sound intensity contrast (Bordier et al., 2013) to control for mere intensity changes over time. These indexes were used to fit the BOLD signal in 16 subjects, who watched the 3D-surround movie during fMRI. The results showed a dissociation between absolute disparity (PPC and V3A) and gradient disparity (V6, STS and IFG). The auditory–spatiality index correlated with activity in auditory cortex, even after accounting for changes of auditory-intensity. We conclude that the combination of computational models and fMRI enables studying brain activity associated with complex spatial signals in naturalistic multisensory environments.

Affiliations: 1: Santa Lucia Foundation, Italy

A major challenge for fMRI studies is to use realistic stimuli, which is relevant for the understanding of multisensory interactions in the real world. Computational models (Itti et al., 1998; Kayser et al., 2005) have been used to track brain activity during viewing of complex audio-visual stimuli (Bartels et al., 2007; Bordier et al., 2013). We extended this approach to investigate activity associated with high-order spatial aspects in both vision and audition. We utilized a 3D-surround movie that included visual disparity cues and multiple sound sources (centre, front left/right, back left/right). For each visual frame, we computed a disparity map (Liu et al., 2011) and indexed absolute disparity (sum over the entire map) and gradient disparity (local contrast, Bordier et al., 2013). For audition, we indexed sound-spatiality by computing the correlation between each of the 5 external channels and a sixth channel that was delivered over headphones during fMRI (this contained primarily the ‘centre’ sound). We also indexed the sound intensity contrast (Bordier et al., 2013) to control for mere intensity changes over time. These indexes were used to fit the BOLD signal in 16 subjects, who watched the 3D-surround movie during fMRI. The results showed a dissociation between absolute disparity (PPC and V3A) and gradient disparity (V6, STS and IFG). The auditory–spatiality index correlated with activity in auditory cortex, even after accounting for changes of auditory-intensity. We conclude that the combination of computational models and fMRI enables studying brain activity associated with complex spatial signals in naturalistic multisensory environments.

Loading

Full text loading...

/deliver/22134808/26/10/22134808_026_00_S88_text.html?itemId=/content/journals/10.1163/22134808-000s0088&mimeType=html&fmt=ahah
/content/journals/10.1163/22134808-000s0088
Loading

Data & Media loading...

http://brill.metastore.ingenta.com/content/journals/10.1163/22134808-000s0088
Loading
Loading

Article metrics loading...

/content/journals/10.1163/22134808-000s0088
2013-05-16
2017-04-26

Sign-in

Can't access your account?
  • Key

  • Full access
  • Open Access
  • Partial/No accessInformation