Cookies Policy
X

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies.

I accept this policy

Find out more here

Investigating the In-Between: Multisensory Integration of Auditory and Visual Motion Streams

No metrics data to plot.
The attempt to load metrics for this article has failed.
The attempt to plot a graph for these metrics has failed.
The full text of this article is not currently available.

Brill’s MyBook program is exclusively available on BrillOnline Books and Journals. Students and scholars affiliated with an institution that has purchased a Brill E-Book on the BrillOnline platform automatically have access to the MyBook option for the title(s) acquired by the Library. Brill MyBook is a print-on-demand paperback copy which is sold at a favorably uniform low price.

Access this article

+ Tax (if applicable)
Add to Favorites
You must be logged in to use this functionality

image of Seeing and Perceiving
For more content, see Multisensory Research and Spatial Vision.

We investigated audiovisual interactions in motion perception by behavioral experiments testing both, the influence of visual stimuli on auditory apparent motion and the influence of auditory stimuli on visual apparent motion perception. A set of loudspeakers with an LED mounted in the middle of each speaker cone was arranged in a semicircle. Apparent motion streams were presented for each modality alone in the unimodal conditions. In the bimodal conditions, stimuli of the second modality were added to fill the temporal and spatial gaps of the sampled trajectory of the reference stream. The participants’ task was to observe the quasi-naturalistic stimulus sequences and to perform a standard classification. The addition of stimuli of the second modality indeed facilitated apparent motion perception. Bimodal presentation increased the upper temporal interval up to which the stimuli could be separated in time while still being perceived as continuous motion. We interpret these results as evidence for an ecologically advantageous audiovisual motion integration mechanism which operates beyond the constraints of strict spatiotemporal coincidence. Functional considerations suggest that this mechanism may represent an amodal stage suited for the processing of both unimodal and bimodal signals.

Affiliations: 1: Institute of Brain Research - Human Neurobiology, Bremen University, Hochschulring 18, Bremen, Germany; 2: Cognitive Neuroinformatics, Bremen University, Enrique-Schmidt-Straße 5, Bremen, Germany

Loading

Full text loading...

/content/journals/10.1163/187847611x620919
Loading

Data & Media loading...

1. Alain C. , Arnott S. R. , Hevenor S. , Graham S. , Grady C. L. ( 2001). "‘What’ and ‘where’ in the human auditory system", Proc. Natl. Acad. Sci. USA Vol 98, 1230112306. http://dx.doi.org/10.1073/pnas.211209098
2. Alais D. , Burr D. ( 2004). "The ventriloquist effect results from near-optimal bimodal integration", Curr. Biol. Vol 14, 257262. http://dx.doi.org/10.1016/j.cub.2004.01.029
3. Allen P. G. , Kolers P. A. ( 1981). "Sensory specificy of apparent motion", J. Exp. Psychol. Hum. Percept. Perform. Vol 7, 13101328. http://dx.doi.org/10.1037/0096-1523.7.6.1318
4. Blauert J. ( 1996). Spatial Hearing — Revised Edition: The Psychophysics of Human Sound Localization. Bradford/The MIT Press, Cambridge, MA.
5. Briggs R. , Perrott D. R. ( 1972). "Auditory apparent movement under dichotic listening conditions", J. Exp. Psychol. Vol 92, 8391. http://dx.doi.org/10.1037/h0032148
6. Bruns P. , Getzmann S. ( 2008). "Audiovisual influences on the perception of visual apparent motion: exploring the effect of a single sound", Acta Psychol. Vol 129, 273283. http://dx.doi.org/10.1016/j.actpsy.2008.08.002
7. Burtt H. E. ( 1917a). "Auditory illusions of movement — a preliminary study", J. Exp. Psychol. Vol 2, 6375. http://dx.doi.org/10.1037/h0070778
8. Burtt H. E. ( 1917b). "Tactual illusions of movement", J. Exp. Psychol. Vol 2, 371385. http://dx.doi.org/10.1037/h0074614
9. Cappe C. , Thut G. , Romei V. , Murray M. M. ( 2009). "Selective integration of auditory–visual looming cues by humans", Neuropsychologia Vol 47, 10451052. http://dx.doi.org/10.1016/j.neuropsychologia.2008.11.003
10. Carriere B. N. , Royal D. W. , Wallace M. T. ( 2008). "Spatial heterogenity of cortical receptive fields and its impact on multisensory interactions", J. Neurophysiol. Vol 99, 23572368. http://dx.doi.org/10.1152/jn.01386.2007
11. Fahle M. , Biester A. , Morrone C. ( 2001). "Spatiotemporal interpolation and quality of apparent motion", J. Opt. Soc. Amer. Vol 18, 26682678. http://dx.doi.org/10.1364/JOSAA.18.002668
12. Fahle M. , Poggio T. ( 1981). "Visual hyperacuity: spatiotemporal interpolation in human vision", Proc. Roy. Soc. London B Vol 313, 451477. http://dx.doi.org/10.1098/rspb.1981.0075
13. Fastl H. , Zwicker E. ( 2007). Psychoacoustics: Facts and Models, 3rd edn. Springer, Berlin. http://dx.doi.org/10.1007/978-3-540-68888-4
14. Getzmann S. ( 2007). "The effect of brief auditory stimuli on visual apparent motion", Perception Vol 36, 10891103. http://dx.doi.org/10.1068/p5741
15. Harrar V. , Winter R. , Harris L. R. ( 2008). "Visuotactile apparent motion", Perception Vol 70, 807817.
16. Harrison N. R. , Wuerger S. M. , Meyer G. F. ( 2010). "Reaction time facilitation for horizontally moving auditory–visual stimuli", J. Vision Vol 10, 121.
17. Hofbauer M. , Wuerger S. M. , Meyer G. F. , Roehrbein F. , Schill K. , Zetzsche C. ( 2004). "Catching audiovisual mice: predicting the arrival time of auditory–visual motion signals", Cogn. Affect. Behav. Neurosci. Vol 4, 241250. http://dx.doi.org/10.3758/CABN.4.2.241
18. Kim H. , Pasieka B. , McCourt M. ( 2003). "Auditory ‘capture’ of visual motion (Abstract)", J. Vision Vol 3, 777. http://dx.doi.org/10.1167/3.9.777
19. Kolers P. A. ( 1972). Aspects of Motion Perception. Pergamon, Oxford, UK.
20. Korte A. ( 1915). "Kinematoskopische Untersuchungen", Zeit. Psychol. Vol 72, 193296.
21. Lakatos S. , Shepard R. N. ( 1997). "Constraints common to apparent motion in visual, tactile, and auditory space", J. Exp. Psychol. Hum. Percept. Perform. Vol 23, 10501060. http://dx.doi.org/10.1037/0096-1523.23.4.1050
22. Mateeff S. , Hohnsbein J. , Noack T. ( 1985). "Dynamic visual capture: apparent auditory motion induced by a moving visual target", Perception Vol 14, 721727. http://dx.doi.org/10.1068/p140721
23. Meredith M. A. , Nemitz J. W. , Stein B. E. ( 1987). "Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors", J. Neurosci. Vol 7, 32153229.
24. Meredith M. A. , Stein B. E. ( 1983). "Interactions among converging sensory inputs in the superior colliculus", Science Vol 221, 389391. http://dx.doi.org/10.1126/science.6867718
25. Meredith M. A. , Stein B. E. ( 1986). "Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration", J. Neurophysiol. Vol 56, 640662.
26. Meyer G. F. , Wuerger S. M. ( 2001). "Cross-modal integration of auditory and visual motion signals", NeuroReport Vol 12, 25572560. http://dx.doi.org/10.1097/00001756-200108080-00053
27. Meyer G. F. , Wuerger S. M. , Röhrbein F. , Zetzsche C. ( 2005). "Low-level integration of auditory and visual motion signals requires spatial co-localisation", Exp. Brain Res. Vol 166, 538547. http://dx.doi.org/10.1007/s00221-005-2394-7
28. Mullette-Gillman O. A. , Cohen Y. E. , Groh J. M. ( 2005). "Eye-centered, head-centered, and complex coding of visual and auditory targets in the intraparietal sulcus", J. Neurophysiol. Vol 94, 23312352. http://dx.doi.org/10.1152/jn.00021.2005
29. Perrot D. R. ( 1974). "Auditory apparent movement", J. Audit. Res. Vol 14, 163169.
30. Perrott D. R. , Musicant A. D. ( 1977). "Minimum auditory movement angle: binaural localization of moving sound sources", J. Acoust. Soc. Amer. Vol 62, 14631466. http://dx.doi.org/10.1121/1.381675
31. Rauschecker J. P. , Tian B. ( 2000). "Mechanisms and streams for processing of ‘what’ and ‘where’ in auditory cortex", Proc. Natl. Acad. Sci. USA Vol 97, 1180011806. http://dx.doi.org/10.1073/pnas.97.22.11800
32. Soto-Faraco S. , Kingstone A. ( 2004). "Multisensory integration of dynamic information", in: Handbook of Multisensory Processes, Calvert C. , Spence C. , Stein B. E. (Eds), pp.  4967. Bradford/The MIT Press, Cambridge, MA.
33. Soto-Faraco S. , Kingstone A. , Spence C. ( 2003). "Multisensory contributions to the perception of motion", Neuropsychologia Vol 41, 18471862. http://dx.doi.org/10.1016/S0028-3932(03)00185-4
34. Soto-Faraco S. , Lyons J. , Gazzaniga M. , Spence C. , Kingstone A. ( 2002). "The ventriloquist in motion: illusory capture of dynamic information across sensory modalities", Cognit. Brain Res. Vol 14, 139146. http://dx.doi.org/10.1016/S0926-6410(02)00068-X
35. Stein B. E. , Meredith M. A. ( 1993). The Merging of the Senses. Bradford/The MIT Press, Cambridge, MA.
36. Stricanne B. , Andersen R. A. , Mazzoni P. ( 1996). "Eye-centered, head-centered, and intermediate coding of remembered sound locations in area LIP", J. Neurophysiol. Vol 76, 20712076.
37. Strybel T. Z. , Manligas C. L. , Chan O. , Perrott D. R. ( 1990). "A comparison of the effects of spatial separation on apparent motion in the auditory and visual modalities", Percept. Psychophys. Vol 47, 439448. http://dx.doi.org/10.3758/BF03208177
38. Strybel T. Z. , Manligas C. L. , Perrott D. R. ( 1989). "Auditory apparent motion under binaural and monaural listening conditions", Percept. Psychophys. Vol 45, 371377. http://dx.doi.org/10.3758/BF03204951
39. Strybel T. Z. , Menges M. L. ( 1998). "Auditory apparent motion between sine waves differing in frequency", Perception Vol 27, 483495. http://dx.doi.org/10.1068/p270483
40. Strybel T. Z. , Neale W. ( 1994). "The effect of burst duration, interstimulus onset interval and loudspeaker arrangement on auditory apparent motion in the free field", J. Acoust. Soc. Amer. Vol 96, 34633475. http://dx.doi.org/10.1121/1.411452
41. Strybel T. Z. , Vatakis A. ( 2004). "A comparison of auditory and visual apparent motion presented individually and with crossmodal moving distractors", Perception Vol 33, 10331048. http://dx.doi.org/10.1068/p5255
42. Strybel T. Z. , Witty A. M. , Perrott D. R. ( 1992). "Auditory apparent motion in the free field: the effects of stimulus duration and separation", Percept. Psychophys. Vol 52, 139143. http://dx.doi.org/10.3758/BF03206766
43. Tian B. , Reser D. , Durham A. , Kustov A. , Rauschecker J. P. ( 2001). "Functional specialization in rhesus monkey auditory cortex", Science Vol 292, 290293. http://dx.doi.org/10.1126/science.1058911
44. Wertheimer M. ( 1912). "Experimentelle Studien über das Sehen von Bewegung", Zeit. Psychol. Vol 61, 161265.
45. Wuerger S. , Meyer G. , Hofbauer M. , Zetzsche C. , Schill K. ( 2010). "Motion extrapolation of auditory–visual targets", Information Fusion Vol 11, 4550. http://dx.doi.org/10.1016/j.inffus.2009.04.005
46. Zapparoli G. C. , Reatto L. L. ( 1969). "The apparent movement between visual and acoustic stimulus and the problem of intermodal relations", Acta Psychol. Vol 29, 256267. http://dx.doi.org/10.1016/0001-6918(69)90019-5
http://brill.metastore.ingenta.com/content/journals/10.1163/187847611x620919
Loading

Article metrics loading...

/content/journals/10.1163/187847611x620919
2012-01-01
2016-12-09

Sign-in

Can't access your account?
  • Tools

  • Add to Favorites
  • Printable version
  • Email this page
  • Subscribe to ToC alert
  • Get permissions
  • Recommend to your library

    You must fill out fields marked with: *

    Librarian details
    Your details
    Why are you recommending this title?
    Select reason:
     
    Seeing and Perceiving — Recommend this title to your library
  • Export citations
  • Key

  • Full access
  • Open Access
  • Partial/No accessInformation