Cookies Policy

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies.

I accept this policy

Find out more here

The Efficacy of Single-Trial Multisensory Memories

No metrics data to plot.
The attempt to load metrics for this article has failed.
The attempt to plot a graph for these metrics has failed.
The full text of this article is not currently available.

Brill’s MyBook program is exclusively available on BrillOnline Books and Journals. Students and scholars affiliated with an institution that has purchased a Brill E-Book on the BrillOnline platform automatically have access to the MyBook option for the title(s) acquired by the Library. Brill MyBook is a print-on-demand paperback copy which is sold at a favorably uniform low price.

Access this article

+ Tax (if applicable)
Add to Favorites
You must be logged in to use this functionality

image of Multisensory Research
For more content, see Seeing and Perceiving and Spatial Vision.

This review article summarizes evidence that multisensory experiences at one point in time have long-lasting effects on subsequent unisensory visual and auditory object recognition. The efficacy of single-trial exposure to task-irrelevant multisensory events is its ability to modulate memory performance and brain activity to unisensory components of these events presented later in time. Object recognition (either visual or auditory) is enhanced if the initial multisensory experience had been semantically congruent and can be impaired if this multisensory pairing was either semantically incongruent or entailed meaningless information in the task-irrelevant modality, when compared to objects encountered exclusively in a unisensory context. Processes active during encoding cannot straightforwardly explain these effects; performance on all initial presentations was indistinguishable despite leading to opposing effects with stimulus repetitions. Brain responses to unisensory stimulus repetitions differ during early processing stages (∼100 ms post-stimulus onset) according to whether or not they had been initially paired in a multisensory context. Plus, the network exhibiting differential responses varies according to whether or not memory performance is enhanced or impaired. The collective findings we review indicate that multisensory associations formed via single-trial learning exert influences on later unisensory processing to promote distinct object representations that manifest as differentiable brain networks whose activity is correlated with memory performance. These influences occur incidentally, despite many intervening stimuli, and are distinguishable from the encoding/learning processes during the formation of the multisensory associations. The consequences of multisensory interactions thus persist over time to impact memory retrieval and object discrimination.

Affiliations: 1: 1The Functional Electrical Neuroimaging Laboratory, Neuropsychology and Neurorehabilitation Service, Department of Clinical Neurosciences, Centre Hospitalier Universitaire Vaudois and University of Lausanne, Switzerland


Full text loading...


Data & Media loading...

1. Amedi A. , von Kriegstein K. , van Atteveldt N. M. , Beauchamp M. S. , Naumer M. J. ( 2005). "Functional imaging of human crossmodal identification and object reocognition", Exp. Brain Res. Vol 166, 559571. [Crossref]
2. Beer A. L. , Plank T. , Meyer G. , Greenlee M. W. ( 2013). "Combined diffusion-weighted and functional magnetic resonance imaging reveals a temporal-occipital network involved in auditory–visual object processing", Front. Integr. Neurosci. Vol 7, 5. [Crossref]
3. Butler A. J. , James K. H. ( 2011). "Cross-modal versuswithin-modal recall: Differences in behavioral and brain responses", Behav. Brain Res. Vol 224, 387396.
4. Cappe C. , Rouiller E. M. , Barone P. ( 2009). "Multisensory anatomical pathways", Hearing Res. Vol 258, 2836. [Crossref]
5. Cappe C. , Thut G. , Romei V. , Murray M. M. ( 2010). "Auditory–visual multisensory interactions in humans: timing, topography, directionality, and sources", J. Neurosci. Vol 30, 1257212580. [Crossref]
6. Cappe C. , Thelen A. , Romei V. , Thut G. , Murray M. M. ( 2012). "Looming signals reveal synergistic principles of multisensory interactions", J. Neurosci. Vol 32, 11711182. [Crossref]
7. Chen Y. C. , Yeh S. L. ( 2008). "Visual events modulated by sound in repetition blindness", Psychon. B. Rev. Vol 15, 404408. [Crossref]
8. Colombo M. , Gross C. G. ( 1994). "Responses of inferior temporal cortex and hippocampal neurons during delayed matching to sample in monkeys ( Macaca fascicularis)", Behav. Neurosci. Vol 108, 443455. [Crossref]
9. Ghazanfar A. A. , Schroeder C. E. ( 2006). "Is neocortex essentially multisensory?" Tr. Cogn. Sci. Vol 10, 278285. [Crossref]
10. Gibson J. R. , Maunsell J. H. R. ( 1997). "Sensory modality specificity of neural activity related to memory in visual cortex", J. Neurophysiol. Vol 78, 12631275.
11. Gottfried J. A. , Smith A. P. R. , Rugg M. D. , Dolan R. J. ( 2004). "Remembrance of odors past: human olfactory cortex in cross-modal recognition memory", Neuron Vol 42, 687695. [Crossref]
12. Guo J. , Guo A. ( 2005). "Crossmodal interactions between olfactory and visual learning in Drosophila ", Science Vol 309, 307310. [Crossref]
13. Haenny P. E. , Maunsell J. H. R. , Schiler P. H. ( 1988). "State dependent activity in monkey visual cortex: II. Retinal and extraretinal factors in V4", Exp. Brain Res. Vol 69, 245259. [Crossref]
14. Hamilton W. ( 1859). Lectures on Metaphysics and Logic. Gould & Lincoln, Boston, MA, USA. [Crossref]
15. James T. W. , Humphrey G. K. , Gati J. S. , Servos P. , Menon R. S. , Goodale M. A. ( 2002). "Haptic study of three-dimensional objects activates extrastriate visual areas", Neuropsychologia Vol 40, 17061714. [Crossref]
16. Johansson B. B. ( 2012). "Multisensory stimulation in stroke rehabilitation", Front. Hum. Neurosci. Vol 6, 60. [Crossref]
17. Lehmann S. , Murray M. M. ( 2005). "The role of multisensory memories in unisensory object discrimination", Cogn. Brain Res. Vol 24, 326334. [Crossref]
18. Maunsell J. H. R. , Sclar G. , Nealey T. A. , Depriest D. D. ( 1991). "Extraretinal representations in area V4 in the macaque monkey", Visual Neurosci. Vol 7, 561573. [Crossref]
19. Meylan R. V. , Murray M. M. ( 2007). "Auditory–visual multisensory interactions attenuate subsequent visual responses in humans", Neuroimage Vol 35, 244254. [Crossref]
20. Michel C. M. , Murray M. M. ( 2012). "Towards the utilization of EEG as a brain imaging tool", Neuroimage Vol 61, 371385. [Crossref]
21. Michel C. M. , Murray M. M. , Lantz G. , Gonzalez S. , Spinelli L. , Grave de Peralta R. ( 2004). "EEG source imaging", Clin. Neurophysiol. Vol 115, 21952222. [Crossref]
22. Murray E. A. , Bussey T. J. ( 1999). "Perceptual–mnemonic functions of the perirhinal cortex", Tr. Cogn. Sci. Vol 3, 142151. [Crossref]
23. Murray E. A. , Gaffan D. ( 1994). "Removal of the amygdala plus subjacent cortex disrupts the retention of both intramodal and crossmodal associative memories in monkeys", Behav. Neurosci. Vol 108, 494500. [Crossref]
24. Murray M. M. , Cappe C. , Romei V. , Martuzzi R. , Thut G. ( 2012). "Auditory–visual multisensory interactions in humans: a synthesis of findings from behavior, ERPs, fMRI, and TMS", in: The New Handbook of Multisensory Processes, Stein B. E. (Ed.), pp.  223238. MIT Press, Cambridge, MA, USA.
25. Murray M. M. , Sperdin H. F. ( 2010). "Single-trial multisensory learning and memory retrieval", in: Multisensory Object Perception in the Primate Brain, Kaiser J. , Naumer M. J. (Eds), pp.  191208. Springer, Heidelberg, Germany. [Crossref]
26. Murray M. M. , Brunet D. , Michel C. M. ( 2008). "Topographic ERP analyses: a step-by-step tutorial review", Brain Topogr. Vol 20, 249264. [Crossref]
27. Murray M. M. , Foxe J. J. , Wylie G. R. ( 2005). "The brain uses single-trial multisensory memories to discriminate without awareness", Neuroimage Vol 27, 473478. [Crossref]
28. Murray M. M. , Michel C. M. , Grave de Peralta R. , Ortigue S. , Brunet D. , Andino S. G. , Schnider A. ( 2004). "Rapid discrimination of visual and multisensory memories revealed by electrical neuroimaging", Neuroimage Vol 21, 125135. [Crossref]
29. Naci L. , Taylor K. I. , Cusack R. , Tyler L. K. ( 2012). "Are the senses enough for sense? Early high-level feedback shapes our comprehension of multisensory objects", Front. Integr. Neurosci. Vol 6, 82. [Crossref]
30. Naue N. , Rach S. , Struber D. , Huster R. J. , Zaehle T. , Korner U. , Herrmann C. S. ( 2011). "Auditory event-related response in visual cortex modulates subsequent visual responses in humans", J. Neurosci. Vol 31, 77297736. [Crossref]
31. Naumer M. J. , Doehrmann O. , Müller N. G. , Muckli L. , Kaiser J. , Hein G. ( 2009). "Cortical plasticity of audio-visual object representations", Cereb. Cortex Vol 19, 16411653. [Crossref]
32. Nyberg L. , Habib R. , McIntosh A. R. , Tulving E. ( 2000). "Reactivation of encoding-related brain activity during memory retrieval", Proc. Natl Acad. Sci. USA Vol 97, 1112011124. [Crossref]
33. Ranganath C. , Rainer G. ( 2003). "Neural mechanisms for detecting and remembering novel events", Nat. Rev. Neurosci. Vol 4, 193202. [Crossref]
34. Shams L. , Wozny D. R. , Kim R. , Seitz A. ( 2011). "Influences of multisensory experience on subsequent unisensory processing", Front. Psychol. Vol 2, 264. [Crossref]
35. Shams L. , Seitz A. R. ( 2008). "Benefits of multisensory learning", Tr. Cogn. Sci. Vol 12, 411417. [Crossref]
36. Spence C. , Nicholls M. E. , Driver J. ( 2001). "The cost of expecting events in the wrong sensory modality", Percept. Psychophys. Vol 63, 330336. [Crossref]
37. Tanabe H. C. , Honda M. , Sadato N. ( 2005). "Functionally segregated neural substrates for arbitrary audiovisual paired-association learning", J. Neurosci. Vol 25, 64096418. [Crossref]
38. Taylor K. I. , Moss H. E. , Stamatakis E. A. , Tyler L. K. ( 2006). "Binding crossmodal object features in perirhinal cortex", Proc. Natl Acad. Sci. USA Vol 103, 82398244. [Crossref]
39. Taylor K. I. , Stamatakis E. A. , Tyler L. K. ( 2009). "Crossmodal integration of object features: voxel-based correlations in brain-damaged patients", Brain Vol 132, 671683. [Crossref]
40. Thelen A. , Cappe C. , Murray M. M. ( 2012). "Electrical neuroimaging of memory discrimination based on single-trial multisensory learning", Neuroimage Vol 62, 14781488. [Crossref]
41. Thelen A. , Murray M. M. ( 2013). Predicting individual differences in the impact of multisensory, single-trial exposure upon subsequent object recognition. Neuroscience 2013 Abstracts, Program No. 765.15483. Society for Neuroscience, San Diego, California, USA.
42. Thelen A. , Talsma D. , Murray M. M. ( submitted). "The efficacy of single-trial multisensory memories for visual and auditory object recognition", J. Cognitive Neurosci.
43. Tsivilis D. , Otten L. J. , Rugg M. D. ( 2001). "Context effects on the neural correlates of recognition memory: an electrophysiological study", Neuron Vol 31, 497505. [Crossref]
44. Tzovara A. , Murray M. M. , Plomp G. , Herzog M. , Michel C. M. , De Lucia M. ( 2012a). "Decoding stimulus-related information from single-trial EEG responses based on voltage topographies", Pattern Recogn. Vol 45, 21092122. [Crossref]
45. Tzovara A. , Murray M. M. , Michel C. M. , De Lucia M. ( 2012b). "A tutorial review of electrical neuroimaging from group-average to single-trial event-related potentials", Dev. Neuropsychol. Vol 37, 518544. [Crossref]
46. van der Linden M. , van Turennout M. , Indefrey P. ( 2010). "Formation of category representations in superior temporal sulcus", J. Cognitive Neurosci. Vol 22, 12701282. [Crossref]
47. Von Kriegstein K. , Giraud A. L. ( 2006). "Implicit multisensory associations influence voice recognition", PLoS Biol. Vol 4, e326.
48. Wheeler M. E. , Petersen S. E. , Buckner R. L. ( 2000). "Memory’s echo: vivid remembering reactivates sensory-specific cortex", Proc. Natl Acad. Sci. USA Vol 97, 1112511129. [Crossref]
49. Wozny D. R. , Shams L. ( 2011). "Computational characterization of visually induced auditory spatial adaptation", Front. Integr. Neurosci. Vol 5, 75. [Crossref]
50. Zangenehpour S. , Zatorre R. J. ( 2010). "Crossmodal recruitment of primary visual cortex following brief exposure to bimodal audiovisual stimuli", Neuropsychologia Vol 48, 591600. [Crossref]

Article metrics loading...



Can't access your account?
  • Tools

  • Add to Favorites
  • Printable version
  • Email this page
  • Subscribe to ToC alert
  • Get permissions
  • Recommend to your library

    You must fill out fields marked with: *

    Librarian details
    Your details
    Why are you recommending this title?
    Select reason:
    Multisensory Research — Recommend this title to your library
  • Export citations
  • Key

  • Full access
  • Open Access
  • Partial/No accessInformation