Cookies Policy

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies.

I accept this policy

Find out more here

Individual Differences in Sensory Substitution

No metrics data to plot.
The attempt to load metrics for this article has failed.
The attempt to plot a graph for these metrics has failed.
The full text of this article is not currently available.

Brill’s MyBook program is exclusively available on BrillOnline Books and Journals. Students and scholars affiliated with an institution that has purchased a Brill E-Book on the BrillOnline platform automatically have access to the MyBook option for the title(s) acquired by the Library. Brill MyBook is a print-on-demand paperback copy which is sold at a favorably uniform low price.

Access this article

+ Tax (if applicable)
Add to Favorites
You must be logged in to use this functionality

image of Multisensory Research
For more content, see Seeing and Perceiving and Spatial Vision.

Sensory substitution devices were developed in the context of perceptual rehabilitation and they aim at compensating one or several functions of a deficient sensory modality by converting stimuli that are normally accessed through this deficient sensory modality into stimuli accessible by another sensory modality. For instance, they can convert visual information into sounds or tactile stimuli. In this article, we review those studies that investigated the individual differences at the behavioural, neural, and phenomenological levels when using a sensory substitution device. We highlight how taking into account individual differences has consequences for the optimization and learning of sensory substitution devices. We also discuss the extent to which these studies allow a better understanding of the experience with sensory substitution devices, and in particular how the resulting experience is not akin to a single sensory modality. Rather, it should be conceived as a multisensory experience, involving both perceptual and cognitive processes, and emerging on each user’s pre-existing sensory and cognitive capacities.

Affiliations: 1: Institut des Systèmes Intelligents et de Robotique, CNRS UMR 7222, Université Pierre et Marie Curie, 4 place Jussieu, 75005 Paris, France

*To whom correspondence should be addressed. E-mail:

Full text loading...


Data & Media loading...

1. Amedi A., Stern W. M., Camprodon J. A., Bermpohl F., Merabet L., Rotman S., Hemond C., Meijer P., Pascual-Leone A. (2007). "Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex", Nat. Neurosci. Vol 10, 687689. [Crossref]
2. Arnold G., Spence C., Auvray M. (2016). "Taking someone else’s spatial perspective: natural stance or effortful decentring?" Cognition Vol 148, 2733. [Crossref]
3. Auvray M., Farina M. (2017). "Patrolling the boundaries of synaethesia: a critical appraisal of transient and artificially-acquired forms of synaesthetic experiences", in: Sensory Blendings: New Essays on Synaesthesia, Deroy O. (Ed.). Oxford University Press, Oxford, UK. (Publication scheduled for April 2017).
4. Auvray M., Myin E. (2009). "Perception with compensatory devices. From sensory substitution to sensorimotor extension", Cogn. Sci. Vol 33, 10361058. [Crossref]
5. Auvray M., Hanneton S., O’Regan J. K. (2007). "Learning to perceive with a visuo-auditory substitution system: localization and object recognition with the voice", Perception Vol 36, 416430. [Crossref]
6. Bach-y-Rita P., Collins C. C., Saunders F. A., White B., Scadden L. (1969). "Vision substitution by tactile image projection", Nature Vol 221(5184), 963964. [Crossref]
7. Bach-y-Rita P., Tyler M. E., Kaczmarek K. A. (2003). "Seeing with the brain", Int. J. Hum-Comput. Interact. Vol 2, 285295. [Crossref]
8. Bavelier D., Neville H. J. (2002). "Cross-modal plasticity: where and how?" Nat. Rev. Neurosci. Vol 3, 443452.
9. Block N. (2003). "Tactile sensation via spatial perception", Trends Cogn. Sci. Vol 7, 285286. [Crossref]
10. Bologna G., Deville B., Pun T. (2009). "On the use of the auditory pathway to represent image scenes in real-time", Neurocomputing Vol 72, 839849. [Crossref]
11. Boussaoud D., Bremmer F. (1999). "Gaze effects in the cerebral cortex: reference frames for space coding and action", Exp. Brain Res. Vol 128, 170180. [Crossref]
12. Brown D. J., Macpherson T., Ward J. (2011). "Seeing with sound? Exploring different characteristics of a visual-to-auditory sensory substitution device", Perception Vol 40, 11201135. [Crossref]
13. Cohen Y. E., Andersen R. A. (2002). "A common reference frame for movement plans in the posterior parietal cortex", Nat. Rev. Neurosci. Vol 3, 553562. [Crossref]
14. Danilov Y., Tyler M., Skinner K., Hogle R., Bach-y-Rita P. (2007). "Efficacy of electrotactile vestibular substitution in patients with peripheral and central vestibular loss", J. Vestib. Res. Vol 17, 119130.
15. Deroy O., Auvray M. (2012). "Reading the world through the skin and ears: a new perspective on sensory substitution", Fron. Psychol. Vol 3, 457. DOI:10.3389/fpsyg.2012.00457.
16. Deroy O., Auvray M. (2014). "A crossmodal perspective on sensory substitution", in: Perception and Its Modalities, Biggs S., Matthen M., Stokes D. (Eds), pp.  327349. Oxford University Press, Oxford, UK. [Crossref]
17. Deroy O., Fasiello I., Hayward V., Auvray M. (2016). "Differentiated audio-tactile correspondences in sighted and blind individuals", J. Exp. Psychol. Hum. Percept. Perform. Vol 42, 12041214. [Crossref]
18. Diot B., Halavackova P., Demongeot J., Vuillerme N. (2014). "Sensory substitution for balance control using a vestibular-to-tactile device", Multisens. Res. Vol 27, 313336. [Crossref]
19. Elli G. V., Benetti S., Collignon O. (2014). "Is there a future for sensory substitution outside academic laboratories?" Multisens. Res. Vol 27, 271291. [Crossref]
20. Farcy R., Leroux R., Jucha A., Damaschini R., Grégoire C., Zogaghi A. (2006). "Electronic travel aids and electronic orientation aids for blind people: technical, rehabilitation and everyday life points of view", in: Proceedings of the Conference and Workshop on Assistive Technology for Vision and Hearing Impairment, Hersh M. A. (Ed.).
21. Farina M. (2013). "Neither touch nor vision: sensory substitution as artificial synaesthesia?" Biol. Philos. Vol 28, 639655. [Crossref]
22. Gallay M., Denis M., Auvray M. (2013). "Navigation assistance for blind pedestrians: guidelines for the design of devices and implications for spatial cognition", in: Representing Space in Cognition: Interrelations of Behaviour, Language, and Formal Models, Tenbrink T., Wiener J., Claramunt C. (Eds). Oxford University Press, Oxford, UK.
23. Goeke C., Kornpetpanee S., Köster M., Fernández-Revelles A., Gramann K., König P. (2015). "Cultural background shapes spatial reference frame proclivity", Sci. Rep. Vol 5, 11426. DOI:10.1038/srep11426. [Crossref]
24. Guarniero G. (1974). "Experience of tactile vision", Perception Vol 3, 101104. [Crossref]
25. Haigh A., Brown D. J., Meijer P., Proulx M. J. (2013). "How well do you see what you hear? The acuity of visual-to-auditory sensory substitution", Front. Psychol. Vol 4, 330. [Crossref]
26. Hanneton S., Auvray M., Durette B. (2010). "The vibe: a versatile vision-to-audition sensory substitution device", Appl. Bionics Biomech. Vol 7, 269276. [Crossref]
27. Hanneton S., Herquel P., Auvray M. (2015). "Intermodal recoding of a video game: learning to process signals for motion perception in a pure auditory environment", Int. J. Adapt. Contr. Signal Proc. Vol 29, 14751483. [Crossref]
28. Harrar V., Harris L. R. (2010). "Touch used to guide action is partially coded in a visual reference frame", Exp. Brain Res. Vol 203, 615620. [Crossref]
29. Hartcher-O’Brien J., Auvray M. (2014). "The process of distal attribution illuminated through studies of sensory substitution", Multisens. Res. Vol 27, 421441. [Crossref]
30. Heimler B., Striem-Amit E., Amedi A. (2015). "Origins of task-specific sensory-independent organization in the visual and auditory brain: neuroscience evidence, open questions and clinical implications", Curr. Opin. Neurobiol. Vol 35, 169177. [Crossref]
31. Ho C., Spence C. (2008). The Multisensory Driver: Implications for Ergonomic Car Interface Design. Ashgate Publishing, Aldershot, UK.
32. Hötting K., Röder B. (2004). "Hearing cheats touch, but less in congenitally blind than in sighted individuals", Psychol. Sci. Vol 15, 6064. [Crossref]
33. Hötting K., Rösler F., Röder B. (2004). "Altered auditory-tactile interactions in congenitally blind humans: an event-related potential study", Exp. Brain Res. Vol 159, 370381. [Crossref]
34. Humphrey N. (1992). A History of the Mind. Simon and Schuster, New York, NY, USA. [Crossref]
35. Hurley S., Noë A. (2003). "Neural plasticity and consciousness", Biol. Philos. Vol 18, 131168. [Crossref]
36. Jones L. A., Kunkel J., Piateski E. (2009). "Vibrotactile pattern recognition on the arm and back", Perception Vol 38, 5268. [Crossref]
37. Klatzky R. L. (1998). "Allocentric and egocentric spatial representations: definitions, distinctions, and interconnections", in: Spatial Cognition I, Freksa C., Habel C., Wender K. F. (Eds), pp.  118. Springer, New York, NY, USA.
38. Kristjánsson Á., Moldoveanu A., Jóhannesson Ó., Balan O., Spagnol S., Valgeirsdóttir V. V., Unnpórsson R. (2016). "Designing sensory-substitution devices: principles, pitfalls and potential", Restor. Neurol. Neurosci. Vol 34, 769787.
39. Kupers R., Fumal A., Maertens de Noordhout A., Gjedde A., Schoenen J., Ptito M. (2006). "Transcranial magnetic stimulation of the visual cortex induces somatotopically organized qualia in blind subjects", Proc. Natl Acad. Sci. Vol 35, 1325613260. [Crossref]
40. Levy-Tzedek S., Novick I., Arbel R., Abboud S., Maidenbaum S., Vaadia E., Amedi A. (2012). "Cross-sensory transfer of sensory-motor information: visuomotor learning affects performance on an audiomotor task, using sensory-substitution", Sci. Rep. Vol 2, 949. DOI:10.1038/srep00949. [Crossref]
41. Levy-Tzedek S., Riemer D., Amedi A. (2014). "Color improves “visual” acuity via sound", Front. Neurosci. Vol 8, 358. DOI:10.3389/fnins.2014.00358. [Crossref]
42. Levy-Tzedek S., Maidenbaum S., Amedi A., Lackner J. (2016). "Aging and sensory substitution in a virtual navigation task", PloS One Vol 11, e0151593. DOI:10.1371/journal.pone.0151593. [Crossref]
43. Loomis J. M., Klatzky R. L., Golledge R. G., Cicinelli J. G., Pellegrino J. W., Fry P. A. (1993). "Nonvisual navigation by blind and sighted: assessment of path integration ability", J. Exp. Psychol. Gen. Vol 122, 7391. [Crossref]
44. Loomis J. M., Klatzky R. L., Giudice N. A. (2012). "Sensory substitution of vision: importance of perceptual and cognitive processing", in: Assistive Technology for Blindness and Low Vision, Manduchi R., Kurniawan S. (Eds), pp.  162191. CRC Press, Boca Raton, FL, USA.
45. Maidenbaum S., Hanassy S., Abboud S., Buchs G., Chebat D. R., Levy-Tzedek S., Amedi A. (2014). "The “EyeCane”, a new electronic travel aid for the blind: technology, behavior and Swift learning", Restor. Neurol. Neurosci. Vol 32, 813824.
46. Marks D. F. (1973). "Visual imagery in the recall of pictures", Br. J. Psychol. Vol 64, 1724. [Crossref]
47. Marks L. E. (1989). "On cross-modal similarity: the perceptual structure of pitch, loudness, and brightness", J. Exp. Psychol. Hum. Percept. Perform. Vol 15, 586602. [Crossref]
48. Meijer P. B. L. (1992). "An experimental system for auditory image representations", IEEE Trans. Biomed. Eng. Vol 39, 112121. [Crossref]
49. Millar S. (1994). Understanding and Representing Space: Theory and Evidence From Studies With Blind and Sighted Children. Clarendon Press, Oxford, UK. [Crossref]
50. Murphy M. C., Nau A. C., Fisher C., Kim S. G., Schuman J. S., Chan K. C. (2016). "Top-down influence on the visual cortex of the blind during sensory substitution", NeuroImage Vol 125, 932940. [Crossref]
51. Noë A. (2004). Action in Perception. MIT Press, Cambridge, MA, USA.
52. Noordzij M. L., Zuidhoek S., Postma A. (2006). "The influence of visual experience on the ability to form spatial mental models based on route and survey descriptions", Cognition Vol 100, 321342. [Crossref]
53. O’Keefe J., Nadel L. (1978). The Hippocampus as a Cognitive Map. Oxford University Press, London, UK.
54. O’Regan J. K. (2011). Why Red Doesn’t Sound Like a Bell. Understanding the Feel of Consciousness. Oxford University Press, Oxford, UK. [Crossref]
55. Ortiz T., Poch J., Santos J. M., Requena C., Martínez A. M., Ortiz-Terán L., Turrero A., Barcia J., Nogales R., Calvo A., Martínez J. M., Córdoba J. L., Pascual-Leone A. (2011). "Recruitment of occipital cortex during sensory substitution training linked to subjective experience of seeing in people with blindness", PLoS One Vol 6, e23264. DOI:10.1371/journal.pone.0023264.
56. Pascual-Leone A., Hamilton R. (2001). "The metamodal organization of the brain", Prog. Brain Res. Vol 134, 427445. [Crossref]
57. Pasqualotto A., Esenkaya T. (2016). "Sensory substitution: the spatial updating of auditory scenes “mimics” the spatial updating of visual scenes", Front. Behav. Neurosci. Vol 10, 79. DOI:10.3389/fnbeh.2016.00079. [Crossref]
58. Pasqualotto A., Proulx M. J. (2012). "The role of visual experience for the neural basis of spatial cognition", Neurosci. Biobehav. Rev. Vol 36, 11791187. [Crossref]
59. Pasqualotto A., Spiller M. J., Jansari A. S., Proulx M. J. (2013). "Visual experience facilitates allocentric spatial representation", Behav. Brain Res. Vol 236, 175179. [Crossref]
60. Pesnot-Lerousseau J., Arnold G., Auvray M. (subm.). Training induced plasticity with visual-to-auditory sensory substitution devices.
61. Poirier C., De Volder A. G., Scheiber C. (2007). "What neuroimaging tells us about sensory substitution", Neurosci. Biobehav. Rev. Vol 31, 10641070. [Crossref]
62. Pollok B., Schnitzler I., Stoerig P., Mierdorf T., Schnitzler A. (2005). "Image-to-sound conversion: experience-induced plasticity in auditory cortex of blindfolded adults", Exp. Brain Res. Vol 167, 287291. [Crossref]
63. Prinz J. (2006). "Putting the brakes on enactive perception", Psyche Vol 12, 119.
64. Proulx M. J., Stoerig P., Ludowig E., Knoll I. (2008). "Seeing ‘where’ through the ears: effects of learning-by-doing and long-term sensory deprivation on localization based on image-to-sound substitution", PloS One Vol 3, e1840. DOI:10.1371/journal.pone.0001840. [Crossref]
65. Proulx M. J., Brown D. J., Pasqualotto A., Meijer P. (2014). "Multisensory perceptual learning and sensory substitution", Neurosci. Biobehav. Rev. Vol 41, 1625. [Crossref]
66. Proulx M. J., Gwinnutt J., Dell’Erba S., Levy-Tzedek S., de Sousa A. A., Brown D. J. (2016). "Other ways of seeing: from behavior to neural mechanisms in the online “visual” control of action with sensory substitution", Restor. Neurol. Neurosci. Vol 34, 2944.
67. Ptito M., Moesgaard S. M., Gjedde A., Kupers R. (2005). "Cross-modal plasticity revealed by electrotactile stimulation of the tongue in the congenitally blind", Brain Vol 128, 606614. [Crossref]
68. Renier L., Laloyaux C., Collignon O., Tranduy D., Vanlierde A., Bruyer R., De Volder A. G. (2005). "The Ponzo illusion using auditory substitution of vision in sighted and early blind subjects", Perception Vol 34, 857867. [Crossref]
69. Rieser J. J., Gum D. A., Hill E. W. (1986). "Sensitivity to perspective structure while walking without vision", Perception Vol 15, 173188. [Crossref]
70. Sathian K., Stilla R. (2010). "Cross-modal plasticity of tactile perception in blindness", Restor. Neurol. Neurosci. Vol 28, 271281.
71. Shore D. I., Spry E., Spence C. (2002). "Confusing the mind by crossing the hands", Cogn. Brain Res. Vol 14, 153163. [Crossref]
72. Spence C., Deroy O. (2013). "Crossmodal mental imagery", in: Multisensory Imagery, Lacey S., Lawson R. (Eds), pp.  157183. Springer, New York, NY, USA. [Crossref]
73. Stiles N. R., Shimojo S. (2015). "Sensory substitution", in: The Oxford Handbook of Perceptual Organization, Wagemans J. (Ed.), pp.  775798. Oxford University Press, Oxford, UK.
74. Stiles N. R., Zheng Y., Shimojo S. (2015). "Length and orientation constancy learning in 2-dimensions with auditory sensory substitution: the importance of self-initiated movement", Front. Psychol. Vol 6, 842. DOI:10.3389/fpsyg.2015.00842. [Crossref]
75. Striem-Amit E., Amedi A. (2014). "Visual cortex extrastriate body-selective area activation in congenitally blind people “seeing” by using sounds", Curr. Biol. Vol 6, 687692. [Crossref]
76. Striem-Amit E., Guendelman M., Amedi A. (2012). "‘Visual’ acuity of the congenitally blind using visual-to-auditory sensory substitution", PloS One Vol 7, e33136. DOI:10.1371/journal.pone.0033136. [Crossref]
77. Tversky B. (2003). "Structures on mental spaces: how people think about space", Environ. Behav. Vol 35, 6680. [Crossref]
78. Vandenberg S. G. (1971). Mental Rotation Test. University of Colorado, Boulder, CO, USA.
79. Wan C. Y., Wood A. G., Reutens D. C., Wilson S. J. (2010). "Early but not late-blindness leads to enhanced auditory perception", Neuropsychologia Vol 48, 344348. [Crossref]
80. Ward J., Meijer P. (2010). "Visual experiences in the blind induced by an auditory sensory substitution device", Consc. Cogn. Vol 19, 492500. [Crossref]
81. West R. L. (1996). "An application of prefrontal cortex function theory to cognitive aging", Psychol. Bull. Vol 120, 272292. [Crossref]
82. White B. W., Saunders F. A., Scadden L., Bach-y-Rita P., Collins C. C. (1970). "Seeing with the skin", Percept. Psychophys. Vol 7, 2327. [Crossref]
83. Zwiers M. P., Van Opstal A. J., Paige G. D. (2003). "Plasticity in human sound localization induced by compressed spatial vision", Nat. Neurosci. Vol 6, 175181. [Crossref]

Article metrics loading...



Can't access your account?
  • Tools

  • Add to Favorites
  • Printable version
  • Email this page
  • Subscribe to ToC alert
  • Get permissions
  • Recommend to your library

    You must fill out fields marked with: *

    Librarian details
    Your details
    Why are you recommending this title?
    Select reason:
    Multisensory Research — Recommend this title to your library
  • Export citations
  • Key

  • Full access
  • Open Access
  • Partial/No accessInformation