Cookies Policy

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies.

I accept this policy

Find out more here

Touch-Screen Technology for the Dynamic Display of 2D Spatial Information Without Vision: Promise and Progress

No metrics data to plot.
The attempt to load metrics for this article has failed.
The attempt to plot a graph for these metrics has failed.
The full text of this article is not currently available.

Brill’s MyBook program is exclusively available on BrillOnline Books and Journals. Students and scholars affiliated with an institution that has purchased a Brill E-Book on the BrillOnline platform automatically have access to the MyBook option for the title(s) acquired by the Library. Brill MyBook is a print-on-demand paperback copy which is sold at a favorably uniform low price.

Access this article

+ Tax (if applicable)
Add to Favorites
You must be logged in to use this functionality

image of Multisensory Research
For more content, see Seeing and Perceiving and Spatial Vision.

Many developers wish to capitalize on touch-screen technology for developing aids for the blind, particularly by incorporating vibrotactile stimulation to convey patterns on their surfaces, which otherwise are featureless. Our belief is that they will need to take into account basic research on haptic perception in designing these graphics interfaces. We point out constraints and limitations in haptic processing that affect the use of these devices. We also suggest ways to use sound to augment basic information from touch, and we include evaluation data from users of a touch-screen device with vibrotactile and auditory feedback that we have been developing, called a vibro-audio interface.

Affiliations: 1: 1Department of Psychology, Carnegie Mellon University, Pittsburgh, PA 15213, USA; 2: 2Spatial Informatics Program, School of Computing and Information Science, Orono, ME 04469, USA; 3: 3Department of Psychological and Brain Sciences, University of California, Santa Barbara, CA 93106, USA


Full text loading...


Data & Media loading...

1. Apkarian-Stielau P. M. , Loomis J. M. ( 1975). "A comparison of tactile and blurred visual form perception", Percept. Psychophys. Vol 18, 362368.
2. Auvray M. , Hanneton S. , Lenay C. , O’Regan K. ( 2005). "There is something out there: distal attribution in sensory substitution, twenty years later", J. Integr. Neurosci. Vol 4, 505521.
3. Auvray M. , Hanneton S. , O’Regan J. K. ( 2007). "Learning to perceive with a visuo-auditory substitution system: localisation and object recognition with ‘the vOICe’", Perception Vol 36, 416430.
4. Bach-y-Rita P. ( 1967). "Sensory plasticity: applications to a vision substitution system", Acta Neurol. Scand. Vol 43, 417426.
5. Bach-y-Rita P. ( 1972). Brain Mechanisms in Sensory Substitution. Academic Press, New York, NY, USA.
6. Bach-y-Rita P. , Kaczmarek K. A. , Tyler M. E. , Garcia-Lara J. ( 1998). "Form perception with a 49-point electrotactile stimulus array on the tongue: a technical note", J. Rehabil. Res. Dev. Vol 35, 427430.
7. Bach-y-Rita P. , Kercel S. ( 2003). "Sensory substitution and the human machine interface", Trends Cogn. Sci. Vol 7, 541546.
8. Bau O. , Poupyrev I. , Israr A. , Harrison C. ( 2010). TeslaTouch: electrovibration for touch surfaces, in: Proc. 23rd Ann. ACM Symp. User interface Software and Technology, pp. 283–292. ACM, New York, NY, USA.
9. Berla E. ( 1973). "Strategies in scanning a tactual pseudo-map", Educ. Vis. Handicap. Vol 5, 819.
10. Blattner M. M. , Sumikawa D. A. , Greenberg R. M. ( 1989). "Earcons and icons: their structure and common design principles", Hum. Comput. Interact. Vol 4, 1144.
11. Blauert J. ( 1997). Spatial Hearing: The Psychophysics of Human Sound Localization. MIT Press, Cambridge, MA, USA.
12. Brown D. , Macpherson T. , Ward J. ( 2011). "Seeing with sound? Exploring different characteristics of a visual-to-auditory sensory substitution device", Perception Vol 40, 11201135.
13. Carpenter P. A. , Shah P. ( 1998). "A model of the perceptual and conceptual processes in graph comprehension", J. Exp. Psychol. Appl. Vol 4, 75100.
14. Chebat D. R. , Rainville C. , Kupers R. , Ptito M. ( 2007). "Tactile–‘visual’ acuity of the tongue in early blind individuals", Neuroreport Vol 18, 19011904.
15. Chebat D. R. , Schneider F. C. , Kupers R. , Ptito M. ( 2011). "Navigation with a sensory substitution device in congenitally blind individuals", Neuroreport Vol 22, 342347.
16. Cheng M. F. ( 1968). "Tactile–kinesthetic perception of length", Am. J. Psychol. Vol 81, 7482.
17. Collins C. C. ( 1970). Tactile television: mechanical and electrical image projection, IEEE Trans. Man–Mach. Syst. MMS-11, 65–71.
18. Collins C. C. ( 1985). "On mobility aids for the blind", in: Electronic Spatial Sensing for the Blind, Warren D. H. , Strelow E. R. (Eds), pp.  3564. Martinus Nijhoff, Boston, MA, USA.
19. Collins C. C. , Madey J. M. ( 1974). "Tactile sensory replacement", Proc. San Diego Biomed. Symp. Vol 13, 1526.
20. Craig J. C. ( 1999). "Grating orientation as a measure of tactile spatial acuity", Somatosens. Mot. Res. Vol 16, 197206.
21. Cronly-Dillon J. , Persaud K. C. , Blore R. ( 2000). "Blind subjects construct conscious mental images of visual scenes encoded in musical form", Proc. R. Soc. Lond. B Biol. Sci. Vol 267, 22312238.
22. Eitan Z. , Timmers R. ( 2010). "Beethoven’s last piano sonata and those who follow crocodiles: cross-domain mappings of auditory pitch in a musical context", Cognition Vol 114, 405422.
23. Ferres L. , Lindgaard G. , Sumegi L. ( 2010). Evaluating a tool for improving accessibility to charts and graphs, in: Proc. 12th Int. ACM SIGACCESS Conf. Computers and Accessibility( ASSETS ’10), pp. 83–90. ACM, New York, NY, USA.
24. Franks F. , Baird R. ( 1971). "Geographical concepts and the visually handicapped", Except. Child. Vol 38, 321324.
25. Galesic M. , Garcia-Retamero R. ( 2011). "Graph literacy: a cross-cultural comparison", Med. Decis. Making Vol 31, 444457.
26. Gardony A. L. , Brunye T. T. , Taylor H. A. ( 2013). Gardony map drawing analyzer: a novel approach to the analysis of sketch maps, poster at the Annual Convention of the Association for Psychology Science, Washington, D.C., May 25.
27. Gaver W. W. ( 1986). "Auditory icons: using sound in computer interfaces", Hum. Comp. Interact. Vol 2, 167177.
28. Giudice N. A. , Legge G. E. ( 2008). "Blind navigation and the role of technology", in: Engineering Handbook of Smart Technology for Aging, Disability and Independence, Helal A. , Mokhtari M. , Abdulrazak B. (Eds), pp.  479500. John Wiley and Sons, New York, NY, USA.
29. Giudice N. A. , Palani H. P. , Brenner E. , Kramer K. M. ( 2012). Learning non-visual graphical information using a touch-based vibro-audio interface, in: Proc. 14th Int. ACM SIGACCESS Conf. Computers and Accessibility(ASSETS ’12), pp. 103–110. ACM, New York, NY, USA.
30. Golledge R. G. ( 1991). "Tactual strip maps as navigational aids", J. Vis. Impair. Blind. Vol 85, 296301.
31. Goncu C. , Marriott K. ( 2011). "GraVVITAS: generic multi-touch presentation of accessible graphics", Lect. Notes Comput. Sci. Vol 6946, 3048.
32. Hoggan E. , Brewster S. A. , Johnston J. ( 2008). Investigating the effectiveness of tactile feedback for mobile touchscreens, in: Proc. SIGCHI Conf Human Factors in Computing Systems (CHI ’08), pp. 1573–1582. ACM, New York, NY, USA.
33. Jansson G. ( 1983). "Tactile guidance of movement", Int. J. Neurosci. Vol 19, 3746.
34. Johansson R. S. , Vallbo A. B. ( 1983). "Tactile sensory coding in the glabrous skin of the human hand", Trends Neurosci. Vol 6, 2732.
35. Kappers A. M. L. ( 2007). "Haptic spatial processing: allocentric and egocentric reference frames", Can. J. Exp. Psychol. Vol 61, 208218.
36. Kaye H. S. , Kang T. , LaPlante M. P. ( 2000). Mobility device use in the United States. Disability Statistics Report(14). Dept. of Education, National Institute on Disability and Rehabilitation Research, Washington, D.C., USA.
37. Klatzky R. L. , Lederman S. J. ( 2003). "Representing spatial location and layout from sparse kinesthetic contacts", J. Exp. Psychol.: Hum. Percept. Perform. Vol 29, 310325.
38. Kramer G. ( 1994). "An introduction to auditory display", in: Auditory Display, Kramer G. (Ed.), pp.  178. Addison-Wesley, Reading, MA, USA.
39. Lederman S. J. , Campbell J. I. ( 1982). "Tangible graphs for the blind", Hum. Fact. Vol 24, 85100.
40. Lederman S. J. , Jones L. A. ( 2011). "Tactile and haptic illusions", IEEE Trans. Haptics Vol 4, 273294.
41. Lederman S. J. , Klatzky R. L. ( 2009). "Haptic perception: a tutorial", Atten. Percept. Psychophys. Vol 71, 14391459.
42. Lederman S. J. , Klatzky R. L. , Chataway C. , Summers C. ( 1990). "Visual mediation and the haptic recognition of two-dimensional pictures of common objects", Percept. Psychophys. Vol 47, 5464.
43. Legge G. E. , Madison C. , Vaughn B. N. , Cheong A. M. Y. , Miller J. C. ( 2008). "Retention of high tactile acuity throughout the life span in blindness", Percept. Psychophys. Vol 70, 14711488.
44. Levy-Tzedek S. , Hanassy S. , Abboud S. , Maidenbaum S. , Amedi A. ( 2012). "Fast, accurate reaching movements with a visual-to-auditory sensory substitution device", Restor. Neurol. Neurosci. Vol 30, 313323.
45. Loomis J. L. , Klatzky R. L. , Giudice N. A. ( 2012). "Sensory substitution of vision: importance of perceptual and cognitive processing", in: Assistive Technology for Blindness and Low Vision, Manduchi R. , Kurniawan S. (Eds), pp.  162191. CRC Press, Boca Raton, FL, USA.
46. Loomis J. M. ( 1981). "Tactile pattern perception", Perception Vol 10, 527.
47. Loomis J. M. ( 1990). "A model of character recognition and legibility", J. Exp. Psychol. Hum. Percept. Perform. Vol 16, 106120.
48. Loomis J. M. , Klatzky R. L. , Lederman S. J. ( 1991). "Similarity of tactual and visual picture recognition with limited field of view", Perception Vol 20, 167177.
49. MacLean K. E. ( 2008). "Haptic interaction design for everyday interfaces", in: Reviews of Human Factors and Ergonomics, Carswell M. (Ed.), Vol Vol. 4, pp.  149194. Human Factors and Ergonomics Society, Santa Monica, CA, USA.
50. Meijer P. B. L. ( 1992). "An experimental system for auditory image representations", IEEE Tran. Biomed. Eng. Vol 39, 112121.
51. Merriam E. P. , Genovese C. R. , Colby C. L. ( 2007). "Remapping in human visual cortex", J. Neurophysiol. Vol 97, 17381755.
52. Millar S. ( 1981). "Self-referent and movement cues in coding spatial location by blind and sighted children", Perception Vol 10, 255264.
53. Millar S. ( 2008). Space and Sense. Psychology Press, New York, NY, USA.
54. Nees M. A. , Walker B. N. ( 2009). "Auditory interfaces and sonification", in: The Universal Access Handbook, Stephanidis C. (Ed.), pp.  507521. CRC Press, New York, NY, USA.
55. Nolan C. Y. , Morris J. E. ( 1971). Improvement of Tactual Symbols for Blind Children: Final Report, American Printing House for the Blind, Louisville, KY, USA.
56. Perkins Maps( 2013).
57. Perkins Museum( 2013).
58. Petit G. , Dufresne A. , Levesque V. , Hayward V. , Trudeau N. ( 2008). Refreshable tactile graphics applied to schoolbook illustrations for students with visual impairment, in: Proc. 10th Int. ACM SIGACCESS Conf. Computers and Accessibility(ASSETS ’08), pp. 89–96. ACM, New York, NY, USA.
59. Pollok B. , Schnitzler I. , Mierdorf T. , Stoerig P. , Schnitzler A. ( 2005). "Image-to-sound conversion: experience-induced plasticity in auditory cortex of blindfolded adults", Exp. Brain Res. Vol 167, 287291.
60. Poppinga B. , Magnusson C. , Pielot M. , Rassmus-Gröhn K. ( 2011). TouchOver map: audio-tactile exploration of interactive maps, in: Proc. 13th Int. Conf. Human Computer Interaction with Mobile Devices and Services( MobileHCI ’11), pp. 545–550. ACM, New York, NY, USA.
61. Pratt C. C. ( 1930). "The spatial character of high and low tones", J. Exp. Psychol. Vol 13, 278285.
62. Ptito M. , Kupers R. ( 2005). "Cross-modal plasticity in early blindness", J. Integr. Neurosci. Vol 4, 479488.
63. Ptito M. , Moesgaard S. M. , Gjedde A. , Kupers R. ( 2005). "Cross-modal plasticity revealed by electrotactile stimulation of the tongue in the congenitally blind", Brain Vol 128, 606614.
64. Raja M. K. ( 2011). The development and validation of a new smartphone based non-visual spatial interface for learning indoor layouts, Unpublished Masters thesis, University of Maine, Orono, MN, USA.
65. Rastogi R. , Pawluk T. V. D. , Ketchum J. ( 2013). "Intuitive tactile zooming for graphics accessed by individuals who are blind and visually impaired", IEEE Trans. Neural Syst. Rehabil. Eng. Vol 21, 655663.
66. Renier L. , Collignon O. , Poirier C. , Tranduy D. , Vanlierde A. , Bol A. , Veraart C. , De Volder A. G. ( 2005). "Cross-modal activation of visual cortex during depth perception using auditory substitution of vision", NeuroImage Vol 26, 573580.
67. Rosenbaum D. A. , Dawson A. M. , Challis J. H. ( 2006). "Haptic tracking permits bimanual independence", J. Exp. Psychol. Hum. Percept. Perform. Vol 32, 12661275.
68. Sampaio E. , Maris S. , Bach-y-Rita P. ( 2001). "Brain plasticity: ‘visual’ acuity of blind persons via the tongue", Brain Res. Vol 908, 204207.
69. Sanders A. F. J. , Kappers A. M. L. ( 2007). "Haptically straight lines", Perception Vol 36, 16821697.
70. Segond H. , Weiss D. , Sampiao E. ( 2005). "Human spatial navigation via a visuo-tactile sensory substitution system", Perception Vol 34, 12311249.
71. Striem-Amit E. , Dehaene S. , Cohen L. , Amedi A. ( 2012). "Reading with sounds: sensory substitution selectively activates the visual word form area in the blind", Neuron Vol 76, 640652.
72. Tversky B. ( 2003). "Some ways graphics communicate", in: Mobile Communication: Essays on Cognition and Community, Nyiri K. (Ed.), pp.  143156. Passagen Verlag, Wien, Austria.
73. Walker B. N. ( 2002). "Magnitude estimation of conceptual data dimensions for use in sonification", J. Exp. Psychol. Appl. Vol 8, 211221.
74. Walker B. N. , Mauney L. ( 2010). "Universal design of auditory graphs: a comparison of sonification mappings for visually impaired and sighted listeners", ACM Trans. Access. Comput. Vol 2, Article 12, 16 pages.
75. Wijntjes M. W. A. , van Lienen T. , Verstijnen I. M. , Kappers A. M. L. ( 2008). "Look what I have felt: unidentified haptic line drawings are identified after sketching", Acta Psychol. Vol 128, 255263.
76. Winfield L. , Glassmire J. , Colgate J. E. , Peshkin M. ( 2007). T-PaD: tactile pattern display through variable friction reduction, in: Proc. 2nd Joint EuroHaptics Conf. Symp. Haptic Interfaces for Virtual Environment and Teleoperator Systems, pp. 421–426. IEEE, New York, NY, USA.
77. World Health Organization( 2011). Visual Impairment and Blindness Fact Sheet.
78. Xu C. , Israr A. , Poupyrev I. , Bau O. , Harrison C. ( 2011). Tactile display for the visually impaired using TeslaTouch, in: CHI ’11 Extended Abstracts on Human Factors in Computing Systems, pp. 317–322. ACM, New York, NY, USA.
79. Yu W. , Brewster S. A. ( 2002a). Comparing two haptic interfaces for multimodal graph rendering, in: Proc. 10th Symp. Haptic Interfaces for Virtual Environment and Teleoperator Systems, pp. 3–9. IEEE, New York, NY, USA.
80. Yu W. , Brewster S. A. ( 2002b). "Evaluation of multimodal graphs for blind people", Univers. Access Inform. Soc. Vol 2, 105124.
81. Zacks J. , Tversky B. ( 1999). "Bars and lines: a study of graphic communication", Mem. Cogn. Vol 27, 10731079.
82. Zahorik P. , Brungart D. S. , Bronkhorst A. W. ( 2005). "Auditory distance perception in humans: a summary of past and present research", Acta Acustica United w. Acustica Vol 91, 409420.

Article metrics loading...



Can't access your account?
  • Tools

  • Add to Favorites
  • Printable version
  • Email this page
  • Subscribe to ToC alert
  • Get permissions
  • Recommend to your library

    You must fill out fields marked with: *

    Librarian details
    Your details
    Why are you recommending this title?
    Select reason:
    Multisensory Research — Recommend this title to your library
  • Export citations
  • Key

  • Full access
  • Open Access
  • Partial/No accessInformation