Cookies Policy

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies.

I accept this policy

Find out more here

The Effect of Visual and Auditory Information on the Perception of Pleasantness and Roughness of Virtual Surfaces

No metrics data to plot.
The attempt to load metrics for this article has failed.
The attempt to plot a graph for these metrics has failed.
The full text of this article is not currently available.

Brill’s MyBook program is exclusively available on BrillOnline Books and Journals. Students and scholars affiliated with an institution that has purchased a Brill E-Book on the BrillOnline platform automatically have access to the MyBook option for the title(s) acquired by the Library. Brill MyBook is a print-on-demand paperback copy which is sold at a favorably uniform low price.

Access this article

+ Tax (if applicable)
Add to Favorites
You must be logged in to use this functionality

image of Multisensory Research
For more content, see Seeing and Perceiving and Spatial Vision.

Despite the large number of studies on the multisensory aspects of tactile perception, very little is known regarding the effects of visual and auditory sensory modalities on the tactile hedonic evaluation of textures, especially when the presentation of the stimuli is mediated by a haptic device. In this study, different haptic virtual surfaces were rendered by varying the static and dynamic frictional coefficients of a Geomagic® Touch device. In Experiment 1, the haptic surfaces were paired with pictures representing everyday materials (glass, plastic, rubber and steel); in Experiment 2, the haptic surfaces were paired with sounds resulting from the haptic exploration of paper or sandpaper. In both the experiments, participants were required to rate the pleasantness and the roughness of the virtual surfaces explored. Exploration times were also recorded. Both pleasantness and roughness judgments, as well as the durations of exploration, varied as a function of the combinations of the visuo-tactile and the audio-tactile stimuli presented. Taken together, these results suggest that vision and audition modulate haptic perception and hedonic preferences when tactile sensations are provided through a haptic device. Importantly, these results offer interesting suggestions for designing more pleasant, and even more realistic, multisensory virtual surfaces.

Affiliations: 1: 1Department of Psychology and NeuroMi, Milan Center for Neuroscience, University of Milan-Bicocca, Piazza dell’Ateneo Nuovo 1, 20126 Milan, Italy ; 2: 2Department of Mechanical Engineering, Politecnico di Milano, Milan, Italy ; 3: 3Center for Mind/Brain Sciences, CIMeC, University of Trento, Trento, Italy ; 4: 4Department of Psychology and Cognitive Science, University of Trento, Trento, Italy

*To whom correspondence should be addressed. E-mail:

Full text loading...


Data & Media loading...

1. Altinsoy M. E. (2008). "The effect of auditory cues on the audiotactile roughness perception: modulation frequency and sound pressure level", in: International Workshop on Haptic and Audio Interaction Design, HAID 2008, Pirhonen A., Brewster S. (Eds), Lecture Notes in Computer Science, Vol Vol. 5270, pp.  120129. Springer, Berlin and Heidelberg, Germany.
2. Avanzini F., Crosato P. (2006). "Haptic-auditory rendering and perception of contact stiffness", in: International Workshop on Haptic and Audio Interaction Design, HAID 2006, McGookin D., Brewster S. (Eds), Lecture Notes in Computer Science, Vol Vol. 4129, pp.  2435. Springer, Berlin and Heidelberg, Germany.
3. Bergmann Tiest W. M., Kappers A. M. (2006). "Analysis of haptic perception of materials by multidimensional scaling and physical measurements of roughness and compressibility", Acta Psychol. Vol 121, 120. [Crossref]
4. Brieber D., Nadal M., Leder H., Rosenberg R. (2014). "Art in time and space: context modulates the relation between art experience and viewing time", PLoS One Vol 9, e99019. DOI:10.1371/journal.pone.0099019. [Crossref]
5. Campion G., Gosline A. H., Hayward V. (2008). "Does judgment of haptic virtual texture roughness scale monotonically with lateral force modulation?", in: International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, Ferre M. (Ed.), Lecture Notes in Computer Science, Vol Vol. 5024, pp.  718723. Springer, Berlin and Heidelberg, Germany.
6. Colavita F. B. (1974). "Human sensory dominance", Percept. Psychophys. Vol 16, 409412. [Crossref]
7. Colavita F. B., Weisberg D. (1979). "A further investigation of visual dominance", Percept. Psychophys. Vol 25, 345347. [Crossref]
8. Croy I., D’Angelo S., Olausson H. (2014). "Reduced pleasant touch appraisal in the presence of a disgusting odor", PLoS One Vol 9, e92975. DOI:10.1371/journal.pone.0092975. [Crossref]
9. Demattè M. L., Sanabria D., Sugarman R., Spence C. (2006). "Cross-modal interactions between olfaction and touch", Chem. Senses Vol 31, 291300. [Crossref]
10. Ellingsen D. M., Wessberg J., Chelnokova O., Olausson H., Laeng B., Leknes S. (2014). "In touch with your emotions: oxytocin and touch change social impressions while others’ facial expressions can alter touch", Psychoneuroendocrinology Vol 39, 1120. [Crossref]
11. Ernst M. O., Banks M. S. (2002). "Humans integrate visual and haptic information in a statistically optimal fashion", Nature Vol 415(6870), 429433. [Crossref]
12. Ernst M. O., Bülthoff H. H. (2004). "Merging the senses into a robust percept", Trends Cogn. Sci. Vol 8, 162169. [Crossref]
13. Essick G. K., McGlone F., Dancer C., Fabricant D., Ragin Y., Phillips N., Jones T., Guest S. (2010). "Quantitative assessment of pleasant touch", Neurosci. Biobehav. Rev. Vol 34, 192203. [Crossref]
14. Etzi R., Gallace A. (2016). "The arousing power of everyday materials: an analysis of the physiological and behavioral responses to visually and tactually presented textures", Exp. Brain Res. Vol 234, 16591666. [Crossref]
15. Etzi R., Spence C., Gallace A. (2014). "Textures that we like to touch: an experimental study of aesthetic preferences for tactile stimuli", Consc. Cogn. Vol 29, 178188. [Crossref]
16. Gallace A. (2011). "Verso una definizione dell’estetica tattile [Towards a definition of tactile aesthetics]", in: Vedere con Mano, Argenton A. (Ed.), pp.  93108. Edizioni Erickson, Trento, Italy.
17. Gallace A. (2015). "Touch to sell: when packaging design meets cognitive neuroscience", Packaging Eur. Vol 10, 4749.
18. Gallace A., Spence C. (2011). "Tactile aesthetics: towards a definition of its characteristics and neural correlates", Soc. Semiotics Vol 21, 569589. [Crossref]
19. Gallace A., Spence C. (2014). In Touch With the Future: the Sense of Touch From Cognitive Neuroscience to Virtual Reality. Oxford University Press, Oxford, UK. [Crossref]
20. Gallace A., Ngo M. K., Sulaitis J., Spence C. (2011). "Multisensory presence in virtual reality: possibilities and limitations", in: Multiple Sensorial Media Advances and Applications: New Developments in MulSeMedia, Ghinea G., Andres F., Gulliver S. (Eds), pp.  138. IGI Global, Vancouver, BC, Canada.
21. Guest S., Spence C. (2003a). "Tactile dominance in speeded discrimination of textures", Exp. Brain Res. Vol 150, 201207. [Crossref]
22. Guest S., Spence C. (2003b). "What role does multisensory integration play in the visuotactile perception of texture?" Int. J. Psychophysiol. Vol 50, 6380. [Crossref]
23. Guest S., Catmur C., Lloyd D., Spence C. (2002). "Audiotactile interactions in roughness perception", Exp. Brain Res. Vol 146, 161171. [Crossref]
24. Guest S., Dessirier J. M., Mehrabyan A., McGlone F., Essick G., Gescheider G., Fontana A., Xiong R., Ackerley R., Blot K. (2011). "The development and validation of sensory and emotional scales of touch perception", Atten. Percept. Psychophys. Vol 73, 531550. [Crossref]
25. Hartcher-O’Brien J., Gallace A., Krings B., Koppen C., Spence C. (2008). "When vision ‘extinguishes’ touch in neurologically-normal people: extending the Colavita visual dominance effect", Exp. Brain Res. Vol 186, 643658. [Crossref]
26. Heller M. A. (1982). "Visual and tactual texture perception: intersensory cooperation", Percept. Psychophys. Vol 31, 339344. [Crossref]
27. Heller M. A. (1985). "Effect of magnification on texture perception", Percept. Mot. Skills Vol 61, 1242. [Crossref]
28. Ho H. N., Iwai D., Yoshikawa Y., Watanabe J., Nishida S. Y. (2014). "Combining colour and temperature: a blue object is more likely to be judged as warm than a red object", Sci. Rep. Vol 4, 5527.
29. Hollins M., Faldowski R., Rao S., Young F. (1993). "Perceptual dimensions of tactile surface texture: a multidimensional scaling analysis", Percept. Psychophys. Vol 54, 697705. [Crossref]
30. Hollins M., Bensmaïa S., Karlof K., Young F. (2000). "Individual differences in perceptual space for tactile textures: evidence from multidimensional scaling", Percept. Psychophys. Vol 62, 15341544. [Crossref]
31. Jones B., O’Neil S. (1985). "Combining vision and touch in texture perception", Percept. Psychophys. Vol 37, 6672. [Crossref]
32. Jousmäki V., Hari R. (1998). "Parchment-skin illusion: sound-biased touch", Curr. Biol. Vol 8, R190R191. [Crossref]
33. Klatzky R. L., Lederman S. J. (2006). "The perceived roughness of resistive virtual textures: I. Rendering by a force-feedback mouse", ACM Trans. Appl. Percept. Vol 3, 114. [Crossref]
34. Klatzky R. L., Lederman S. J. (2010). "Multisensory texture perception", in: Multisensory Object Perception in the Primate Brain, Naumer M. J., Kaiser J. (Eds), pp.  211230. Springer, New York, NY, USA. [Crossref]
35. Koppen C., Spence C. (2007). "Seeing the light: exploring the Colavita visual dominance effect", Exp. Brain Res. Vol 180, 737754. [Crossref]
36. Kornbrot D., Penn P., Petrie H., Furner S., Hardwick A. (2007). "Roughness perception in haptic virtual reality for sighted and blind people", Percept. Psychophys. Vol 69, 502512. [Crossref]
37. Lederman S. J., Abbott S. G. (1981). "Texture perception: studies of intersensory organization using a discrepancy paradigm, and visual versus tactual psychophysics", J. Exp. Psychol. Hum. Percept. Perform. Vol 7, 902915. [Crossref]
38. Lederman S. J., Klatzky R. L. (2004). "Multisensory texture perception", in: The Handbook of Multisensory Processes, Calvert G. A., Spence C., Stein B. E. (Eds), pp.  107123. MIT Press, Cambridge, MA, USA.
39. Lederman S. J., Thorne G., Jones B. (1986). "Perception of texture by vision and touch: multidimensionality and intersensory integration", J. Exp. Psychol. Hum. Percept. Perform. Vol 12, 169180. [Crossref]
40. Lederman S. J., Klatzky R. L., Tong C., Hamilton C. (2006). "The perceived roughness of resistive virtual textures: II. Effects of varying viscosity with a force-feedback device", ACM Trans. Appl. Percept. Vol 3, 1530. [Crossref]
41. McCabe D. B., Nowlis S. M. (2003). "The effect of examining actual products or product descriptions on consumer preference", J. Consum. Psychol. Vol 13, 431439. [Crossref]
42. Meredith M. A., Stein B. E. (1986). "Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration", J. Neurophysiol. Vol 56, 640662. [Crossref]
43. Okamoto S., Nagano H., Yamada Y. (2013). "Psychophysical dimensions of tactile perception of textures", IEEE Trans. Haptics Vol 6, 8193. [Crossref]
44. Picard D., Dacremont C., Valentin D., Giboreau A. (2003). "Perceptual dimensions of tactile textures", Acta Psycholog. Vol 114, 165184. [Crossref]
45. Posner M. I., Nissen M. J., Klein R. M. (1976). "Visual dominance: an information-processing account of its origins and significance", Psychol. Rev. Vol 83, 157171. [Crossref]
46. Punpongsanon P., Iwai D., Sato K. (2015). "Softar: visually manipulating haptic softness perception in spatial augmented reality", IEEE Trans. Vis. Comput. Graph. Vol 21, 12791288. [Crossref]
47. Rock I., Victor J. (1964). "Vision and touch: an experimentally created conflict between the two senses", Science Vol 143, 594596. [Crossref]
48. Sinnett S., Spence C., Soto-Faraco S. (2007). "Visual dominance and attention: the Colavita effect revisited", Percept. Psychophys. Vol 69, 673686. [Crossref]
49. Spence C. (2009). "Explaining the Colavita visual dominance effect", Progr. Brain Res. Vol 176, 245258. [Crossref]
50. Spence C., Gallace A. (2008). "Making sense of touch", in: Touch in Museums: Policy and Practice in Object Handling, Chatterjee H. E. (Ed.), pp.  2140. Berg, London, UK.
51. Spence C., Gallace A. (2011). "Multisensory design: reaching out to touch the consumer", Psychol. Marketing Vol 28, 267308. [Crossref]
52. Stein B. E. (Ed.) (2012). The New Handbook of Multisensory Processing. MIT Press, Cambridge, MA, USA.
53. Stein B. E., Stanford T. R. (2008). "Multisensory integration: current issues from the perspective of the single neuron", Nat. Rev. Neurosci. Vol 9, 255266. [Crossref]
54. Tanaka Y., Tanaka M., Chonan S. (2006). "Development of a sensor system for measuring tactile sensation", in: Proceedings of IEEE Conference on Sensors, pp.  554557. Daegu, South Korea.
55. Ujitoko Y., Ban Y., Narumi T., Tanikawa T., Hirota K., Hirose M. (2015). "Yubi-Toko: finger walking in snowy scene using pseudo-haptic technique on touchpad", in: SA ’15 SIGGRAPH Asia 2015 Emerging Technologies, Kobe, Japan, art. 29.
56. Weisenberger J. M., Poling G. L. (2004). "Multisensory roughness perception of virtual surfaces: effects of correlated cues", in: Proceedings of the 12th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systs, pp.  161168. Chicago, IL, USA.
57. Welch R. B., Warren D. H. (1980). "Immediate perceptual response to intersensory discrepancy", Psychol. Bull. Vol 88, 638667. [Crossref]
58. Whitaker T. A., Simões-Franklin C., Newell F. N. (2008). "Vision and touch: independent or integrated systems for the perception of texture?" Brain Res. Vol 1242, 5972. [Crossref]
59. World Medical Association (2013). "World Medical Association declaration of Helsinki: ethical principles for medical research involving human subjects", JAMA Vol 310, 21912194. [Crossref]
60. Zampini M., Spence C. (2004). "The role of auditory cues in modulating the perceived crispness and staleness of potato chips", J. Sens. Stud. Vol 19, 347363. [Crossref]

Article metrics loading...



Can't access your account?
  • Tools

  • Add to Favorites
  • Printable version
  • Email this page
  • Subscribe to ToC alert
  • Get permissions
  • Recommend to your library

    You must fill out fields marked with: *

    Librarian details
    Your details
    Why are you recommending this title?
    Select reason:
    Multisensory Research — Recommend this title to your library
  • Export citations
  • Key

  • Full access
  • Open Access
  • Partial/No accessInformation