Multisensory Perception
How the brain combines uncertain signals during perception and action
Multisensory perception studies how people infer the state of the world from signals that are useful but imperfect. Vision, touch, audition, and proprioception each carry partial information, and the brain must decide when signals belong together, how reliable they are, and how they should influence perception and action.
The work spans cue integration, visual-haptic perception, body ownership, spatial perception, depth, material appearance, object shape, and crossmodal timing. It treats perception as a process of interpretation under uncertainty: sensory signals are noisy, sometimes ambiguous, and often affected by movement, context, prior experience, and recent adaptation.
This topic connects basic perceptual science with technology design. Understanding how multisensory signals are combined helps explain why delayed visual feedback can change perceived stiffness, why a virtual hand can feel more or less like one’s own, why synchrony matters for integration, and how XR and haptic systems can be made more intelligible by respecting perceptual constraints.
Key Questions
When does the brain treat signals from different senses as evidence about the same event?
How are uncertain visual, tactile, auditory, and proprioceptive cues weighted during perception and action?
How do synchrony, delay, context, and prior experience reshape multisensory integration?
What can multisensory science tell us about body ownership, XR interaction, and haptic design?
Related Keywords
Portfolio and Resources
Tactile iPhone experiment
Mobile tactile perception experiment translating multisensory texture research into an accessible public app.
TactileMirror code
Code for a wearable sensory substitution device, supporting reproducible haptics and tactile augmentation research.
Simulation of a virtual stiffness
Archived demonstration of virtual stiffness and visual-haptic interaction.
Ultrasound/visual virtual viscosity
Archived demo exploring perceived viscosity through combined ultrasound and visual stimulation.
Aging Touch website
Public research resource on tactile perception, ageing, and surface texture perception, linked to BBSRC-supported work.
Oculus Connect haptic demo
Interactive haptic demonstration translating perceptual research into an industry-facing XR experience.Featured Publications
(2018).
Peri-personal space as a prior in coupling visual and proprioceptive signals.
Scientific Reports.
(2016).
Optimal Perceived Timing: Integrating Sensory Information with Dynamically Updated Expectations.
Scientific Reports.
(2014).
The duration of uncertain times: Audiovisual information about intervals is integrated in a statistically optimal fashion.
PLoS ONE.
(2011).
Effects of visual-haptic asynchronies and loading-unloading movements on compliance perception.
Brain Research Bulletin.
(2011).
The rubber hand illusion: Feeling of ownership and proprioceptive drift Do not go hand in hand.
PLoS ONE.
(2009).
Recalibration of multisensory simultaneity: Cross-modal transfer coincides with a change in perceptual latency.
Journal of Vision.
Articles in multisensory perception 32
(2026).
Temporal sensitivity and perceived timing in unimodal and crossmodal isochronous sequences.
Experimental Brain Research, 244, Article 173.
(2021).
Tactile Echoes: Multisensory Augmented Reality for the Hand.
IEEE Transactions on Haptics.
(2020).
Causality shifts the perceived temporal order of audiovisual events..
Journal of Experimental Psychology: Human Perception and Performance.
(2019).
Virtual grasping feedback and virtual hand ownership.
Proceedings - SAP 2019: ACM Conference on Applied Perception.
(2019).
Perceptual Limits of Visual-Haptic Simultaneity in Virtual Reality Interactions.
2019 IEEE World Haptics Conference (WHC).
(2018).
Modality-specific temporal constraints for state-dependent interval timing.
Scientific Reports.
(2018).
Peri-personal space as a prior in coupling visual and proprioceptive signals.
Scientific Reports.
(2018).
Touch with foreign hands: The effect of virtual hand appearance on visual-haptic integration.
Proceedings - SAP 2018: ACM Symposium on Applied Perception.
(2016).
Optimal Perceived Timing: Integrating Sensory Information with Dynamically Updated Expectations.
Scientific Reports.
(2016).
The Consistency of Crossmodal Synchrony Perception Across the Visual, Auditory, and Tactile Senses.
Journal of Experimental Psychology: Human Perception and Performance.
(2016).
Depth: The Forgotten Dimension in Multisensory Research.
Multisensory Research.
(2014).
Computational aspects of softness perception.
Multisensory softness.
(2014).
Duration perception in crossmodally-defined intervals.
Acta Psychologica.
(2014).
Multisensory Softness.
Springer London.
(2014).
The duration of uncertain times: Audiovisual information about intervals is integrated in a statistically optimal fashion.
PLoS ONE.
(2013).
Multisensory contributions to spatial perception.
Handbook of Spatial Cognition.
(2012).
Multisensory simultaneity recalibration: storage of the aftereffect in the absence of counterevidence.
Experimental Brain Research.
(2012).
Multisensory Perception: From Integration to Remapping.
Sensory Cue Integration.
(2011).
Effects of visual-haptic asynchronies and loading-unloading movements on compliance perception.
Brain Research Bulletin.
(2011).
Audiovisual asynchrony detection in human speech..
Journal of experimental psychology. Human perception and performance.
(2011).
The rubber hand illusion: Feeling of ownership and proprioceptive drift Do not go hand in hand.
PLoS ONE.
(2010).
Inconsistency of perceived 3D shape.
Vision Research.
(2010).
Within- and cross-modal distance information disambiguate visual size-change perception.
PLoS Computational Biology.
(2009).
Influence of visual and haptic delays on stiffness perception in augmented reality.
Science and Technology Proceedings - IEEE 2009 International Symposium on Mixed and Augmented Reality, ISMAR 2009.
(2009).
Recalibration of multisensory simultaneity: Cross-modal transfer coincides with a change in perceptual latency.
Journal of Vision.
(2007).
Temporal calibration between the visual, auditory, and tactile senses: A psychophysical approach.
Peach Summer School 2007.