Multisensory softness

Compliance is the amount of physical deformation of an object for the amount of force applied. Compliance perception (i.e., object softness) is a case of active sensing, because we need to interact with the object in order to collect sensory information. For this, there is no single sensor that captures compliance. Our brain needs to combine several sensory signals to obtain an estimate. Some of these signals are redundant (position of the fingers is sensed proprioceptively and visually) and other are complementary (both force and position are required) and they vary over time. How are multiple sensory signals used to perceive object compliance? Redundant information about the amount of deformation of the object surfaces is obtained from vision and haptic sensory modalities. Even when the viewed fingers position is artificially made to be discrepant with the proprioceptive position of the finger, both senses are still used for perception. To obtain an estimate of compliance the two positions are combined through a weighed average but the weights do not change depending on the manipulation (they are “sticky”, Kuschel, Di Luca, & Klatzky 2010). Because compliance can be sensed only through changes in the force and position, there can be no estimate at one time-point. The brain needs to integrate information over time, possibly during the whole interaction. Information during loading (pressing onto the object) is more informative about the object compliance, and it also contributes more to the perceptual estimate (Di Luca, Knörlein, Ernst, & Harders, 2011). The imbalanced contribution of information during loading and unloading is responsible for the distortion of perceived compliance when visual information is delayed (Knörlein, Di Luca, & Harders, 2009). A delay makes the object appear less deformed than it is during loading movements, creating the illusion of a harder object. Moreover, the brain also needs to combine information coming from each of the fingers. In a pinch grasp the sensed force is common between the fingers (i.e. the same force is sensed at the index and thumb) but the fingers are free to move independently. The independence in movement can make the sensory information to differ substantially in particular situations (i.e., when objects composed of different materials are grasped). In such cases, the contribution of the individual fingers to the final percept depends on the movement performed, so that the finger that presses more contributes more to the estimate of compliance (Di Luca, 2011). This makes sense because the finger that applies more force also senses compliance more reliably, but it can also give rise to unexpected phenomena, like the fact that such composite objects are perceived to be differently soft depending on which finger is in contact with which side.