Visual delays are unavoidable in augmented reality setups and occur in different steps of the rendering pipeline. In the context of haptic interaction with virtual objects, it has been shown that delayed force feedback can alter the perception of object stiffness. We hypothesize that delays in augmented reality systems can have similar consequences. To test this, we carried out a user study to investigate the effect of visual and haptic delays on the perception of stiffness. The experiment has been performed in an optimized visuo-haptic augmented reality setup, which allows to artificially manipulate delays during visual and haptic rendering. In line with previous results, delays for haptic feedback resulted in decreased perceived stiffness. In contrast, visual delays caused an increase in perceived stiffness. However, the simultaneous occurrence of delays in both sensory channels led to a partial compensation of these effects. This could potentially help to correct stiffness perception of virtual objects in visuo-haptic augmented reality systems.