Multimodal interaction

When going into the detailed interaction design for wearables, you need to consider three relevant human senses: sight, hearing, and touch.

These senses can be communicated with through several main interaction channels (multimodal interaction):

  • ? Visual (display)
  • ? Audio (sound/voice)
  • ? Tactile (touch/vibration)
  • ? Physical keys

These interaction channels serve two main information flows: output (user feedback) and input (data entry). Let’s explore each one through the lens of the main interaction channels.


As discussed throughout this chapter, most wearable devices today focus on the output information flow — providing feedback to the user (based on data these devices collect seamlessly).

Given that wearable devices are by definition worn on the body and thus come in direct (or close) contact with the skin, tactile feedback becomes a primary interaction channel.

In some cases, tactile is even more important than the visual. Why?

Let’s analyze this by going back to the wearable display types, which handle the visual feedback:

? Wearables that have no display cannot rely on visual feedback at all. Consequently, tactile feedback becomes the main communication channel.

Audio (whether as a sound effect or voice output) is also an option that might be an effective feedback channel in certain contexts. However, it carries several caveats:

It can be heard publicly-, and thus takes away the advantage of discreteness many wearables provide.

To set audio feedback to a minimum (to keep it private), the device needs to be located in close proximity to the ears. This puts a significant constraint regarding the device placement.

Incorporating sound requires adding a way (setting/button/gesture) to turn it on and off, for cases in which it might disturb the user or her surroundings (work meetings, cinema, going to sleep, and so on)[]

Furthermore, between the tactile and audio channels, the tactile one (vibration) is often much more salient than sound, which brings up the question whether an additional feedback channel is even needed. Be aware that both vibration and sound are interruptive — they immediately grab the user’s attention and disrupt what she’s doing. For the sake of simplicity and delight, if you can have a single feedback channel, one that is subtle yet effective, you should prefer it and avoid any added disruptions (especially where with some wearables, the frequency of alerts can become pretty high).

  • ? Wearables that have minimal display should definitely utilize the visual feedback channel. However, visual clarity should also be accompanied by tactile feedback that is more immediate and noticeable. This is especially important when the feedback is a result of a system-triggered alert, and not initiated by the user. Remember that in many cases, the wearable is peripheral to or completely out of the user’s visual field; therefore, the feedback could be easily missed, without actively drawing the user’s attention to it.[]
  • ? Wearables that have full interactive displays have less clear-cut guidelines with regard to the feedback channels and require closer analysis on a per-case basis. In smartglasses, for example, the visual display is obviously the primary feedback channel, as it’s integrated into the user’s field of vision. In this case, there is no real need to add yet another feedback channel in the form of vibration. Also, the head area — and especially around the eyes — is not the best area for tactile feedback. In addition, because smartglasses are placed close to the ears, the audio channel becomes an effective channel, as well. You can use this channel for sound effects indicating certain events as well as for voice output (reading the information presented on the screen, for example).

With smartwatches, however, both visual and tactile feedbacks are important for information output. Similar to minimal-display wearables, the device is generally away from the visual field, and although a visual effect in a full-screen display draws more attention compared to a minimal display, it can still be easily missed.

< Prev   CONTENTS   Source   Next >