Musical Instrument Design
Experience Design and Musical Instruments
Experience Design (XD) extends the field of user experience (UX), just as UX extended the older concept of usability. Usability appeared in the 1980s, applying ergonomic principles to that era’s emerging technologies: word processors, spreadsheets, and others. In turn, the emerging technology of the 1990s — online shopping, SMS (texting), early camera phones, pre-smartphone PDAs — brought forth UX.
Again, today’s emerging technology demands broader design, as previously separate elements combine to form new experiences, such as our society where almost everyone, from awaking to falling asleep, carries a smartphone for browsing and posting to websites for social media. Hence, the need for XD: as the experience is broader than one person using one device, so the designers of the phone’s components (camera, operating system, apps, network infrastructure, even its fee schedule) must look beyond their individual specialties to construct the overall experience of the individual and the society. For example, the public was unprepared for the calamity of texting while driving. Only recently has this danger prompted one telecom company to commission a famous movie director to make a documentary about it, distributed for free; but as long as the always- online lifestyle is promoted in advertisements that are also distributed for free, such films will be taken just as seriously as the brewery billboard footnotes that advise consumers to “drink in moderation.”
The point of this admittedly sensational example is that texting while driving is not a Designed eXperience. There is a need for XD. To that end, I offer an example from which XD can learn: the field of musical instruments. It clearly connects to human-device interaction. Its social roles are well studied. Less obvious is its long history of disruptive high technology. Already three centuries ago, the interface for the pipe organ had grown to several keyboards, a pedalboard, and a dizzying array of other buttons, drawknobs, and levers, as depicted in Figure 11-1. So complex were these controls that page-turning assistants were needed to operate them. Organists were the space shuttle pilots of the day, and much effort was spent to help them get the best possible sound out of something that had been built at proportional space shuttle expense.
The design of musical instruments illuminates XD, because these devices present interfaces that are sophisticated, elaborate, refined over centuries, beautiful, and exquisitely adapted to the shape, capabilities, and senses of the human body. (To prepare for rapid change in the future, it helps to take a long view of history.) The interfaces of musical instruments also demand — and reward — tens of thousands of hours of continued use and study. One cannot say the same of handheld electronic doodads. Even if these facts alone warrant the study of musical instruments, the parallels to XD go deeper still. Let’s begin with the organ.
Figure 11-1. Console of the pipe organ at St. John’s Catholic Chapel, Champaign, Illinois
The organ’s primary feedback mechanism is not acoustic, but haptic. (The pipes can be so far from the console that organists joke about playing a fugue and then sitting back to listen to it.) Precise standards are set for the keys and pedals: required force, traveling distance, travel point where air gets admitted to the pipes, surface friction. Physically, keyboards have not grown in width like the piano has, because the lowest and highest notes are attained through other means than mere reach. On large theatre organs, the upper keyboards tilt down, sometimes almost vertically. Cognitive issues relate to which pipes are sounded by a particular keyboard. (To the newcomer, the organ’s profusion of modes may be its most terrifying aspect.) Individual drawknobs enable particular timbres, such as clarinets, flutes, or trumpets. “Mixture” drawknobs enable several timbres (ranks of pipes) simultaneously. Combination pistons set an entire collection of drawknobs with a single push, from a thumb or a toe, whichever is free at that moment during a performance. Couplers connect one keyboard’s pipes to another keyboard, or sound pipes an octave higher or lower than usual. A bass coupler connects the pedalboard’s pipes to the lowest note currently held on a keyboard. Most revealingly, even with recent computerization and touchscreens, organs don’t try to helpfully draw a diagram of active connections. That would only be a cluttersome distraction; instead, visual feedback is limited to indicating which couplers are active. The organist memorizes how these are wired together, a task not much more complicated than memorizing the layout of a car’s stick shift.
Speaking of computers, the parallels to XD grow stronger for newer musical instruments that incorporate software.
Later in the chapter, we’ll see how adding a tilt sensor to a pitch-tracked electric guitar can make it either easier to play or impossibly harder, depending only on the software. We’ll also see the many ways that dancers can affect the music to which they’re dancing by using software to interpret their movements sensed by a motion capture rig.
Even if a software-based instrument lacks the organ’s combinatorial explosion of mapping keyboards to ranks of pipes, software’s sheer flexibility seduces the designer into making the instrument so reconfigurable that the player has little attention left for actual playing. (In the limit, live coding replaces all traditional performance with writing software. But this chapter deals only with instruments that still respond to real-time physical gestures.) The player’s cognitive limits must be respected but also challenged: the kazoo has spawned no Mephistophelean virtuosos, no grand concertos. A reasonable challenge rewards the player with repeated levels of mastery, like a well-paced video game.
Software lets us arbitrarily connect the player’s physical gestures to the resulting sounds. Properly exploiting such richness requires guidance from XD. Only then can that realm of infinite possibility be concentrated into one instrument, one that is not merely enjoyable to listen to, but also worth playing and worth mastering.