Conducting “In The Box”
Musicality is the intersection between movement and sound. Music and dance have been bound to each other since before our species can remember itself, and the two cannot be split without a loss of efficacy. They are two sides of the same phenomenon—like electricity and magnetism. Musicality is motion. Motion is musicality. To say it plainly: without an understanding and fluency in motion, one cannot be musical.
A computer’s interface is generally dumb to motion; therefore, it cannot understand or engender musicality. Sure, people make fantastic music on computers, but we wage an interface battle with computers in order to encode our inherent musicality into an inhospitable environment. Even so, more and more musicians and music professionals on both the young and experienced ends of the spectrum have been turning to the computer to inspire, create, and mix their music. It’s called “mixing in the box.” I have to think hard to find a less appealing phrase.
Conductors were the first audio engineers. With a deftly rising, falling, and swooping hand accompanied by a metronomic wand they controlled what listeners heard. Volume, speed, dynamics, character, and sometimes content. They shaped raw music into experience, into motion by using nothing more than motion. Modern engineers do it with mixing boards, pan pots, reverb units, delays, flangers, EQs, compressors, expanders, and—god save us—computers. In order to interface with a single piece of music, an audio engineer may stare at and tweak hundreds of disparate, clunky, asynchronous interfaces, each one a dozen times over.
I’d like to investigate the role gesture might play in educating computerized musical engineering interfaces as to the natural motions of music. Possible applications could include:
- Humanizing a beat sequencer to generate a more dynamic “performance” and a deeper kinetic response to computer generated beats.
- Re-imagining the interfaces of a mixing engineer to perform more like those between conductor and orchestra
- Approaching audio effects such as compression, EQ, reverbs, delays, et al using gestural sculpting
There have been inroads to educating computers about motion in music. Wii Music is a great example of an interface that allows for a playful physicality to be returned to a previously arranged piece of music as the participants control dynamics through motion activated Wiimotes. Tod Machover’s Media Lab group, Opera of the Future, is creating richly physical instruments which can be played intuitively through gesture and other types of physical computing. A central theme in Tod’s work is breaking down barriers for non-musicians with unusual instruments so that they can begin creating rich music without formal training. He’s got a great TED talk.
