In this digital age, new interfaces for musical expression provide much broader musical possibilities than have ever existed before. There is a constant quest to be in harmony with one’s instrument so that music can flow freely from the imagination and take form effortlessly. This sparks an interest in new ways to interact with instruments, because we may be able to achieve more fluid methods for creating music. There are many new digital musical interfaces, but most are based on traditional musical instruments or are at least designed as a tangible object. This project aims to eliminate the physical “instrument” altogether. The sensor system enables the use of one’s own body as a musical instrument through detection of movement, freeing the artist from traditional requirements of producing live music. The ability to create and manipulate sound through movement provides the potential for immediate intuitive control of musical pieces.
Dance and music are quite obviously intertwined. One seems empty without the other. Dance and music deserve a close relationship, with no strings attached. The goal of this project is to blur the line between the two, and open up an avenue for them to mingle more intimately. Instead of dancing to music, it is now possible to create music by dancing. This project is a tool, an interface between motion and music, a new musical instrument. It is designed to be highly configurable to allow the artist as free a form of expression as possible. It is also designed to be fun and comfortable to use. This is a powerful tool and a fun toy.
High Level Design
This project implements a wireless, wearable sensor array which operates fast enough to generate MIDI (Musical Instrument Digital Interface) data in real-time based on the user’s motions. Four sensors attached to one’s body wirelessly send data to a MIDI host. The system is designed to be flexible, so it can be attached to a personal computer (PC), digital audio workstation (DAW), or stand-alone synthesizer, sampler, etc, and configured at will as an air guitar, input to an algorithmic composition, moshing interpreter, a live soundtrack to your daily life, or any other scheme you can imagine.
The user attaches four accelerometers and a wireless transmitter to his or her body to detect movement, which is used as input to the system producing music as an audio output. By wearing these sensors, the user gains physical control over musical parameters such as volume, pitch, timbre, and tempo through movement and dance. In addition to continuous control of musical aspects, the system can recognize gestures made by the user. Gestures can trigger musical events like playing an audio sample or non-musical events like switching between modes of operation. Multimodal operation is in fact a key design aspect to ensure maximum flexibility, so that the user can make decisions on the fly about what aspects of the musical piece to control and what types of movement control them. During a performance, mobility and independence is essential, so the user’s only input via this system is through the accelerometers. The design is such that all operations can be performed with no human-computer interface other than the wearable sensors.
An emphasis is placed on flexibility so that the sensor system may be used in multiple contexts. Therefore, the system can be configured for multiple modes of operation.
First, the system provides the ability to either create or manipulate music. The former mode of operation allows the user to essentially act as a solo instrument. The later mode allows real-time modification a musical piece played by either a computer or another live musician, much like a conductor uses movement and gestures to give direction to the performance of a piece.
Secondly, the system allows for either direct or indirect mapping of sensor data to musical parameters. The former mode of operation maps mostly unprocessed sensor data to musical parameters. The later provides a layer of algorithmic interpretation between sensor input and musical output, allowing the user to control musical parameters through cumulative multi-sensor movements.
There are many possibilities for configurable modes of operation, but the following table details the designations described above and lists some examples.
This project is constructed with an embedded system and a general-purpose computer or other MIDI host like a synthesizer. The embedded system is composed of wearable accelerometers and a radio frequency (RF) transmitter run by the first microcontroller unit (MCU), with the RF receiver run by the second microcontroller that also generates MIDI output. The MIDI data is then used as input to a host program on a personal computer (PC) for further data interpretation and audio generation. Although using a PC as the MIDI host is a more flexible option, the design of the embedded system is able to act as a standalone MIDI controller for interaction with other industry standard MIDI equipment. The following diagram describes the system from a high level.
|Description||Manufacturer||Manufacturer No.||Vendor||Vendor Number||Quantity||Unit Price||Total|
|ATmega644 + AT86RF230||Atmel||ATMEGA64RZAV-10PU||Digikey||ATMEGA64RZAV-10PU-ND||2||9.72||19.44|
|Prototype Board||Bruce Land||2||5.00||10.00|
|Tri-axis digital accelerometers||Kionix||KXP74-1051||Kionix||KXP74-1051||4||sampled||0.00|
|MIDI jack||CUI Inc||SDS-50J||Digikey||CP-7050-ND||1||0.90||0.90|
|0805 12pF capacitors||Digikey||4||0.50||2.00|
|AA battery holder||Radioshack||1||1.50||1.50|
|MIDI-USB interface||previously owned||0.00||0.00|
|USB Cable||previously owned||0.00||0.00|
|Recycled Ocean Spray Juice Container||previously owned||0.00||0.00|
|Microkorg synthesizer||previously owned|
For more detail: Movement to Music: A Wearable Wireless Motion Sensor system