Mister Gloves – A Wireless USB Gesture Input System Using Atmega644

High Level Design

Rationale | Feature Set | Logical Structure | Tradeoffs | Standards

Rationale and Sources

The motivation for our project was to create an intuitive input system that would be easy and fun to use for computer applications that are not particularly keystroke-intensive, such as surfing the web and playing (certain) video games.
We were particularly inspired by a few commercial products, hobby projects, and past ECE 4760 final projects:
Clove and Peregrine are both glove controllers that rely on various finger, thumb, and palm touch actions. We decided to improve upon this concept by incorporating accelerometers to detect the tilt of the hand as well, thus permitting wrist movements to control the keystroke input in combination with the touch actions. This also allows for a comparable assortment of possible keystrokes with a reduced number of touch or contact sensors, which in turn enables us to define larger contact zones to allow easier activation.
The ECE 4760 Cornell University Airmouse Initiative is a glove controller that controls the cursor through up/down right hand movement and rotation of the wrist. We incorporated this feature and implemented mouse smoothing and additional gesture-sensing elements to provide a higher degree of control for the user.
Additionally, the ECE 4760 USB Magnetic Mouse project gave us confidence about the feasibility and reliability of using software USB.

Feature Set

Input Mapping:
We first list the gestures that can be captured by our system as well as the corresponding keyboard/mouse inputs. Pictures here are shown with bare hands since the current controller hardware is somewhat bulky (this is a prototype/proof-of-concept, after all) and may obscure anatomical parts.
There are four starting points for a right hand gesture, each giving different input possibilities. In the right hand “finger gun” position, curling the index finger performs a left click while thumbing down performs a right click. The wrist can be rotated left and right to move the cursor horizontally, and the hand can be tilted up or down to move the cursor vertically.
n the right hand “downward fist” position, moving the hand upwards scrolls up and moving the hand downwards scrolls down.
n the right hand “upward fist” position, curling the index finger causes an alt key to be typed, while thumbing down causes a ctrl key to be typed.
Finally, in the right hand “karate chop” position, thumbing down causes a shift key to be typed.
Note that the ctrl, shift, and alt motions are meant to be used in combination with the left hand controller.
There are 4 tilt states for the left hand and 14 contact spots on the hands and fingers that can be touched by the thumb, making for a total of 56 keys
Audio Feedback:
There is a piezoelectric buzzer on the left hand controller that can play a square wave at various frequencies corresponding to different tilt states. When a contact is activated, audio is played at double the frequency (one octave up).
Note that these frequencies correspond to four of the open strings on a guitar with standard tuning (as well as those of the notes in an E minor 7 chord). The notes are D (293.66 Hz), G (392 Hz), B (493.88 Hz), E (659.26 Hz), for the “back up”, “thumb up”, “palm up”, and “fingers up” positions respectively. It has been brought to our attention, however, that the audio sounds more like it is coming from a bagpipe. The volume can be adjusted via a trimpot on the left hand controller next to the buzzer.
Settings:
We also included calibration/setup hardware. There are four pushbuttons on the left hand controller, each used to reset a different tilt state to the user’s preference. To reset the hand position for a tilt state, the button is simply pressed while keeping the left hand in the desired position.
There are two trimpots on the right hand controller for controlling horizontal and vertical mouse sensitivity. There is also a calibration trimpot for controlling how much the index finger must be curled to activate a click. Additionally, the right hand controller includes a big red button for resetting the right hand tilt position (for the “finger gun” gesture) as well as for turning on the entire system and activating the base station.

Logical Structure

Our input system is separated into three basic components. The MCU in the left hand controller takes measurements from a pair of accelerometers to calculate the current tilt position of the hand. Note again that the tilt positions can be reset via pushbuttons. Additionally, it determines which finger or hand contact is currently activated. These readings are then passed over to the MCU in the right hand controller, which is connected by wire. Optionally, the left hand controller can output audio through a piezoelectric buzzer, where the frequency of the audio corresponds to the current tilt of the hand as mentioned above.

Simultaneously, the right hand MCU takes measurements from a pair of accelerometers within a 5 degrees of freedom sensor package, flex sensors, and pushbuttons to determine the current gesture. Next, the right hand MCU processes all readings from both hands them and converts them into the corresponding keyboard or mouse inputs. The requested keyboard and mouse inputs are then wirelessly transmitted to the base station via a packet radio.

Parts List:

Total: $55.40

For more detail: Mister Gloves – A Wireless USB Gesture Input System Using Atmega644