Mister Gloves is a wireless USB gesture input system that enables a person to use a computer by performing intuitive hand and finger motions in the air. While wearing a glove controller on the right hand, the user can move the cursor by forming a pointing gesture and click by curling the index finger or thumb. Other right hand gestures enable scrolling and provide access to various keyboard shortcuts. Wearing a glove device on the left hand allows the user to type different keys through a combination of tilting the hand and touching different portions of the palm and fingers with the thumb. The left and right controllers are attached to each other but communicate to the computer wirelessly through a base station. Mister Gloves uses a low-speed USB 1.1 interface and does not require the installation of additional drivers.
Our gesture input system can be conveniently used by anyone who wishes not to be tied down to a desk when using a computer, making it perfect for giving presentations or web surfing from the couch. The intuitive hand motion controls also allow it to serve as an alternative video game controller. Additionally, since our input system does not exert pressure on the median nerve at the wrist while in use, it may prevent the development of carpal tunnel syndrome and other repetitive stress injuries.
The motivation for our project was to create an intuitive input system that would be easy and fun to use for computer applications that are not particularly keystroke-intensive, such as surfing the web and playing (certain) video games.
We were particularly inspired by a few commercial products, hobby projects, and past ECE 4760 final projects:
Clove and Peregrine are both glove controllers that rely on various finger, thumb, and palm touch actions. We decided to improve upon this concept by incorporating accelerometers to detect the tilt of the hand as well, thus permitting wrist movements to control the keystroke input in combination with the touch actions. This also allows for a comparable assortment of possible keystrokes with a reduced number of touch or contact sensors, which in turn enables us to define larger contact zones to allow easier activation.
The ECE 4760 Cornell University Airmouse Initiative is a glove controller that controls the cursor through up/down right hand movement and rotation of the wrist. We incorporated this feature and implemented mouse smoothing and additional gesture-sensing elements to provide a higher degree of control for the user.
Additionally, the ECE 4760 USB Magnetic Mouse project gave us confidence about the feasibility and reliability of using software USB.
We first list the gestures that can be captured by our system as well as the corresponding keyboard/mouse inputs. Pictures here are shown with bare hands since the current controller hardware is somewhat bulky (this is a prototype/proof-of-concept, after all) and may obscure anatomical parts.
There are four starting points for a right hand gesture, each giving different input possibilities. In the right hand “finger gun” position, curling the index finger performs a left click while thumbing down performs a right click. The wrist can be rotated left and right to move the cursor horizontally, and the hand can be tilted up or down to move the cursor vertically.
n the right hand “downward fist” position, moving the hand upwards scrolls up and moving the hand downwards scrolls down.
n the right hand “upward fist” position, curling the index finger causes an alt key to be typed, while thumbing down causes a ctrl key to be typed.
Finally, in the right hand “karate chop” position, thumbing down causes a shift key to be typed.
Note that the ctrl, shift, and alt motions are meant to be used in combination with the left hand controller.
There are 4 tilt states for the left hand and 14 contact spots on the hands and fingers that can be touched by the thumb, making for a total of 56 keys
There is a piezoelectric buzzer on the left hand controller that can play a square wave at various frequencies corresponding to different tilt states. When a contact is activated, audio is played at double the frequency (one octave up).
Note that these frequencies correspond to four of the open strings on a guitar with standard tuning (as well as those of the notes in an E minor 7 chord). The notes are D (293.66 Hz), G (392 Hz), B (493.88 Hz), E (659.26 Hz), for the “back up”, “thumb up”, “palm up”, and “fingers up” positions respectively. It has been brought to our attention, however, that the audio sounds more like it is coming from a bagpipe. The volume can be adjusted via a trimpot on the left hand controller next to the buzzer.
We also included calibration/setup hardware. There are four pushbuttons on the left hand controller, each used to reset a different tilt state to the user’s preference. To reset the hand position for a tilt state, the button is simply pressed while keeping the left hand in the desired position.
There are two trimpots on the right hand controller for controlling horizontal and vertical mouse sensitivity. There is also a calibration trimpot for controlling how much the index finger must be curled to activate a click. Additionally, the right hand controller includes a big red button for resetting the right hand tilt position (for the “finger gun” gesture) as well as for turning on the entire system and activating the base station.
Our input system is separated into three basic components. The MCU in the left hand controller takes measurements from a pair of accelerometers to calculate the current tilt position of the hand. Note again that the tilt positions can be reset via pushbuttons. Additionally, it determines which finger or hand contact is currently activated. These readings are then passed over to the MCU in the right hand controller, which is connected by wire. Optionally, the left hand controller can output audio through a piezoelectric buzzer, where the frequency of the audio corresponds to the current tilt of the hand as mentioned above.
Simultaneously, the right hand MCU takes measurements from a pair of accelerometers within a 5 degrees of freedom sensor package, flex sensors, and pushbuttons to determine the current gesture. Next, the right hand MCU processes all readings from both hands them and converts them into the corresponding keyboard or mouse inputs. The requested keyboard and mouse inputs are then wirelessly transmitted to the base station via a packet radio.
|Component||Part Number||Qty||Cost||Acquire Method|
|ECE 4760 prototype board||—||3||$12.00||Lab stock|
|Spectra Symbol Flex sensor||FS-L-0112-103-ST||1||$12.95||Purchased|
|Solder board||—||2||$5.00||Lab Stock|
|Duracell AAA Batteries||—||4||$3.00||Purchased|
|Machine/header pins||—||350 (est.)||$17.50||Lab stock|
|ST Microelectronics 3.3V Voltage regulator||LD1117V33||1||$1.95||Purchased|
|Sparkfun 5 degrees of freedom sensor||—||1||Donated by Prof. Land|
|Nike Weightlifting glove||—||1||Pre-owned|
|Atmel ATmega16 Microcontroller||ATMEGA16||4||Lab stock|
|Mini protoboard||—||1||Borrowed from CSL Lab|
|Freescale Single-axis accelerometers||MMA2260D, MMA1260D||1 of each||Lab stock|
|Radiotronix Transceiver||WI.232FHSS-25-R||2||Donated by Prof. Land|
|Resistors, trimpots, capacitors||—||—||Lab Stock|
|National Semiconductor 5V Voltage regulator||LM340T5||1||Lab Stock|
|3.6V Zener diode||—||2||Pre-owned|
|National Semiconductor Op Amp||LM358||1||Lab Stock|
|Battery case||—||1||Lab stock|
|Microcontroller ribbon cable||—||1||Lab stock|
For more detail: Mister Gloves – A Wireless USB Gesture Input System Using Atmega644