The Cornell University Airmouse Initiative is a motion sensing glove with buttons on it that plugs into your computer to function as a mouse.
Many tasks that are performed on the computer require the use of both a keyboard on the mouse, and many people find it frustrating and awkward to have to switch back and forth between them. So, we’ve created what we believe to be the solution to this dilemma, as you can wear the glove on your hand while you’re typing and then when you want to move the mouse cursor, simply push a button that is already on your hand and point your finger at the screen and move it around. The Airmouse is comfortable to wear, and does not significantly inhibit typing. It functions completely as a two button serial mouse, and even implements variable rate vertical scrolling.
II. High Level Design
Our project was inspired by two sources. Firstly, when Professor Land exclaimed, “I have free accelerometers, and they [can do cool things],” we felt as if we should explore the possibility of using them. We thought for a while about what kind of awe-inspiring device we could construct with acceleration sensors, and then we found our second fountain of inspiration – the mouse-glove that Tom Cruise used in Minority Report would not only be very neat were it real, and it was not impossible to make it real. Accelerometers seemed to be the ideal sensor for constructing a hand-mounted pointing device.
Our original plan used accelerometers to measure the acceleration of a user’s hand and integrate that acceleration into a change in position. The math behind this is very easy – using Verlet Integration, we would approximate the integral of the acceleration to the second degree. Verlet integration interpolates between to measured accelerations, using the average slope between them to derive velocity. This is sometimes called “trapezoidal integraton.” Here are the equations, courtesy of Professor Land’s CS417 webpage:
This approach, however, turned out to be a practical impossibility. While we successfully implemented the Verlet scheme and watched a mouse cursor controlled by the scheme move on the screen as we expected it to, the glove had to be held exactly at 0g (or 1g for earth-normal) in all three directions. If gravity were allowed to affect the accelerometers at all, this acceleration added into the integral, and with no negative acceleration to remove it, the change in position that we calculated grew boundlessly. We needed gyroscopes to measure gravity so that we could remove its effect from our calculations, but gyroscopes are not inexpensive, and the cheapest ones cannot be soldered by human hands.
Because of this, we decided to construct a tilt mouse instead. While not as impressive as a position tracking device, the tilt mouse is easy to use (after a bit of practice) and almost as neat as a position tracking mouse. The math behind this scheme is very easy – measure the acceleration due to gravity on the mouse, and multiply this by some constant to scale your output to a desirable level. The Microsoft serial mouse protocol uses an 8-bit twos complement number scheme to send data to the computer, and the numbers outputted by the Atmel Mega32’s analog-to-digital converter can be conveniently represented in the same number of bits. However, the numbers outputted by the accelerometer had to be altered to normalize voltage extremes to -128 and 127 and the accelerometer’s neutral output to 0. We will discuss in the next section, Program/Hardware Design, exactly how this was done. In addition to scaling the output, we also used a step filter on the data to make the mouse easier to use. Our accelerometer was so sensitive that the slightest motion of one’s hand would cause the device to output a nonzero acceleration. To give the user a more stable region around the zero point, we quantized all outputs below a certain level to 0, then normalized any outputs out of this cutoff range by the breadth of the cutoff range. For example, if the cutoff were abs(10), the intersection of all numbers > -10 and all numbers < 10 defined the cutoff region, and any outputs outside of this range would be normalized by +10 or -10, depending on whether the output were negative or positive, respectively.
In addition to position changes, mice detect button presses. We implemented 4 buttons in our mouse – output on/off, left click, right click, and scroll enable. Each pushbutton is connected to a port pin on our microcontroller. When the output on/off button is pressed, the serial mouse output to the computer is either [re]enabled or disabled. This allows the user to move his or her hand and not have the motion affect the mouse pointer. Left click functions as a mouse left click, right click functions as a mouse right click, and scroll enable disables all motion and other button outputs while it is held down, and the y-output of the accelerometer is translated into a scroll-wheel output to the computer.
The only hardware/software tradeoff with which we had to deal was the sensitivity of our analog to digital converter (ADC). The granularity per “click” of the ADC is approximately 19.5mV/click. When we used accelerometers with sensitivities of approximately 50mV/g for our position mouse the outputs of the devices had to be normalized to the full ADC range of 0-5V so that we would have good sensitivity on our device. We will explain in the next section exactly how this was done. For our tilt mouse, we used accelerometers that outputted 1000mV/g, so the sensitivity was much higher, but we decided to normalize the range to 2000mV/g to gain even higher sensitivity, especially since we already had the necessary circuits designed and constructed. Another hardware/software issue one must usually take into consideration is the “bouncing,” or inconsistent, state in which some pushbuttons can be after a transition, but our buttons were internally debounced, as can be seen in the below figure, so we did not have to worry about polling them at certain intervals.
Our design is related to two standards. We use the Microsoft serial mouse protocol and an RS232 serial connection. RS232 uses -+12V for high and low signals, respectively. RS232 idles at +12V. Our microcontroller outputs 5V high, 0V low serial (TTL level), so we had to use a serial level shifter chip to allow the microcontroller to properly speak with the computer. The Microsoft serial mouse protocol works as such: when the RTS line of the serial connection is pulled logic high then falls logic low again, the computer expects you to send an identifier string to it so that it can determine what type of device you are. The protocol expects data at 1200 baud, 8N1. Our controller sends it “MZ” for Microsoft serial scroll mouse. The computer may query you for this information multiple times, but after 5 or so times you can stop sending it the identifier code and it will still consider you to be identified. The packet format is as follows:
L, R, and M are the left, right, and middle mouse buttons. A 1 indicates pushed and a 0 not pushed. (most significant) Y7-Y0 (least significant) represents the twos complement mouse delta Y, with positive Y pointing to the bottom of the screen. X is the same format as Y, with positive X pointing to the right of the screen. Z is also in the same format as Y, with positive Z being scroll up. The first 1 bit in every packet is an extra stop bit. From what we’ve seen, some implementations set it 1, and some implementations set it low. We listened to the data coming off of a genuine Microsoft serial mouse, and it set the bit as 1, so we chose to set it as 1.
At http://patft.uspto.gov, we searched for “accelerometer AND mouse” and “mouse AND glove,” and we found one invention very similar to our project. US patent #6,870,526 is a “Glove mouse with virtual tracking ball.” It uses roll of the hand to measure direction of mouse traversal and bending sensors in the thumb and index finger apparatuses to determine whether the mouse will move in a negative or positive direction on the indicated line of orientation. This mouse functions differently than ours, as ours uses magnitude of roll and pitch to determine the direction and speed of the mouse motion. No mention of mouse click buttons was made in the description of this invention.
III. Program/Hardware Design
First we wait for the computer to toggle the RTS line. When it does, we send it “MZ,” the indicator that we are a Microsoft serial scroll mouse. We respond to 5 queries, then disable query response and start mouse functioning.
After the mouse has started functioning, the buttons are sampled on every cycle. If the mouse is not enabled, the program cycles through the button sampling until the mouse is enabled. After the buttons are sampled and the mouse is enabled, a check is done to see which acceleration axes have been sampled. If X has not been sampled, set the sample multiplexer to Y, sleep until X is ready, and sample X. The multiplexer is set to Y before we sample X because the current sample being processed is always taken off of the current multiplexer value, which is X if X has not been sampled. If X has been sampled, we check to see if Y has been sampled. If it has not been, set the multiplexer to X, sleep until Y is ready, and sample Y. If Y has been sampled, modify the samples to the correct twos complement numbers, scale them, and create the serial packets. Send the serial packets, indicate that X needs to be sampled, and return to the beginning of the loop. The mouse is enabled and disabled by one of the sampled buttons. If the button value is read as “pressed” and its previous value was “not pressed,” the mouse enable/disable is toggled.
The tricky parts of this program were the timing and the conversion of our accelerometer output to the proper twos complement number. We had some problems with our accelerometer readings – they became erratic on certain types of reads, specifically when we moved the mouse on the NW-SE plane. We could not figure out how to get steady readings until we googled our problem and found a page put up by previous ECE476 students. These students (credited in our conclusion) built a mouse similar to ours, and to get good readings they slept their microprocessor until the reading was ready. We simply added sleeps before we read our values, and the entire setup worked as we expected it to. As for our number formatting, when we formatted the numbers by performing mathematical operations on them, we found that our math often caused unexpected overflows. These overflows, coupled with the fact that we could never tell if we were casting the numbers properly, prompted us to devise an almost purely bit-manipulative method of converting our accelerometer outputs to numbers that we could send to the computer. Please see our commented code in the Appendix for a detailed description of our number conversion process.
|Part Num.||Description||Qty.||Unit Price|
|—||9V DC Power supply||1||$5.00|
|AMP0452||DB-9 connector for serial port||1||$2.00|
|MAX233A||TTL to RS232 convertor||1||$0.00 (sampled)|
|—||Custom PC board||1||$5.00|
|LM358P||Op amp DIP||1||$0.45|
|—||Trim potentiometers from lab||2||$0.50|
|UA78M33CKC||3.3V fixed voltage regulators||2||$0.56|
|ADXL203||Analog Devices Accelerometer||1||$0.00 (sampled)|
|—||Pair of girly prom gloves||1||$8.00|
|—||Old unmated sock||1||$0.00 (from Joe’s sock drawer)|
|***Never Received 2x RC4136N Quad Op-Amp, Were Told to Use LM358P Instead|
For more detail: AirMouse using ATMega32