Easy Input is a head-controlled keyboard and mouse input device for disabled users.
The system uses accelerometers to detect the user’s head tilt in order to direct mouse movement on the monitor. The clicking of the mouse is activated by the user’s eye blinking through a reflective sensor. The keyboard function is implemented by allowing the user to scroll through letters with head tilt and with eye blinking as the selection mechanism.
This project was a five week design lab for ECE 476 at Cornell University. Video of the final demonstration will be available in the Results section.
High Level Design
Project Inspiration
This project was motivated by the novella/film �The Diving Bell and the Butterfly� based on the life of Jean-Dominique Bauby. At the age of 42 Bauby, the then French editor the magazine ELLE, suffered a massive and spontaneous stroke while driving down a road in rural France. After several weeks in a coma, he awoke with what�s known as �locked-in syndrome�, with full muscular paralysis, and with only the control of his left eye remaining. Despite the loss of almost all of his physical ability, his sight, hearing, and especially mental function remained unchanged. He (the butterfly) was essentially trapped inside his body (the diving bell). With the aid of an extra-ordinarily patient speech therapist, he gained the ability to communicate again by blinking as she orally read a frequency sorted alphabet to him. In this way he would slowly form words and then sentences, and over the period of months, with extreme difficulty, was able to write the aforementioned account of his experience. This primitive form of communication was excruciatingly tedious to both Bauby and his �interpreter�, requiring extreme patience from both individuals as letter entry was quite slow and any mistakes were not easily corrected.
Overall Concept
We constructd an interface system that would allow a similarly paralyzed user to interact with a computer with almost full functional capability. That is, the system operates as a mouse initially, but the user has the ability to toggle in and out of a keyboard mode allowing the entry of text. This is achieved by using the control from a single eye, tracking the position of the pupil for direction, and using blinking as an input. As detection of eye motion proved too challenging, we built an accelerometer based tilt detector to determine head motion, so that, although not as applicable in this particular case, it might be use by a quadriplegic individual.
Relevant Standards
PS/2 Standard for Keyboard and Mouse
Nowadays, most mice and keyboards use Universal Serial Bus (USB). However, we decided to simulate a mouse and a keyboard using PS/2 instead of USB because it is easier and seems to be more doable within one month time. Because the computers in the laboratory do not have PS/2 ports on the mother board, we will need a PS/2 to USB cable to connect to the computer. PS/2 transmits serial data at a rate of 10 to 16kHz with 1 stop bit, 1 start bit and 1 odd parity bit. There are 6 pins in a PS/2 connector. The pin specifications are as follow.
PS/2 for Keyboard
When a key is pressed, a unique make code for that key is sent to the computer. When a key is released, another unique break code for that key is sent. According to Keyboard Scan Codes: Set 2 which is used by all modern keyboards, a make code is either 1 or 2 bytes and a break code is either 2 or 3 bytes. There is a simple relationship between the make code and break code. For example, the make code for an ‘A’ is 0x1C and the corresponding break code is 0xF01C. Besides the key press event, the computer may send some commands to the keyboard. We also need to handle these commands such as RESET and RESEND in order to make it works.
PS/2 for Mouse
There are many extensions on how the mouse communicates with the computer. We stuck with the most basic one because we do not need features such as more buttons and scrolling.
The standard PS/2 mouse sends movement and button information to the computer periodically using a 3-byte packet. The specifications of the packet are shown below. The mouse samples the movement at a frequency from 10Hz to 200Hz. Similar to a PS/2 keyboard, we need to handle command from the computer.
For more detail: Head-Controlled Keyboard And Mouse using ATmega32
