It is becoming increasingly difficult for users to interact with the slew of portable gadgets they carry, especially in the area of text entry. Although miniature displays and keyboards make some portable devices, such as cell phones and PDAs, amazingly small, users’ hands do not shrink accordingly.
To solve this problem, we proposed a Virtual Keyboard. This device will replace a physical keypad with a customizable keyboard printed on a standard A3 size paper whose “keystrokes” are read and translated to real input. This virtual keyboard can be placed on any flat surface, such as desktops, airplane tray tables, kitchen counters, etc. and can theoretically be interfaced with any computing device that requires text entry. This would eliminate the need to carry anything around and also prevent any chance of mechanical damage to the keypad in harsh environments if a simple lamination is used to protect the paper. In addition, buttons on this device can be reconfigured on-the-fly to give a new keyboard layout using a GUI we built in Java and then transferring that data to the device using a computer’s serial port.
High Level Design
The Virtual Keyboard has three main components: the laser, camera, and printed keyboard. The laser beam is simply a conventional off-the-shelf red laser with a line-generating diffractive optical element attached to it. This assembly generates an invisible plane of red light hovering a few millimeters above the typing surface. When a finger passes through this plane, it shines bright red in that region.
The CMOS camera continuously captures images of the region where the printed keyboard is supposed to be placed and checks these images for red color data above a specified threshold. The threshold concept works in this case because the laser shining on a typical human finger generates saturating values of red color data, which is very easily distinguishable from its surroundings.
Lastly, the printed keyboard is simply a standard A3 size paper that contains a custom keyboard layout. After rigorous testing, we decided to use a black background and blue letters for the printed keyboard because our device doesn’t use its own light source. Therefore, proper contrast is necessary to distinguish the typing finger from the surrounding area in various lighting conditions. The actual programming of the printed keyboard layout into the device can be done using a serial port and a GUI we developed in Java. This GUI basically gives the user a blank grid of buttons and the user can choose to assign any button to any letter or number he/she desires.
The software component was split into 5 main components:
- Implementing the I2C protocol to read and write registers from camera
- Reading values from camera to obtain 6 frames every second
- Processing the images to obtain a pressed key
- Converting the pressed key into a scan code which is then transmitted using the PS/2 protocol
- Sending serial data from a java application to update the array of scan codes in the Mega32
At first we initialize PORTA on the Mega32 to take UV input from the camera and PORTC to communicate with the camera over the I2C interface. The baud rate is set to 19,200bps for serial communication. We then run the calibrate function on the camera, which looks at a black keyboard to determine a distinguishable value for red color threshold. Then we call a function called “init_cam” which performs a soft reset on the camera before writing the required values to corresponding camera registers. These registers change the frame size to 176×144, turn on auto white balance, set the frame rate to 6 fps, and set the output format to 16-bit on the Y/UV mode with Y=G G G G and UV = B R B R. The code then enters an infinite loop which checks for the status of the PS2 transmitting queue and tries to process the next captured frame if the queue is empty. If not, the queue is updated and the PS2 transmission is allowed to continue.
The getRedIndex function captures rows of data from the camera and processes each of them. We first wait until a neg edge on the VSYNC, which indicates the arrival of new frame data on the UV and Y lines. We then wait for one HREF to go by since the first row of data is invalid. At this point, we can clock in 176 pixels of data for a given vertical line in the Bayer format.
For more detail: Design a Customizable Virtual Keyboard using ATmega32