Accelerometer Based Hand Action Recognition using Atmega644

Introduction

We created a wearable game controller that uses accelerometers to acquire action of the hand and then maps an action to an arbitrary keystroke. The types of actions we are trying to recognize should be suitable as input control for video games.

We placed 3 z-axis accelerometers on tips of the thumb, the index finger and the middle finger, and three accelerometers on the back of the hand for x,y, and z acceleration. The Atmega644 microcontroller read the output of the accelerometers and simulates a finite state machine to compute the gesture and the motion of the hand. The gesture and motion information is then transmitted to PC through serial connection, and a Java program is used to read the information and map it to an arbitrary keystroke.

High level design

Rationale

Some current game controllers such as the Wii remote take use of sensors and are capable of using the player’s motion as input, to some extent. However, none of the controlling method can take advantage of the expressibilities of natural gestures. As avid gamers, we determined that a hand-action-based controller would be a novel and fun input device.
By observation, we have found that many meaningful actions performed by a hand (e.g. smash, swing or push) can be described by the hand gesture and the movement of the palm. Furthermore, the hand gesture should be independent with the hand movement and orientation, but can only be changed from one to another by movements of fingers. On the other hand, the orientation is only dependent on the movement of the hand. Therefore in our project, we propose a prototype of real time hand action recognition by 8-bit microcontroller using acceleration data from finger tips and back of the hand.

Background Math and Physics

Accelerometers: the accelerometers we used measures acceleration from a capacitive sensing cell (g-cell) which forms two back-to-back capacitors. As shown in Figure 1, when the center plate deflects due to acceleration, the value of capacitors will change and acceleration data can be extracted.
Each accelerometer’s reading consists of dynamic acceleration and static acceleration, namely the gravity. For a z-axis accelerometer placed flat, Figure 2 shows the sign of the reading. The static acceleration is +1g, where as the dynamic acceleration reading is positive if the direction of the acceleration is upward.
By placing three accelerometers on the back of the hand, we set up a coordinate system. If the hand is still, we can measure direction of the hand plane. Moreover, if the finger moves slowly, then at each reading its acceleration are mostly due to gravity.
If the hand moves very rapidly, the process is a sequence of acceleration-deceleration. Therefore the output will show a large peak and afterwards a peak with opposite sign. Figure 4 shows the z-axis acceleration output when a hand suddenly moves downward and goes back.
he high level design of our project is shown in Figure 5. This diagram shows the flow of data from when the user end Java Application opens the port for communication to when then information is transmitted to the Application and mapped to a keystroke.

Hardware/Software Tradeoffs

One of the major limitations for our project is the functionalities of the accelerometers. Limited by budget, available materials and soldering technique, we only used one-axis low-g analog accelerometers. Although these accelerometers require simpler hardware and software than a digital or multi-axis accelerometers does, their outputs are very noisy. In addition, they have a measurement range of only 1.5g, meaning that fast movements will cause the output to reach the rails. It is difficult to integrate the acceleration to get velocity, not to mention position. However, we applied several techniques to reduce the noise and were still able to acquire much useful information from the accelerations. Specifics are covered in Software and Hardware sections.
Another tradeoff is expandability of the system versus usability. We cannot hard code every possible action. However, adding more actions meaning adding states in the state machine, which poses a difficulty for the user. We have to define a system that is ready to go, but can also be easily expanded.

Hardware Design

Our hardware consists of three parts: the custom PCB for Atmega644, the accelerometer circuit, and the Pololu USB AVR programmer, which we used as the serial connection between PC and the custom PCB. Schematics for all hardware can be found in the Appendix III: Hardware Schematics.

ECE 4760 Custom PCB

We used the custom PCB designed by ECE 4760 instructor Bruce Land to interface Atmega644 with our circuits.

Accelerometer

The accelerometer circuit consists of four MMA1260D z-axis analog accelerometers, three MMA2260D x-axis analog accelerometers, and the external circuit, which consists of decoupling capacitors and low pass filtering circuits as the datasheet of MMA1260D and MMA2260D suggests. From our tests, the circuit decreased the magnitude of the noise in the output by 0.03V, which is desirable

Parts List:

For more detail: Accelerometer Based Hand Action Recognition using Atmega644