Guitar Synthesizer and Game Using Atmega32

Introduction

Compose your own virtual guitar masterpiece or follow along with a preprogrammed classic. No experience needed!
We developed a guitar synthesizer with video component inspired by the popular video game Guitar Hero. The original game consisted of only reproducing popular rock and roll songs but we wanted to allow players to be able to create their own music. The game uses direct digital synthesis to replicate guitar tones and outputs the sound to a set of computer speakers. A black and white television is used to view a video image which is produced using NTSC non-interlacing video code. The video shows the treble staff resembling actual sheet music as well the corresponding guitar tab. The user interacts using a guitar shaped PlayStation 2 controller to play notes and select from the menu. To imitate playing a guitar as realistically as possible, notes are played when the player “strums” the guitar controller and not when “fret” buttons on the neck are pressed. Another feature of our game that any musician will appreciate is that each note displayed on the staff corresponds to the correct pitch and frequency of the actual sound.

High Level Design

When we began looking at final projects, we wanted to do something fun. Other groups were planning clever and interesting projects, but only a few of them allowed for much user interaction. One of our group members owns the game Guitar Hero for the PlayStation 2 (PS2). The controller, shaped like a guitar, inspired us to make a game that utilized its unique look and button layout. After researching the PS2 communication protocol, we decided that we could use the Mega32 to talk to the controller, and the idea for our game began to take shape. At first, we just wanted to create a synthesizer that would produce realistic electric guitar sounds. Unfortunately, the input capabilities of the controller limited the number of actual chords or notes we could play and the size and power of the Mega32 limited sound generation to DDS, meaning that full guitar chords were out of the question. We also decided that we wanted a video component for the game so that the user could have visual feedback on the generated music, as well as follow along with preprogrammed songs for practice.
Our project’s most complicated calculations are part of the sound generation unit. We perform two relatively complex calculations. First is the generation of a sine table to simulate the frequencies of musical notes. Through trial and error, as well as a bit of research, we decided that the first four harmonics of each fundamental musical note were needed to generate acceptable acoustic guitar sounds. During the initialization routine on the audio board, scaled versions of each of these harmonics are summed for every entry of the sine table. Since these calculations only take place during initialization, they do not have a significant effect on our timing or microprocessor load. The other set of significant calculations that we perform are for the envelope of the guitar output. In order to mimic the “pluck” sound and then resonate for a moment, we modulate the amplitude of the sine values to grow rapidly and then slowly die out. Ideally, the amplitude would decay exponentially, but we chose to keep it linear and are satisfied with the sound produced. In order to modulate, we modified Professor Land’s optimized multiplication function to accept an integer and an 8-bit fixed point fraction. The result is an envelope that grows to 0.FF amplitude over two samples, and then dies out to nothing over 3/4 of a second.
The core of our project is communication. The game itself resides in the audio board, but it must collect data from the PS2 controller and transmit data to the video board in order to do anything meaningful. Both of these devices communicate with the master over a Serial Peripheral Interface (SPI) bus. For video communication, the audio board waits for the video board to request attention. In response, the audio board transmits a single byte of information with an opcode in the high nibble and data in the lower nibble. The opcodes cover all of the commands that the video board needs to accomplish, including shifting notes onto the staff, showing the menu, and selecting the menu options. This interaction occurs every two frames, when the video code reaches the user component and guarantees that the data is ready on the next frame.
To communicate with the PS2 controller, the audio board requests the attention of the controller’s internal logic. Following the protocol of the PS2, the audio board commands the controller to send its button states, and the controller responds with 4 bytes of data that contain the status of the buttons, each stored as a bit. These bytes are stored for reference by the code that controls game play.
With this information, the audio board can update the state of the game. In freeplay or song mode, if the user is pressing a fret key and has just hit the strum bar, the DDS for the appropriate note is initialized. If the start button is pressed the menu appears regardless of the current point within a song. In menu mode, the strum bar toggles between the options, and the select button chooses one. Each of these state updates involves a corresponding video command which is transmitted during the next video attention request.
One of the places where we had to choose between hardware and software was our chip-to-chip and PS2 communication protocol. We could have written up our own bit-banged code to talk to our slave devices, but we chose to do it using the SPI hardware instead. For chip-to-chip communication, it was an obvious choice, as the hardware is built into the Mega32, and software transmission of data would add an unacceptable amount of overhead to our video code. Using the PS2 controller complicated things, because it meant that the bus would have to be shared. In addition, the PS2 protocol is much more complex than the single-byte opcode system that we developed for the video commands. In the end, we used the software to select between two formats of SPI communication, and were able to achieve concurrent communication on the bus.

Program/Hardware Design

Our code is split into two segments, one for the audio board that takes care of communication, sound generation and general game play, and one for the video board that takes care of the TV display.

Audio Code Details

The main board is responsible for generating the audio, communicating with the guitar controller, and running the actual game. Audio is generated using a simple DDS system based on Professor Land’s examples. A lookup table contains 256 samples across one full period of the guitar waveform and is built by combining the first four harmonics. The first harmonic is weighted the most, and each successive harmonic is weighted less than the one before it. Values are extracted from the lookup table using a 32-bit accumulator, but only the most significant byte is used to reference the values in the table.
The DDS is controlled by timer 0, which is set to interrupt 62,500 times per second. Every time it interrupts, it outputs the current sample from the lookup table to the R/2R DAC. The accumulator is then incremented by a known amount determined by the frequency of the generated sound. Therefore, different notes can be produced simply by setting an appropriate increment. The increment is calculated using the following equation, with f being the desired frequency:

In order to make the produced tone sound more like a guitar pluck, the amplitude of the wave is modulated by an envelope. The envelope has three parts: attack, decay, and sustain. During the attack, the envelope increases very rapidly (from zero to maximum in only a few samples). The envelope then decreases rapidly during the decay portion until some critical value has been reached. Finally, in sustain the envelope decreases very slowly until it reaches zero.
The envelope is stored as a 16-bit unsigned integer representing a fraction. That is, the radix point is all the way on the left of the MSB. We wrote a specialized routine that multiplies a 16-bit signed integer by this fraction. Thus, a sample from the lookup table can be multiplied by a value from zero to (slightly less than) one in order to modulate the amplitude. Unfortunately, it turns out that we only ended up using an 8-bit signed char to hold a sample, therefore making our routine somewhat more complex than is strictly necessary. Regardless, our routine works exactly as desired.
The audio board is also responsible for communicating with the Guitar Hero PS2 controller, which uses a specialized digital protocol to communicate. The controller communicates over five lines: command (CMND), data (DATA), clock (CLK), attention (ATT), and acknowledge (ACK). It turns out, however, that the protocol used is similar enough to SPI that the controller can be directly connected to the Mega32’s SPI pins. In our project, CMND goes to MOSI, DATA goes to MISO, CLK goes to SCK, and ATT goes to SS. Since SPI doesn’t support acknowledge, we connected the ACK line to the INT0 pin on the Mega32, an external interrupt.
The microcontroller initiates communication with the guitar controller by pulling the ATT line low. Then, after a slight delay (experimentally determined to be about 7us), the start byte 0x01 is transmitted on CMND. The rest of the transmission is handled in the EXT_INT0 routine which is triggered by a rising edge of the ACK line (that is, when the guitar controller releases said line). The second byte transmitted is 0x42 which instructs the controller to send back button information. During the remaining three bytes of the packet, the microcontroller sends only 0x00 since it is only interested in receiving data.
While the second byte of the packet is being transmitted to the controller, the controller is sending back a mode identifier. In this case, the mode identifier is 0x41, indicating the controller is in “digital” mode (normal PS2 controllers also have an “analog” mode which allows for variable reading of the two joysticks). During the third byte, the controller sends back 0x5A which indicates that it is ready to transmit actual button data. Finally, during the last two bytes, the controller sends back the actual button presses. The whole communication packet is illustrated below:

Finally, the audio board is responsible for the actual game play. This is handled in a giant state machine in the main loop with the following states: Freeplay, Song, Menu, LoadSong, and Transition. In the Freeplay state, the user is in complete control of what notes are played. When the user strums, the microcontroller reads the fret buttons. If a valid combination was held down when the guitar was strummed, the accumulator increment is set accordingly and the audio microcontroller sends a SETSL command to the video board containing the note played. Between strums, the microcontroller is constantly sending the SET command with the current note being pressed, so that the user has real-time feedback before strumming.
In the Song state, the user can only play the next note in the selected song. When the correct note is strummed, the audio microcontroller sets the accumulator increment accordingly. It then loads the fifth note in the future from EEPROM and sends it to the video board with the SLSET command. In this manner, the current note and the next five are always displayed on the screen (until the end of the song). Entire songs are stored in the EEPROM as 0xFF-terminated strings of notes. The first song is stored starting at location 0 in EEPROM, the second song at location 64, and the third at location 128.
The Menu state is entered when the start button on the controller is pressed while in Freeplay or Song states. The audio microcontroller sends the MENU command with the appropriate menu ID (1, in this case). In the Menu state, the audio microcontroller keeps track of the current menu entry selected. When the strum bar is pushed up or down, the menu item selected is either decremented or incremented, respectively, and the new selection is sent to the video board in the MENUS command. If the start button on the guitar controller is pressed, the CLRM command is sent and the state machine returns to its previous state. If the select button is pressed, the current menu item is acted on. In all cases, the first command sent is CLRM. The next state is determined by the selected menu item. If it was “Resume,” the state machine returns to its previous state. If it was “Freeplay” or any of the songs, the next state is the Transition state, with the following state specified.
Based on the specified next state, the Transition state sets the mode of video board. If the next state is Freeplay, the mode is set to freeplay. If the next state is LoadSong, the mode is set to song. The state is then updated according to the specified next state.
The LoadSong state is used to load the first six notes onto the screen. It iterates through those first six notes and sends them to the video microcontroller with the LOAD command, which loads the note buffer without updating the screen. When all six notes have been loaded, the audio microcontroller sends the video board the DISP command which forces it to display its buffer. The state machine then enters the Song state in the next cycle.
The commands are sent to the video board over SPI. The data request line (RQ) from the video microcontroller is connected to the INT1 pin on the audio microcontroller. The EXT_INT1 routine is set to run on the rising edge of that signal. Thus, the video board requests a new data packet by pulling the line high. When the interrupt is executed, the audio board transmits the one byte of video command over the SPI bus.
Because both the guitar controller and the video board communicate with the audio board over SPI, the audio board needs to keep track of which slave it is communicating with. It does this with a simple state machine with three states: Idle, PSX, and Video. In the PSX and Video states, it is communicating with the PS2 controller and the video board, respectively. In the Idle state, no communication is currently happening. Communication with either slave is initiated by setting a flag (PSX and Video have separate flags). Then, in the main loop, if a flag has been set and the SPI bus is currently in the Idle state, the appropriate transmission is initiated.

Video Code Details

The video code is the more complicated of the two programs, but only due to the complexity of conforming to the NTSC format. Our display implementation was not interlaced, which is generally used when each frame is split into separate half-frames each consisting of all the odd or even lines. In our case, the even lines just duplicate the odd lines rather than contain unique information. Fortunately for us, the protocol is taken care of by Professor Land’s sample code. The parts of the video code that we wrote deal with decoding transmissions from the audio board, and using those decoded transmissions to write to the screen.
Our video display consists of a musical staff with a treble clef, 6 positions that can each take on one of 9 notes, 5 bubbles that indicate which fret buttons the user is or should be pressing, and dots that show the fret buttons that correspond to the displayed notes. The clef is very expensive to draw, time wise, so we only do it during initialization. The staff lines are drawn every frame, so that note erasure does not eat away at them, but they are drawn quickly by directly modifying the screen array.
The most important thing that the video code does is bring its SPI data request line (RQ) high at the end of every second frame. Until it does this it will not receive any data from the audio board. While a frame is being drawn to the screen, the video SPI buffer receives a 1 byte transmission from the audio board. The upper nibble consists of an opcode and the lower nibble consists of any data that the corresponding operation needs. The opcodes cover all of the various desired draw functions that the game requires. The most used of these is the NOP command. Usually, the audio board just wants the video board to keep a static display. Depending on user input, any one of the 10 other opcodes may be transmitted.
When MENU code is received, the video board pauses the display, clears the bottom of the screen in one frame and prints 5 menu options with a box around the top option in the next frame. When the MENUS code is received, the video board erases the old menu selection box and draws a new one around the option that the lower nibble indicates. When the CLRM code is received the video board removes the menu from the screen and redraws the bubbles and the dots. The bubble location is based on data transmitted in the most recent MODE code, which tells the video board whether the game is in Freeplay mode or in Song mode. MODE also clears the notes from the staff.

If Freeplay mode has been selected, the bubbles are drawn on the right hand of the screen. When the SET code is received, the video board erases any previous note over the bubbles and draws in the note that is indicated in the data sent along with SET. When the SETSL code is received, the note to the right of the bubbles is set to the value included with SETSL, and the rest of the notes are shifted one position to the left. This command takes two frames to complete because only 6 notes total can be drawn or erased per frame. To avoid flickering, the left three notes are handled in the first frame and the right three in the second.

Parts List:

 
For more detail:  Guitar Synthesizer and Game Using Atmega32