Our project consisted of an elementary eight neuron network that used Hebbian Learning to train a robot to respond intelligently to input light stimuli.
First, we decided upon a task that would accurately denote Hebbian learning. One of the most common examples of conditional learning such as Hebbian learning is seen in Pavlovs experiment with his dog. In this experiment, when food was offered to the dog, it caused the dog to salivate. At first the sound of a doorbell elicited no such response. However, Pavlov decided to sound the bell when he offered food to the dog. After a few repetitions of this experiment, the dog began to salivate at the sound of the bell even when no food was present. Here food was the unconditioned stimulus, and the doorbell was the conditioned stimulus. Similarly, in our experiment we first show that shining light in front of and or behind the robot elicits no response but pressing the push button causes the robot to move forward or backward. We then press the button while shining the light on the robot and the neural network programmed into the robot causes it to associate the light input with the push button input. Soon the robot moves forward or backward depending on whether the light is shined behind or in front of it in the absence of push button input. There are other neurons in this network that play an inhibitory role and prevent the robot from going too close to the light. They too display learning. Initially, the robot goes very close to a light source before it decides to move in the opposite direction. As time passes by the robot gets more responsive to the light and does not get too close to either light source.
In order to reach our end goal, we first programmed a three neuron neural network in C and thoroughly tested it using LEDs and hyperterm. We then extended this to a four neuron network and finally to an eight neuron network, with thorough testing at each level of complexity. Following this, we added the hardware interface. This involved integration of stepper motor control code into our neural network such that the stepper motors would step when the motor neurons fired. We then built the chassis and added all the motors, LEDs, pushbuttons and MCU board to the design.
A neuron consists of a cell body with dendrites and an axon which terminates onto a muscle fiber or the synapse of another neuron. It receives signals in the form of charges (sodium, potassium, calcium ions) from other neurons that have their axons terminals sharing synapses with its dendrites. These charges are integrated spatially (across number of neurons that synapse onto it) and temporally (charges received over time) and change the membrane potential of the neuron. The membrane potential usually increases with the influx of sodium ions and the efflux of potassium and calcium ions. An increase in membrane potential is also referred to as a depolarization. Once the membrane potential increases beyond a certain threshold potential, specific for the particular neuron, an action potential is fired. Action potentials are characterized by a strong depolarization, followed by a hyperpolarization (decrease in membrane potential). The action potential is then transmitted down the axon to the axon terminals where it is passed onto the next neuron through a synapse.
Action potentials are all or none events, and their method of generation can be characterized by an integrate and fire approach. That is, a cell integrates charge signals it receives from other neurons and only after its threshold is reached it fires an action potential, thereby transmitting a signal to its postsynaptic neuron. After firing this action potential, the firing neuron goes through a refractory period. During this time, it cannot fire another action potential even if the signals it receives push its membrane potential beyond its threshold value. At the end of the refractory period the membrane potential is at its resting state. All of the charge entering a neuron from other cells does not contribute to the rise in membrane potential. Some of the charge is constantly leaking out of the neuron. This is called leakage current.
Neurons connected in a network can be used to display Hebbian Learning. Donald O. Hebb a Canadian Neuropsychologist proposed the following postulate:
When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A’s efficiency, as one of the cells firing B, is increased.
The principles underlying his postulate came to be known as Hebbian Learning. Therefore, if two neurons in a network (a post-synaptic and pre-synaptic) neuron fire repeatedly in the correct order, the connection between them is strengthened. The order in which neurons fire is very important because it can also cause the weights between neurons to decrease. If a presynaptic neuron fires shortly before the post-synaptic neuron their connection is strengthened, however if the post-synaptic neuron fires shortly before a presynaptic neuron their connection is weakened. If the time interval between the firings of these two neurons is very large, they cannot be correlated. The following diagram shows how the timing of action potential (also referred to as spikes) affect the weights between a two neurons.
High Level Design
At first, we had a hard time choosing a final project idea. Professor Land had talked to us about building an elementary neural network as a possible idea of a final project. After doing research on the idea, we found it very interesting because both of us had taken highly enjoyable neurobiology courses. Building a neural network would be the software basis for our project, but we wanted to apply it some real world phenomena. So we decided to build a robot that would use a neural network to learn to react to certain stimuli. At first we wanted to use heat or ultrasound as an input into the system, but after doing further research we realized that doing these wouldnt be as very feasible to implement with our neural network. Finally, we settled on using a light-guided robot as the real world interface for the project.Light turned out to be a input because LEDs were cheap and reliable to be used as light sensors. They served our needs perfectly, and we had a large supply of them in the lab.Their directionality was also good enough to guide our robot. Our inspiration for using light came from how a moth is always attracted to light, but when it gets too close to a light, it will quickly fly away due to in the intense heat. The attraction to the light shows excitation while the aversion at close proximity shows an inhibitory effect. We decided to model this type of system where there would be multiple layers of Hebbian learning.
Integrate and Fire Equations:
Temporal & Spatial Integration of voltage:
Where τ is our decay constant. Vrest is the resting membrane potential which we set to 0; Vexc is the voltage inputs due to excitatory connection and Vinh is the voltage inputs due to inhibitory connections.
The weights were updated according to the following equation:
where i corresponds to the presynaptic neuron, j corresponds to the postsynaptic neuron and Wij is weight of the connection; Tij is the weight of the connection * Learning / Unlearning rate.
We did the design process in many stages. The first stage was to simply getting the neural network working.We first started with a simple 3 neuron network and made sure we got this to work. We then added a fourth inhibitory neuron to make sure we understood how to code both excitatory and inhibitory effects. We then added four more neurons to give bi-directionality to the neural network. We used the STK500 board, hyperterm, and the LEDs to make sure we got this working first. Next, we separately tried to integrate the stepper motors both in hardware and software. After getting the motor control working, we integrated this with the neural network to make sure the motors only moved whenever an output neuron spiked. Finally, we had to build a robot and solder parts for the final product. During demo, our final project would consist of demonstrating learning first through hyperterm and show how the weights are changing. Then, we would shine light both backwards and forwards and show that the robot can move in both directions and there exists both excitatory and inhibitory effects. There will also be real-time learning as the robot is moving through its light course.
The following is an overview block diagram of the overall project.
The neural network has 4 inputs (forward and backward light sensors and pushbuttons) and two outputs (moving forward or moving backward). The following diagram shows the network, where solid lines indicate excitatory connections and dashed lines indicating inhibitory connections.
For more detail: Neural net robot