# Simple BackPropagation Algorithm

• 10-16-2009
like_no_other
Simple BackPropagation Algorithm
I've read some neural net tutorials and decided to build a simple app: create simple perceptrons capable to recognize 2D black&white block representations of digits.
My problem comes with the weights' updating - i didn't fully understand the mechanics. On the tutorial page the formula was http://www.generation5.org/content/1...ges/pn_eq2.png. I've highlighted the part of code i need help with.
How should i update the weights?

Code:

```int zero[]= {0,0,1,1,0,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,0,1,1,0}; int one[]=  {0,             0,0,1,0,             0,1,1,0,             0,0,1,0,             0,0,1,0,             0,0,1,0,             0,1,1,1}; int one_blurred[]=  {0,             0,1,1,0,             0,0,1,0,             1,0,0,0,             1,0,1,0,             0,0,1,0,             0,1,1,1};             int two[]=  {0,             0,1,1,0,             1,0,0,1,             0,0,0,1,             0,0,1,0,             0,1,0,0,             1,1,1,1}; .. //Neuron class Neuron_c {     private:     int iNumInputs;     int i;     int p;     vector<double> vWeights;     vector<double>::iterator it;     double delta;         public:         Neuron_c(int iNumInputs);         ~Neuron_c();         void Update(int * iInput, int iOutput);         double Check(int * iInput);         double Sigmoid(double iActivation); }; //Constructor Neuron_c::Neuron_c(int iNumInputs) {     p = 1;     this->iNumInputs = iNumInputs +1;     for( i = 0; i < iNumInputs; i++)     {         vWeights.push_back((i*i)/(e*(i-e*e)));         random_shuffle(vWeights.begin(), vWeights.end());             }     for( i = 0; i <iNumInputs+1; i++)     {         cout<<vWeights[i]<<" ";     } } //Destructor Neuron_c::~Neuron_c() { } //Sigmoid double Neuron_c::Sigmoid(double iActivation) {     return 1/(1 + pow(e,-iActivation/p)); } void Neuron_c::Update(int * iInput, int iOutput) {     //Get current output     delta = iOutput - Check(iInput);     cout<<"Desired output: "<<iOutput<<"\n";     cout<<"Error: "<<delta<<"\n";         i = 0;     for ( it = vWeights.begin(); it <vWeights.end(); it++)     {                 i++;         *it = *it + 0.9 * ( iInput[i] * delta) ;     }     } double Neuron_c::Check(int * iInput) {             double activation = 0;     i = 0;     for ( it = vWeights.begin(); it < vWeights.end(); it++)     {         i++;         activation =activation + iInput[i]* *it;              }         cout<<"Activation: "<<activation<<"\n";         return Sigmoid(activation);             int main() {     /*     initwindow(width, height, title);         Image_c Image(width/2, height/2, 30, two);     Image.Paint();     */     Neuron_c Perceptron_one(24);     cout<<"Initial check: "<<Perceptron_one.Check( one )<<"\n";     int x = 30;     while( x> 0 )     {     Perceptron_one.Update(one, 1);     Perceptron_one.Update(two, 0);     Perceptron_one.Update(zero, 0);     x--;         }         cout<<"Check Neuron_one for 0: "<<Perceptron_one.Check( zero )<<"\n";      cout<<"Check Neuron_one for 1: "<<Perceptron_one.Check( one )<<"\n";     cout<<"Check Neuron_one for 1_blurred: "<<Perceptron_one.Check( one_blurred )<<"\n";     cout<<"Check Neuron_one for 2: "<<Perceptron_one.Check( two )<<"\n";                 cin.get();             return 0; }     }```
Output:
Code:

```Check Neuron_one for 0: 3.01565e-011 Check Neuron_one for 1: 1 Check Neuron_one for 1_blurred: 1 Check Neuron_one for 2: 0.00355031```
• 10-16-2009
abachler
Put simply, the change applied to the weight is proportional to the input times the error. The error in this case is positive if the weight needs to be more positive, and negative if it needs to be more negative.