Thread: Simple BackPropagation Algorithm

  1. #1
    Registered User
    Join Date
    Sep 2008
    Posts
    48

    Simple BackPropagation Algorithm

    I've read some neural net tutorials and decided to build a simple app: create simple perceptrons capable to recognize 2D black&white block representations of digits.
    My problem comes with the weights' updating - i didn't fully understand the mechanics. On the tutorial page the formula was . I've highlighted the part of code i need help with.
    How should i update the weights?

    Code:
    int zero[]= {0,0,1,1,0,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,0,1,1,0};
    int one[]=  {0,
                0,0,1,0,
                0,1,1,0,
                0,0,1,0,
                0,0,1,0,
                0,0,1,0,
                0,1,1,1};
    int one_blurred[]=  {0,
                0,1,1,0,
                0,0,1,0,
                1,0,0,0,
                1,0,1,0,
                0,0,1,0,
                0,1,1,1};
                
    int two[]=  {0,
                0,1,1,0,
                1,0,0,1,
                0,0,0,1,
                0,0,1,0,
                0,1,0,0,
                1,1,1,1};
    ..
    //Neuron
    class Neuron_c
    {
        private:
        int iNumInputs;
        int i;
        int p;
        vector<double> vWeights;
        vector<double>::iterator it;
        double delta;
        
        public:
            Neuron_c(int iNumInputs);
            ~Neuron_c();
            void Update(int * iInput, int iOutput);
            double Check(int * iInput);
            double Sigmoid(double iActivation);
    };
    //Constructor
    Neuron_c::Neuron_c(int iNumInputs)
    {
        p = 1;
        this->iNumInputs = iNumInputs +1;
        for( i = 0; i < iNumInputs; i++)
        {
            vWeights.push_back((i*i)/(e*(i-e*e)));
            random_shuffle(vWeights.begin(), vWeights.end());
            
        }
        for( i = 0; i <iNumInputs+1; i++)
        {
            cout<<vWeights[i]<<" ";
        }
    }
    //Destructor
    Neuron_c::~Neuron_c()
    {
    }
    
    //Sigmoid
    double Neuron_c::Sigmoid(double iActivation)
    {
        return 1/(1 + pow(e,-iActivation/p));
    }
    
    void Neuron_c::Update(int * iInput, int iOutput)
    {
        //Get current output
        delta = iOutput - Check(iInput);
        cout<<"Desired output: "<<iOutput<<"\n";
        cout<<"Error: "<<delta<<"\n";
        
        i = 0;
        for ( it = vWeights.begin(); it <vWeights.end(); it++)
        {
            
            i++;
            *it = *it + 0.9 * ( iInput[i] * delta) ;
        }
        
    }
    
    double Neuron_c::Check(int * iInput)
    {
            
        double activation = 0;
        i = 0;
        for ( it = vWeights.begin(); it < vWeights.end(); it++)
        {
            i++;
            activation =activation + iInput[i]* *it;          
        }
        
        cout<<"Activation: "<<activation<<"\n";
        
        return Sigmoid(activation);
        
        
        int main()
    {
        /*
        initwindow(width, height, title);
        
        Image_c Image(width/2, height/2, 30, two);
        Image.Paint();
        */
        Neuron_c Perceptron_one(24);
        cout<<"Initial check: "<<Perceptron_one.Check( one )<<"\n";
        int x = 30;
        while( x> 0 )
        {
        Perceptron_one.Update(one, 1);
        Perceptron_one.Update(two, 0);
        Perceptron_one.Update(zero, 0);
        x--;
        
        }
        
        cout<<"Check Neuron_one for 0: "<<Perceptron_one.Check( zero )<<"\n";   
        cout<<"Check Neuron_one for 1: "<<Perceptron_one.Check( one )<<"\n";
        cout<<"Check Neuron_one for 1_blurred: "<<Perceptron_one.Check( one_blurred )<<"\n";
        cout<<"Check Neuron_one for 2: "<<Perceptron_one.Check( two )<<"\n";
        
        
        
        cin.get();
        
        
        return 0;
    }
        
    }
    Output:
    Code:
    Check Neuron_one for 0: 3.01565e-011
    Check Neuron_one for 1: 1
    Check Neuron_one for 1_blurred: 1
    Check Neuron_one for 2: 0.00355031

  2. #2
    Malum in se abachler's Avatar
    Join Date
    Apr 2007
    Posts
    3,195
    Put simply, the change applied to the weight is proportional to the input times the error. The error in this case is positive if the weight needs to be more positive, and negative if it needs to be more negative.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Simple Shift errors...
    By Junior89 in forum C++ Programming
    Replies: 5
    Last Post: 06-26-2007, 05:55 AM
  2. Request for comments
    By Prelude in forum A Brief History of Cprogramming.com
    Replies: 15
    Last Post: 01-02-2004, 10:33 AM
  3. Simple simple graphics
    By triplem in forum C Programming
    Replies: 2
    Last Post: 05-19-2003, 02:52 AM
  4. a simple algorithm and questions
    By ustuzou in forum C++ Programming
    Replies: 0
    Last Post: 02-18-2002, 11:12 AM
  5. Simple File Creation Algorithm
    By muffin in forum C Programming
    Replies: 13
    Last Post: 08-24-2001, 03:28 PM