C Board  

Go Back   C Board > Cprogramming.com and AIHorizon.com's Artificial Intelligence Boards > General AI Programming

Reply
 
LinkBack Thread Tools Display Modes
Old 10-16-2009, 05:16 AM   #1
Registered User
 
Join Date: Sep 2008
Posts: 27
Simple BackPropagation Algorithm

I've read some neural net tutorials and decided to build a simple app: create simple perceptrons capable to recognize 2D black&white block representations of digits.
My problem comes with the weights' updating - i didn't fully understand the mechanics. On the tutorial page the formula was . I've highlighted the part of code i need help with.
How should i update the weights?

Code:
int zero[]= {0,0,1,1,0,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,0,1,1,0};
int one[]=  {0,
            0,0,1,0,
            0,1,1,0,
            0,0,1,0,
            0,0,1,0,
            0,0,1,0,
            0,1,1,1};
int one_blurred[]=  {0,
            0,1,1,0,
            0,0,1,0,
            1,0,0,0,
            1,0,1,0,
            0,0,1,0,
            0,1,1,1};
            
int two[]=  {0,
            0,1,1,0,
            1,0,0,1,
            0,0,0,1,
            0,0,1,0,
            0,1,0,0,
            1,1,1,1};
..
//Neuron
class Neuron_c
{
    private:
    int iNumInputs;
    int i;
    int p;
    vector<double> vWeights;
    vector<double>::iterator it;
    double delta;
    
    public:
        Neuron_c(int iNumInputs);
        ~Neuron_c();
        void Update(int * iInput, int iOutput);
        double Check(int * iInput);
        double Sigmoid(double iActivation);
};
//Constructor
Neuron_c::Neuron_c(int iNumInputs)
{
    p = 1;
    this->iNumInputs = iNumInputs +1;
    for( i = 0; i < iNumInputs; i++)
    {
        vWeights.push_back((i*i)/(e*(i-e*e)));
        random_shuffle(vWeights.begin(), vWeights.end());
        
    }
    for( i = 0; i <iNumInputs+1; i++)
    {
        cout<<vWeights[i]<<" ";
    }
}
//Destructor
Neuron_c::~Neuron_c()
{
}

//Sigmoid
double Neuron_c::Sigmoid(double iActivation)
{
    return 1/(1 + pow(e,-iActivation/p));
}

void Neuron_c::Update(int * iInput, int iOutput)
{
    //Get current output
    delta = iOutput - Check(iInput);
    cout<<"Desired output: "<<iOutput<<"\n";
    cout<<"Error: "<<delta<<"\n";
    
    i = 0;
    for ( it = vWeights.begin(); it <vWeights.end(); it++)
    {
        
        i++;
        *it = *it + 0.9 * ( iInput[i] * delta) ;
    }
    
}

double Neuron_c::Check(int * iInput)
{
        
    double activation = 0;
    i = 0;
    for ( it = vWeights.begin(); it < vWeights.end(); it++)
    {
        i++;
        activation =activation + iInput[i]* *it;          
    }
    
    cout<<"Activation: "<<activation<<"\n";
    
    return Sigmoid(activation);
    
    
    int main()
{
    /*
    initwindow(width, height, title);
    
    Image_c Image(width/2, height/2, 30, two);
    Image.Paint();
    */
    Neuron_c Perceptron_one(24);
    cout<<"Initial check: "<<Perceptron_one.Check( one )<<"\n";
    int x = 30;
    while( x> 0 )
    {
    Perceptron_one.Update(one, 1);
    Perceptron_one.Update(two, 0);
    Perceptron_one.Update(zero, 0);
    x--;
    
    }
    
    cout<<"Check Neuron_one for 0: "<<Perceptron_one.Check( zero )<<"\n";   
    cout<<"Check Neuron_one for 1: "<<Perceptron_one.Check( one )<<"\n";
    cout<<"Check Neuron_one for 1_blurred: "<<Perceptron_one.Check( one_blurred )<<"\n";
    cout<<"Check Neuron_one for 2: "<<Perceptron_one.Check( two )<<"\n";
    
    
    
    cin.get();
    
    
    return 0;
}
    
}
Output:
Code:
Check Neuron_one for 0: 3.01565e-011
Check Neuron_one for 1: 1
Check Neuron_one for 1_blurred: 1
Check Neuron_one for 2: 0.00355031
like_no_other is offline   Reply With Quote
Old 10-16-2009, 06:55 AM   #2
Rampaging 35 Stone Welsh
 
abachler's Avatar
 
Join Date: Apr 2007
Posts: 2,924
Put simply, the change applied to the weight is proportional to the input times the error. The error in this case is positive if the weight needs to be more positive, and negative if it needs to be more negative.
__________________
He is free, you say. Ah! That is his misfortune… These men… [have] the most terrible, the most imperious of masters, that is, need. … They must therefore find someone to hire them, or die of hunger. Is that to be free? - Simon Linguet
abachler is offline   Reply With Quote
Reply

Thread Tools
Display Modes

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Simple Shift errors... Junior89 C++ Programming 5 06-26-2007 05:55 AM
Request for comments Prelude A Brief History of Cprogramming.com 15 01-02-2004 10:33 AM
Simple simple graphics triplem C Programming 2 05-19-2003 02:52 AM
a simple algorithm and questions ustuzou C++ Programming 0 02-18-2002 11:12 AM
Simple File Creation Algorithm muffin C Programming 13 08-24-2001 03:28 PM


All times are GMT -6. The time now is 11:05 PM.


Powered by vBulletin® Version 3.8.1
Copyright ©2000 - 2009, Jelsoft Enterprises Ltd.
Search Engine Optimization by vBSEO 3.3.0 RC2

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22