Questions about the Backpropogation Algorithm

Posted by Colemangrill on Stack Overflow See other posts from Stack Overflow or by Colemangrill
Published on 2012-09-24T03:33:47Z Indexed on 2012/09/24 3:37 UTC
Read the original article Hit count: 233

I have a few questions concerning backpropogation. I'm trying to learn the fundamentals behind neural network theory and wanted to start small, building a simple XOR classifier. I've read a lot of articles and skimmed multiple textbooks - but I can't seem to teach this thing the pattern for XOR.

Firstly, I am unclear about the learning model for backpropogation. Here is some pseudo-code to represent how I am trying to train the network. [Lets assume my network is setup properly (ie: multiple inputs connect to a hidden layer connect to an output layer and all wired up properly)].

SET guess = getNetworkOutput() // Note this is using a sigmoid activation function.
SET error = desiredOutput - guess
SET delta = learningConstant * error * sigmoidDerivative(guess)
For Each Node in inputNodes
   For Each Weight in inputNodes[n]
      inputNodes[n].weight[j] += delta;

// At this point, I am assuming the first layer has been trained.
// Then I recurse a similar function over the hidden layer and output layer.
// The prime difference being that it further divi's up the adjustment delta.

I realize this is probably not enough to go off of, and I will gladly expound on any part of my implementation. Using the above algorithm, my neural network does get trained, kind of. But not properly. The output is always

XOR 1 1 [smallest number]
XOR 0 0 [largest number]
XOR 1 0 [medium number]
XOR 0 1 [medium number]

I can never train the [1,1] [0,0] to be the same value.

If you have any suggestions, additional resources, articles, blogs, etc for me to look at I am very interested in learning more about this topic. Thank you for your assistance, I appreciate it greatly!

© Stack Overflow or respective owner

Related posts about c#

Related posts about algorithm