Input list always comes with a bias value of 1, for sigmoid function to be able to shift left and right.

Single perceptron, important pseudo-code:

function heaviside(X) {
  if (X<0)
    return 0;
  else
    return 1;
}

Single neuron, important pseudo-code (just like perceptron, instead of using ‘heaviside’ function as activation function, use ‘sigmoid’ function as activation function):

function sigmoid(X) {
  return tanh(X); //tanh is just 1 of many sigmoid functions
}

function dsigmoid(X) {
  return (double)1-(X*X);
}

Multi-layer feedforward neural network, important pseudo-code:

/*
2 important functions: feedforward, backpropagate
Initial weights: Random
Initial weight changes: Zeros
*/

//feedforward
Output = sigmoid(Input1*Weight1+...);

//backpropagate (calculate errors)
E10 = sum(W20a*E20 + W21a*E21 + W22a*E22) * dsigmoid(O10)

//update weights
W11a_Change = I11a*E11*Learning_Rate + Last_W11a_Change*Momentum
W11a       += W11a_Change
C11a        = W11a_Change

Abbreviations in the featured image:

Is = Inputs
Ws = Weights
Cs = Weight changes
O  = Output
E  = Error
Advertisements