Types of neurons & cells:

  • Backfed input cell
  • Input cell
  • Noisy input cell
  • Hidden cell
  • Probablistic hidden cell
  • Spiking hidden cell
  • Output cell
  • Match input output cell
  • Recurrent cell
  • Memory cell
  • Open memory cell
  • Scanning filter
  • Convolution

Types of neural network forms:

  • Feedforward AND
  • Feedforward XOR
  • Radial basis network
  • Deep feedforward
  • Recurrent neural network (bi)
  • Long/short term memory (bi)
  • Gated recurrent unit (bi)
  • Auto encoder
  • Variational auto encoder
  • Denoising auto encoder
  • Sparse auto encoder
  • Markov chain
  • Hopfield network
  • Boltzmann machine
  • Restricted boltz machine
  • Deep belief network
  • Deep convolutional network
  • Deconvolutional network
  • Deep convolutional inverse graphics network
  • Generative adversarial network
  • Liquid state machine
  • Echo state network
  • Kohonen network
  • Deep residual network
  • Support vector machine
  • Neural turing machine

Interesting pick:

In simplest manner, svm without kernel is a single neural network neuron but with different cost function. If you add a kernel function, then it is comparable with 2 layer neural nets. First layer is able to project data into some other space and next layer classifies the projected data. If you force to have one more layer then you might ensemble multiple kernel svms then you mimics 3 layer nn.