Convolutional Neural Network

A convolutional neural network (CNN) is a type of neural network frequently used to solve computer vision problems such as image recognition and image classification. Like any neural network, it solves the problem by finding the best possible approximation to a function. This is achieved using weights and bias that are updated via a backpropagation algorithm.

In a typical neural network each layer consists of a set of neurons connected to all neurons in the previous layer. CNNs utilize 3-dimensional layers where only some of the neurons are connected to the previous layer. CNNs are formed by stacking different layers that transform the input; convolutional layer, pooling layer, rectified linear unit (ReLU) layer, and fully connected layer.

The convolution layer forms the foundation of a CNN. It contains a set of filters (kernels). Unlike other image processing algorithms, CNNs require minimal pre-processing and can learn the filters that would otherwise need to be created by hand.   

In CNN architecture, a pooling layer is often inserted between successive convolution layers. The pooling or subsampling layer reduces the number of parameters and computational requirements, while controlling overfitting.

A ReLU layer typically accompanies a convolution layer with the purpose of injecting non-linearity into the neural net. It does this by replacing all negative pixels values with zero.

The Fully Connected Layer is a Multi Layer Perceptron where every neuron in the previous layer is connected with every neuron on the next one. It serves to classify the image input based on training data.