Deep Learning 8: Gradient Descent vs Stochastic Gradient Descent vs Mini-Batch Gradient Descent

Three ways of backpropagation are gradient descent, stochastic gradient descent and mini-batch gradient descent.
In gradient descent, the entire training set is propagated forward in the network and then the sum of the error in individual example is calculated. Then based on this total error, the learnable parameters are updated.
In stochastic gradient descent, every time a single example is taken from the training data, and forward propagation is performed, then error is found and based on this individual error, the weights and biases are updated, then second example is taken and the same procedure is repeated until the training data ends and this makes one complete epoch.
In min-batch gradient descent, the training data set is divided into batches of some examples (called batch size), and every batch is propagated forward in the network and upon completion of a single batch, the weights and biases are updated, then another batch is taken and the same process is repeated until the training dataset ends. This completes one epoch.
In mini-batch if the batch size is 1 then it becomes stochastic gradient descent and if it is equal to the size of the input dataset then it becomes gradient descent.