Neural Network Learning Visualization

The tool offers a new interpretation of the feed-forward mechanism and gradient descent. I wanted to address questions such as: why does introducing nonlinearities through activation functions lead to input space warping (e.g., spirals)? How can we visualize the matrix multiplications within the network, as these represent linear projection transformations into smaller hyper-spaces? What happens when the loss function has a plateau, but the network configuration continues to evolve? Is the function stuck in a local minimum? How can we interpret gradient descent steps (the movement of projection vectors, represented by matrix lines between layers, in the optimal direction for the desired shape of the final activation layer—such as a triangle for softmax with 3 classes)?

A few million parameters can be reduced to a few hundred vectors and a reasonably small number of transformations with the created technique.

The tool can be used for removing unnecessary intermediate layers, stopping training earlier, selecting a better topology, understanding the learning mechanism, and choosing or creating optimizers.

#neuralnetworks #science #machinelearning #ai #artificialintelligence #ml #datascience #deeplearning