Convolution It is a matrix operation in which we add each element with local neighbors with respect to the weight of the kernels. Mathematically, it is the element-wise product of each element of the kernel with the image-piece followed by a sum. These filters can be used to detect various things like edges etc. Filters/Kernels Filters/Kernels capture features in their receptive field using matrices containing values (weights) with convolution. A higher result of this operation implies that the feature captured by the kernel is in the image, and a lower score implies the opposite.
The ridiculous effectiveness of Deep Learning has lead to research on tools that help to analyze these Deep Neural Network based “black boxes”. Recent research papers by the Information Theory community to analyze has rise to a new tool, The Information Plane, which can help analyze and answer various questions about these networks. This article, provides a brief overview of the concepts from information theory required to develop an understanding of the Information Plane, followed by a replication study of the implementation of the paper that introduces this theory with respect to Deep Neural Networks.
I attended HackTheNorth, which is Canada’s Biggest Hackathon and takes place at University of Waterloo campus. I was glad to join over 1000 students from around the world at the University of Waterloo to collaborate and create something extraordinary in 36 hours. I didn’t go in as a team as the visa process was a bit delayed and so none of the participants from India knew if they would be able to go.