Convolutional Neural Network Basics

3 minute read Modified:

Convolution It is a matrix operation in which we add each element with local neighbors with respect to the weight of the kernels. Mathematically, it is the element-wise product of each element of the kernel with the image-piece followed by a sum. These filters can be used to detect various things like edges etc. Filters/Kernels Filters/Kernels capture features in their receptive field using matrices containing values (weights) with convolution. A higher result of this operation implies that the feature captured by the kernel is in the image, and a lower score implies the opposite.

Deep Learning Through the Lens of the Information Plane

13 minute read Modified:

The ridiculous effectiveness of Deep Learning has lead to research on tools that help to analyze these Deep Neural Network based “black boxes”. Recent research papers by the Information Theory community to analyze has rise to a new tool, The Information Plane, which can help analyze and answer various questions about these networks. This article, provides a brief overview of the concepts from information theory required to develop an understanding of the Information Plane, followed by a replication study of the implementation of the paper that introduces this theory with respect to Deep Neural Networks.