🌟 Advanced Concepts About Convolutional Neural Networks

Important Terms

Term

Description

πŸ”· Padding

Adding additional border(s) to the image before convolution

🌠 Strided Convolution

Convolving by s steps

🏐 Convolutions Over Volume

Applying convs on n-dimensional input (such as an RGB image)

πŸ™Œ Padding

Adding an additional one border or more to the image so the image is n+2 x n+2 and after convolution we end up with n x n image which is the original size of the image

p = number of added borders

For convention: it is filled by 0

πŸ€” How much to pad?

For better understanding let's say that we have two concepts:

πŸ•΅οΈβ€β™€οΈ Valid Convolutions

It means no padding so:

n x n * f x f ➑ n-f+1 x n-f+1

πŸ₯½ Same Convolutions

Pad so that output size is the same as the input size.

So we want that 🧐:

n+2p-f+1 = n

Hence:

p = (f-1)/2

For convention f is chosen to be odd πŸ‘©β€πŸš€

πŸ‘€ Visualization

πŸ”’ Strided Convolution

Another approach of convolutions, we calculate the output by applying filter on regions by some value s.

πŸ‘€ Visualization

πŸ€— To Generalize

For an n x n image and f x f filter, with p padding and stride s; the output image size can be calculated by the following formula

β€‹βŒŠn+2pβˆ’fs+1βŒ‹Γ—βŒŠn+2pβˆ’fs+1βŒ‹\left \lfloor{\frac{n+2p-f}{s}+1}\right \rfloor \times \left \lfloor{\frac{n+2p-f}{s}+1}\right \rfloor​

πŸš€ Convolutions Over Volume

To apply convolution operation on an RGB image; for example on 10x10 px RGB image, technically the image's dimension is 10x10x3 so we can apply for example a 3x3x3 filter or fxfx3 🀳

Filters can be applied on a special color channel 🎨

πŸ‘€ Visualization

πŸ€Έβ€β™€οΈ Multiple Filters

🎨 Types of Layer In A Convolutional Network

Layer

Description

πŸ’« Convolution CONV

Filters to extract features

πŸŒ€ Pooling POOL

A technique to reduce size of representation and to speed up the computations

β­• Fully Connected FC

Standard single neural network layer (one dimensional)

πŸ‘©β€πŸ« Usually when people report number of layers in an NN they just report the number of layers that have weights and params

Convention: CONV1 + POOL1 = LAYER1

πŸ€” Why Convolotions?

  • Better performance since they decrease the parameters that will be tuned πŸ’«

🧐 References