πCommon Concepts
Basic Concepts of ANN
π Basic Neural Network

Convention: The NN in the image called to be a 2-layers NN since input layer is not being counted π’β
π Common Terms
Term
Description
π Input Layer
A layer that contains the inputs to the NN
π Hidden Layer
The layer(s) where computational operations are being done
π Output Layer
The final layer of the NN and it is responsible for generating the predicted value yΜ
π§ Neuron
A placeholder for a mathematical function, it applies a function on inputs and provides an output
π₯ Activation Function
A function that converts an input signal of a node to an output signal by applying some transformation
πΆ Shallow NN
NN with few number of hidden layers (one or two)
πͺ Deep NN
NN with large number of hidden layers
Number of units in l layer
π§ What does an artificial neuron do?
It calculates a weighted sum of its input, adds a bias and then decides whether it should be fired or not due to an activaiton function
My detailed notes on activation functions are here π©βπ«
π©βπ§ Parameters Dimension Control
Parameter
Dimension
Making sure that these dimensions are true help us to write better and bug-free :bug: codes
π Summary of Forward Propagation Process
Input:
Output:
π©βπ§ Vectorized Equations
π Summary of Back Propagation Process
Input:
Output:
π©βπ§ Vectorized Equations
β°β° To Put Forward Prop. and Back Prop. Together
π΅π€
β¨ Parameters vs Hyperparameters
π©βπ« Parameters
......
π©βπ§ Hyperparameters
Learning rate
Number of iterations
Number of hidden layers
Number of hidden units
Choice of activation function
......
We can say that hyperparameters control parameters π€
Last updated
Was this helpful?