πCommon Concepts
Basic Concepts of ANN
π Basic Neural Network
Convention: The NN in the image called to be a 2-layers NN since input layer is not being counted π’β
π Common Terms
Term | Description |
π Input Layer | A layer that contains the inputs to the NN |
π Hidden Layer | The layer(s) where computational operations are being done |
π Output Layer | The final layer of the NN and it is responsible for generating the predicted value yΜ |
π§ Neuron | A placeholder for a mathematical function, it applies a function on inputs and provides an output |
π₯ Activation Function | A function that converts an input signal of a node to an output signal by applying some transformation |
πΆ Shallow NN | NN with few number of hidden layers (one or two) |
πͺ Deep NN | NN with large number of hidden layers |
Number of units in l layer |
π§ What does an artificial neuron do?
It calculates a weighted sum of its input, adds a bias and then decides whether it should be fired or not due to an activaiton function
My detailed notes on activation functions are here π©βπ«
π©βπ§ Parameters Dimension Control
Parameter | Dimension |
Making sure that these dimensions are true help us to write better and bug-free :bug: codes
π Summary of Forward Propagation Process
Input: | |
Output: |
π©βπ§ Vectorized Equations
π Summary of Back Propagation Process
Input: | |
Output: |
π©βπ§ Vectorized Equations
β°β° To Put Forward Prop. and Back Prop. Together
π΅π€
β¨ Parameters vs Hyperparameters
π©βπ« Parameters
......
π©βπ§ Hyperparameters
Learning rate
Number of iterations
Number of hidden layers
Number of hidden units
Choice of activation function
......
We can say that hyperparameters control parameters π€
Last updated