πThe Problem in General
Given a dataset like:
We want:
π Basic Concepts and Notations
Concept | Description |
| Number of examples in dataset |
| |
| Predicted output |
Loss Function | A function to compute the error for a single training example |
Cost Function | The average of the loss functions of the entire training set |
Convex Function | A function that has one local value |
Non-Convex Function | A function that has lots of different local values |
Gradient Descent | An iterative optimization method that we use to converge to the global optimum of |
In other words: The
Cost Function
measures how well our parametersw
andb
are doing on the training set, so the bestw
andb
are the values that minimizeπΉ(w, b)
as possible
π Gradient Descent
General Formula:
Ξ±
(alpha) is the Learning Rate
π₯½ Learning Rate
It is a positive scalar determining the size of the step of each iteration of gradient descent due to the corresponded estimated error each time the model weights are updated, so, it controls how quickly or slowly a neural network model learns a problem.
π Good Learning Rate
π’ Bad Learning Rate
π§ References
Last updated