๐ฉโ๐ซ Implementation Guidelines
Implementation guidelines and error anlysis
๐ Common Terms
Term
Description
๐ฉโ๐ Bayes Error
The lowest possible error rate for any classifier (The optimal error ๐ค)
๐ฉโ๐ซ Human Level Error
The error rate that can be obtained by a human
๐ฎโโ๏ธ Avoidable Bias โ
The difference between Bayes error and human level error
I did my best, my project is still doing bad, what shall I do? ๐ฅ
Well, in this stage we have a criteria, is your model doing worse than humans (Because humans are quite good at a lot of tasks ๐ฉโ๐)? If yes, you can:
๐ฉโ๐ซ Get labeled data from humans
๐ Gain insight from manual error analysis; (Why did a person get this right? ๐)
๐ Better analysis of bias / variance ๐
๐ค Note: knowing how well humans can do on a task can help us to understand better how much we should try to reduce bias and variance
๐ง Is your model doing better than humans?
Processes are less clear ๐ฅ
Suitable techniques will be added here
๐ค Study case
Let's assume that we have these two situations:
Case1
Case2
Human Error
1%
7.5%
Training Error
8%
8%
Dev Error
10%
10%
Even though training and dev errors are same we will apply different tactics for better performance
In Case1, We have
High Bias
so we have to focus on bias reduction techniques ๐ค, in other words we have to reduce the difference between training and human errors the avoidable errorBetter algorithm, better NN structure, ......
In Case2, We have
High Variance
so we have to focus on variance reduction techniques ๐, in other words we have to reduce the difference between training and dev errorsAdding regularization, getting more data, ......
We call this procedure of analysis Error analysis ๐ต๏ธโ
๐ Error Types Visualization

In computer vision issues,
human-level-error โ bayes-error
because humans are good in vision tasks
๐ค Problems that ML surpasses human level performance
Online advertising
Product recommendations
Logistics
Loan approvals
.....
โจ My Detailed Notes on Bias / Variance and Related Procedures
๐คธโโ๏ธ It is recommended to
When we have a new project it is recommended to produce an initial model and then iterate over it until you get the best model, this is more practical than spending time building model theoretical and thinking about the best hyperparameter -which is almost impossible ๐-
So, just don't overthink! (In both ML problems and life problems ๐ค๐โ)
Last updated
Was this helpful?