Understanding AUC
To\r\n understand the calculation of AUC, a few basic concepts must be \r\nintroduced. For a binary choice prediction, there are four possible \r\noutcomes:
\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n
\r\n\r\n
The true positive rate, or recall, is çalculated as the number of true positives divided by the total number of positives. When identifying aircraft from radar signals, it is proportion that are correctly identified.
The false positive rate is çalculated as the number of false positives divided by the total \r\nnumber of negatives. When identifying aircraft from radar signals, it is the rate of false alarms.
If\r\n somebody makes random guesses, the ROC curve will be a diagonal line \r\nstretching from (0,0) to (1,1) - see the blue line in the figure below. \r\nTo understand this consider:
![\"area](\"@Url.GeneralImage('AUC.png')\")
While\r\n ROC is a two-dimensional representation of a model's performance, the \r\nAUC distils this information into a single scalar. As the name implies, \r\nit is calculated as the area under the ROC curve. A perfect model will \r\nscore an AUC of 1, while random guessing will score an AUC of around of\r\n 0.5. In practice, almost all models will fit somewhere in between.
To\r\n understand the calculation of AUC, a few basic concepts must be \r\nintroduced. For a binary choice prediction, there are four possible \r\noutcomes:
- true positive - a positive instance that is correctly classified as positive;
- false positive - a negative instance that is incorrectly\r\n classified as positive;
- true negative - a negative instance that is correctly\r\n classified as negative;
- false negative - a positive instance that is incorrectly\r\n classified as negative);
\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n
| actual class | \r\n|||
| \r\n | P | \r\nN | \r\n|
| predicted class | \r\np | \r\ntrue positive | \r\nfalse positive | \r\n
| n | \r\nfalse negative | \r\ntrue negative | \r\n
The true positive rate, or recall, is çalculated as the number of true positives divided by the total number of positives. When identifying aircraft from radar signals, it is proportion that are correctly identified.
The false positive rate is çalculated as the number of false positives divided by the total \r\nnumber of negatives. When identifying aircraft from radar signals, it is the rate of false alarms.
If\r\n somebody makes random guesses, the ROC curve will be a diagonal line \r\nstretching from (0,0) to (1,1) - see the blue line in the figure below. \r\nTo understand this consider:
- Somebody\r\n who randomly guesses that 10 per cent of all radar signals point to \r\nplanes. The false positive rate and the false alarm rate will be 10 per \r\ncent.
- Somebody who randomly guesses that 90 per cent of \r\nall radar signals point to planes. The false positive rate and the false\r\n alarm rate will be 90 per cent.
While\r\n ROC is a two-dimensional representation of a model's performance, the \r\nAUC distils this information into a single scalar. As the name implies, \r\nit is calculated as the area under the ROC curve. A perfect model will \r\nscore an AUC of 1, while random guessing will score an AUC of around of\r\n 0.5. In practice, almost all models will fit somewhere in between.