Conditional probability for binary classifier

Conditional Probabilities

Quantities

\(\mathrm{TP}\)	-	number of true positive samples
\(\mathrm{TN}\)	-	number of true negative samples
\(\mathrm{FP}\)	-	number of false positive samples
\(\mathrm{FN}\)	-	number of false negative samples
\((+)\)	-	number of positive samples in population
\((-)\)	-	number of negative samples in population

Probabilities

\(P(+)\)	-	probability that the sample is positive
\(P(-)\)	-	probability that the sample is negative
\(P(C+)\)	-	probability that test classifies the sample as positive
\(P(C-)\)	-	probability that test classifies the sample as negative
\(P(A | B)\)	-	conditional probability: probability of A under the condition B

Precision

Probability that the sample is positive, given being classified as positive. Also known as PPV (Positive Predictive Value). $$\mathrm{PREC} = \frac{\mathrm{TP}}{\mathrm{TP} + \mathrm{FP}} = P(+ | C+)$$ See: Precision and recall on Wikipedia.

Recall

Probability that the test classifies sample as positive, given sample being positive. Also known as Sensitivity or TPR (True Positive Rate). $$\mathrm{REC} = \frac{\mathrm{TP}}{\mathrm{TP} + \mathrm{FN}} = P(C+ | +)$$ See: Precision and recall on Wikipedia.

Specificity

Probability that test classifies sample as negative, given sample being negative. Also known as TNR (True Negative Rate). $$\mathrm{SPEC} = \frac{\mathrm{TN}}{\mathrm{TN} + \mathrm{FP}} = P(C- | -)$$ See: Specificity and sensitivity on Wikipedia.

Accuracy

$$\mathrm{ACC} = \frac{\mathrm{TP} + \mathrm {TN}}{\mathrm{TP} + \mathrm{TN} + \mathrm{FP} + \mathrm{FN}}$$ See: Accuracy and precision on Wikipedia.

Negative predictive value

$$\mathrm{NPV} = \frac{\mathrm{TN}}{\mathrm{TN} + \mathrm{FN}} = P(- | C-)$$ See: Positive and negative predictive values on Wikipedia.

False omission rate

$$\mathrm{FOR} = \frac{\mathrm{FN}}{\mathrm{FN} + \mathrm{TN}} = 1 - \mathrm{NPV} = P(+ | C-)$$ See: Positive and negative predictive values on Wikipedia.

False discovery rate

$$\mathrm{FDR} = \frac{\mathrm{FP}}{\mathrm{FP} + \mathrm{TP}} = 1 - \mathrm{PREC} = P(- | C+)$$ See: False discovery rate on Wikipedia.

False positive rate

$$\mathrm{FPR} = \frac{\mathrm{FP}}{\mathrm{FP} + \mathrm{TN}} = \frac{\mathrm{FP}}{(+)} = 1 - \mathrm{SPEC} = P(C+ | -)$$ See False positive rate on Wikipedia.

False negative rate

$$\mathrm{FNR} = \frac{\mathrm{FN}}{\mathrm{FN} + \mathrm{TP}} = \frac{\mathrm{FN}}{\mathrm{(+)}} = 1 - \mathrm{REC} = P(C- | +) $$ See False positives and negatives on Wikipedia.

F1

F1 is a harmonic mean of Precision and Recall, which gives: $$\mathrm{F1} = \frac{\mathrm{TP}}{\mathrm{TP} + \frac{1}{2} \left( \mathrm{FP} + \mathrm{FN} \right)}$$ See: F1 score on Wikipedia.

FM

FM - Fowlkes–Mallows index is a geometric mean of Precision and Recall: $$\mathrm{F1} = \sqrt{\mathrm{PREC} \cdot \mathrm{REC}}$$ See: Fowlkes–Mallows index on Wikipedia.

P4

P4 - probabilistic harmonic mean - is a harmonic mean of Precision, Recall, Specificity and NPV: $$\mathrm{P}_4 = \frac{4}{\frac{1}{\mathrm{PREC}} + \frac{1}{\mathrm{REC}} + \frac{1}{\mathrm{SPEC}} + \frac{1}{\mathrm{NPV}}} = \frac{4\cdot\mathrm{TN}\cdot\mathrm{TP}}{4\cdot\mathrm{TP}\cdot\mathrm{TN} + \mathrm{TP}\cdot\mathrm{FP} + \mathrm{TP}\cdot\mathrm{FN} + \mathrm{TN}\cdot\mathrm{FP} + \mathrm{TN}\cdot\mathrm{FN}}$$

Youden Index

$$\mathrm{J} = \mathrm{REC} + \mathrm{SPEC} - 1 = \frac{\mathrm{TP}}{\mathrm{TP} + \mathrm{FN}} + \frac{\mathrm{TN}}{\mathrm{TN} + \mathrm{FP}} - 1$$ See: Youden's statistic on Wikipedia.

Markedness

$$\mathrm{MK} = \mathrm{PREC} + \mathrm{NPV} - 1$$ See: Powers (2020) article.

MCC

MCC - Matthews correlation coefficient, also known as Phi coefficient. $$\mathrm{MCC} = \frac{\mathrm{TP} \cdot \mathrm{TN} - \mathrm{FP} \cdot \mathrm{FN}}{\sqrt{(\mathrm{TP} + \mathrm{FP}) (\mathrm{TP} + \mathrm{FN}) (\mathrm{TN} + \mathrm{FP}) (\mathrm{TN} + \mathrm{FN})}}$$ See: Phi coefficient on Wikipedia.