https://stanford.edu/~shervine CS 229 – Machine Learning VIP Cheatsheet: Machine Learning Tips r ROC – The receiver operating curve, also noted ROC, is the plot of TPR versus FPR by varying the threshold. These metrics are are summed up in the table below: Metric Formula Equivalent True Positive Rate TP TP + FN Recall, sensitivity FP TN + FP 1-specificity Afshine Amidi and Shervine Amidi August 12, 2018 TPR False Positive Rate FPR Metrics Given a set of data points {x(1) , ..., x(m) }, where each x(i) has n features, associated to a set of outcomes {y (1) , ..., y (m) }, we want to assess a given classifier that learns how to predict y from x. r AUC – The area under the receiving operating curve, also noted AUC or AUROC, is the area below the ROC as shown in the following figure: Classification In a context of a binary classification, here are the main metrics that are important to track to assess the performance of the model. r Confusion matrix – The confusion matrix is used to have a more complete picture when assessing the performance of a model. It is defined as follows: Predicted class + + TP FN False Negatives Regression Type II error r Basic metrics – Given a regression model f , the following metrics are commonly used to assess the performance of the model: True Positives Actual class FP False Positives Type I error TN Total sum of squares True Negatives SStot = r Main metrics – The following metrics are commonly used to assess the performance of classification models: Metric Formula Accuracy TP + TN TP + TN + FP + FN Precision TP TP + FP How accurate the positive predictions are Recall TP TP + FN Coverage of actual positive sample Specificity TN TN + FP Coverage of actual negative sample F1 score 2TP 2TP + FP + FN Sensitivity Stanford University Explained sum of squares m X (yi − y)2 SSreg = X i=1 Interpretation Residual sum of squares m (f (xi ) − y)2 SSres = i=1 m X (yi − f (xi ))2 i=1 r C