Machine learning tips and tricks (Tổng hợp mẹo về học máy)

Uni24h·3 months ago·2669 documents

PDF526 KB

Free download

0 downloads

0 likes

0 dislikes

Generating preview...

Next by this authorRefresher probabilities statistics (Xác suất và thống kê cho Machine Learning) - Afshine Amidi and Shervine Amidi Previous in courseEval cv notes (10) (Đánh giá mô hình và kiểm tra chéo trong học máy) - Sebastian Raschka Next in courseRefresher probabilities statistics (Xác suất và thống kê cho Machine Learning) - Afshine Amidi and Shervine Amidi

What is this document?

Tài liệu tổng hợp mẹo về học máy, bao gồm các độ đo đánh giá mô hình phân loại và hồi quy, lựa chọn mô hình, và chính quy hoá.

Auto-generated · quick read before downloading

Document name: Machine learning tips and tricks (Tổng hợp mẹo về học máy)
School / Course: Stanford University · Machine learning
Author (in document): Afshine Amidi and Shervine Amidi
Content: Tài liệu cung cấp các mẹo và thủ thuật học máy, tập trung vào các chỉ số đánh giá hiệu suất cho bài toán phân loại và hồi quy, cùng với các kỹ thuật lựa chọn mô hình như kiểm định chéo và điều chuẩn hóa.
Table of contents
Pages: 2 pages
Uploaded by: Uni24h

Document content

Trang 1

CS 229 – Machine Learning https://stanford.edu/~shervine VIP Cheatsheet: Machine Learning Tips r ROC – The receiver operating curve, also noted ROC, is the plot of TPR versus FPR by varying the threshold. These metrics are are summed up in the table below: Afshine Amidi and Shervine Amidi Metric Formula Equivalent TP True Positive Rate Recall, sensitivity August 12, 2018 TP + FN TPR FP False Positive Rate 1-specificity TN + FP Metrics FPR Given a set of data points {x(1), ..., x(m)}, where each x(i) has n features, associated to a set of r AUC – The area under the receiving operating curve, also noted AUC or AUROC, is the outcomes {y(1), ..., y(m)}, we want to assess a given classifier that learns how to predict y from area below the ROC as shown in the following figure: x. Classification In a context of a binary classification, here are the main metrics that are important to track to assess the performance of the model. r Confusion matrix – The confusion matrix is used to have a more complete picture when assessing the performance of a model. It is defined as follows: Predicted class + – TP FN Regression + False Negatives True Positives Type II error r Basic metrics – Given a regression model f, the following metrics are commonly used to Actual class assess the performance of the model: FP TN – False Positives Total sum of squares Explained sum of squares Residual sum of squares True Negatives Type I error ∑m ∑m ∑m SS = (y − y)2 SS = (f(x ) − y)2 SS = (y − f(x ))2 tot i reg i res i i r Main metrics – The following metrics are commonly used to assess the performance of classification models: i=1 i=1 i=1 Metric Formula Interpretation r Coefficient of determination – The coefficient of determination, often noted R2 or r2, TP + TN provides a measure of how well the observed outcomes are replicated by the model and is defined Accuracy Overall performance of model as follows: TP + TN + FP + FN 2 SSres TP R = 1 − Precision How accurate the positive predictions are SStot TP + FP TP r Main metrics – The following metrics are commonly used to assess the performance of Recall Coverage of actual positive sample regression models, by taking into account the number of variables n that they take into consid- TP + FN Sensitivity eration: TN Specificity Coverage of actual negative sample Mallow’s Cp AIC BIC Adjusted R2 TN + FP 2TP SS + 2(n + 1)̂σ2 [ ] (1 − R2)(m − 1) F1 score Hybrid metric useful for unbalanced classes res 2 (n + 2) − log(L) log(m)(n + 2) − 2 log(L) 1 − 2TP + FP + FN m m − n − 1 Stanford University 1 Fall 2018

Trang 2

CS 229 – Machine Learning https://stanford.edu/~shervine Model selection Underfitting Just right Overfitting r Vocabulary – When selecting a model, we distinguish 3 different parts of the dataset as follows: - High training error - Low training error • Training set: this is the part of the dataset on which the model is trained. Symptoms - Training error close - Training error slightly - Training error much to test error lower than test error lower than test error • Validation set: also called hold-out, development set or dev set, it is used to assess the - High bias - High variance performance of the previously trained model. • Test set: once the model has been chosen, it is trained on the training+validation set and tested on the unseen test set. r Cross-validation – Cross-validation is a method that is used to select a model that does Regression not rely on the initial training set too much. The most commonly used method is called k-fold cross-validation and splits the training data into k folds to validate the model on one fold while training the model on the k − 1 other folds, all of this k times. The error is then averaged over the k folds and is named cross-validation error. r Regularization – The regularization procedure aims at avoiding the model to overfit the data and thus deals with high variance issues. The following table sums up the different types of commonly used regularization techniques: LASSO Ridge Elastic Net Classification - Shrinks coefficients to 0 Makes coefficients smaller Tradeoff between variable - Good for variable selection selection and small coefficients Deep learning [ ] - Complexify model - Regularize ... + λ||θ|| ... + λ||θ||2 ... + λ (1 − α)||θ|| + α||θ||2 Remedies - Add more features - Get more data 1 2 1 2 λ ∈ R λ ∈ R λ ∈ R, α ∈ [0,1] - Train longer r Model selection – Train model on training set, then evaluate on the development set, then r Error analysis – Error analysis is analyzing the root cause of the difference in performance pick best performance model on the development set, and retrain all of that model on the whole between the current and the perfect models. training set. r Ablative analysis – Ablative analysis is analyzing the root cause of the difference in perfor- mance between the current and the baseline models. Diagnostics r Bias – The bias of a model is the difference between the expected prediction and the correct model that we try to predict for given data points. r Variance – The variance of a model is the variability of the model prediction for given data points. r Bias/variance tradeoff – The simpler the model, the higher the bias, and the more complex the model, the higher the variance. Stanford University 2 Fall 2018

Showing the first 2 pages. Download the document to see it all.

Description

Document excerpt

https://stanford.edu/~shervine CS 229 – Machine Learning VIP Cheatsheet: Machine Learning Tips r ROC – The receiver operating curve, also noted ROC, is the plot of TPR versus FPR by varying the threshold. These metrics are are summed up in the table below: Metric Formula Equivalent True Positive Rate TP TP + FN Recall, sensitivity FP TN + FP 1-specificity Afshine Amidi and Shervine Amidi August 12, 2018 TPR False Positive Rate FPR Metrics Given a set of data points {x(1) , ..., x(m) }, where each x(i) has n features, associated to a set of outcomes {y (1) , ..., y (m) }, we want to assess a given classifier that learns how to predict y from x. r AUC – The area under the receiving operating curve, also noted AUC or AUROC, is the area below the ROC as shown in the following figure: Classification In a context of a binary classification, here are the main metrics that are important to track to assess the performance of the model. r Confusion matrix – The confusion matrix is used to have a more complete picture when assessing the performance of a model. It is defined as follows: Predicted class + + TP FN False Negatives Regression Type II error r Basic metrics – Given a regression model f , the following metrics are commonly used to assess the performance of the model: True Positives Actual class FP False Positives Type I error TN Total sum of squares True Negatives SStot = r Main metrics – The following metrics are commonly used to assess the performance of classification models: Metric Formula Accuracy TP + TN TP + TN + FP + FN Precision TP TP + FP How accurate the positive predictions are Recall TP TP + FN Coverage of actual positive sample Specificity TN TN + FP Coverage of actual negative sample F1 score 2TP 2TP + FP + FN Sensitivity Stanford University Explained sum of squares m X (yi − y)2 SSreg = X i=1 Interpretation Residual sum of squares m (f (xi ) − y)2 SSres = i=1 m X (yi − f (xi ))2 i=1 r C

Frequently asked questions

Is this document free?

Yes. “Machine learning tips and tricks (Tổng hợp mẹo về học máy)” is free — just sign in and click Download to get the original file.

How many pages is this document?

The document has 2 pages, for the course Machine learning. You can preview it online before downloading.

Can I preview before downloading?

Yes. You can preview this document right on this page with the online reader, then decide whether to download.

Comments (0)

No comments yet. Be the first!