STAT 479: Machine Learning Lecture Notes Sebastian Raschka Department of Statistics University of Wisconsin–Madison http://stat.wisc.edu/∼sraschka/teaching/stat479-fs2018/ Fall 2018 Contents 11 Statistical Tests and Algorithm Comparison 1 11.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 11.2 Testing the Difference of Proportions . . . . . . . . . . . . . . . . . . . . . . . 1 11.3 Comparing Two Models with the McNemar Test . . . . . . . . . . . . . . . . 3 11.4 Exact p-Values via the Binomial Test . . . . . . . . . . . . . . . . . . . . . . . 5 11.5 Multiple Hypotheses Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 11.6 Cochran’s Q Test for Comparing the Performance of Multiple Classifiers . . . 7 11.7 The F -test for Comparing Multiple Classifiers . . . . . . . . . . . . . . . . . . 10 11.8 Comparing Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 11.9 Resampled Paired t-Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 11.10k-fold Cross-validated Paired t-Test . . . . . . . . . . . . . . . . . . . . . . . . 13 11.11Dietterich’s 5x2-Fold Cross-Validated Paired t-Test . . . . . . . . . . . . . . . 13 11.12Alpaydin’s Combined 5x2cv F -test . . . . . . . . . . . . . . . . . . . . . . . . 14 11.13Effect size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 11.14Nested Cross-Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 11.15Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 STAT 479: Machine Learning Lecture Notes Sebastian Raschka Department of Statistics University of Wisconsin–Madison http://stat.wisc.edu/∼sraschka/teaching/stat479-fs2018/ Fall 2018 11 Statistical Tests and Algorithm Comparison 11.1 Overview This fourth lecture in the model evaluation part of this machine learning class provides overviews of several statistical hypothesis testin

… Tải file gốc để đọc toàn bộ tài liệu.