CS361 (Software Engineering Program) Artificial Intelligence II - Applied Machine Learning Lecture 7 A Basic Introduction to Ensemble Learning [ Bagging, Random Forests, Boosting, AdaBoost ] Amr S. Ghoneim (Assistant Professor, Computer Science Dept.) Helwan University Fall 2019 Lecture is based on its counterparts in the following courses (and the following resources): o Ensemble Learning, University of Szeged "Szegedi Tudományegyetem" (Hungary), Institute of Informatics. o Web-Mining Agents: Classification with Ensemble Methods, Universität zu Lübeck (Germany), R. Möller, at the Institute of Information Systems. o Machine Learning CS165B, UCSB University of California Santa Barbara (California USA), Department of Computer Science. o Explaining AdaBoost, Princeton University (New Jersey USA), Rob E. Schapire, Department of Computer Science: https://www.cs.princeton.edu/~schapire/papers/explaining-adaboost.pdf Today’s Key Concepts o Basic Idea o Condorcet’s Jury Theorem o Strong versus Weak Learners o Ensemble Learning .. a Generic Approach o Conditions o How to produce Diverse Classifiers? o Randomization of Decision Trees o Random Forests o Ensemble-Based Methods specifically invented for Ensemble Learning o Bagging o Bootstrap Resampling o Random Forests o Boosting o Boosting by Sampling o Boosting by Weighting o Adaboost (Adaptive Boosting) Machine Learning? {Artificial Intelligence} Machine Learning Map 3 Recap: Supervised Learning Goal: learn predictor h(x): o High accuracy (low error). o Using training data { (x1, y1), .., (xn, yn) }. Recap: Supervised Learning Male? Yes 1 Person Age Male? Height > 55” Alice 14 0 1 Yes No Bob 10 1 1 Age>9? Age>10? Carol 13 0 1 Dave 8 1 0 Erin 11 0 0 Frank 9 1 1 Gena 8 0 0 No 0 Yes 1 No 0 é ù age ú x =ê êë 1[gender=male] úû ìï 1 height > 55" y=í ïî 0 height £ 55" Basic Idea .. Condorcet’s Jury Theorem Concordet’s jury theorem (1785) is a political science theorem about the