CS361 (Artificial Intelligence) Lecture 7 Supervised Machine Learning: Decision Trees via ID3 Dr. Hala Abdel-Galil & Dr. Amr S. Ghoneim (Computer Science Dept.) Helwan University Fall 2019 Lecture is based on its counterparts in the following courses: o Applied Machine Learning, University of Pennsylvania (School of Engineering and Applied Science) Today’s Key Concepts o Machine Learning o Classical Supervised Machine Learning o Why use learning? o Supervised Learning: Training & Testing o Decision Trees (the ID3 algorithm; a Greedy heuristic – based on information gain – originally developed for discrete features) o o o o o o Decision Trees: An Introduction Basic Decision Trees Learning Algorithm Picking the Root Attribute: Entropy & Information Gain Which feature to split on? An Illustrative Example Decision Trees Induction o Weka; the Waikato Environment for Knowledge Analysis (Machine Learning Software in Java) o An Application to Business Intelligence o What is Business Intelligence (BI) & Business Analytics (BA) .. ? o Why BI? .. Business Management Issues o A Framework for Business Intelligence (BI’s Architecture & Components) o A Case Study: The German Credit Data Analysis (Statlog Dataset) Machine Learning? {Artificial Intelligence} Machine Learning Map 3 Classical Supervised Machine Learning 4 Supervised Learning Input 𝒙∈ 𝑿 An item 𝒙 drawn from an input space 𝑿 Output System 𝑦 = 𝑓(𝒙) 𝑦∈ 𝒀 An item 𝑦 drawn from an output space 𝒀 We consider systems that apply a function 𝒇( ) to input items x and return an output 𝒚 = 𝒇(𝒙). Supervised Learning Input 𝒙∈ 𝑿 An item 𝒙 drawn from an input space 𝑿 Output System 𝑦 = 𝑓(𝒙) 𝑦∈ 𝒀 An item 𝑦 drawn from an output space 𝒀 In (supervised) machine learning, we deal with systems whose 𝒇(𝒙) is learned from examples. Why use learning? We typically use machine learning when the function 𝒇(𝒙) we want the system to apply is unknown to us, and we cannot “think” about it. The fu