E6893 Big Data Analytics Lecture 3: Big Data Analytics Algorithms Ching-Yung Lin, Ph.D. Adjunct Professor, Dept. of Electrical Engineering and Computer Science September 22, 2023 1 E6893 Big Data Analytics – Lecture 3: Big Data Analytics Algorithms © 2023 CY Lin, Columbia University Key Components of Spark MLlib 2 E6893 Big Data Analytics – Lecture 3: Big Data Analytics Algorithms © 2023 CY Lin, Columbia University Spark ML Classification and Regression 3 E6893 Big Data Analytics – Lecture 3: Big Data Analytics Algorithms © 2023 CY Lin, Columbia University Classification — definition 4 E6893 Big Data Analytics – Lecture 3: Big Data Analytics Algorithms © 2023 CY Lin, Columbia University Classification example: using SVM to recognize a Toyota Camry Non-ML Rule 1.Symbol has something like bull’s head Rule 2.Big black portion in front of car. Rule 3. …..???? ML — Support Vector Machine Feature Space Positive SVs Negative SVs 5 E6893 Big Data Analytics – Lecture 3: Big Data Analytics Algorithms © 2023 CY Lin, Columbia University Classification example: using SVM to recognize a Toyota Camry ML — Support Vector Machine Positive SVs PCamry > 0.95 Feature Space Negative SVs 6 E6893 Big Data Analytics – Lecture 3: Big Data Analytics Algorithms © 2023 CY Lin, Columbia University When to use Big Data System for classification? 7 E6893 Big Data Analytics – Lecture 3: Big Data Analytics Algorithms © 2023 CY Lin, Columbia University The advantage of using Big Data System for classification 8 E6893 Big Data Analytics – Lecture 3: Big Data Analytics Algorithms © 2023 CY Lin, Columbia University How does a classification system work? 9 E6893 Big Data Analytics – Lecture 3: Big Data Analytics Algorithms © 2023 CY Lin, Columbia University Key terminology for classification 10 E6893 Big Data Analytics – Lecture 3: Big Data Analytics Algorithms © 2023 CY Lin, Columbia University Input and Output of a classification model 11 E6893