E6893 Big Data Analytics Lecture 3: Big Data Analytics Algorithms Ching-Yung Lin, Ph.D. Adjunct Professor, Dept. of Electrical Engineering and Computer Science September 22, 2023 1 © 2023 CY Lin, Columbia University E6893 Big Data Analytics – Lecture 3: Big Data Analytics Algorithms
Key Components of Spark MLlib 2 E6893 Big Data Analytics – Lecture 3: Big Data Analytics Algorithms © 2023 CY Lin, Columbia University
Spark ML Classification and Regression 3 E6893 Big Data Analytics – Lecture 3: Big Data Analytics Algorithms © 2023 CY Lin, Columbia University
Classification — definition 4 E6893 Big Data Analytics – Lecture 3: Big Data Analytics Algorithms © 2023 CY Lin, Columbia University
Classification example: using SVM to recognize a Toyota Camry ML — Support Vector Machine Non-ML Feature Space Positive SVs Rule 1.Symbol has something like bull’s head Rule 2.Big black portion in front of car. Rule 3. …..???? Negative SVs 5 E6893 Big Data Analytics – Lecture 3: Big Data Analytics Algorithms © 2023 CY Lin, Columbia University
Classification example: using SVM to recognize a Toyota Camry ML — Support Vector Machine Positive SVs PCamry > 0.95 Feature Space Negative SVs 6 E6893 Big Data Analytics – Lecture 3: Big Data Analytics Algorithms © 2023 CY Lin, Columbia University
When to use Big Data System for classification? 7 E6893 Big Data Analytics – Lecture 3: Big Data Analytics Algorithms © 2023 CY Lin, Columbia University
The advantage of using Big Data System for classification 8 E6893 Big Data Analytics – Lecture 3: Big Data Analytics Algorithms © 2023 CY Lin, Columbia University
How does a classification system work? 9 E6893 Big Data Analytics – Lecture 3: Big Data Analytics Algorithms © 2023 CY Lin, Columbia University
Key terminology for classification 10 E6893 Big Data Analytics – Lecture 3: Big Data Analytics Algorithms © 2023 CY Lin, Columbia University
Input and Output of a classification model 11 E6893 Big Data Analytics – Lecture 3: Big Data Analytics Algorithms © 2023 CY Lin, Columbia University
Four types of values for predictor variables 12 E6893 Big Data Analytics – Lecture 3: Big Data Analytics Algorithms © 2023 CY Lin, Columbia University
Sample data that illustrates all four value types 13 E6893 Big Data Analytics – Lecture 3: Big Data Analytics Algorithms © 2023 CY Lin, Columbia University
Supervised vs. Unsupervised Learning 14 E6893 Big Data Analytics – Lecture 3: Big Data Analytics Algorithms © 2023 CY Lin, Columbia University
Work flow in a typical classification project 15 E6893 Big Data Analytics – Lecture 3: Big Data Analytics Algorithms © 2023 CY Lin, Columbia University
Classification Example 1 — Color-Fill Position looks promising, especially the x-axis ==> predictor variable. Shape seems to be irrelevant. Target variable is “color-fill” label. 16 E6893 Big Data Analytics – Lecture 3: Big Data Analytics Algorithms © 2023 CY Lin, Columbia University
Classification Example 2 — Color-Fill (another feature) 17 E6893 Big Data Analytics – Lecture 3: Big Data Analytics Algorithms © 2023 CY Lin, Columbia University
Fundamental classification algorithms Example of fundamental classification algorithms: • Naive Bayesian • Complementary Naive Bayesian • Stochastic Gradient Descent (SDG) • Random Forest • Support Vector Machines 18 E6893 Big Data Analytics – Lecture 3: Big Data Analytics Algorithms © 2022 CY Lin, Columbia University
Choose algorithm 19 E6893 Big Data Analytics – Lecture 3: Big Data Analytics Algorithms © 2023 CY Lin, Columbia University
Stochastic Gradient Descent (SGD) 20 E6893 Big Data Analytics – Lecture 3: Big Data Analytics Algorithms © 2023 CY Lin, Columbia University
Affichage des 20 premières pages. Téléchargez le document pour le voir en entier.
Aucun commentaire pour le moment. Soyez le premier !
Commentaires (0)
Aucun commentaire pour le moment. Soyez le premier !