Trang 1
CS361 (Software Engineering Program) Artificial Intelligence II-Applied Machine Learning Lecture 9 A Basic Introduction to Deep Learning & Convolutional Neural Networks [CNNs] Amr S. Ghoneim (Assistant Professor, Computer Science Dept.) Helwan University Fall 2019 Lecture is based on its counterparts in the following courses (& the following resources): o CS231n: Convolutional Neural Networks for Visual Recognition, Stanford University (California USA), Stanford School of Engineering. o CS 898: Deep Learning and Its Applications, University of Waterloo (Waterloo - Ontario, Canada), David R. Cheriton School of Computer Science o Introduction to Deep Learning, UIUC University of Illinois at Urbana - Champaign (Illinois USA), Computer Science Department. o CS231A: Computer Vision, From 3D Reconstruction to Recognition, Stanford University (California USA), Computer Science Dept. -The Computational Vision & Geometry Lab (CVGL)
Trang 2
Today’s Key Concepts • Machine Learning • Deep Learning • CNNs: A Bit of History • First Strong Results • Hierarchical Organization of early Visual Pathways • ImageNet Classification with Deep CNNs • AlexNet • LeNet-5 • Basic Concepts of CNNs • Overall CNN Architecture • Convolution Layer & Convolution Filters • Activation Maps & Activation Functions (ReLU) • ConvNet: a Sequence of Convolution Layers • Spatial Dimensions • Pooling Layer & Padding the Borders • Fully Connected Layer (FC layer) & Flattening
Trang 3
Machine Learning? {A r t i f i c i a l I n t e l l i g e n c e} M a c h i n e L e a r n i n g M a p 3
Trang 5
Recap: Machine Learning Basics Machine learning is a field of computer science that gives computers the ability to learn without being explicitly programmed Machine Learning Labeled Data Algorithm Training Prediction Learned Labeled Data Model Prediction Methods that can learn from and make predictions on data.
Trang 6
Recap: Types of Learning Supervised: Learning with a labeled training set. Example: email classification with already labeled emails. Unsupervised: Discover patterns in unlabeled data. Example: cluster similar documents based on text. Reinforcement learning: learn to act based on feedback/reward Example: learn to play Go, reward: win or lose. class A class A Classification Regression Clustering Anomaly Detection Sequence labeling …
Trang 7
Recap: ML vs. Deep Learning • Most machine learning methods work well because of human- designed representations and input features. • ML becomes just optimizing weights to best make a final prediction.
Trang 8
Recap: What is Deep Learning (DL) ? • A machine learning subfield of learning representations of data. • Exceptionally effective at learning patterns. • Deep learning algorithms attempt to learn (multiple levels of) representation by using a hierarchy of multiple layers. • If you provide the system tons of information, it begins to understand it and respond in useful ways. https://www.xenonstack.com/blog/static/public/uploads/media/machine-learning-vs-deep-learning.png
Trang 9
Convolutional Neural Networks (CNN, ConvNet) is a class of deep, feed-forward (not recurrent) artificial neural networks that areapplied “mostly”to analyzing visualimagery. Convolutional Neural Networks CNNs A BRIEF HISTORY & APPLICATIONS
Trang 11
A Bit of History.. Rumelhart et al., 1986: First time back-propagation became popular. Illustration of Rumelhart et al., 1986 by Lane McIntosh, copyright CS231n 2017
Trang 12
First Strong Results .. A Bit of History.. Acoustic Modeling using Deep Belief Networks Abdel-rahman Mohamed, George Dahl, Geoffrey Hinton, 2010 Context-Dependent Pre-trained Deep Neural Networks for Large Vocabulary Speech Recognition George Dahl, Dong Yu, Li Deng, Alex Acero, 2012 Illustration of Dahl et al. 2012 by Lane McIntosh, copyright CS231n 2017 Imagenet classification with deep convolutional neural networks Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton, 2012 Figures copyright Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton, 2012. Reproduced withpermission.
Trang 13
A Bit of History .. Using electrodes .. Hubel & Wiesel, 1959 RECEPTIVE FIELDS OF SINGLE NEURONES IN THE CAT'S STRIATE CORTEX 1962 RECEPTIVE FIELDS, BINOCULAR INTERACTION AND FUNCTIONAL ARCHITECTURE IN THE CAT'S VISUAL CORTEX 1968..
Trang 14
A Bit of History .. Hubel & Wiesel, 1959 RECEPTIVE FIELDS OF SINGLE NEURONES IN THE CAT'S STRIATE CORTEX 1962 RECEPTIVE FIELDS, BINOCULAR INTERACTION AND FUNCTIONAL ARCHITECTURE IN THE CAT'S VISUAL CORTEX 1968.. Topographical mapping in the cortex: nearby cells in cortex represent nearby regions in the visual field ..
Trang 15
A Bit of History .. Hierarchical Organization? Illustration of hierarchical organization in early visual pathways by Lane McIntosh, copyright CS231n2017
Trang 16
A Bit of History .. Neocognitron [Fukushima 1980] “sandwich” architecture (SCSCSC…) simple cells: modifiable parameters complex cells: perform pooling
Trang 17
A Bit of History .. Gradient-based learning applied to document recognition [LeCun, Bottou, Bengio, Haffner 1998] LeNet-5
Trang 18
Image Maps 1998 Input LeCun et al. K Output Convolutions Fully Connected Subsampling # of transistors # of pixels used in training 106 107 2012 Krizhevsky et al. # of transistors GPUs # of pixels used in training 109 1014 Figure copyright Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton, 2012. Reproduced with permission.
Trang 19
www.image-net.org 22K categories and 14M images • Animals • Plants • Structures • Person • Bird • Tree • Artifact • Scenes • Fish • Flower • Tools • Indoor • Mammal • Food • Appliances • Geological • Invertebrate • Materials • Structures Formations • SportActivities Deng, Dong, Socher, Li, Li, & Fei-Fei, 2009
Trang 20
The Image Classification Challenge: 1,000 object classes 1,431,167 images Output: Output: Scale ✔ Scale ✗ T-shirt T-shirt Steel drum Giant panda Drumstick Drumstick Mud turtle Mud turtle Russakovskyet al. arXiv, 2014
Hiển thị 20 trang đầu. Tải tài liệu để xem đầy đủ.
Bình luận (0)
Chưa có bình luận nào. Hãy là người đầu tiên!