Trang 1
deep learning interviewsDEEP LEARNING Deep Learning Interviews is home to hundreds of fully-solved problems, from a wide range of key topics in AI. It is designed to both rehearse interview or exam-specific topics and provide machine learning M.Sc./Ph.D. INTERVIEWS students, and those awaiting an interview a well-organized overview of the field. The problems it poses are tough enough to cut your teeth on and to dramatically improve your skills-but they’re framed within thought- provoking questions and engaging stories. REAL-WORLD DEEP LEARNING INTERVIEW That is what makes the volume so specifically valuable to students and job PROBLEMS & SOLUTIONS seekers: it provides them with the ability to speak confidently and quickly on any relevant topic, to answer technical questions clearly and correctly, and to second edition fully understand the purpose and meaning of interview questions and answers. These are powerful, indispensable advantages to have when walking into the interview room. The book’s contents is a large inventory of numerous topics relevant to DL job interviews and graduate-level exams. That places this work at the forefront of the growing trend in science to teach a core set of practical mathematical and computational skills. It is widely accepted that the training of every computer scientist must include the fundamental theorems SHLOMO KASHANI of ML, and AI appears in the curriculum of nearly every university. This volume is designed as an excellent reference for graduates of such programs. Shlomo Kashani, Author. Amir Ivry,Chief Editor. • Logistic Regression • Information Theory • Calculus • Algorithmic Differentiation • Bayesian Deep Learning • Probabilistic Programming • Ensemble Learning • CNN Feature Extraction • Deep Learning: Expanded Chapter www.interviews.ai SHLOMO KASHANI DEEP LEARNING INTERVIEWS
Trang 2
SHLOMO KASHANI DEEP LEARNING INTERVIEWS By Shlomo Kashani, M.Sc, QMUL, UK. H1 θ1 H2 γ1 θ2 H3 Published by Shlomo Kashani, Tel-Aviv, ISRAEL. Visit: http://www.interviews.ai Copyright, 2020 This book is protected by copyright. No part may be reproduced in any manner without written permission from the publisher. Printing version: VER. 26TH OCTOBER 2021 Printed in the United States of America. Library of Congress Cataloging-in-Publication Data A catalog record for this book is available from the Library of Congress
Trang 3
COPYRIGHT. © 2016-2020 Shlomo Kashani, entropy@interviews.ai ALL RIGHTS RESERVEDreproduced, duplicated or transmitted without direct written permission.The content contained within this book may not be from the author or the publisher. Under no circumstances will any blame or legal responsibility be held against the publisher, or author, for any dam- ages, reparation, or monetary loss due to the information contained within this book. Either directly or indirectly. This book is copyright protected. This book is only for personal use. You cannot amend, distribute, sell, use, quote or paraphrase any part, or the content within this book, without the consent of the author or publisher. Please note the information contained within this document is for educational and entertainment purposes only. All effort has been executed to present accurate, up to date, and reliable, complete information. No warranties of any kind are declared or implied. Readers acknowledge that the author is not engaging in the rendering of legal, financial, medical or professional advice. The content within this book has been derived from various sources. Please consult a licensed professional before attempt- ing any techniques outlined in this book. By reading this document, the reader agrees that under no circumstances is the author responsible for any losses, direct or indirect, which are incurred as a result of the use of information contained within this docu- ment, including, but not limited to errors, omissions, or inaccuracies. No part of this publication may be reproduced, stored in a retrieval system, or trans- mitted in any form or by any means, electronic, mechanical, photocopying, record- ing, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without the prior written permission of the Publisher. Limit of Liability/Disclaimer of Warranty. While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and spe- cifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or writ- ten sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither
Trang 4
the publisher nor author shall be liable for any loss of profit or any other commer- cial damages, including but not limited to special, incidental, consequential, or other damages. Notices. Knowledge and best practice in this field are constantly changing. As new research and experience broaden our understanding, changes in research methods, professional practices, or medical treatment may become necessary. Practitioners and researchers must always rely on their own experience and knowledge in evaluating and using any information, methods, compounds, or experiments described herein. In using such information or methods they should be mindful of their own safety and the safety of others, including parties for whom they have a professional responsibil- ity. To the fullest extent of the law, neither the Publisher nor the authors, contributors, or editors, assume any liability for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use or operation of any methods, products, instructions, or ideas contained in the material herein.
Trang 5
FOREWORD. We will build a machine that will fly. — Joseph Michael Montgolfier, French Inventor/Aeronaut (1740-1810) DEEP learning interviews are technical, dense, and thanks to the fields com-petitiveness, often high-stakes. The prospect of preparing for one can be daunting, and the fear of failure can be paralyzing and many interviewees find their ideas slipping away alongside their confidence. This book was written for you: an aspiring data scientist with a quantitative back- ground, facing down the gauntlet of the interview process in an increasingly competit- ive field. For most of you, the interview process is the most significant hurdle between you and a dream job. Even though you have the ability, the background, and the mo- tivation to excel in your target position, you might need some guidance on how to get your foot in the door. Though this book is highly technical it is not too dense to work through quickly. It aims to be comprehensive, including many of the terms and topics involved in modern data science and deep learning. That thoroughness makes it unique; no other single work offers such breadth of learning targeted so specifically at the demands of the interview. Most comparable information is available in a variety of formats, locations, struc- tures, and resourcesblog posts, tech articles, and short books scattered across the inter- net. Those resources are simply not adequate to the demands of deep learning inter- view or exam preparation and were not assembled with this explicit purpose in mind. TIt is hoped that this book does not suffer the same shortcomings. HIS books creation was guided by a few key principles: clarity and depth, thoroughness and precision, interest and accuracy. The volume was de- signed for use by job seekers in the fields of machine learning and deep learning whose abilities and background locate them firmly within STEM (science, technology, engineering, and mathematics). The book will still be of use to other readers, such as those still undergoing their initial education in a STEM field. However, it is tailored most directly to the needs of active job seekers and stu- dents attending M.Sc/Ph.D programmes in AI. It is, in any case, a book for engineers, mathematicians, and computer scientists: nowhere does it include the kind of very basic background material that would allow it to be read by someone with no prior
Trang 6
knowledge of quantitative and mathematical processes. The books contents are a large inventory of numerous topics relevant to deep learn- ing job interviews and graduate level exams. Ideas that are interesting or pertinent have been excluded if they are not valuable in that context. That places this work at the forefront of the growing trend in education and in business to emphasize a core set of practical mathematical and computational skills. It is now widely understood that the training of every computer scientist must include a course dealing with the fundamental theorems of machine learning in a rigorous manner; Deep Learning ap- pears in the curriculum of nearly every university; and this volume is designed as a convenient ongoing reference for graduates of such courses and programs. The book is grounded in both academic expertise and on-the-job experience and thus has two goals. First, it compresses all of the necessary information into a coher- ent package. And second, it renders that information accessible and makes it easy to navigate. As a result, the book helps the reader develop a thorough understanding of the principles and concepts underlying practical data science. None of the textbooks I read met all of those needs, which are: 1. Appropriate presentation level. I wanted a friendly introductory text accessible to graduate students who have not had extensive applied experience as data scientists. 2. A text that is rigorous and builds a solid understanding of the subject without getting bogged down in too many technicalities. 3. Logical and notational consistency among topics. There are intimate connec- tions between calculus, logistic regression, entropy, and deep learning theory, which I feel need to be emphasized and elucidated if the reader is to fully under- stand the field. Differences in notation and presentation style in existing sources make it very difficult for students to appreciate these kinds of connections. 4. Manageable size. It is very useful to have a text compact enough that all of the material in it can be covered in few weeks or months of intensive review. Most candidates will have only that much time to prepare for an interview, so a longer text is of no use to them. The text that follows is an attempt to meet all of the above challenges. It will inevitably prove more successful at handling some of them than others, but it has at least made a sincere and devoted effort.
Trang 7
A note about Bibliography The book provides a carefully curated bibliography to guide further study, whether for interview preparation or simply as a matter of interest or job-relevant research. A comprehensive bibliography would be far too long to include here, and would be of little immediate use, so the selections have been made with deliberate attention to the value of each included text. Only the most important books and articles on each topic have been included, and only those written in English that I personally consulted. Each is given a brief annota- tion to indicate its scope and applicability. Many of the works cited will be found to include very full bibliographies of the particular subject treated, and I recommend turning there if you wish to dive deeper into a specific topic, method, or process. We have a web page for this book, where we list errata, examples, and any ad- ditional information. You can access this page at: http://www.interviews.ai. To comment or ask technical questions about this book, send email to: entropy@ interviews.ai. I would also like to solicit corrections, criticisms, and suggestions from students and other readers. Although I have tried to eliminate errors over the multi year process of writing and revising this text, a few undoubtedly remain. In particular, some typographical infelicities will no doubt find their way into the final version. I hope you will forgive them. THE AUTHOR. TEL AVIV ISRAEL, DECEMBER, 2020. FIRST PRINTING, DECEMBER 2020.
Trang 8
ACKNOWLEDGEMENTS. The thanks and acknowledgements of the publisher are due to the following: My dear son, Amir Ivry, Matthew Isaac Harvey, Sandy Noymer, Steve foot and Velimir Gayevskiy.
Trang 9
AUTHOR’S BIOGRAPHY. When Shlomo typed his book in LATEX, he wanted it to reflect some of his passions: AI, design, typography, and most notably coding. On a typical day, his two halves - the scientist and the artist - spend hours meticulously design- ing AI systems, from epilepsy prediction and pulmonary nodule detection, to training a computer-vision model on a cluster. Shlomo spends whole days in a lab full of GPUs work- ing on his many interesting research projects. Though re- search satisfies his itch for discovery, his most important scientific contribution, he says, is helping other researchers. And the results are evident in his publications. But, al- though theoretical studies are important, practical experi- ence has many great virtues. As the Head of AI at DeepOncology, he developed uses of Deep Learning for precise tumour detection, expanding and refining what human experts are capable of. The work, which relies on CNN’s, marks the culmination of a career spent applying AI techniques to problems in medical AI. Shlomo holds an MSc in Digital Signal Processing (Distinction) from the University of London. A PERSONAL NOTE: In this first volume, I purposely present a coherent, cumu- lative, and content-specific core curriculum of the data science field, including topics such as information theory, Bayesian statistics, algorithmic differentiation, logistic re- gression, perceptrons, and convolutional neural networks. I hope you will find this book stimulating. It is my belief that you the postgradu- ate students and job-seekers for whom the book is primarily meant will benefit from reading it; however, it is my hope that even the most experienced researchers will find it fascinating as well. SHLOMO KASHANI,TEL-AVIV,ISRAEL.
Trang 10
ABOUT THE CHIEF EDITOR. Amir Ivry has been an applied research scientist in the fields of deep learning and speech signal processing since 2015. A direct PhD candidate in the Electrical and Computer Engineering Fac- ulty in the Technion - Israel Institute of Technology, Amir is the author of over a dozen academic papers in leading IEEE journ- als and top-tier conferences. For his contribution to the field of hands-free speech communication using deep neural networks, Amir has received more than a dozen awards and honors, in- cluding back-to-back Jacobs citations for research excellence, and most recently the international speech communication associ- ation grant. Being only 28 years old, he has been cemented as a popular lecturer in the machine learning community, and delivered technological sessions for MIT, Google for startups, Alibaba, and more. Amir is currently holding a position as an applied research intern in Microsoft Advanced Technology Labs.
Trang 12
Contents I Rusty Nail 1 HOW-TO USE THIS BOOK 3 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 What makes this book so valuable . . . . . . . . . . . . . . . . . . . . . . 3 What will I learn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 How to Work Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Types of Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 II Kindergarten 9 LOGISTIC REGRESSION 11 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 General Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Odds, Log-odds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 The Sigmoid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 Truly Understanding Logistic Regression . . . . . . . . . . . . . . . . . . 16 The Logit Function and Entropy . . . . . . . . . . . . . . . . . . . . . . . 22 Python/PyTorch/CPP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 General Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 Odds, Log-odds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 The Sigmoid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 Truly Understanding Logistic Regression . . . . . . . . . . . . . . . . . . 33 The Logit Function and Entropy . . . . . . . . . . . . . . . . . . . . . . . 38 Python, PyTorch, CPP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
Trang 13
PROBABILISTIC PROGRAMMING & BAYESIAN DL 41 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 Expectation and Variance . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 Conditional Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 Bayes Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 Maximum Likelihood Estimation . . . . . . . . . . . . . . . . . . . . . . . 51 Fisher Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 Posterior & prior predictive distributions . . . . . . . . . . . . . . . . . . 54 Conjugate priors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 Bayesian Deep Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 Expectation and Variance . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 Conditional Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 Bayes Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 Maximum Likelihood Estimation . . . . . . . . . . . . . . . . . . . . . . . 71 Fisher Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 Posterior & prior predictive distributions . . . . . . . . . . . . . . . . . . 76 Conjugate priors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 Bayesian Deep Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 III High School 83 INFORMATION THEORY 85 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 Logarithms in Information Theory . . . . . . . . . . . . . . . . . . . . . . 87 Shannon's Entropy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 Kullback-Leibler Divergence (KLD) . . . . . . . . . . . . . . . . . . . . . . 93 Classification and Information Gain . . . . . . . . . . . . . . . . . . . . . 94 Mutual Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 Mechanical Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 Jensen's inequality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 Logarithms in Information Theory . . . . . . . . . . . . . . . . . . . . . . 101 Shannon's Entropy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
Trang 14
Kullback-Leibler Divergence . . . . . . . . . . . . . . . . . . . . . . . . . . 108 Classification and Information Gain . . . . . . . . . . . . . . . . . . . . . 110 Mutual Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 Mechanical Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 Jensen's inequality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 DEEP LEARNING: CALCULUS, ALGORITHMIC DIFFERENTIATION 121 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 AD, Gradient descent & Backpropagation . . . . . . . . . . . . . . . . . . 124 Numerical differentiation . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 Directed Acyclic Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 The chain rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 Taylor series expansion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 Limits and continuity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130 Partial derivatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130 Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 The Gradient descent algorithm . . . . . . . . . . . . . . . . . . . . . . . . 132 The Backpropagation algorithm . . . . . . . . . . . . . . . . . . . . . . . . 134 Feed forward neural networks . . . . . . . . . . . . . . . . . . . . . . . . 135 Activation functions, Autograd/JAX . . . . . . . . . . . . . . . . . . . . . 136 Dual numbers in AD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 Forward mode AD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 Forward mode AD table construction . . . . . . . . . . . . . . . . . . . . 142 Symbolic differentiation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 Simple differentiation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 The Beta-Binomial model . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146 Algorithmic differentiation, Gradient descent . . . . . . . . . . . . . . . . 146 Numerical differentiation . . . . . . . . . . . . . . . . . . . . . . . . . . . 146 Directed Acyclic Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 The chain rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 Taylor series expansion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150 Limits and continuity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 Partial derivatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152 Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 The Gradient descent algorithm . . . . . . . . . . . . . . . . . . . . . . . . 155
Trang 15
The Backpropagation algorithm . . . . . . . . . . . . . . . . . . . . . . . . 156 Feed forward neural networks . . . . . . . . . . . . . . . . . . . . . . . . 158 Activation functions, Autograd/JAX . . . . . . . . . . . . . . . . . . . . . 158 Dual numbers in AD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163 Forward mode AD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 Forward mode AD table construction . . . . . . . . . . . . . . . . . . . . 168 Symbolic differentiation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172 Simple differentiation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172 The Beta-Binomial model . . . . . . . . . . . . . . . . . . . . . . . . . . . 174 IV Bachelors 183 DEEP LEARNING: NN ENSEMBLES 185 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186 Bagging, Boosting and Stacking . . . . . . . . . . . . . . . . . . . . . . . . 186 Approaches for Combining Predictors . . . . . . . . . . . . . . . . . . . . 190 Monolithic and Heterogeneous Ensembling . . . . . . . . . . . . . . . . . 191 Ensemble Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194 Snapshot Ensembling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 Multi-model Ensembling . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196 Learning-rate Schedules in Ensembling . . . . . . . . . . . . . . . . . . . 197 Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198 Bagging, Boosting and Stacking . . . . . . . . . . . . . . . . . . . . . . . . 198 Approaches for Combining Predictors . . . . . . . . . . . . . . . . . . . . 199 Monolithic and Heterogeneous Ensembling . . . . . . . . . . . . . . . . . 200 Ensemble Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201 Snapshot Ensembling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201 Multi-model Ensembling . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202 Learning-rate Schedules in Ensembling . . . . . . . . . . . . . . . . . . . 202 DEEP LEARNING: CNN FEATURE EXTRACTION 205 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206 CNN as Fixed Feature Extractor . . . . . . . . . . . . . . . . . . . . . . . . 206 Fine-tuning CNNs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
Trang 16
Neural style transfer, NST . . . . . . . . . . . . . . . . . . . . . . . . . . . 214 Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216 CNN as Fixed Feature Extractor . . . . . . . . . . . . . . . . . . . . . . . . 216 Fine-tuning CNNs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222 Neural style transfer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224 DEEP LEARNING 227 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231 Cross Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231 Convolution and correlation . . . . . . . . . . . . . . . . . . . . . . . . . . 234 Similarity measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241 Perceptrons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246 Activation functions (rectification) . . . . . . . . . . . . . . . . . . . . . . 253 Performance Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260 NN Layers, topologies, blocks . . . . . . . . . . . . . . . . . . . . . . . . . 263 Training, hyperparameters . . . . . . . . . . . . . . . . . . . . . . . . . . . 280 Optimization, Loss . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286 Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289 Cross Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289 Convolution and correlation . . . . . . . . . . . . . . . . . . . . . . . . . . 291 Similarity measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296 Perceptrons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299 Activation functions (rectification) . . . . . . . . . . . . . . . . . . . . . . 306 Performance Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316 NN Layers, topologies, blocks . . . . . . . . . . . . . . . . . . . . . . . . . 318 Training, hyperparameters . . . . . . . . . . . . . . . . . . . . . . . . . . . 327 Optimization, Loss . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331 V Practice Exam 339 JOB INTERVIEW MOCK EXAM 341 Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343 Perceptrons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343 CNN layers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343
Trang 17
Classification, Logistic regression . . . . . . . . . . . . . . . . . . . . . . . 345 Information theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347 Feature extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349 Bayesian deep learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352 VI Volume two 357 VOLUME TWO - PLAN 359 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360 AI system design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360 Advanced CNN topologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360 1D CNN’s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360 3D CNN’s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360 Data augmentations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360 Object detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360 Object segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360 Semantic segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360 Instance segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360 Image classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360 Image captioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360 NLP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360 RNN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361 LSTM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361 GANs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361 Adversarial attacks and defences . . . . . . . . . . . . . . . . . . . . . . . . . . 361 Variational auto encoders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361 FCN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361 Seq2Seq . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361 Monte carlo, ELBO, Re-parametrization . . . . . . . . . . . . . . . . . . . . . . 361 Text to speech . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361 Speech to text . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361 CRF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361 Quantum computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361 RL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361 xvi
Trang 20
CHAPTER 1 HOW-TO USE THIS BOOK The true logic of this world is in the calculus of probabilities. — James C. Maxwell Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 What makes this book so valuable . . . . . . . . . . . . . . . . . . . . . 3 What will I learn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Starting Your Career . . . . . . . . . . . . . . . . . . . . . . . . . 4 Advancing Your Career . . . . . . . . . . . . . . . . . . . . . . . 5 Diving Into Deep Learning . . . . . . . . . . . . . . . . . . . . . 5 How to Work Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Types of Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.1 Introduction First of all, welcome to world of Deep Learning Interviews. 1.1.1 What makes this book so valuable TARGETED advertising. Deciphering dead languages. Detecting malignanttumours. Predicting natural disasters. Every year we see dozens of new uses for deep learning emerge from corporate R&R, academia, and plucky entrepreneurs. Increasingly, deep learning and artificial intelligence are in- grained in our cultural consciousness. Leading universities are dedicating programs to teaching them, and they make the headlines every few days. That means jobs. It means intense demand and intense competition. It means a generation of data scientists and machine learning engineers making their way into
Hiển thị 20 trang đầu. Tải tài liệu để xem đầy đủ.
Bình luận (0)
Chưa có bình luận nào. Hãy là người đầu tiên!