Trang 2
Accessing the E-book edition of INTRODUCTION TO PROBABILITY Using the VitalSource® ebook DOWNLOAD AND READ OFFLINE Access to the VitalBookTM ebook accompanying this book is To use your ebook ofline, download BookShelf to your PC, via VitalSource® Bookshelf – an ebook reader which allows Mac, iOS device, Android device or Kindle Fire, and log in to you to make and share notes and highlights on your ebooks your Bookshelf account to access your ebook: and search across all of the ebooks that you hold on your On your PC/Mac VitalSource Bookshelf. You can access the ebook online or Go to http://bookshelf.vitalsource.com/ and follow the ofline on your smartphone, tablet or PC/Mac and your notes instructions to download the free VitalSource Bookshelf and highlights will automatically stay in sync no matter where app to your PC or Mac and log into your Bookshelf account. you make them. On your iPhone/iPod Touch/iPad 1. Create a VitalSource Bookshelf account at Download the free VitalSource Bookshelf App available https://online.vitalsource.com/user/new or log into via the iTunes App Store and log into your Bookshelf your existing account if you already have one. account. You can ind more information at https://support. vitalsource.com/hc/en-us/categories/200134217- 2. Redeem the code provided in the panel below Bookshelf-for-iOS to get online access to the ebook. Log in to Bookshelf and click the Account menu at the top right On your Android™ smartphone or tablet of the screen. Select Redeem and enter the redemption Download the free VitalSource Bookshelf App available code shown on the scratch-off panel below in the Code via Google Play and log into your Bookshelf account. You can To Redeem box. Press Redeem. Once the code has ind more information at https://support.vitalsource.com/ been redeemed your ebook will download and appear in hc/en-us/categories/200139976-Bookshelf-for-Android- your library. and-Kindle-Fire On your Kindle Fire Download the free VitalSource Bookshelf App available from Amazon and log into your Bookshelf account. You can ind more information at https://support.vitalsource.com/ hc/en-us/categories/200139976-Bookshelf-for-Android- and-Kindle-Fire N.B. The code in the scratch-off panel can only be used once. When you have created a Bookshelf account and redeemed the code you will be able to access the ebook online or ofline on your smartphone, tablet or PC/Mac. SUPPORT If you have any questions about downloading Bookshelf, creating your account, or accessing and using your ebook edition, please visit http://support.vitalsource.com/
Trang 4
CHAPMAN & HALL/CRC Texts in Statistical Science Series Series Editors Francesca Dominici, Harvard School of Public Health, USA Julian J. Faraway, University of Bath, UK Martin Tanner, Northwestern University, USA Jim Zidek, University of British Columbia, Canada Statistical heory: A Concise Introduction Modelling Survival Data in Medical Research, F. Abramovich and Y. Ritov Second Edition Practical Multivariate Analysis, Fifth Edition D. Collett A. Aii, S. May, and V.A. Clark Introduction to Statistical Methods for Practical Statistics for Medical Research Clinical Trials D.G. Altman T.D. Cook and D.L. DeMets Interpreting Data: A First Course Applied Statistics: Principles and Examples in Statistics D.R. Cox and E.J. Snell A.J.B. Anderson Multivariate Survival Analysis and Competing Introduction to Probability with R Risks K. Baclawski M. Crowder Linear Algebra and Matrix Analysis for Statistical Analysis of Reliability Data Statistics M.J. Crowder, A.C. Kimber, S. Banerjee and A. Roy T.J. Sweeting, and R.L. Smith Statistical Methods for SPC and TQM An Introduction to Generalized D. Bissell Linear Models, hird Edition A.J. Dobson and A.G. Barnett Introduction to Probability J. K. Blitzstein and J. Hwang Nonlinear Time Series: heory, Methods, and Applications with R Examples Bayesian Methods for Data Analysis, R. Douc, E. Moulines, and D.S. Stofer hird Edition B.P. Carlin and T.A. Louis Introduction to Optimization Methods and heir Applications in Statistics Second Edition B.S. Everitt R. Caulcutt Extending the Linear Model with R: he Analysis of Time Series: An Introduction, Generalized Linear, Mixed Efects and Sixth Edition Nonparametric Regression Models C. Chatield J.J. Faraway Introduction to Multivariate Analysis Linear Models with R, Second Edition C. Chatield and A.J. Collins J.J. Faraway Problem Solving: A Statistician’s Guide, A Course in Large Sample heory Second Edition T.S. Ferguson C. Chatield Multivariate Statistics: A Practical Statistics for Technology: A Course in Applied Approach Statistics, hird Edition B. Flury and H. Riedwyl C. Chatield Readings in Decision Analysis Bayesian Ideas and Data Analysis: An S. French Introduction for Scientists and Statisticians R. Christensen, W. Johnson, A. Branscum, Markov Chain Monte Carlo: and T.E. Hanson Stochastic Simulation for Bayesian Inference, Second Edition Modelling Binary Data, Second Edition D. Gamerman and H.F. Lopes D. Collett
Trang 5
Bayesian Data Analysis, hird Edition Introduction to the heory of Statistical A. Gelman, J.B. Carlin, H.S. Stern, D.B. Dunson, Inference A. Vehtari, and D.B. Rubin H. Liero and S. Zwanzig Multivariate Analysis of Variance and Statistical heory, Fourth Edition Repeated Measures: A Practical Approach for B.W. Lindgren Behavioural Scientists Stationary Stochastic Processes: heory and D.J. Hand and C.C. Taylor Applications Practical Data Analysis for Designed Practical G. Lindgren Longitudinal Data Analysis he BUGS Book: A Practical Introduction to D.J. Hand and M. Crowder Bayesian Analysis Logistic Regression Models D. Lunn, C. Jackson, N. Best, A. homas, and J.M. Hilbe D. Spiegelhalter Richly Parameterized Linear Models: Introduction to General and Generalized Additive, Time Series, and Spatial Models Linear Models Using Random Efects H. Madsen and P. hyregod J.S. Hodges Time Series Analysis Statistics for Epidemiology H. Madsen N.P. Jewell Pólya Urn Models Stochastic Processes: An Introduction, H. Mahmoud Second Edition Randomization, Bootstrap and Monte Carlo P.W. Jones and P. Smith Methods in Biology, hird Edition he heory of Linear Models B.F.J. Manly B. Jørgensen Introduction to Randomized Controlled Principles of Uncertainty Clinical Trials, Second Edition J.B. Kadane J.N.S. Matthews Graphics for Statistics and Data Analysis with R Statistical Methods in Agriculture and K.J. Keen Experimental Biology, Second Edition Mathematical Statistics R. Mead, R.N. Curnow, and A.M. Hasted K. Knight Statistics in Engineering: A Practical Approach Introduction to Multivariate Analysis: A.V. Metcalfe Linear and Nonlinear Modeling Beyond ANOVA: Basics of Applied Statistics S. Konishi R.G. Miller, Jr. Nonparametric Methods in Statistics with SAS A Primer on Linear Models Applications J.F. Monahan O. Korosteleva Applied Stochastic Modelling, Second Edition Modeling and Analysis of Stochastic Systems, B.J.T. Morgan Second Edition Elements of Simulation V.G. Kulkarni B.J.T. Morgan Exercises and Solutions in Biostatistical heory Probability: Methods and Measurement L.L. Kupper, B.H. Neelon, and S.M. O’Brien A. O’Hagan Exercises and Solutions in Statistical heory Introduction to Statistical Limit heory L.L. Kupper, B.H. Neelon, and S.M. O’Brien A.M. Polansky Design and Analysis of Experiments with SAS Applied Bayesian Forecasting and Time Series J. Lawson Analysis A Course in Categorical Data Analysis A. Pole, M. West, and J. Harrison T. Leonard Statistics in Research and Development, Statistics for Accountants Time Series: Modeling, Computation, and S. Letchford Inference R. Prado and M. West
Trang 6
Introduction to Statistical Process Control Data Driven Statistical Methods P. Qiu P. Sprent Sampling Methodologies with Applications Generalized Linear Mixed Models: P.S.R.S. Rao Modern Concepts, Methods and Applications A First Course in Linear Model heory W. W. Stroup N. Ravishanker and D.K. Dey Survival Analysis Using S: Analysis of Essential Statistics, Fourth Edition Time-to-Event Data D.A.G. Rees M. Tableman and J.S. Kim Stochastic Modeling and Mathematical Applied Categorical and Count Data Analysis Statistics: A Text for Statisticians and W. Tang, H. He, and X.M. Tu Quantitative Elementary Applications of Probability heory, F.J. Samaniego Second Edition Statistical Methods for Spatial Data Analysis H.C. Tuckwell O. Schabenberger and C.A. Gotway Introduction to Statistical Inference and Its Bayesian Networks: With Examples in R Applications with R M. Scutari and J.-B. Denis M.W. Trosset Large Sample Methods in Statistics Understanding Advanced Statistical Methods P.K. Sen and J. da Motta Singer P.H. Westfall and K.S.S. Henning Decision Analysis: A Bayesian Approach Statistical Process Control: heory and J.Q. Smith Practice, hird Edition G.B. Wetherill and D.W. Brown Analysis of Failure and Survival Data P. J. Smith Generalized Additive Models: An Introduction with R Applied Statistics: Handbook of GENSTAT S. Wood Analyses E.J. Snell and H. Simpson Epidemiology: Study Design and Data Analysis, hird Edition Applied Nonparametric Statistical Methods, M. Woodward Fourth Edition P. Sprent and N.C. Smeeton Experiments B.S. Yandell
Trang 7
Texts in Statistical Science Introduction to Probability Joseph K. Blitzstein Harvard University Cambridge, Massachusetts, USA Jessica Hwang Stanford University Stanford, California, USA
Trang 8
CRC Press Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487-2742 © 2015 by Taylor & Francis Group, LLC CRC Press is an imprint of Taylor & Francis Group, an Informa business No claim to original U.S. Government works Version Date: 20140609 International Standard Book Number-13: 978-1-4665-7559-2 (eBook - PDF) This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot assume responsibility for the valid- ity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may rectify in any future reprint. Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or uti- lized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopy- ing, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers. For permission to photocopy or use material electronically from this work, please access www.copyright.com (http:// www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com and the CRC Press Web site at http://www.crcpress.com
Trang 11
Contents 1 Probability and counting 1 1.1 Why study probability? . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Sample spaces and Pebble World . . . . . . . . . . . . . . . . . . . . 3 1.3 Naive definition of probability . . . . . . . . . . . . . . . . . . . . . 6 1.4 How to count . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1.5 Story proofs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 1.6 Non-naive definition of probability . . . . . . . . . . . . . . . . . . . 20 1.7 Recap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 1.8 R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 1.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 2 Conditional probability 41 2.1 The importance of thinking conditionally . . . . . . . . . . . . . . . 41 2.2 Definition and intuition . . . . . . . . . . . . . . . . . . . . . . . . . 42 2.3 Bayes’ rule and the law of total probability . . . . . . . . . . . . . . 47 2.4 Conditional probabilities are probabilities . . . . . . . . . . . . . . . 53 2.5 Independence of events . . . . . . . . . . . . . . . . . . . . . . . . . 56 2.6 Coherency of Bayes’ rule . . . . . . . . . . . . . . . . . . . . . . . . 59 2.7 Conditioning as a problem-solving tool . . . . . . . . . . . . . . . . 60 2.8 Pitfalls and paradoxes . . . . . . . . . . . . . . . . . . . . . . . . . . 66 2.9 Recap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 2.10 R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 2.11 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 3 Random variables and their distributions 91 3.1 Random variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 3.2 Distributions and probability mass functions . . . . . . . . . . . . . 94 3.3 Bernoulli and Binomial . . . . . . . . . . . . . . . . . . . . . . . . . 100 3.4 Hypergeometric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 3.5 Discrete Uniform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 3.6 Cumulative distribution functions . . . . . . . . . . . . . . . . . . . 108 3.7 Functions of random variables . . . . . . . . . . . . . . . . . . . . . 110 3.8 Independence of r.v.s . . . . . . . . . . . . . . . . . . . . . . . . . . 117 3.9 Connections between Binomial and Hypergeometric . . . . . . . . . 121 3.10 Recap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 3.11 R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 3.12 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 ix
Trang 12
x Contents 4 Expectation 137 4.1 Definition of expectation . . . . . . . . . . . . . . . . . . . . . . . . 137 4.2 Linearity of expectation . . . . . . . . . . . . . . . . . . . . . . . . . 140 4.3 Geometric and Negative Binomial . . . . . . . . . . . . . . . . . . . 144 4.4 Indicator r.v.s and the fundamental bridge . . . . . . . . . . . . . . 151 4.5 Law of the unconscious statistician (LOTUS) . . . . . . . . . . . . . 156 4.6 Variance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 4.7 Poisson . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 4.8 Connections between Poisson and Binomial . . . . . . . . . . . . . . 165 4.9 *Using probability and expectation to prove existence . . . . . . . . 168 4.10 Recap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174 4.11 R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175 4.12 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178 5 Continuous random variables 195 5.1 Probability density functions . . . . . . . . . . . . . . . . . . . . . . 195 5.2 Uniform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201 5.3 Universality of the Uniform . . . . . . . . . . . . . . . . . . . . . . . 205 5.4 Normal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211 5.5 Exponential . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217 5.6 Poisson processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222 5.7 Symmetry of i.i.d. continuous r.v.s . . . . . . . . . . . . . . . . . . . 225 5.8 Recap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226 5.9 R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228 5.10 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231 6 Moments 243 6.1 Summaries of a distribution . . . . . . . . . . . . . . . . . . . . . . 243 6.2 Interpreting moments . . . . . . . . . . . . . . . . . . . . . . . . . . 248 6.3 Sample moments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252 6.4 Moment generating functions . . . . . . . . . . . . . . . . . . . . . . 255 6.5 Generating moments with MGFs . . . . . . . . . . . . . . . . . . . . 259 6.6 Sums of independent r.v.s via MGFs . . . . . . . . . . . . . . . . . . 261 6.7 *Probability generating functions . . . . . . . . . . . . . . . . . . . 262 6.8 Recap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267 6.9 R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267 6.10 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272 7 Joint distributions 277 7.1 Joint, marginal, and conditional . . . . . . . . . . . . . . . . . . . . 278 7.2 2D LOTUS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298 7.3 Covariance and correlation . . . . . . . . . . . . . . . . . . . . . . . 300 7.4 Multinomial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306 7.5 Multivariate Normal . . . . . . . . . . . . . . . . . . . . . . . . . . . 309 7.6 Recap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316
Trang 13
Contents xi 7.7 R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318 7.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320 8 Transformations 339 8.1 Change of variables . . . . . . . . . . . . . . . . . . . . . . . . . . . 341 8.2 Convolutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346 8.3 Beta . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351 8.4 Gamma . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356 8.5 Beta-Gamma connections . . . . . . . . . . . . . . . . . . . . . . . . 365 8.6 Order statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367 8.7 Recap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370 8.8 R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373 8.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375 9 Conditional expectation 383 9.1 Conditional expectation given an event . . . . . . . . . . . . . . . . 383 9.2 Conditional expectation given an r.v. . . . . . . . . . . . . . . . . . 392 9.3 Properties of conditional expectation . . . . . . . . . . . . . . . . . 394 9.4 *Geometric interpretation of conditional expectation . . . . . . . . . 399 9.5 Conditional variance . . . . . . . . . . . . . . . . . . . . . . . . . . . 400 9.6 Adam and Eve examples . . . . . . . . . . . . . . . . . . . . . . . . 402 9.7 Recap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407 9.8 R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408 9.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 410 10 Inequalities and limit theorems 421 10.1 Inequalities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 422 10.2 Law of large numbers . . . . . . . . . . . . . . . . . . . . . . . . . . 431 10.3 Central limit theorem . . . . . . . . . . . . . . . . . . . . . . . . . . 435 10.4 Chi-Square and Student-t . . . . . . . . . . . . . . . . . . . . . . . . 441 10.5 Recap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445 10.6 R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 447 10.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 450 11 Markov chains 459 11.1 Markov property and transition matrix . . . . . . . . . . . . . . . . 459 11.2 Classification of states . . . . . . . . . . . . . . . . . . . . . . . . . . 465 11.3 Stationary distribution . . . . . . . . . . . . . . . . . . . . . . . . . 469 11.4 Reversibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 475 11.5 Recap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 480 11.6 R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 481 11.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 484 12 Markov chain Monte Carlo 495 12.1 Metropolis-Hastings . . . . . . . . . . . . . . . . . . . . . . . . . . . 496 12.2 Gibbs sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 508
Trang 14
xii Contents 12.3 Recap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 515 12.4 R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 515 12.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 517 13 Poisson processes 519 13.1 Poisson processes in one dimension . . . . . . . . . . . . . . . . . . 519 13.2 Conditioning, superposition, thinning . . . . . . . . . . . . . . . . . 521 13.3 Poisson processes in multiple dimensions . . . . . . . . . . . . . . . 532 13.4 Recap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 534 13.5 R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 534 13.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 536 A Math 541 A.1 Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 541 A.2 Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 545 A.3 Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 550 A.4 Difference equations . . . . . . . . . . . . . . . . . . . . . . . . . . . 552 A.5 Differential equations . . . . . . . . . . . . . . . . . . . . . . . . . . 553 A.6 Partial derivatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . 554 A.7 Multiple integrals . . . . . . . . . . . . . . . . . . . . . . . . . . . . 554 A.8 Sums . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 556 A.9 Pattern recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . 558 A.10 Common sense and checking answers . . . . . . . . . . . . . . . . . 558 B R 561 B.1 Vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 561 B.2 Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 562 B.3 Math . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 563 B.4 Sampling and simulation . . . . . . . . . . . . . . . . . . . . . . . . 563 B.5 Plotting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 564 B.6 Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 564 B.7 Summary statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . 564 B.8 Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 565 C Table of distributions 567 Bibliography 569 Index 571
Trang 15
Preface This book provides a modern introduction to probability and develops a foundation for understanding statistics, randomness, and uncertainty. A variety of applications and examples are explored, from basic coin-tossing and the study of coincidences to Google PageRank and Markov chain Monte Carlo. As probability is often considered to be a counterintuitive subject, many intuitive explanations, diagrams, and practice problems are given. Each chapter ends with a section showing how to explore the ideas of that chapter in R, a free software environment for statistical calculations and simulations. Lecture videos from Stat 110 at Harvard, the course which gave rise to this book (and which has been taught by Joe every year since 2006), are freely available at stat110.net. Additional supplementary materials, such as R code and solutions to exercises marked with ©s , are also available at this site. Calculus is a prerequisite for this book; there is no statistics prerequisite. The main mathematical challenge lies not in performing technical calculus derivations, but in translating between abstract concepts and concrete examples. Some major themes and features are listed below. 1. Stories. Throughout this book, definitions, theorems, and proofs are pre- sented through stories: real-world interpretations that preserve mathemat- ical precision and generality. We explore probability distributions using the generative stories that make them widely used in statistical modeling. When possible, we refrain from tedious derivations and instead aim to give interpretations and intuitions for why key results are true. Our expe- rience is that this approach promotes long-term retention of the material by providing insight instead of demanding rote memorization. 2. Pictures. Since pictures are thousand-word stories, we supplement defini- tions with illustrations so that key concepts are associated with memorable diagrams. In many fields, the difference between a novice and an expert has been described as follows: the novice struggles to memorize a large number of seemingly disconnected facts and formulas, whereas the expert sees a unified structure in which a few principles and ideas connect these facts coherently. To help students see the structure of probability, we em- phasize the connections between ideas (both verbally and visually), and at the end of most chapters we present recurring, ever-expanding maps of concepts and distributions. xiii
Trang 16
xiv Preface 3. Dual teaching of concepts and strategies. Our intent is that in reading this book, students will learn not only the concepts of probability, but also a set of problem-solving strategies that are widely applicable outside of probability. In the worked examples, we explain each step of the solution but also comment on how we knew to take the approach we did. Often we present multiple solutions to the same problem. We explicitly identify and name important strategies such as symmetry and pattern recognition, and we proactively dispel common misunder- standings, which are marked with the h (biohazard) symbol. 4. Practice problems. The book contains about 600 exercises of varying dif- ficulty. The exercises are intended to reinforce understanding of the ma- terial and strengthen problem-solving skills instead of requiring repetitive calculations. Some are strategic practice problems, grouped by theme to facilitate practice of a particular topic, while others are mixed practice, in which several earlier topics may need to be synthesized. About 250 exercises have detailed online solutions for practice and self-study. 5. Simulation, Monte Carlo, and R. Many probability problems are too dif- ficult to solve exactly, and in any case it is important to be able to check one’s answer. We introduce techniques for exploring probability via sim- ulation, and show that often a few lines of R code suffice to create a simulation for a seemingly complicated problem. 6. Focus on real-world relevance and statistical thinking. Examples and ex- ercises in this book have a clear real-world motivation, with a particu- lar focus on building a strong foundation for further study of statistical inference and modeling. We preview important statistical ideas such as sampling, simulation, Bayesian inference, and Markov chain Monte Carlo; other application areas include genetics, medicine, computer science, and information theory. Our choice of examples and exercises is intended to highlight the power, applicability, and beauty of probabilistic thinking. Acknowledgments We thank our colleagues, the Stat 110 teaching assistants, and several thousand Stat 110 students for their comments and ideas related to the course and the book. In particular, we thank Alvin Siu, Angela Fan, Anji Tang, Carolyn Stein, David Jones, David Rosengarten, David Watson, Johannes Ruf, Kari Lock, Keli Liu, Kevin Bartz, Lazhi Wang, Martin Lysy, Michele Zemplenyi, Peng Ding, Rob Phillips, Sam Fisher, Sebastian Chiu, Sofia Hou, Theresa Gebert, Valeria Espinosa, Viktoriia Liublinska, Viviana Garcia, William Chen, and Xander Marcus for their feedback. We especially thank Bo Jiang, Raj Bhuptani, Shira Mitchell, and the anonymous reviewers for their detailed comments on draft versions of the book, and Andrew
Trang 17
Preface xv Gelman, Carl Morris, Persi Diaconis, Stephen Blyth, Susan Holmes, and Xiao-Li Meng for countless insightful discussions about probability. John Kimmel at Chapman and Hall/CRC Press provided wonderful editorial exper- tise throughout the writing of this book. We greatly appreciate his support. Finally, we would like to express our deepest gratitude to our families for their love and encouragement. Joe Blitzstein and Jessica Hwang Cambridge, MA and Stanford, CA May 2014
Trang 19
1 Probability and counting Luck. Coincidence. Randomness. Uncertainty. Risk. Doubt. Fortune. Chance. You’ve probably heard these words countless times, but chances are that they were used in a vague, casual way. Unfortunately, despite its ubiquity in science and ev- eryday life, probability can be deeply counterintuitive. If we rely on intuitions of doubtful validity, we run a serious risk of making inaccurate predictions or over- confident decisions. The goal of this book is to introduce probability as a logical framework for quantifying uncertainty and randomness in a principled way. We’ll also aim to strengthen intuition, both when our initial guesses coincide with logical reasoning and when we’re not so lucky. 1.1 Why study probability? Mathematics is the logic of certainty; probability is the logic of uncertainty. Prob- ability is extremely useful in a wide variety of fields, since it provides tools for understanding and explaining variation, separating signal from noise, and modeling complex phenomena. To give just a small sample from a continually growing list of applications: 1. Statistics: Probability is the foundation and language for statistics, en- abling many powerful methods for using data to learn about the world. 2. Physics: Einstein famously said “God does not play dice with the uni- verse”, but current understanding of quantum physics heavily involves probability at the most fundamental level of nature. Statistical mechanics is another major branch of physics that is built on probability. 3. Biology: Genetics is deeply intertwined with probability, both in the in- heritance of genes and in modeling random mutations. 4. Computer science: Randomized algorithms make random choices while they are run, and in many important applications they are simpler and more efficient than any currently known deterministic alternatives. Proba- bility also plays an essential role in studying the performance of algorithms, and in machine learning and artificial intelligence. 1
Trang 20
2 Introduction to Probability 5. Meteorology: Weather forecasts are (or should be) computed and expressed in terms of probability. 6. Gambling: Many of the earliest investigations of probability were aimed at answering questions about gambling and games of chance. 7. Finance: At the risk of redundancy with the previous example, it should be pointed out that probability is central in quantitative finance. Modeling stock prices over time and determining “fair” prices for financial instru- ments are based heavily on probability. 8. Political science: In recent years, political science has become more and more quantitative and statistical. For example, Nate Silver’s successes in predicting election results, such as in the 2008 and 2012 U.S. presidential elections, were achieved using probability models to make sense of polls and to drive simulations (see Silver [25]). 9. Medicine: The development of randomized clinical trials, in which patients are randomly assigned to receive treatment or placebo, has transformed medical research in recent years. As the biostatistician David Harrington remarked, “Some have conjectured that it could be the most significant advance in scientific medicine in the twentieth century. . . . In one of the delightful ironies of modern science, the randomized trial ‘adjusts’ for both observed and unobserved heterogeneity in a controlled experiment by in- troducing chance variation into the study design.” [17] 10. Life: Life is uncertain, and probability is the logic of uncertainty. While it isn’t practical to carry out a formal probability calculation for every deci- sion made in life, thinking hard about probability can help us avert some common fallacies, shed light on coincidences, and make better predictions. Probability provides procedures for principled problem-solving, but it can also pro- duce pitfalls and paradoxes. For example, we’ll see in this chapter that even Got- tfried Wilhelm von Leibniz and Sir Isaac Newton, the two people who independently discovered calculus in the 17th century, were not immune to basic errors in prob- ability. Throughout this book, we will use the following strategies to help avoid potential pitfalls. 1. Simulation: A beautiful aspect of probability is that it is often possible to study problems via simulation. Rather than endlessly debating an answer with someone who disagrees with you, you can run a simulation and see empirically who is right. Each chapter in this book ends with a section that gives examples of how to do calculations and simulations in R, a free statistical computing environment. 2. Biohazards: Studying common mistakes is important for gaining a stronger understanding of what is and is not valid reasoning in probability. In this
Hiển thị 20 trang đầu. Tải tài liệu để xem đầy đủ.
Bình luận (0)
Chưa có bình luận nào. Hãy là người đầu tiên!