https://stanford.edu/~shervine CS 229 – Machine Learning VIP Refresher: Probabilities and Statistics Remark: for any event B in the sample space, we have P (B) = n X P (B|Ai )P (Ai ). i=1 r Extended form of Bayes’ rule – Let {Ai , i ∈ [[1,n]]} be a partition of the sample space. We have: Afshine Amidi and Shervine Amidi August 6, 2018 P (Ak |B) = P (B|Ak )P (Ak ) n X P (B|Ai )P (Ai ) i=1 Introduction to Probability and Combinatorics r Independence – Two events A and B are independent if and only if we have: r Sample space – The set of all possible outcomes of an experiment is known as the sample space of the experiment and is denoted by S. P (A ∩ B) = P (A)P (B) r Event – Any subset E of the sample space is known as an event. That is, an event is a set consisting of possible outcomes of the experiment. If the outcome of the experiment is contained in E, then we say that E has occurred. Random Variables r Axioms of probability – For each event E, we denote P (E) as the probability of event E occuring. By noting E1 ,...,En mutually exclusive events, we have the 3 following axioms: r Random variable – A random variable, often noted X, is a function that maps every element in a sample space to a real line. (1) 0 ⩽ P (E) ⩽ 1 (2) P (S) = 1 (3) n [ P ! Ei i=1 = n X r Cumulative distribution function (CDF) – The cumulative distribution function F , which is monotonically non-decreasing and is such that lim F (x) = 0 and lim F (x) = 1, is P (Ei ) x→+∞ F (x) = P (X ⩽ x) r Permutation – A permutation is an arrangement of r objects from a pool of n objects, in a given order. The number of such arrangements is given by P (n, r), defined as: Remark: we have P (a < X ⩽ B) = F (b) − F (a). n! P (n, r) = (n − r)! r Probability density function (PDF) – The probability density function f is the probability that X takes on values between two adjacent realizations of the random variable. r Relationships involving the PDF and CDF – Here are the impor