Review of Discrete Probability

The sample space Ω is the set of possible outcomes of an experiment. Points ω in Ω are called sample outcomes, realizations, or elements. Subsets of Ω are called events.

An event is denoted by a capital letter near the beginning of the alphabet (A, B, . . .). The probability that A occurs is denoted by P(A).


If we toss a coin twice then Ω = {HH, HT, TH, TT}. The event that the first toss is heads is A = {HH, HT}.

Probability satisfies the following elementary properties, called axioms; all other properties can be derived from these.

  1. 0 ≤ P(A) ≤ 1 for any event A;
  2. P(not A) = 1 − P(A);
  3. P(A or B) = P(A) + P(B) if A and B are mutually exclusive events (i.e. A and B cannot both happen simultaneously).

More generally, if A and B are any events then

P(A or B) = P(A) + P(B) − P(A and B). (1)

If A and B are mutually exclusive, then P(A and B) = 0 and (1) reduces to axiom 3.

Conditional Probability

If B is known to have occurred, then this knowledge may affect the probability of another event A. The probability of A once B is known to have occurred is written P(A|B) and called “the conditional probability of A given B,” or, more simply, “the probability of A given B.” It is defined as

P(A|B) = P(A and B)/P(B) (2)

provided that P(B) ≠ 0.


The events A and B are said to be independent if

P(A and B) = P(A) P(B). (3)

By (2), this implies P(A|B) = P(A) and P(B|A) = P(B). Intuitively, independence means that knowing A has occurred provides no information about whether or not B has occurred and vice-versa.