The sample space Ω is the set of possible outcomes of an experiment. Points ω in Ω are called sample outcomes, realizations, or elements. Subsets of Ω are called events.
An event is denoted by a capital letter near the beginning of the alphabet (A, B, . . .). The probability that A occurs is denoted by P(A).
Example
If we toss a coin twice then Ω = {HH, HT, TH, TT}. The event that the first toss is heads is A = {HH, HT}.
Probability satisfies the following elementary properties, called axioms; all other properties can be derived from these.
- 0 ≤ P(A) ≤ 1 for any event A;
- P(not A) = 1 − P(A);
- P(A or B) = P(A) + P(B) if A and B are mutually exclusive events (i.e. A and B cannot both happen simultaneously).
More generally, if A and B are any events then
P(A or B) = P(A) + P(B) − P(A and B). (1)
If A and B are mutually exclusive, then P(A and B) = 0 and (1) reduces to axiom 3.
Conditional Probability
If B is known to have occurred, then this knowledge may affect the probability of another event A. The probability of A once B is known to have occurred is written P(A|B) and called “the conditional probability of A given B,” or, more simply, “the probability of A given B.” It is defined as
P(A|B) = P(A and B)/P(B) (2)
provided that P(B) ≠ 0.
Independence
The events A and B are said to be independent if
P(A and B) = P(A) P(B). (3)
By (2), this implies P(A|B) = P(A) and P(B|A) = P(B). Intuitively, independence means that knowing A has occurred provides no information about whether or not B has occurred and vice-versa.