Probability basics

Begin with a set Ω—the sample space.

This is all the possible things that could happen.

‚ 6possiblerollsofadie.

‚ HowmanyifIhavetwodice?

ω P Ω is a sample point, atomic event.

(pngimg.com)

⃝c -Trenn, King’s College London 2

Probability basics

A probability space or probability model is a sample space with an assignment Ppωq for every ω P Ω such that:

0 ď Ppωq ď 1 ÿ

Ppωq “ 1 ω

For a typical die: Pp1q“Pp2q“Pp3q“Pp4q“Pp5q“Pp6q“1{6.

(pngimg.com) (hartsport.com.au)

⃝c -Trenn, King’s College London 3

Probability basics

An event A is any subset of Ω

PpAq “

ÿ

tωPAu

Ppωq

Again, for a regular die:

P pdie roll ă 4q “ P p1q ` P p2q ` P p3q “ 1{6 ` 1{6 ` 1{6 “ 1{2

(pngimg.com) (hartsport.com.au)

⃝c -Trenn, King’s College London 4

Random variables

A random variable is a function from sample points to some range. ‚ rainingpLondonqPttrue,falseu.

‚ temperatureplectureroomqPt0,1,…,30u.

P induces a probability distribution for any r.v. X: ÿ

PpX “xiq “

In our dice example, we could set ω “ die shows an odd number:

PpOdd“trueq “ Pp1q`Pp3q`Pp5q “ 1{6`1{6`1{6

“ 1{2

⃝c -Trenn, King’s College London

5

tω:Xpωq“xiu

Ppωq

Propositions

We describe the world in terms of propositions, which are mathematical statements such as “the die shows an odd number’ or “it is raining”.

Think of a proposition as the event (set of sample points) where the proposition is true

Given Boolean random variables A and B:

event a = set of sample ω points where Apωq “ true

event ␣a = set of sample points ω where Apωq “ f alse event a ^ b = points ω where Apωq“true and Bpωq“true

Example:

The set of sample points is Ω “ t1,2,3,4,5,6u.

Apωq “ true or simply a is the event that the number is odd, i.e., t1, 3, 5u. Bpωq “ true or simply b is the event that the number is ă 4, i.e., t1, 2, 3u. a ^ b is given by the sample points in t1, 3u.

⃝c -Trenn, King’s College London 6

Propositions

A state can be defined by a set of Boolean variables.

a ^ b _ ␣c A “ true, B “ true, C “ f alse This is then just a sample point.

⃝c -Trenn, King’s College London

7

Union and Intersection

The definitions imply that certain logically related events must have related probabilities

P pa _ bq “ P paq ` P pbq ́ P pa ^ bq

(lucidchart.com)

Example: Let a denote the event that the die shows an odd number. Let b denote the event that the die shows a number ă 4. Hence, the probability that we get an odd number or a number ă 4 is

Ppa_bq“Ppaq`Ppbq ́Ppa^bq“21 `12 ́Pproll1or3q“2{3.

⃝c -Trenn, King’s College London 8

Prior and posterior probability

Prior or unconditional probabilities of propositions

P pCavity “ trueq “ 0.1 and P pW eather “ sunnyq “ 0.72 correspond to belief before (prior) to arrival of any (new) evidence.

In contrast, P pCavity “ true | toothacheq “ 0.2 is the posterior or conditional probability. Here, we have additional evidence.

⃝c -Trenn, King’s College London 9

Conditional Probabilities

Conditional Probabilities

assuming P pbq ą 0.

Ppa|bq “ Ppa ^ bq P pbq

(lucidchart.com)

Example: Let a denote the event that the die shows an odd number. Let b denote

the event that the die shows a number ă 4. Hence, the probability that we get an

odd number given that the is number ă 4 is P pa|bq “ P pa^bq “ 1{3 “ 2{3. P pbq 1{2

⃝c -Trenn, King’s College London

10

Syntax for propositions

Propositional or Boolean random variables ‚ Cavity (do I have a cavity?)

‚ Cavity“trueisaproposition,alsowrittencavity

‚ Cavity“falseisaproposition,alsowritten␣cavityorcavity

Discrete random variables (finite or infinite)

‚ Weatherisoneoftsunny,rain,cloudy,snowu ‚ Weather“rainisaproposition

Continuous random variables (bounded or unbounded) ‚ Temp“21.6;alsoallow,e.g.,Tempă22.0.

⃝c -Trenn, King’s College London 11

Syntax for propositions

We allow arbitrary combination of logical operators (AND, OR, NOT) and comparison operators (ă, ď, “, ‰, . . . q.

E.g., T emp ă 22.0 AND Cavity “ true

⃝c -Trenn, King’s College London 12

Notation

Probability distribution gives values for all possible assignments (assumes a fixed ordering):

̈ ̨

0.72 ̊‹

̊ 0.1 ‹

PpW eatherq “ ̊‹ (1)

means

‚ PpWeather“sunnyq“0.72,

‚ PpWeather“rainq“0.1,

‚ PpWeather“cloudyq“0.08and

‚ PpWeather“snowq“0.1

Values must be exhaustive (everything covered) and mutually exclusive (no overlap)

Values have to sums to 1

Note that the book uses the notation x0.72, 0.1, 0.08, 0.1y

⃝c -Trenn, King’s College London 13

̊0.08‹ ̋‚

0.1

Quiz

There is a KEATS quiz to see if you understood. Have a look!

⃝c -Trenn, King’s College London 14