Introduction to Probability Theory and Statistics

Introduction to Probability Theory and Statistics

Copyright @ Javier R. Movellan, 2004-2008 August 21, 2008

2

Contents

1 Probability

7

1.1 Intuitive Set Theory . . . . . . . . . . . . . . . . . . . . . . . . . 10

1.2 Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

1.3 Probability measures . . . . . . . . . . . . . . . . . . . . . . . . 14

1.4 Joint Probabilities . . . . . . . . . . . . . . . . . . . . . . . . . 16

1.5 Conditional Probabilities . . . . . . . . . . . . . . . . . . . . . 16

1.6 Independence of 2 Events . . . . . . . . . . . . . . . . . . . . . 17

1.7 Independence of n Events . . . . . . . . . . . . . . . . . . . . . 17

1.8 The Chain Rule of Probability . . . . . . . . . . . . . . . . . . 19

1.9 The Law of Total Probability . . . . . . . . . . . . . . . . . . . 20

1.10 Bayes' Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . 21

1.11 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

2 Random variables

25

2.1 Probability mass functions. . . . . . . . . . . . . . . . . . . . . . 28

2.2 Probability density functions. . . . . . . . . . . . . . . . . . . . . 30

2.3 Cumulative Distribution Function. . . . . . . . . . . . . . . . . . 32

2.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

3 Random Vectors

37

3.1 Joint probability mass functions . . . . . . . . . . . . . . . . . . 37

3.2 Joint probability density functions . . . . . . . . . . . . . . . . . 37

3.3 Joint Cumulative Distribution Functions . . . . . . . . . . . . . . 38

3.4 Marginalizing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

3.5 Independence . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

3.6 Bayes' Rule for continuous data and discrete hypotheses . . . . . 40

3.6.1 A useful version of the LTP . . . . . . . . . . . . . . . . 41

3.7 Random Vectors and Stochastic Processes . . . . . . . . . . . . . 42

3

4

CONTENTS

4 Expected Values

43

4.1 Fundamental Theorem of Expected Values . . . . . . . . . . . . . 44

4.2 Properties of Expected Values . . . . . . . . . . . . . . . . . . . 46

4.3 Variance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

4.3.1 Properties of the Variance . . . . . . . . . . . . . . . . . 48

4.4 Appendix: Using Standard Gaussian Tables . . . . . . . . . . . . 51

4.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

5 The precision of the arithmetic mean

53

5.1 The sampling distribution of the mean . . . . . . . . . . . . . . . 54

5.1.1 Central Limit Theorem . . . . . . . . . . . . . . . . . . . 55

5.2 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

6 Introduction to Statistical Hypothesis Testing

59

6.1 The Classic Approach . . . . . . . . . . . . . . . . . . . . . . . . 59

6.2 Type I and Type II errors . . . . . . . . . . . . . . . . . . . . . . 61

6.2.1 Specifications of a decision system . . . . . . . . . . . . . 63

6.3 The Bayesian approach . . . . . . . . . . . . . . . . . . . . . . . 63

6.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

7 Introduction to Classic Statistical Tests

67

7.1 The Z test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

7.1.1 Two tailed Z test . . . . . . . . . . . . . . . . . . . . . . 67

7.1.2 One tailed Z test . . . . . . . . . . . . . . . . . . . . . . 69

7.2 Reporting the results of a classical statistical test . . . . . . . . . . 71

7.2.1 Interpreting the results of a classical statistical test . . . . 71

7.3 The T-test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

7.3.1 The distribution of T . . . . . . . . . . . . . . . . . . . . 73

7.3.2 Two-tailed T-test . . . . . . . . . . . . . . . . . . . . . . 74

7.3.3 A note about LinuStats . . . . . . . . . . . . . . . . . . . 76

7.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

7.5 Appendix: The sample variance is an umbiased estimate of the

population variance . . . . . . . . . . . . . . . . . . . . . . . . . 78

8 Intro to Experimental Design

81

8.1 An example experiment . . . . . . . . . . . . . . . . . . . . . . . 81

8.2 Independent, Dependent and Intervening Variables . . . . . . . . 83

8.3 Control Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 84

CONTENTS

5

8.4 Useful Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 8.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

9 Experiments with 2 groups

93

9.1 Between Subjects Experiments . . . . . . . . . . . . . . . . . . . 93

9.1.1 Within Subjects Experiments . . . . . . . . . . . . . . . . 96

9.2 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

10 Factorial Experiments

99

10.1 Experiments with more than 2 groups . . . . . . . . . . . . . . . 99

10.2 Interaction Effects . . . . . . . . . . . . . . . . . . . . . . . . . . 101

11 Confidence Intervals

103

A Useful Mathematical Facts

107

B Set Theory

115

B.1 Proofs and Logical Truth . . . . . . . . . . . . . . . . . . . . . . 118

B.2 The Axioms of Set Theory . . . . . . . . . . . . . . . . . . . . . 119

B.2.1 Axiom of Existence: . . . . . . . . . . . . . . . . . . . . 119

B.2.2 Axiom of Equality: . . . . . . . . . . . . . . . . . . . . . 120

B.2.3 Axiom of Pair: . . . . . . . . . . . . . . . . . . . . . . . 120

B.2.4 Axiom of Separation: . . . . . . . . . . . . . . . . . . . . 121

B.2.5 Axiom of Union: . . . . . . . . . . . . . . . . . . . . . . 122

B.2.6 Axiom of Power: . . . . . . . . . . . . . . . . . . . . . . 123

B.2.7 Axiom of Infinity: . . . . . . . . . . . . . . . . . . . . . 124

B.2.8 Axiom of Image: . . . . . . . . . . . . . . . . . . . . . . 124

B.2.9 Axiom of Foundation: . . . . . . . . . . . . . . . . . . . 125

B.2.10 Axiom of Choice: . . . . . . . . . . . . . . . . . . . . . . 125

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download