## An incomplete guide to probability theory

This is a brief introduction/cheatsheet to probability theory. Most of the content is from the book “Statistical Inference” ^{1}

## 1.Basics

I assume you have some basic ideas of set theory such as what is intersection and what is union.

### 1.1 \(\sigma\)-algebra and measure theory

You probably don’t need to know this. But in case you need it, here is the definition of \(\sigma\)-algebra.

A collection of subsets \(\mathcal{S}\) is called **sigma algebra**, denoted by \(\mathcal{B}\), if it satisfies the follwoing three conditions:

- \(\emptyset \in \mathcal{B}\) (the empty set is an element of \(\mathcal{B}\))
- If \(\mathcal{A} \in \mathcal{B}\), then \(\mathcal{A}^{c} \in \mathcal{B}\)
- If \(\mathcal{A}_{1}, \mathcal{A}_{2},... \in \mathcal{B}\), then \(\cup_{i=1}^{\infty} \mathcal{A}_{i} \in \mathcal{B}\)

### 1.2 Rules of probability

The definition of probability function:

Given a sample space \(\mathcal{S}\) and an associated sigma algebra \(\mathcal{B}\), a probability function satisfies:

- \(\mathcal{P}(\mathcal{A}) \geq 0\) for all \(\mathcal{A} \in \mathcal{B}\).
- \(\mathcal{P}(\mathcal{S}) = 1\).
- If \(\mathcal{A}_{1}, \mathcal{A}_{2}, ... \in \mathcal{B}\) are pairwise disjoint, then \(\mathcal{P}(\cup_{i=1}^{\infty} \mathcal{A}_{i}) = \sum_{i=1}^{\infty} \mathcal{P}(A_{i})\).

Some calculus of probability:

If \(\mathcal{P}\) is a probabilty function and \(\mathcal{A}\) is any set in \(\mathcal{B}\), then

- \(\mathcal{P}(\emptyset) = 0\).
- \(\mathcal{P}(A) \leq 1\).
- \(\mathcal{P}(\mathcal{A^{c}}) = 1 - \mathcal{P}(\mathcal{A})\).
- \(\mathcal{P} (\mathcal{A} \cup \mathcal{B}) = \mathcal{P}(\mathcal{A}) + \mathcal{P}(\mathcal{B}) - \mathcal{P} (\mathcal{A} \cap \mathcal{B})\).
- \(\mathcal{P} (\mathcal{B} \cap \mathcal{A}^{c}) = \mathcal{P}(\mathcal{B}) - \mathcal{P} (\mathcal{A} \cap \mathcal{B})\).
- If \(\mathcal{A} \subseteq \mathcal{B}\), then \(\mathcal{P}(\mathcal{A}) \leq \mathcal{P}(\mathcal{B})\).

### 1.3 Independence

### 1.4 Random variables

### 1.5 Distribution functions

### 1.6 Densities

### 1.7 Some inequalities

- Bonferroni inequality
- Jensen’s inequality

## 2. Transformation and Expectation

## 3. Common family of distribution

## 4. Multivariate distributions

## 5. Convergence in probability

You probably don’t need to know too much details about this.

Casella, G., & Berger, R. L. (2021). Statistical inference. Cengage Learning. ↩