Mathematical Statistics Lecture -

The lecture then introduces the concept of a statistical model—a family of probability distributions ( P_\theta : \theta \in \Theta ), where ( \Theta ) is the parameter space. Here, the narrative tension begins. We cannot know ( P_\theta ); we can only hope to learn ( \theta ).

The professor will derive the likelihood function ( L(\theta; x) ), not as a probability, but as a measure of evidence. The famous Likelihood Principle is stated: all evidence from an experiment about ( \theta ) is contained in the likelihood function. This is a philosophical earthquake. It implies that the design of an experiment (stopping rules, optional sampling) is irrelevant after the data are collected.

Then comes the elegant, almost magical concept of sufficiency. A statistic ( T(X) ) is sufficient if the conditional distribution of the sample given ( T(X) ) does not depend on ( \theta ). In plain language: the sufficient statistic captures all information about ( \theta ) contained in the sample. The Neyman-Fisher factorization theorem is derived, and the room feels the power of data reduction without loss of information.

Choose ( \theta ) to maximize the likelihood function: [ L(\theta; x_1,\dots,x_n) = \prod_i=1^n f(x_i; \theta) ] Or equivalently maximize the log-likelihood ( \ell(\theta) = \sum \log f(x_i;\theta) ). mathematical statistics lecture

Why MLE? Under regularity conditions, MLEs are:

Example: For ( X_i \sim \textBernoulli(p) ), the MLE is ( \hatp = \barX ).

Estimation asks "What is $\theta$?" Hypothesis testing asks "Is $\theta$ equal to a specific value?" The lecture then introduces the concept of a

We set up two competing hypotheses:

Idea: Equate the population moments to the sample moments and solve for the parameters.

Procedure: If you have $k$ parameters to estimate, set the first $k$ population moments equal to the first $k$ sample moments and solve the system of equations. Example: For ( X_i \sim \textBernoulli(p) ), the

The distribution of a statistic (over repeated sampling) is its sampling distribution. This is the key to inference.

Example: If ( X_i \stackreli.i.d.\sim N(\mu, \sigma^2) ), then: [ \barX \sim N\left(\mu, \frac\sigma^2n\right) ]

A random variable (RV) is a function that maps outcomes of a random experiment to real numbers.