MLE Consistency and Asymptotic Properties

An estimator is considered consistent if, as the sample size grows, it converges in probability to the true parameter value. The notation θ̂ₙ →ᵖ θ₀ specifically indicates that the estimator θ̂ₙ approaches the true parameter θ₀ in probability, which is the definition of consistency.

Maximum Likelihood Estimates (MLEs) achieve consistency primarily because, as sample size increases, the log-likelihood function becomes increasingly concentrated around the true parameter. This phenomenon occurs due to the law of large numbers, which ensures that sample averages converge to their expected values, thereby leading the MLE to converge to the true parameter value.

I(θ₀)⁻¹ represents the asymptotic variance of the maximum likelihood estimator (MLE). As the sample size increases, the distribution of the MLE approaches a normal distribution centered at the true parameter value, with variance given by the inverse of the Fisher information matrix, indicating the precision of the estimator.

Higher Fisher information indicates that the log-likelihood function is steep and well-defined around the true parameter value, leading to more precise estimates. Consequently, this results in a lower asymptotic variance, meaning that as the sample size increases, the estimates become more concentrated around the true parameter value.

The likelihood being symmetric around θ₀ is not a standard requirement for the consistency of Maximum Likelihood Estimators (MLE). MLE consistency primarily relies on conditions such as compact parameter space, continuity of the log-likelihood, and identifiability, which ensure that estimators converge to the true parameter values as sample size increases.

Maximum Likelihood Estimators (MLE) are considered asymptotically efficient because, as the sample size increases, they achieve the lowest possible variance among all unbiased estimators, as defined by the Cramér-Rao lower bound. This property ensures that MLEs provide increasingly accurate estimates for large samples.

If θ̂ₙ consistently estimates θ₀, it means that as the sample size increases, θ̂ₙ converges in probability to θ₀. The continuous mapping theorem states that if a function g is continuous, then applying g to θ̂ₙ will yield a sequence that converges in probability to g(θ₀), ensuring g(θ̂ₙ) is a consistent estimator of g(θ₀).

An asymptotically normal estimator can converge in distribution to a normal distribution as the sample size increases, but it does not guarantee that the estimator is unbiased. An estimator can be biased and still exhibit asymptotic normality, meaning that its bias may diminish as the sample size grows, but it does not become unbiased.

The convergence rate of the Maximum Likelihood Estimator (MLE) indicates how quickly the estimator approaches the true parameter value as the sample size increases. A rate of √n implies that the error decreases proportionally to 1/√n, which is mathematically represented as O(n⁻¹/²), indicating a faster convergence than linear but slower than logarithmic.

When the log-likelihood function is twice continuously differentiable, setting the first derivative to zero indicates that the maximum likelihood estimator (MLE) is at a critical point. This is known as the first-order condition, as it is the initial criterion for finding optimal parameters in statistical estimation.

The score function, which is the gradient of the log-likelihood, is expected to have a mean of zero under the true parameter value. This property ensures that the average information provided by the data about the parameter is unbiased, leading to the relationship between the Hessian matrix and Fisher information.

An estimator can be consistent and asymptotically normal without being the maximum likelihood estimator (MLE). Consistency and asymptotic normality are properties that can apply to various estimators, not just the MLE. Other methods, such as method of moments or Bayesian estimators, can also yield consistent and asymptotically normal estimates.