Excerpt from Mathematical Statistics with Applications
Limiting approaches, or asymptotic distribution theory is the study of the properties of random variable sequences. While this aspect of statistical theory is a rigorous, and intellectually stimulating body of knowledge in and of itself, its utility in the realm of statistical applications is a question worthy of a detailed answer. There is no doubt that statistical distribution theory with its concepts of limits and weak vs. strong convergence is challenging. The question is whether its study is useful.
Concepts of probability and statistics surround us. One need only casually peruse a local newspaper on any given day to appreciate the role of statistics in conveying information about our environment. Projections commonly appear concerning the impact of global warming on our ecosystem. Pollsters attempt to learn the moods of a populace by taking a survey. Estimates of the financial and human costs of a future war are attempted and reported. Clinical investigators work to learn the effect of a new therapy developed to prevent strokes in patients with diabetes mellitus. For each of these circumstances, the forecasters, the pollsters, the predictors, or the experimenters attempt to discern facts based upon a relatively small database whose primary utility is that it is available and can be currently measured. The environmental scientist works to learn about future temperatures on earth by estimating past temperatures and measuring future ones. The pollster cannot obtain the responses of the entire national population, but he can gauge the mood of a few hundred respondents. The forecaster cannot know the impact of a future conflict with all of its uncertainties. Instead, she attempts to project findings about current economic systems and the destructive power of modern weapons. Finally the clinical experimenter is interested in generalizing a research finding that is based on 600 patients to a population of twenty million patients with diabetes mellitus. In each of these circumstances, the worker is attempting to generalize results from samples they can measure to populations that they cannot measure.
The obvious question one must ask is how reliable are these results that are based on relatively small samples? The answers to these question calibrate us, and tells us how to judge the utility of the ongoing work. If the estimates are not accurate, then the consumer of the statistical information must be appropriately wary of the conclusions, and, perhaps more importantly any actions taken based on these estimates must be disciplined and tightly restrained.
While there are several reasons why an estimate obtained from a sample can be inaccurate, an ultimate explanation lies in the fact that the estimate is based not on the population, but instead on only a sample of the population. The pollster who attempts to register the opinions of the populace on a question of national interest cannot study the entire population. They can only obtain a sample (often times, an infinitesimally small sample) of the entire population. Our intuition tells us that, if the estimate is a good one, then the larger the sample becomes, the closer the estimate based on the sample will be to the true population value. Even the gambler, who argues that his string of bad luck must turn around, is (perhaps unwittingly), using a large sample property to help guide his actions. Thus, statisticians, when confronted with several different and competing estimators of a population measure, e.g. the proportion of eligible citizens who choose to vote, commonly focus on the large sample, or asymptotic properties of the estimators.
For example, they ask whether the estimator under consideration overestimates or underestimates the population measure when the sample size approaches the population size. If there are several estimators, which of them has the smallest variance as the sample size increases? These are each important questions that asymptotic theory addresses. In order to examine these issues, we must understand the use of limits in probability and statistics, and learn some of the classic results of the utilization of this theory. These will be the topic of this chapter.