VARIANCE ESTIMATION IN NONPARAMETRIC MULTIPLE REGRESSION

ABSTRACT

We consider the variance estimation in a general nonparametric regression model with multiple covariates. We extend difference methods to the multivariate setting by introducing an algorithm that orders the design points in higher dimensions. We also consider an adaptive difference estimator which requires much less strict assumptions on the covariate design and can significantly reduce mean squared error for small sample sizes.     

MINIMUM NORM ESTIMATION OF VARIANCE COMPONENTS FOR LIFE INSURANCE DATA

Life insurance companies want to predict the average claimed sums they have to pay in events of death for specific groups of customers in order to derive group specific premiums. This requires estimation of the variability of claims across groups. We derive a corresponding mixed linear model for claim data from many groups of customers that incorporates group-specific age distributions, the Compertz-Makeham mortality function and an unknown group-specific random hazard factor. It takes the form of a generalized replicated model with two variance components where the between blocks variance component depends on the common mean of all observations. Two methods of parameter estimation are derived along the lines of C. R. Rao's MINQUE and generalized least squares estimation. Simulations show both methods to work well for large sets of data.     

NONPARAMETRIC QUANTILE ESTIMATION FROM RECORD-BREAKING DATA

Sometimes, in industrial quality control experiments and destructive stress testing, only values smaller than all previous ones are observed. Here we consider nonparametric quantile estimation, both the ‘sample quantile function’ and kernel-type estimators, from such record-breaking data. For a single record-breaking sample, consistent estimation is not possible except in the extreme tails of the distribution. Hence replication is required, and for m. such independent record-breaking samples the quantile estimators are shown to be strongly consistent and asymptotically normal as m-→∞. Also, for small m, the mean-squared errors, biases and smoothing parameters (for the smoothed estimators) are investigated through computer simulations.     

NONPARAMETRIC ESTIMATION FOR COMPETING RISKS

This paper considers the estimation of the ratio of two scale parameters when the data are censored. It emphasises characteristics of the asymptotic variance under censoring from a practical point of view. The estimator proposed by Padgett & Wei (1982) for the two-sample scale model is extended to the competing risks model. Asymptotic properties of the estimator are studied via its influence function. The use of influence functions permits a unified treatment of both models. Examples show and illustrate that under both models the variance can become infinite under some circumstances.     

NONPARAMETRIC ESTIMATION OF CONDITIONAL CUMULATIVE HAZARDS FOR MISSING POPULATION MARKS

A new function for the competing risks model, the conditional cumulative hazard function, is introduced, from which the conditional distribution of failure times of individuals failing due to cause  j  can be studied. The standard Nelson–Aalen estimator is not appropriate in this setting, as population membership (mark) information may be missing for some individuals owing to random right-censoring. We propose the use of imputed population marks for the censored individuals through fractional risk sets. Some asymptotic properties, including uniform strong consistency, are established. We study the practical performance of this estimator through simulation studies and apply it to a real data set for illustration.     

NONPARAMETRIC ESTIMATORS FOR CENSORED CORRELATED DATA

ABSTRACT

In the context of failure time data, over the long run, dependent observations that might be censored are commonly encountered in practice. The main objective of this paper is to make inference about the common marginal distribution of the failure times. To this end, one nonparametric estimator, namely, the Nelson-Aalen estimator is modified to incorporate the dependence among the observations. The modified estimator is the weighted moving average (WMA) version of the existing estimator used for independent data. It has been shown that the new version is better in the sense of minimizing the one-step ahead forecast errors. Also, the new estimator can be used as a crude measure for checking independence among observations.     

NONPARAMETRIC ESTIMATION OF MULTIVARIATE CONVEX-TRANSFORMED DENSITIES

We study estimation of multivariate densities p of the form p(x) = h(g(x)) for x ∈ ?(d) and for a fixed monotone function h and an unknown convex function g. The canonical example is h(y) = e(-y) for y ∈ ?; in this case, the resulting class of densities [Formula: see text]is well known as the class of log-concave densities. Other functions h allow for classes of densities with heavier tails than the log-concave class.We first investigate when the maximum likelihood estimator p? exists for the class P(h) for various choices of monotone transformations h, including decreasing and increasing functions h. The resulting models for increasing transformations h extend the classes of log-convex densities studied previously in the econometrics literature, corresponding to h(y) = exp(y).We then establish consistency of the maximum likelihood estimator for fairly general functions h, including the log-concave class P(e(-y)) and many others. In a final section, we provide asymptotic minimax lower bounds for the estimation of p and its vector of derivatives at a fixed point x(0) under natural smoothness hypotheses on h and g. The proofs rely heavily on results from convex analysis.     

EXPLICIT NONPARAMETRIC CONFIDENCE INTERVALS FOR THE VARIANCE WITH GUARANTEED COVERAGE

ABSTRACT

In this paper, we provide a method for constructing confidence intervals for the variance which exhibits guaranteed coverage probability for any sample size, uniformly over a wide class of probability distributions. In contrast, standard methods achieve guaranteed coverage only in the limit for a fixed distribution or for any sample size over a very restrictive (parametric) class of probability distributions. Of course, it is impossible to construct effective confidence intervals for the variance without some restriction, due to a result of Bahadur and Savage.[1] Bahadur, R. and Savage, L. 1956. The Nonexistence of Certain Statistical Procedures in Nonparametric Problems. Annals of Mathematical Statistics, 25: 1115–1122.  [Google Scholar] However, it is possible if the observations lie in a fixed compact set. We also consider the case of lower confidence bounds without any support restriction. Our method is based on the behavior of the variance over distributions that lie within a Kolmogorov–Smirnov confidence band for the underlying distribution. The method is a generalization of an idea of Anderson,[2] Anderson, T. 1967. Confidence Limits for the Expected Value of an Arbitrary Bounded Random Variable with a Continuous Distribution Function. Bull. ISI, 43: 249–251.  [Google Scholar] who considered only the case of the mean; it applies to very general parameters, and particularly the variance. While typically it is not clear how to compute these intervals explicitly, for the special case of the variance we provide an algorithm to do so. Asymptotically, the length of the intervals is of order n ^?/2 (in probability), so that, while providing guaranteed coverage, they are not overly conservative. A small simulation study examines the finite sample behavior of the proposed intervals.     

EXAMPLES OF MINIMUM VARIANCE ESTIMATION1

In this sequel to a previous discussion of minimum variance estimation (Bartlett, 1982), the gain with conditional estimation procedures is illustrated for the location parameter for (i) the rectangular distribution; (ii) a triangular distribution (typifying an asymmetric case). This note concludes with further remarks on the multi-sample and multi-parameter cases.     

ON THE ROBUST ANALYSIS OF VARIANCE COMPONENTS MODELS FOR PEDIGREE DATA

Quantitative traits measured over pedigrees of individuals may be analysed using maximum likelihood estimation, assuming that the trait has a multivariate normal distribution. This approach is often used in the analysis of mixed linear models. In this paper a robust version of the log likelihood for multivariate normal data is used to construct M-estimators which are resistant to contamination by outliers. The robust estimators are found using a minimisation routine which retains the flexible parameterisations of the multivariate normal approach. Asymptotic properties of the estimators are derived, computation of the estimates and their use in outlier detection tests are discussed, and a small simulation study is conducted.     

NONPARAMETRIC AUTOCOVARIANCE FUNCTION ESTIMATION

Nonparametric estimators of autocovariance functions for non-stationary time series are developed. The estimators are based on straightforward nonparametric mean function estimation ideas and allow use of any linear smoother (e.g. smoothing spline, local polynomial). The paper studies the properties of the estimators, and illustrates their usefulness through application to some meteorological and seismic time series.     

DENSITY ESTIMATION FOR CLUSTERED DATA

The commonly used survey technique of clustering introduces dependence into sample data. Such data is frequently used in economic analysis, though the dependence induced by the sample structure of the data is often ignored. In this paper, the effect of clustering on the non-parametric, kernel estimate of the density, f(x), is examined. The window width commonly used for density estimation for the case of i.i.d. data is shown to no longer be optimal. A new optimal bandwidth using a higher-order kernel is proposed and is shown to give a smaller integrated mean squared error than two window widths which are widely used for the case of i.i.d. data. Several illustrations from simulation are provided.     

MINIMUM VARIANCE UNBIASED ESTIMATION OF PROBABILITY DENSITIES

The purpose of this note is to point out an elementary method for deriving the minimum variance unbiased (mvu) estimators of probability densities. The method is illustrated by estimating the densities of some well known important distributions. We believe that this note would be of interest especially to both teachers and students of a first course in mathematical statistics.     

VARIANCE ESTIMATION FOR SAMPLE AUTOCOVARIANCES: DIRECT AND RESAMPLING APPROACHES

The usual covariance estimates for data _n-1 from a stationary zero-mean stochastic process {X_t} are the sample covariances Both direct and resampling approaches are used to estimate the variance of the sample covariances. This paper compares the performance of these variance estimates. Using a direct approach, we show that a consistent windowed periodogram estimate for the spectrum is more effective than using the periodogram itself. A frequency domain bootstrap for time series is proposed and analyzed, and we introduce a frequency domain version of the jackknife that is shown to be asymptotically unbiased and consistent for Gaussian processes. Monte Carlo techniques show that the time domain jackknife and subseries method cannot be recommended. For a Gaussian underlying series a direct approach using a smoothed periodogram is best; for a non-Gaussian series the frequency domain bootstrap appears preferable. For small samples, the bootstraps are dangerous: both the direct approach and frequency domain jackknife are better.     

ERROR VARIANCE ESTIMATION FOR THE SINGLE‐INDEX MODEL

Single‐index models provide one way of reducing the dimension in regression analysis. The statistical literature has focused mainly on estimating the index coefficients, the mean function, and their asymptotic properties. For accurate statistical inference it is equally important to estimate the error variance of these models. We examine two estimators of the error variance in a single‐index model and compare them with a few competing estimators with respect to their corresponding asymptotic properties. Using a simulation study, we evaluate the finite‐sample performance of our estimators against their competitors.     

A NOTE ON VARIANCE ESTIMATION FOR THE GENERALIZED REGRESSION PREDICTOR

The generalized regression (GREG) predictor is used for estimating a finite population total when the study variable is well‐related to the auxiliary variable. In 1997, Chaudhuri & Roy provided an optimal estimator for the variance of the GREG predictor within a class of non‐homogeneous quadratic estimators (H) under a certain superpopulation model M. They also found an inequality concerning the expected variances of the estimators of the variance of the GREG predictor belonging to the class H under the model M. This paper shows that the derivation of the optimal estimator and relevant inequality, presented by Chaudhuri & Roy, are incorrect.     

NONPARAMETRIC ESTIMATION OF ANGULAR DISTRIBUTIONS FROM CARTESIAN OBSERVATIONS WITH ERROR

If angular data are obtained from Cartesian observations, then any measurement error in these observations will produce a particular error structure in the angular data. The paper shows how non-parametric density estimation by orthogonal series may be performed in this case.     

FOURIER SERIES ESTIMATION FOR LENGTH BIASED DATA

This paper proposes and investigates Fourier series estimators for length biased data. Specifically, two Fourier series estimators are constructed and studied based on ideas of Jones (1991) and Bhattacharyya et al. (1988) in the case of kernel density estimation. Approximate expressions for mean squared errors and integrated mean squared errors are obtained and compared, and some simulated examples are investigated. The Fourier series estimator based on the proposal of Jones seems to have the more desirable properties of the two. The paper concludes with some comments that put this work in a wider context.     

VARIANCE ESTIMATION IN MULTI-STAGE SAMPLING

In multi-stage sampling with the first stage units (fsu) chosen without replacement (WOR) with varying probability schemes (VPS) unbiased estimators (UE) of variances of homogeneous linear (HL) functions of unbiased estimators (UE) T_i's of fsu totals Y_i's based on selection of subsequent stage units (SSU) from chosen fsu's are derived as homogeneous quadratic (HQ) functions of alternative less efficient UE's, say of T_i';'s of Y_i's. Specific strategies are illustrated.     

VARIANCE ESTIMATION IN TWO-PHASE SAMPLING

Two‐phase sampling is often used for estimating a population total or mean when the cost per unit of collecting auxiliary variables, x, is much smaller than the cost per unit of measuring a characteristic of interest, y. In the first phase, a large sample s₁ is drawn according to a specific sampling design p(s₁) , and auxiliary data x are observed for the units i∈s₁ . Given the first‐phase sample s₁ , a second‐phase sample s₂ is selected from s₁ according to a specified sampling design {p(s₂∣s₁) } , and (y, x) is observed for the units i∈s₂ . In some cases, the population totals of some components of x may also be known. Two‐phase sampling is used for stratification at the second phase or both phases and for regression estimation. Horvitz–Thompson‐type variance estimators are used for variance estimation. However, the Horvitz–Thompson ( Horvitz & Thompson, J. Amer. Statist. Assoc. 1952 ) variance estimator in uni‐phase sampling is known to be highly unstable and may take negative values when the units are selected with unequal probabilities. On the other hand, the Sen–Yates–Grundy variance estimator is relatively stable and non‐negative for several unequal probability sampling designs with fixed sample sizes. In this paper, we extend the Sen–Yates–Grundy ( Sen , J. Ind. Soc. Agric. Statist. 1953; Yates & Grundy , J. Roy. Statist. Soc. Ser. B 1953) variance estimator to two‐phase sampling, assuming fixed first‐phase sample size and fixed second‐phase sample size given the first‐phase sample. We apply the new variance estimators to two‐phase sampling designs with stratification at the second phase or both phases. We also develop Sen–Yates–Grundy‐type variance estimators of the two‐phase regression estimators that make use of the first‐phase auxiliary data and known population totals of some of the auxiliary variables.