Some Matrix Norm Kantorovich Inequalities and Their Applications

In this article, we first present four matrix norm Kantorovich-type inequalities involving non negative definite matrix. Then, based on these inequalities, we propose four new efficiency criteria and present their lower bounds to make efficiency comparisons between the ordinary least squares estimator and the best linear unbiased estimator in a singular linear model.     

A regression estimator for finite population mean of a sensitive variable using an optional randomized response model

Kalucha et al. (Kalucha G., Gupta S., Dass B. K. (accepted). Ratio estimation of finite population mean using optional randomized response models. Journal of Statistical Theory and Practice) introduced an additive ratio estimator for finite population mean of a sensitive variable in simple random sampling without replacement and showed that this estimator performs better than the ordinary mean estimator based on an optional randomized response technique (RRT). In this paper, we introduce a regression estimator that performs better than the ratio estimator even for the modest correlation between the study and the auxiliary variables. A comparison of the proposed estimator with the corresponding ratio estimator and the ordinary RRT mean estimator is carried out theoretically, and is also illustrated with a simulation study.     

Adaptive Estimation Using Weighted Least Squares

This paper proposes an adaptive estimator that is more precise than the ordinary least squares estimator if the distribution of random errors is skewed or has long tails. The adaptive estimates are computed using a weighted least squares approach with weights based on the lengths of the tails of the distribution of residuals. Smaller weights are assigned to those observations that have residuals in the tails of long-tailed distributions and larger weights are assigned to observations having residuals in the tails of short-tailed distributions. Monte Carlo methods are used to compare the performance of the proposed estimator and the performance of the ordinary least squares estimator. The estimates that were studied in this simulation include the difference between the means of two populations, the mean of a symmetric distribution, and the slope of a regression line. The adaptive estimators are shown to have lower mean squared errors than those for the ordinary least squares estimators for short-tailed, long-tailed, and skewed distributions, provided the sample size is at least 20. The ordinary least squares estimator has slightly lower mean squared error for normally distributed errors. The adaptive estimator is recommended for general use for studies having sample sizes of at least 20 observations unless the random errors are known to be normally distributed.     

A TWO-PHASE SAMPLING ESTIMATOR IN SAMPLE SURVEYS

A two-phase sampling estimator of the ratio-type for estimating the mean of a finite population, has been considered where the value of ρC_y/C_x can be guessed or estimated in advance. Here C_y and C_x denote respectively the coefficients of variation of the characteristic under study, y, and the auxiliary characteristic x and ρ denotes the coefficient of correlation between y and x. When the value of ρC_y/C_x is guessed or estimated exactly, the estimator has a smaller large-sample variance compared with either an ordinary ratio estimator or an ordinary linear regression estimator in two-phase sampling in the case where the first-phase sample is drawn independently from the second-phase sample. If the sample at the second phase is a subsample of the first-phase sample, the estimator has variance equal to that of the linear regression estimator. The largest value of the difference between the assumed value and the actual value of ρC_y/C_x has been obtained so as not to result in the variance of the estimator being larger than the variances of either an ordinary ratio estimator or an ordinary linear regression estimator.     

Asymptotic misspecification biases for heckman's two step estimator

Asymptotic bias formulae are obtained for Heckman's two step estimator under misspecification of the single equation Tobit modelj and the two equation sample selection model. Asymptotic biases are also obtained for the ordinary least squares estimator based on uncensored observations only. Omitted variables, errors in variables, and heteroskedasticity are considered as sources of misspecification. The biases are illustrated by numerical examples, in which the Tobit maximum likelihood estimator is also included. Severe consequences for the two step estimator are indicated.     

A likelihood based estimator for vector autoregressive processes

A onestep estimator, which is an approximation to the unconditional maximum likelihood estimator (MLE) of the coefficient matrices of a Gaussian vector autoregressive process is presented. The onestep estimator is easy to compute and can be computed using standard software. Unlike the computation of the unconditional MLE, the computation of the onestep estimator does not require any iterative optimization and the computation is numerically stable. In finite samples the onestep estimator generally has smaller mean square error than the ordinary least squares estimator. In a simple model, where the unconditional MLE can be computed, numerical investigation shows that the onestep estimator is slightly worse than the unconditional MLE in terms of mean square error but superior to the ordinary least squares estimator. The limiting distribution of the onestep estimator for processes with some unit roots is derived.     

Restricted Two Parameter Ridge Estimator

In 2005 Lipovetsky and Conklin proposed an estimator, the two parameter ridge estimator (TRE), as an alternative to the ordinary least squares estimator (OLSE) and the ordinary ridge estimator (RE) in the presence of multicollinearity, and in 2006 Lipovetsky improved the two parameter model. In this paper, we introduce two new estimators, one of which is the modified two parameter ridge estimator (MTRE) defined by following Swindel's paper of 1976. The other one is the restricted two parameter ridge estimator (RTRE) which is derived by setting additional linear restrictions on the parameter vectors. This estimator is a generalization of the restricted least squares estimator (RLSE) and includes the restricted ridge estimator (RRE) proposed by Groß in 2003. A numerical example is provided and a simulation study is conducted for the comparisons of the RTRE with the OLSE, RLSE, RE, RRE and TRE.     

Robust Linearized Ridge M-estimator for Linear Regression Model

In the multiple linear regression, multicollinearity and outliers are commonly occurring problems. They produce undesirable effects on the ordinary least squares estimator. Many alternative parameter estimation methods are available in the literature which deals with these problems independently. In practice, it may happen that the multicollinearity and outliers occur simultaneously. In this article, we present a new estimator called as Linearized Ridge M-estimator which combats the problem of simultaneous occurrence of multicollinearity and outliers. A real data example and a simulation study is carried out to illustrate the performance of the proposed estimator.     

Conditional logistic regression in cluster-specific 1: m matched treatment–control designs

Conditional logistic regression is a popular method for estimating a treatment effect while eliminating cluster-specific nuisance parameters when they are not of interest. Under a cluster-specific 1: m matched treatment–control study design, we present a new closed-form relationship between the conditional logistic regression estimator and the ordinary logistic regression estimator. In addition, we prove an equivalence between the ordinary logistic regression and the conditional logistic regression estimators, when the clusters are replicated infinitely often, which indicates that potential bias concerns when applying conditional logistic regression to complex survey samples.     

A risk set calibration method for failure time regression by using a covariate reliability sample

Regression parameter estimation in the Cox failure time model is considered when regression variables are subject to measurement error. Assuming that repeat regression vector measurements adhere to a classical measurement model, we can consider an ordinary regression calibration approach in which the unobserved covariates are replaced by an estimate of their conditional expectation given available covariate measurements. However, since the rate of withdrawal from the risk set across the time axis, due to failure or censoring, will typically depend on covariates, we may improve the regression parameter estimator by recalibrating within each risk set. The asymptotic and small sample properties of such a risk set regression calibration estimator are studied. A simple estimator based on a least squares calibration in each risk set appears able to eliminate much of the bias that attends the ordinary regression calibration estimator under extreme measurement error circumstances. Corresponding asymptotic distribution theory is developed, small sample properties are studied using computer simulations and an illustration is provided.     

Some conditions for optimality of the almost unbiased estimators of regression coefficients

The purpose of this paper is to examine the asymptotic properties of the operational almost unbiased estimator of regression coefficients which includes almost unbiased ordinary ridge estimator a s a special case. The small distrubance approximations for the bias and mean square error matrix of the estimator are derived. As a consequence, it is proved that, under certain conditions, the estimator is more efficient than a general class of estimators given by Vinod and Ullah (1981). Also it is shown that, if the ordinary ridge estimator (ORE) dominates the ordinary least squares estimator then the almost unbiased ordinary ridge estimator does not dominate ORE under the mean square error criterion.     

A Two-Stage Estimation Method for Random Coefficient Differential Equation Models with Application to Longitudinal HIV Dynamic Data

We propose a two-stage estimation method for random coefficient ordinary differential equation (ODE) models. A maximum pseudo-likelihood estimator (MPLE) is derived based on a mixed-effects modeling approach and its asymptotic properties for population parameters are established. The proposed method does not require repeatedly solving ODEs, and is computationally efficient although it does pay a price with the loss of some estimation efficiency. However, the method does offer an alternative approach when the exact likelihood approach fails due to model complexity and high-dimensional parameter space, and it can also serve as a method to obtain the starting estimates for more accurate estimation methods. In addition, the proposed method does not need to specify the initial values of state variables and preserves all the advantages of the mixed-effects modeling approach. The finite sample properties of the proposed estimator are studied via Monte Carlo simulations and the methodology is also illustrated with application to an AIDS clinical data set.     

Superiority of the r–k Class Estimator Over Some Estimators In A Linear Model

In regression analysis, to overcome the problem of multicollinearity, the r ? k class estimator is proposed as an alternative to the ordinary least squares estimator which is a general estimator including the ordinary ridge regression estimator, the principal components regression estimator and the ordinary least squares estimator. In this article, we derive the necessary and sufficient conditions for the superiority of the r ? k class estimator over each of these estimators under the Mahalanobis loss function by the average loss criterion. Then, we compare these estimators with each other using the same criterion. Also, we suggest to test to verify if these conditions are indeed satisfied. Finally, a numerical example and a Monte Carlo simulation are done to illustrate the theoretical results.     

A new estimator of proportion with a linear function using data from two-decks randomized response model

A new estimator for estimating the proportion of a potentially sensitive attribute in survey sampling has been introduced by solving a linear equation. The proposed estimator has been compared with the estimator proposed by Odumade and Singh (2009) with equal protection to all of the respondents. The asymptotic properties of the proposed estimator are investigated through exact numerical illustrations for different choices of parameters. A non randomized response approach has been suggested. A scope for further research has also been pointed out.     

Measuring the degree of severity of heteroskedasticity and the choice between the ols estimator and the 2sae

This paper dwells on the choice between the ordinary least squares and the estimated generalized least squares estimators when the presence of heteroskedasticity is suspected. Since the estimated generalized least squares estimator does not dominate the ordinary least squares estimator completely over the whole parameter space, it is of interest to the researcher to know in advance whether the degree of severity of heteroskedasticity is such that OLS estimator outperforms the estimated generalized least squares (or 2SAE). Casting the problem in the non-spherical error mold and exploiting the principle underlying the Bayesian pretest estimator, an intuitive non-mathematical procedure is proposed to serve as an aid to the researcher in deciding when to use either the ordinary least squares (OLS) or the estimated generalized least squares (2SAE) estimators.     

Estimating Percentiles of Time-to-Failure Distribution Obtained from a Linear Degradation Model Using Kernel Density Method

In this article, we propose a nonparametric estimator for percentiles of the time-to-failure distribution obtained from a linear degradation model using the kernel density method. The properties of the proposed kernel estimator are investigated and compared with well-known maximum likelihood and ordinary least squares estimators via a simulation technique. The mean squared error and the length of the bootstrap confidence interval are used as the basis criteria of the comparisons. The simulation study shows that the performance of the kernel estimator is acceptable as a general estimator. When the distribution of the data is assumed to be known, the maximum likelihood and ordinary least squares estimators perform better than the kernel estimator, while the kernel estimator is superior when the assumption of our knowledge of the data distribution is violated. A comparison among different estimators is achieved using a real data set.     

On weak consistency in linear models with equi-correlated random errors

A new estimator in linear models with equi-correlated random errors is postulated. Consistency properties of the proposed estimator and the ordinary least squares estimator are studied. It is shown that the new estimator has smaller variance than the usual least squares estimator under some mild conditions. In addition, it is observed that the new estimator tends to be weakly consistent in many cases where the usual least squares estimator is not.     

The small sample properties of the restricted principal component regression estimator in linear regression model

In regression analysis, to deal with the problem of multicollinearity, the restricted principal components regression estimator is proposed. In this paper, we compared the restricted principal components regression estimator, the principal components regression estimator, and the ordinary least-squares estimator with each other under the Pitman's closeness criterion. We showed that the restricted principal components regression estimator is always superior to the principal components regression estimator, under certain conditions the restricted principal components regression estimator is superior to the ordinary least-squares estimator under the Pitman's closeness criterion and under certain conditions the principal components regression estimator is superior to the ordinary least-squares estimator under the Pitman's closeness criterion.     

New estimation and feature selection methods in mixture‐of‐experts models

We study estimation and feature selection problems in mixture‐of‐experts models. An $l_2$ ‐penalized maximum likelihood estimator is proposed as an alternative to the ordinary maximum likelihood estimator. The estimator is particularly advantageous when fitting a mixture‐of‐experts model to data with many correlated features. It is shown that the proposed estimator is root‐$n$ consistent, and simulations show its superior finite sample behaviour compared to that of the maximum likelihood estimator. For feature selection, two extra penalty functions are applied to the $l_2$ ‐penalized log‐likelihood function. The proposed feature selection method is computationally much more efficient than the popular all‐subset selection methods. Theoretically it is shown that the method is consistent in feature selection, and simulations support our theoretical results. A real‐data example is presented to demonstrate the method. The Canadian Journal of Statistics 38: 519–539; 2010 © 2010 Statistical Society of Canada     

Matrix Euclidean norm Wielandt inequalities and their applications to statistics

In this paper we shall establish a new matrix inequality which will fill the gap that there has not been any matrix Euclidean norm version of the Wielandt inequality in the literature yet. This inequality can be used to present an upper bound of a new measure of association which plays a very important role in statistics, especially in multivariate analysis. A new alternative based on Euclidean norm for relative gain of the covariance adjusted estimator of parameters is provided.