Approaches to regression analysis with multiple measurements from individual sampling units

Application of ordinary least-squares regression to data sets which contain multiple measurements from individual sampling units produces an unbiased estimator of the parameters but a biased estimator of the covariance matrix of the parameter estimates. The present work considers a random coefficient, linear model to deal with such data sets: this model permits many senses in which multiple measurements are taken from a sampling unit, not just when it is measured at several times. Three procedures to estimate the covariance matrix of the error term of the model are considered. Given these, three procedures to estimate the parameters of the model and their covariance matrix are considered; these are ordinary least-squares, generalized least-squares, and an adjusted ordinary least-squares procedure which produces an unbiased estimator of the covariance matrix of the parameters with small samples. These various procedures are compared in simulation studies using three examples from the biological literature. The possibility of testing hypotheses about the vector of parameters is also considered. It is found that all three procedures for regression estimation produce estimators of the parameters with bias of no practical consequence, Both generalized least-squares and adjusted ordinary least-squares generally produce estimators of the covariance matrix of the parameter estimates with bias of no practical consequence, while ordinary least-squares produces a negatively biased estimator. Neither ordinary nor generalized least-squares provide satisfactory hypothesis tests of the vector of parameter estimates. It is concluded that adjusted ordinary least-squares, when applied with either of two of the procedures used to estimate the error coveriance matrix, shows promise for practical application with data sets of the nature considered here.     

Statistical inference for heteroscedastic semi-varying coefficient EV models

This paper proposes an estimation procedure for a class of semi-varying coefficient regression models when the covariates of the linear part are subject to measurement errors. Initial estimates for the regression and varying coefficients are first constructed by the profile least-squares procedure without input from heteroscedasticity, a bias-corrected kernel estimate for the variance function then is proposed, which in turn is used to define re-weighted bias-corrected estimates of the regression and varying coefficients. Large sample properties of the proposed estimates are thoroughly investigated. The finite-sample performance of the proposed estimates is assessed by an extensive simulation study and an application to the Boston housing data set. The simulation results show that the re-weighted bias-corrected estimates outperform the initial estimates and the naive estimates.     

Analysis of tidal data via the blockwise bootstrap

We analyze tidal data from Port Mansfield, TX, using Kunsch's blockwise bootstrap in the regression setting. In particular, we estimate the variability of parameter estimates in a harmonic analysis via block subsampling of residuals from a least-squares fit. We see that naive least-squares variance estimates can be either too large or too small, depending on the strength of correlation and the design matrix. We argue that the block bootstrap is a simple, omnibus method of accounting for correlation in a regression model with correlated errors.     

Optimum percentile estimating equations for nonlinear random coefficient models

In nonlinear random coefficients models, the means or variances of response variables may not exist. In such cases, commonly used estimation procedures, e.g., (extended) least-squares (LS) and quasi-likelihood methods, are not applicable. This article solves this problem by proposing an estimate based on percentile estimating equations (PEE). This method does not require full distribution assumptions and leads to efficient estimates within the class of unbiased estimating equations. By minimizing the asymptotic variance of the PEE estimates, the optimum percentile estimating equations (OPEE) are derived. Several examples including Weibull regression show the flexibility of the PEE estimates. Under certain regularity conditions, the PEE estimates are shown to be strongly consistent and asymptotic normal, and the OPEE estimates have the minimal asymptotic variance. Compared with the parametric maximum likelihood estimates (MLE), the asymptotic efficiency of the OPEE estimates is more than 98%, while the LS-type of procedures can have infinite variances. When the observations have outliers or do not follow the distributions considered in model assumptions, the article shows that OPEE is more robust than the MLE, and the asymptotic efficiency in the model misspecification cases can be above 150%.     

Different estimation methods for the parameters of the extended Burr XII distribution

The extended three-parameter Burr XII (EBXII) distribution has recently attracted considerable attention for modeling data from various scientific fields since it yields a wide range of skewness and kurtosis values. However, it is well known that the parameter estimates have significant effects on the success of a distribution in real-life applications. In this study, modified moment estimators (MMEs) and modified probability-weighted moments estimators (MPWMEs) are used to estimate the parameters of the EBXII distribution. These two considered estimators are also compared with the commonly used maximum-likelihood, percentiles, least-squares and weighted least-squares estimators in terms of bias and efficiency via an extensive numerical simulation. The MMEs and MPWMEs are observed to perform well in varying sample cases, and the simulation results are supported with application through a real-life data set.     

Limit theory for autoregressive-parameter estimates in an infinite-variance random walk

We consider the asymptotic behaviour of least-squares and M-estimates of the autoregressive parameter when the process is an infinite-variance random walk. It is shown that certain M -estimates converge faster than least-squares estimates and that they are also asymptotically normal.     

Nonparametric regression with additional measurement errors in the dependent variable

Estimation of a regression function from data which consists of an independent and identically distributed sample of the underlying distribution with additional measurement errors in the dependent variable is considered. It is allowed that the measurement errors are not independent and have nonzero mean. It is shown that the rate of convergence of least-squares estimates applied to this data is similar to the rate of convergence of least-squares estimates applied to an independent and identically distributed sample of the underlying distribution as long as the measurement errors are small. As an application, estimation of conditional variance functions from residuals is considered.     

Robust estimation of variance components

New robust estimates for variance components are introduced. Two simple models are considered: the balanced one-way classification model with a random factor and the balanced mixed model with one random factor and one fixed factor. However, the method of estimation proposed can be extended to more complex models. The new method of estimation we propose is based on the relationship between the variance components and the coefficients of the least-mean-squared-error predictor between two observations of the same group. This relationship enables us to transform the problem of estimating the variance components into the problem of estimating the coefficients of a simple linear regression model. The variance-component estimators derived from the least-squares regression estimates are shown to coincide with the maximum-likelihood estimates. Robust estimates of the variance components can be obtained by replacing the least-squares estimates by robust regression estimates. In particular, a Monte Carlo study shows that for outlier-contaminated normal samples, the estimates of variance components derived from GM regression estimates and the derived test outperform other robust procedures.     

On sufficient conditions for the strong consistency of least-squares estimates

The strong consistency of the least-squares estimates in regression models is obtained when the errors are i.i.d. with absolute moment of order r, 0<r? 2. The assumptions presented for the random error sequence will permit us to obtain improvements of the conditions on the regressors in order to obtain the strong consistency of the least-squares estimates in linear and nonlinear regression models.     

Likelihood estimation for exchangeable multinomial data

In this article, maximum likelihood estimates of an exchangeable multinomial distribution using a parametric form to model the parameters as functions of covariates are derived. The non linearity of the exchangeable multinomial distribution and the parametric model make direct application of Newton Rahpson and Fisher's scoring algorithms computationally infeasible. Instead parameter estimates are obtained as solutions to an iterative weighted least-squares algorithm. A completely monotonic parametric form is proposed for defining the marginal probabilities that results in a valid probability model.     

On the estimates of the parameters of the first order autoregressive process

Two methods of estimation for the parameters of an AR(1) process which are based on a non-linear least-squares approach are presented. On the basis of some simulation results they are compared with two maximum likelihood estimates and their relative merits are discussed.     

On the estimation of the most probable number in a serial dilution experiment

The purpose of this paper is to present some alternative estimates for the 'most probable number' of bacteria in a serial dilution experiment. These estimates are directed to be less biased than the ordinary maximum likelihood estimate. A numerical example illustrates the extent to which the variance and the mean square error of these estimates are generally less than those corresponding to the maximum likelihood estimate.     

Local influence in ridge semiparametric models

ABSTRACT

Ridge penalized least-squares estimators has been suggested as an alternative to the minimum penalized sum of squares estimates in the presence of collinearity among the explanatory variables in semiparametric regression models (SPRMs). This paper studies the local influence of minor perturbations on the ridge estimates in the SPRM. The diagnostics under the perturbation of ridge penalized sum of squares, response variable, explanatory variables and ridge parameter are considered. Some local influence diagnostics are given. A Monte Carlo simulation study and a real example are used to illustrate the proposed perturbations.     

Weighted total least-squares joint adjustment with weight correction factors

Abstract

A joint adjustment involves integrating different types of geodetic datasets, or multiple datasets of the same data type, into a single adjustment. This paper applies the weighted total least-squares (WTLS) principle to joint adjustment problems and proposes an iterative algorithm for WTLS joint (WTLS-J) adjustment with weight correction factors. Weight correction factors are used to rescale the weight matrix of each dataset while using the Helmert variance component estimation (VCE) method to estimate the variance components, since the variance components in the stochastic model are unknown. An affine transformation example is illustrated to verify the practical benefit and the relative computational efficiency of the proposed algorithm. It is shown that the proposed algorithm obtains the same parameter estimates as the Amiri-Simkooei algorithm in our example; however, the proposed algorithm has its own computational advantages, especially when the number of data points is large.     

A Note on Nonparametric Regression with β-Mixing Sequences

Statistical risks of least-squares estimates of a regression function (possibly nonlinear) are studied under a β-mixing setup. The assumptions are minimal and are virtually the same as those of the i.i.d. case. The bounds obtained are optimal, up to a term of order log n. Similar results are also obtained for complexity regularized regression estimates in the presence of penalty functions.     

Comparison of estimation methods for the Marshall–Olkin extended Lindley distribution

The aim of this paper is to compare the parameters' estimations of the Marshall–Olkin extended Lindley distribution obtained by six estimation methods: maximum likelihood, ordinary least-squares, weighted least-squares, maximum product of spacings, Cramér–von Mises and Anderson–Darling. The bias, root mean-squared error, average absolute difference between the true and estimate distributions' functions and the maximum absolute difference between the true and estimate distributions' functions are used as comparison criteria. Although the maximum product of spacings method is not widely used, the simulation study concludes that it is highly competitive with the maximum likelihood method.     

Estimation of harmonic component in regression with cyclically dependent errors

This paper deals with the estimation of hidden periodicities in a non-linear regression model with stationary noise displaying cyclical dependence. Consistency and asymptotic normality are established for the least-squares estimates.     

Least circular distance regression for directional data

Least-squares regression is not appropriate when the response variable is circular, and can lead to erroneous results. The reason for this is that the squared difference is not an appropriate measure of distance on the circle. In this paper, a circular analog to least-squares regression is presented for predicting a circular response variable by another circular variable and a set of linear covariates. An alternative maximum-likelihood formulation yields the same regression parameter estimates. Under the maximum-likelihood model, asymptotic standard errors of the parameter estimates are obtained. As an example, the regression model is used to model data from a marine biology study.     

Parameter Estimation Through Weighted Least-Squares Rank Regression with Specific Reference to the Weibull and Gumbel Distributions

Probability plots are often used to estimate the parameters of distributions. Using large sample properties of the empirical distribution function and order statistics, weights to stabilize the variance in order to perform weighted least squares regression are derived. Weighted least squares regression is then applied to the estimation of the parameters of the Weibull, and the Gumbel distribution. The weights are independent of the parameters of the distributions considered. Monte Carlo simulation shows that the weighted least-squares estimators outperform the usual least-squares estimators totally, especially in small samples.     

Exploring spatial dependence in area-level random effect model for disaggregate-level crop yield estimation

This paper describes an application of small area estimation (SAE) techniques under area-level spatial random effect models when only area (or district or aggregated) level data are available. In particular, the SAE approach is applied to produce district-level model-based estimates of crop yield for paddy in the state of Uttar Pradesh in India using the data on crop-cutting experiments supervised under the Improvement of Crop Statistics scheme and the secondary data from the Population Census. The diagnostic measures are illustrated to examine the model assumptions as well as reliability and validity of the generated model-based small area estimates. The results show a considerable gain in precision in model-based estimates produced applying SAE. Furthermore, the model-based estimates obtained by exploiting spatial information are more efficient than the one obtained by ignoring this information. However, both of these model-based estimates are more efficient than the direct survey estimate. In many districts, there is no survey data and therefore it is not possible to produce direct survey estimates for these districts. The model-based estimates generated using SAE are still reliable for such districts. These estimates produced by using SAE will provide invaluable information to policy-analysts and decision-makers.