Liu estimator in partly linear regression models with correlated errors

This article is concerned with the parameter estimation in partly linear regression models when the errors are dependent. To overcome the multicollinearity problem, a generalized Liu estimator is proposed. The theoretical properties of the proposed estimator and its relationship with some existing methods designed for partly linear models are investigated. Finally, a hypothetical data is conducted to illustrate some of the theoretical results.     

Efficiency analysis of ten estimation procedures for quantitative linear models with autocorrelated errors

Many estimation procedures for quantitative linear models with autocorrelated errors have been proposed in the literature. A number of these procedures have been compared in various ways for different sample sizes and autocorrelation parameters values and for structured or random explanatory vaiables. In this paper, we revisit three situations that were considered to some extent in previous studies, by comparing ten estimation procedures: Ordinary Least Squares (OLS), Generalized Least Squares (GLS), estimated Generalized Least Squares (six procedures), Maximum Likelihood (ML), and First Differences (FD). The six estimated GLS procedures and the ML procedure differ in the way the error autocovariance matrix is estimated. The three situations can be defined as follows: Case 1, the explanatory variable x in the simple linear regression is fixed; Case 2,x is purely random; and Case 3x is first-order autoregressive. Following a theoretical presentation, the ten estimation procedures are compared in a Monte Carlo study conducted in the time domain, where the errors are first-order autoregressive in Cases 1-3. The measure of comparison for the estimation procedures is their efficiency relative to OLS. It is evaluated as a function of the time series length and the magnitude and sign of the error autocorrelation parameter. Overall, knowledge of the model of the time series process generating the errors enhances efficiency in estimated GLS. Differences in the efficiency of estimation procedures between Case 1 and Cases 2 and 3 as well as differences in efficiency among procedures in a given situation are observed and discussed.     

Ridge regression estimators in the linear regression models with non-spherical errors

The present paper considers a family of ordinary ridge regression estimators in the linear regression model when the disturbances covariance matrix depends upon a few unknown parameters. An asymptotic expansion for the distribution of the ridge regression estimator is developed and under the quadratic loss function its asymptotic risk is compared with that of the feasible GLS estimator.     

Statistical inference on partial linear additive models with distortion measurement errors

We consider statistical inference for partial linear additive models (PLAMs) when the linear covariates are measured with errors and distorted by unknown functions of commonly observable confounding variables. A semiparametric profile least squares estimation procedure is proposed to estimate unknown parameter under unrestricted and restricted conditions. Asymptotic properties for the estimators are established. To test a hypothesis on the parametric components, a test statistic based on the difference between the residual sums of squares under the null and alternative hypotheses is proposed, and we further show that its limiting distribution is a weighted sum of independent standard chi-squared distributions. A bootstrap procedure is further proposed to calculate critical values. Simulation studies are conducted to demonstrate the performance of the proposed procedure and a real example is analyzed for an illustration.     

Modeling of covariance structures of random effects and random errors in linear mixed models

Abstract

In this paper, we discuss how to model the mean and covariancestructures in linear mixed models (LMMs) simultaneously. We propose a data-driven method to modelcovariance structures of the random effects and random errors in the LMMs. Parameter estimation in the mean and covariances is considered by using EM algorithm, and standard errors of the parameter estimates are calculated through Louis’ (1982 Louis, T.A. (1982). Finding observed information using the EM algorithm. J. Royal Stat. Soc. B 44:98–130. [Google Scholar]) information principle. Kenward’s (1987 Kenward, M.G. (1987). A method for comparing profiles of repeated measurements. Appl. Stat. 36:296–308.[Crossref], [Web of Science ®] , [Google Scholar]) cattle data sets are analyzed for illustration,and comparison to the literature work is made through simulation studies. Our numerical analysis confirms the superiority of the proposed method to existing approaches in terms of Akaike information criterion.     

Shrinkage and LASSO strategies in high-dimensional heteroscedastic models

ABSTRACT

In this paper, we consider the estimation problem of the parameter vector in the linear regression model with heteroscedastic errors. First, under heteroscedastic errors, we study the performance of shrinkage-type estimators and their performance as compared to theunrestricted and restricted least squares estimators. In order to accommodate the heteroscedastic structure, we generalize an identity which is useful in deriving the risk function. Thanks to the established identity, we prove that shrinkage estimators dominate the unrestricted estimator. Finally, we explore the performance of high-dimensional heteroscedastic regression estimator as compared to classical LASSO and shrinkage estimators.     

Ridge regression methodology in partial linear models with correlated errors

In this article, a generalized restricted difference-based ridge estimator is defined for the vector parameter in a partial linear model when the errors are dependent. It is suspected that some additional linear constraints may hold on to the whole parameter space. The estimator is a generalization of the well-known restricted least-squares estimator and is confined to the (affine) subspace which is generated by the restrictions. The risk functions of the proposed estimators are derived under balanced loss function. Finally, the performance of the new estimators is evaluated by a simulated data set.     

Strong consistency of least squares estimates with i.i.d. errors with mean values not necessarily defined

We establish strong consistency of the least squares estimates in multiple regression models discarding the usual assumption of the errors having null mean value. Thus, we required them to be i.i.d. with absolute moment of order r, 0<r<2, and null mean value when r>1. Only moderately restrictive conditions are imposed on the model matrix. In our treatment, we use an extension of the Marcinkiewicz–Zygmund strong law to overcome the errors mean value not being defined. In this way, we get a unified treatment for the case of i.i.d. errors extending the results of some previous papers.     

Linear trimmed means for the linear regression with AR(1) errors model

For the linear regression with AR(1) errors model, the robust generalized and feasible generalized estimators of Lai et al. (2003) of regression parameters are shown to have the desired property of a robust Gauss Markov theorem. This is done by showing that these two estimators are the best among classes of linear trimmed means. Monte Carlo and data analysis for this technique have been performed.     

Asymptotic results for estimation and testing variances in regression models

Usually the variance of independent observations resulting from a linear or a nonlinear relationship is estimated by the Least-Squares residual estimator. In this paper its asymptotic properties are investigated. Further the asymptotic behaviour of tests for one-sided hypotheses on the variance is studied. The paper splits into two parts, the first one concerned with linear and the second one with nonlinear models.     

The strong consistency of M-estimates in linear models with extended negatively dependent errors

In this paper, we first establish the strong convergence for weighted sums of extended negatively dependent (END) random variables. Based on the strong convergence and Bernstein inequality, we obtain the strong consistency of M-estimates of the regression parameters in a linear model for END random errors under some mild moment conditions. The results generalize and improve the ones obtained in the literature to the case of END random errors.     

On discrete-domain multidimensional sinusoidal models

We consider three-dimensional sinusoidal frequency model in a random field. Three-dimensional frequency model has wide applications in statistical signal processing. In this article, we mainly consider the usual least squares estimators and the estimators that can be obtained by maximizing the periodogram function. We obtain consistency and asymptotic normality property of both the estimators. It is observed that they are asymptotically equivalent. Finally we generalize the results to the multidimensional case.     

Feasible robust estimator in restricted semiparametric regression models based on the LTS approach

Under some nonstochastic linear restrictions based on either additional information or prior knowledge in a semiparametric regression model, a family of feasible generalized robust estimators for the regression parameter is proposed. The least trimmed squares (LTS) method proposed by Rousseeuw as a highly robust regression estimator is a statistical technique for fitting a regression model based on the subset of h observations (out of n) whose least-square fit possesses the smallest sum of squared residuals. The coverage h may be set between n/2 and n. The LTS estimator involves computing the hyperplane that minimizes the sum of the smallest h squared residuals. For practical purpose, it is assumed that the covariance matrix of the error term is unknown and thus feasible estimators are replaced. Then, we develop an algorithm for the LTS estimator based on feasible methods. Through the Monte Carlo simulation studies and a real data example, performance of the feasible type of robust estimators is compared with the classical ones in restricted semiparametric regression models.     

Missing data in linear models with correlated errors

One common method for analyzing data in experimental designs when observations are missing was devised by Yates (1933), who developed his procedure based upon a suggestion by R. A. Fisher. Considering a linear model with independent, equi-variate errors, Yates substituted algebraic values for the missing data and then minimized the error sum of squares with respect to both the unknown parameters and the algebraic values. Yates showed that this procedure yielded the correct error sum of squares and a positively biased hypothesis sum of squares.

Others have elaborated on this technique. Chakrabarti (1962) gave a formal proof of Fisher's rule that produced a way to simplify the calculations of the auxiliary values to be used in place of the missing observations. Kshirsagar (1971) proved that the hypothesis sum of squares based on these values was biased, and developed an easy way to compute that bias. Sclove     

A new test for random effects in linear mixed models with longitudinal data

As it is known, testing the existence of random effects is often transferred to testing their zero variances/covariance matrices. It is a nonstandard testing problem because the hypothetical values are on the boundary of the whole space. In the literature, a difference-based test was proposed, which has asymptotically tractable null distribution and is then easy to implement. However, the projection method on which the difference-based test relies may affect and deteriorate its performance when covariates associated with fixed effects and covariates associated with random effects are highly correlated. In the paper, for linear mixed models (LMM) with longitudinal data, a new test is proposed to avoid this problem. The new test is also asymptotically distribution-free and more powerful than the difference-based test, particularly when the above correlation is high. The new test is consistent against all global alternatives and can detect local alternatives converging to the null at a rate as close as to m^−1/2$m^{-1/2}$ with m being the number of subjects. Simulations are carried out to examine the performance and a real data analysis is performed for illustration.     

Restricted estimation and testing of hypothesis in linear measurement errors models

ABSTRACT

In this article, the linear models with measurement error both in the response and in the covariates are considered. Following Shalabh et al. (2007 Shalabh, Garg, G., Misra, N. (2007). Restricted regression estimation in measurement error models. Comput. Stat. Data Anal. 52:1149–1166.[Crossref], [Web of Science ®] , [Google Scholar], 2009 Shalabh, Garg, G., Misra, N. (2009). Use of prior information in the consistent estimation of regression coefficients in measurement error models. J. Multivariate Anal. 100:1498–1520.[Crossref], [Web of Science ®] , [Google Scholar]), we propose several restricted estimators for the regression coefficients. The consistency and asymptotic normality of the restricted estimators are established. Furthermore, we also discuss the superiority of the restricted estimators to unrestricted estimators under Pitman closeness criterion. We also develop several variance estimators and establish their asymptotic distributions. Wald-type statistics are constructed for testing the linear restrictions. Finally, Monte Carlo simulations are conducted to illustrate the finite-sample properties of the proposed estimators.     

Projectors and linear estimation in general linear models

The paper gives a self-contained account of minimum disper­sion linear unbiased estimation of the expectation vector in a linear model with the dispersion matrix belonging to some, rather arbitrary, set of nonnegative definite matrices. The approach to linear estimation in general linear models recommended here is a direct generalization of some ideas and results presented by Rao (1973, 19 74) for the case of a general Gauss-Markov model

A new insight into the nature of some estimation problems originaly arising in the context of a general Gauss-Markov model as well as the correspondence of results known in the literature to those obtained in the present paper for general linear models are also given. As preliminary results the theory of projectors defined by Rao (1973) is extended.     

Semi-parametric estimation of disequilibrium models

This paper presents an easy-to-compute semi-parametric (SP) method to estimate a simple disequilibrium model proposed by Fair and Jaffee (1972). The proposed approach is based on a non-parametric interpretation of the EM (Expectation and Maximization) principle (Dempster et al; 1977) and the least squares method. The simple disequilibrium model includes the demand equation, the supply equation, and the condition that only the minimum of quantity demanded and quantity supplied is observed. The method used here allows one to consistently estimate the disequilibrium model without fully specifying the distribution of error terms in both demand and supply equations. Our Monte Carlo study suggests that the proposedestimator is better than the normal maximum likelihood estimator under asymmetric error distributions. and comparable to the nlaximunl likelihood estimator under synirnetric error distributions in finite samples. Aggregate U.S. labor market data from Quandt and Rosen (1988) is used to illustrate the procedure.