Focused Information Criterion and Model Averaging in Quantile Regression

In this article, we study model selection and model averaging in quantile regression. Under general conditions, we develop a focused information criterion and a frequentist model average estimator for the parameters in quantile regression model, and examine their theoretical properties. The new procedures provide a robust alternative to the least squares method or likelihood method, and a major advantage of the proposed procedures is that when the variance of random error is infinite, the proposed procedure works beautifully while the least squares method breaks down. A simulation study and a real data example are presented to show that the proposed method performs well with a finite sample and is easy to use in practice.     

Variance least squares estimators for multivariate linear mixed model

Non-iterative, distribution-free, and unbiased estimators of variance components by least squares method are derived for multivariate linear mixed model. A general inter-cluster variance matrix, a same-member only general inter-response variance matrix, and an uncorrelated intra-cluster error structure for each response are assumed. Projection method is suggested when unbiased estimators of variance components are not nonnegative definite matrices. A simulation study is conducted to investigate the properties of the proposed estimators in terms of bias and mean square error with comparison to the Gaussian (restricted) maximum likelihood estimators. The proposed estimators are illustrated by an application of gene expression familial study.     

A simple method for fitting a linear model involving variance components

This paper describes combined maximum likelihood estimation of treatment effects and variance components in a linear mixed model. Estimates are obtained by a simple modification of iterative weighted least squares. The method is illustrated using plating efficiencies of Solanum and Lycopersicon species.     

Maximum likelihood estimation of variance components in a completely random bib design

ABSTRACT

The purposes of this paper are to abstract from a number of articles variance component estimation procedures which can be used for completely random balanced incomplete block designs, to develop an iterated least squares (ITLS) computing algorithm for calculating maximum likelihood estimates, and to compare these procedures by use of simulated experiments. Based on the simulated experiments, the estimated mean square errors of the ITLS estimates are generally less than*those for previously proposed analysis of variance and symmetric sums estimators.     

Improper use of the ordinary least squares estimator in the switching regression model

The paper considers the consequences of incorrectly using the ordinary least squares estimator, when the true but unknown model is a switching regression. Bias and mean square error express ons are given for slope and residual variance estimators. Except for in very specialized cases the estimators are biased. A numerical exarnple illustrates some of the issues raised and provides a conpelison between the ordinary least squares and maximum likelihood estimators.     

Fitting linear regression models to censored data by least squares and maximum likelihood methods

Several approaches have been suggested for fitting linear regression models to censored data. These include Cox's propor­tional hazard models based on quasi-likelihoods. Methods of fitting based on least squares and maximum likelihoods have also been proposed. The methods proposed so far all require special purpose optimization routines. We describe an approach here which requires only a modified standard least squares routine.

We present methods for fitting a linear regression model to censored data by least squares and method of maximum likelihood. In the least squares method, the censored values are replaced by their expectations, and the residual sum of squares is minimized. Several variants are suggested in the ways in which the expect­ation is calculated. A parametric (assuming a normal error model) and two non-parametric approaches are described. We also present a method for solving the maximum likelihood equations in the estimation of the regression parameters in the censored regression situation. It is shown that the solutions can be obtained by a recursive algorithm which needs only a least squares routine for optimization. The suggested procesures gain considerably in computational officiency. The Stanford Heart Transplant data is used to illustrate the various methods.     

Some Recent Developments in the Analysis and Design of Quantitative Genetic Experiments

The design and analysis of experiments to estimate heritability when data are available on both parents and progeny and the offspring have a hierarchical structure is considered. The method of analysis is related to a multivariate analysis of variance and to weighted least squares. It is shown that genetical theory gives a simple interpretation of both maximum likelihood (ML) and Rao's minimum norm quadratic unbiased (MINQUE) methods of estimation of variance components in unbalanced designs.     

Local maximum likelihood estimation and inference

Local maximum likelihood estimation is a nonparametric counterpart of the widely used parametric maximum likelihood technique. It extends the scope of the parametric maximum likelihood method to a much wider class of parametric spaces. Associated with this nonparametric estimation scheme is the issue of bandwidth selection and bias and variance assessment. This paper provides a unified approach to selecting a bandwidth and constructing confidence intervals in local maximum likelihood estimation. The approach is then applied to least squares nonparametric regression and to nonparametric logistic regression. Our experiences in these two settings show that the general idea outlined here is powerful and encouraging.     

Some Recent Developments in Econometric Inference

Recent results in information theory, see Soofi (1996; 2001) for a review, include derivations of optimal information processing rules, including Bayes' theorem, for learning from data based on minimizing a criterion functional, namely output information minus input information as shown in Zellner (1988; 1991; 1997; 2002). Herein, solution post data densities for parameters are obtained and studied for cases in which the input information is that in (1) a likelihood function and a prior density; (2) only a likelihood function; and (3) neither a prior nor a likelihood function but only input information in the form of post data moments of parameters, as in the Bayesian method of moments approach. Then it is shown how optimal output densities can be employed to obtain predictive densities and optimal, finite sample structural coefficient estimates using three alternative loss functions. Such optimal estimates are compared with usual estimates, e.g., maximum likelihood, two-stage least squares, ordinary least squares, etc. Some Monte Carlo experimental results in the literature are discussed and implications for the future are provided.     

Alternative estimators in logistic regression when the data are collinear

Logistic regression using conditional maximum likelihood estimation has recently gained widespread use. Many of the applications of logistic regression have been in situations in which the independent variables are collinear. It is shown that collinearity among the independent variables seriously effects the conditional maximum likelihood estimator in that the variance of this estimator is inflated in much the same way that collinearity inflates the variance of the least squares estimator in multiple regression. Drawing on the similarities between multiple and logistic regression several alternative estimators, which reduce the effect of the collinearity and are easy to obtain in practice, are suggested and compared in a simulation study.     

On generalized least squares estimation of the weibull distribution

A simple estimation procedure, based on the generalized least squares method, for the parameters of the Weibull distribution is described and investigated. Through a simulation study, this estimation technique is compared with maximum likelihood estimation, ordinary least squares estimation, and Menon's estimation procedure; this comparison is based on observed relative efficiencies (that is, the ratio of the Cramer-Rao lower bound to the observed mean squared error). Simulation results are presented for samples of size 25. Among the estimators considered in this simulation study, the generalized least squares estimator was found to be the "best" estimator for the shape parameter and a close competitor to the maximum likelihood estimator of the scale parameter.     

A Weighted Least Squares Approach to Levene's Test of Homogeneity of Variance

In 1960 Levene suggested a potentially robust test of homogeneity of variance based on an ordinary least squares analysis of variance of the absolute values of mean-based residuals. Levene's test has since been shown to have inflated levels of significance when based on the F-distribution, and tests a hypothesis other than homogeneity of variance when treatments are unequally replicated, but the incorrect formulation is now standard output in several statistical packages. This paper develops a weighted least squares analysis of variance of the absolute values of both mean-based and median-based residuals. It shows how to adjust the residuals so that tests using the F -statistic focus on homogeneity of variance for both balanced and unbalanced designs. It shows how to modify the F -statistics currently produced by statistical packages so that the distribution of the resultant test statistic is closer to an F-distribution than is currently the case. The weighted least squares approach also produces component mean squares that are unbiased irrespective of which variable is used in Levene's test. To complete this aspect of the investigation the paper derives exact second-order moments of the component sums of squares used in the calculation of the mean-based test statistic. It shows that, for large samples, both ordinary and weighted least squares test statistics are equivalent; however they are over-dispersed compared to an F variable.     

Parameter estimation in semi-linear models using a maximal invariant likelihood function

In this paper, we consider the problem of estimation of semi-linear regression models. Using invariance arguments, Bhowmik and King [2007. Maximal invariant likelihood based testing of semi-linear models. Statist. Papers 48, 357–383] derived the probability density function of the maximal invariant statistic for the non-linear component of these models. Using this density function as a likelihood function allows us to estimate these models in a two-step process. First the non-linear component parameters are estimated by maximising the maximal invariant likelihood function. Then the non-linear component, with the parameter values replaced by estimates, is treated as a regressor and ordinary least squares is used to estimate the remaining parameters. We report the results of a simulation study conducted to compare the accuracy of this approach with full maximum likelihood and maximum profile-marginal likelihood estimation. We find maximising the maximal invariant likelihood function typically results in less biased and lower variance estimates than those from full maximum likelihood.     

空间回归模型估计中的最小二乘法

空间回归模型由于引入了空间地理信息而使得其参数估计变得复杂,因为主要采用最大似然法,致使一般人认为在空间回归模型参数估计中不存在最小二乘法。通过分析空间回归模型的参数估计技术,研究发现,最小二乘法和最大似然法分别用于估计空间回归模型的不同的参数,只有将两者结合起来才能快速有效地完成全部的参数估计。数理论证结果表明,空间回归模型参数最小二乘估计量是最佳线性无偏估计量。空间回归模型的回归参数可以在估计量为正态性的条件下而实施显著性检验,而空间效应参数则不可以用此方法进行检验。     

Some Recent Developments in Econometric Inference

Abstract

Recent results in information theory, see Soofi (1996; 2001) for a review, include derivations of optimal information processing rules, including Bayes' theorem, for learning from data based on minimizing a criterion functional, namely output information minus input information as shown in Zellner (1988; 1991; 1997; 2002). Herein, solution post data densities for parameters are obtained and studied for cases in which the input information is that in (1) a likelihood function and a prior density; (2) only a likelihood function; and (3) neither a prior nor a likelihood function but only input information in the form of post data moments of parameters, as in the Bayesian method of moments approach. Then it is shown how optimal output densities can be employed to obtain predictive densities and optimal, finite sample structural coefficient estimates using three alternative loss functions. Such optimal estimates are compared with usual estimates, e.g., maximum likelihood, two‐stage least squares, ordinary least squares, etc. Some Monte Carlo experimental results in the literature are discussed and implications for the future are provided.     

Comparison of methods of estimation of parameters of the weibull distribution

The generalised least squares, maximum likelihood, Bain-Antle 1 and 2, and two mixed methods of estimating the parameters of the two-parameter Weibull distribution are compared. The comparison is made using (a) the observed relative efficiency of parameter estimates and (b) themean squared relative error in estimated quantiles, to summarize the results of 1000 simulated samples of sizes 10 and 25. The results are that: generalised least squares is the best method of estimating the shape parameter ß the best method of estimating the scale parameter a depends onthe size of ß for quantile estimation maximum likelihood is best Bain-Antle 2 is uniformly the worst of the methods.     

ASYMPTOTIC PROPERTIES OF RESTRICTED MAXIMUM LIKELIHOOD (REML) ESTIMATES FOR HIERARCHICAL MIXED LINEAR MODELS

This paper explores the asymptotic distribution of the restricted maximum likelihood estimator of the variance components in a general mixed model. Restricting attention to hierarchical models, central limit theorems are obtained using elementary arguments with only mild conditions on the covariates in the fixed part of the model and without having to assume that the data are either normally or spherically symmetrically distributed. Further, the REML and maximum likelihood estimators are shown to be asymptotically equivalent in this general framework, and the asymptotic distribution of the weighted least squares estimator (based on the REML estimator) of the fixed effect parameters is derived.     

Adaptive likelihood estimator of conditional variance function

Modelling volatility in the form of conditional variance function has been a popular method mainly due to its application in financial risk management. Among others, we distinguish the parametric GARCH models and the nonparametric local polynomial approximation using weighted least squares or gaussian likelihood function. We introduce an alternative likelihood estimate of conditional variance and we show that substitution of the error density with its estimate yields similar asymptotic properties, that is, the proposed estimate is adaptive to the error distribution. Theoretical comparison with existing estimates reveals substantial gains in efficiency, especially if error distribution has fatter tails than Gaussian distribution. Simulated data confirm the theoretical findings while an empirical example demonstrates the gains of the proposed estimate.     

On the bias of factor score determinacy coefficients based on different estimation methods of the exploratory factor model

The factor score determinacy coefficient represents the common variance of the factor score predictor with the corresponding factor. The aim of the present simulation study was to compare the bias of determinacy coefficients based on different estimation methods of the exploratory factor model. Overall, determinacy coefficients computed from parameters based on maximum likelihood estimation, unweighted least squares estimation, and principal axis factoring were more precise than determinacy coefficients based on generalized least squares estimation and alpha factoring.     

Efficient estimation of the PDF and the CDF of the inverse Rayleigh distribution

In this paper, we consider the estimation of the probability density function and the cumulative distribution function of the inverse Rayleigh distribution. In this regard, the following estimators are considered: uniformly minimum variance unbiased estimator, maximum likelihood (ML) estimator, percentile estimator, least squares estimator and weighted least squares estimator. To do so, analytical expressions are derived for the mean integrated squared error. As the result of simulation studies and real data applications indicate, when the sample size is not very small the ML estimator performs better than the others.