Efficiency analysis of ten estimation procedures for quantitative linear models with autocorrelated errors

Many estimation procedures for quantitative linear models with autocorrelated errors have been proposed in the literature. A number of these procedures have been compared in various ways for different sample sizes and autocorrelation parameters values and for structured or random explanatory vaiables. In this paper, we revisit three situations that were considered to some extent in previous studies, by comparing ten estimation procedures: Ordinary Least Squares (OLS), Generalized Least Squares (GLS), estimated Generalized Least Squares (six procedures), Maximum Likelihood (ML), and First Differences (FD). The six estimated GLS procedures and the ML procedure differ in the way the error autocovariance matrix is estimated. The three situations can be defined as follows: Case 1, the explanatory variable x in the simple linear regression is fixed; Case 2,x is purely random; and Case 3x is first-order autoregressive. Following a theoretical presentation, the ten estimation procedures are compared in a Monte Carlo study conducted in the time domain, where the errors are first-order autoregressive in Cases 1-3. The measure of comparison for the estimation procedures is their efficiency relative to OLS. It is evaluated as a function of the time series length and the magnitude and sign of the error autocorrelation parameter. Overall, knowledge of the model of the time series process generating the errors enhances efficiency in estimated GLS. Differences in the efficiency of estimation procedures between Case 1 and Cases 2 and 3 as well as differences in efficiency among procedures in a given situation are observed and discussed.     

Bayesian inference for a common scale parameter of several Pareto populations

This article deals with the problem of Bayesian inference concerning the common scale parameter of several Pareto distributions. Bayesian hypothesis testing of, and Bayesian interval estimation for, the common scale parameter is given. Numerical studies including a comparison study, a simulation study, and a practical application study are given in order to illustrate our procedures and to demonstrate the performance, advantages, and merits of the Bayesian procedures over the classical and generalized variable procedures.     

Inference on effect size indices from several two-armed experiments

The statistical inference problem on effect size indices is addressed using a series of independent two-armed experiments from k arbitrary populations. The effect size parameter simply quantifies the difference between two groups. It is a meaningful index to be used when data are measured on different scales. In the context of bivariate statistical models, we define estimators of the effect size indices and propose large sample testing procedures to test the homogeneity of these indices. The null and non-null distributions of the proposed testing procedures are derived and their performance is evaluated via Monte Carlo simulation. Further, three types of interval estimation of the proposed indices are considered for both combined and uncombined data. Lower and upper confidence limits for the actual effect size indices are obtained and compared via bootstrapping. It is found that the length of the intervals based on the combined effect size estimator are almost half the length of the intervals based on the uncombined effect size estimators. Finally, we illustrate the proposed procedures for hypothesis testing and interval estimation using a real data set.     

Model selection and post estimation based on a pretest for logistic regression models

ABSTRACT

This article addresses the problem of parameter estimation of the logistic regression model under subspace information via linear shrinkage, pretest, and shrinkage pretest estimators along with the traditional unrestricted maximum likelihood estimator and restricted estimator. We developed an asymptotic theory for the linear shrinkage and pretest estimators and compared their relative performance using the notion of asymptotic distributional bias and asymptotic quadratic risk. The analytical results demonstrated that the proposed estimation strategies outperformed the classical estimation strategies in a meaningful parameter space. Detailed Monte-Carlo simulation studies were conducted for different combinations and the performance of each estimation method was evaluated in terms of simulated relative efficiency. The results of the simulation study were in strong agreement with the asymptotic analytical findings. Two real-data examples are also given to appraise the performance of the estimators.     

Modeling statistical dependence of Markov chains via copula models

Conditional probability distributions have been commonly used in modeling Markov chains. In this paper we consider an alternative approach based on copulas to investigate Markov-type dependence structures. Based on the realization of a single Markov chain, we estimate the parameters using one- and two-stage estimation procedures. We derive asymptotic properties of the marginal and copula parameter estimators and compare performance of the estimation procedures based on Monte Carlo simulations. At low and moderate dependence structures the two-stage estimation has comparable performance as the maximum likelihood estimation. In addition we propose a parametric pseudo-likelihood ratio test for copula model selection under the two-stage procedure. We apply the proposed methods to an environmental data set.     

The E-Bayesian and hierarchical Bayesian estimations of Pareto distribution parameter under different loss functions

In this paper, we study the E-Bayesian and hierarchical Bayesian estimations of the parameter derived from Pareto distribution under different loss functions. The definition of the E-Bayesian estimation of the parameter is provided. Moreover, for Pareto distribution, under the condition of the scale parameter is known, based on the different loss functions, formulas of the E-Bayesian estimation and hierarchical Bayesian estimations for the shape parameter are given, respectively, properties of the E-Bayesian estimation – (i) the relationship between of E-Bayesian estimations under different loss functions are provided, (ii) the relationship between of E-Bayesian and hierarchical Bayesian estimations under the same loss function are also provided, and using the Monte Carlo method simulation example is given. Finally, combined with the golfers income data practical problem are calculated, the results show that the proposed method is feasible and convenient for application.     

Stabilizing the Performance of Kurtosis Estimator of Multivariate Data

The estimation of the kurtosis parameter of the underlying distribution plays a central role in many statistical applications. The central theme of the article is to improve the estimation of the kurtosis parameter using a priori information. More specifically, we consider the problem of estimating kurtosis parameter of a multivariate population when some prior information regarding the the parameter is available. The rationale is that the sample estimator of the kurtosis parameter has a large estimation error. In this situation we consider shrinkage and pretest estimation methodologies and reappraise their statistical properties. The estimation based on these strategies yield relatively smaller estimation error in comparison with the sample estimator in the candidate subspace. A large sample theory of the suggested estimators are developed and compared. The results demonstrate that suggested estimators outperform the estimator based on the sample data only in the candidate subspace. In an effort to appreciate the relative behavior of the estimators in a finite sample scenario, a Monte-carlo simulation study is planned and performed. The result of simulation study strongly corroborates the asymptotic result. To illustrate the application of the estimators, some example are showcased based on recently published data.     

The Cox Proportional Hazards Model with a Partly Linear Relative Risk Function

The conventional Cox proportional hazards regression model contains a loglinear relative risk function, linking the covariate information to the hazard ratio with a finite number of parameters. A generalization, termed the partly linear Cox model, allows for both finite dimensional parameters and an infinite dimensional parameter in the relative risk function, providing a more robust specification of the relative risk function. In this work, a likelihood based inference procedure is developed for the finite dimensional parameters of the partly linear Cox model. To alleviate the problems associated with a likelihood approach in the presence of an infinite dimensional parameter, the relative risk is reparameterized such that the finite dimensional parameters of interest are orthogonal to the infinite dimensional parameter. Inference on the finite dimensional parameters is accomplished through maximization of the profile partial likelihood, profiling out the infinite dimensional nuisance parameter using a kernel function. The asymptotic distribution theory for the maximum profile partial likelihood estimate is established. It is determined that this estimate is asymptotically efficient; the orthogonal reparameterization enables employment of profile likelihood inference procedures without adjustment for estimation of the nuisance parameter. An example from a retrospective analysis in cancer demonstrates the methodology.     

Model selection: some generalized distributions

Many models have been used to represent the distributions of random variables in statistics, engineering, business, and the physical and social science. This paper considers two, four-parameter generalized bea distributions that include nearly all the models actually used as special or limiting cases. Properties and the interrelationships among these distributions are considered. Expressions are reported that facilitate parameter estimation and the analysis of associated means, variances, hazard functions and other distributional characteristics.

Estimation procedures corresponding to different data types are considered. Maximum likelihood estimation is used and the value of the likelihood function provides and important criterion for model selection. The relative performance of the various models is compared for several data sets.     

Shrinkage and pretest estimators for longitudinal data analysis under partially linear models

In this paper, we develop marginal analysis methods for longitudinal data under partially linear models. We employ the pretest and shrinkage estimation procedures to estimate the mean response parameters as well as the association parameters, which may be subject to certain restrictions. We provide the analytic expressions for the asymptotic biases and risks of the proposed estimators, and investigate their relative performance to the unrestricted semiparametric least-squares estimator (USLSE). We show that if the dimension of association parameters exceeds two, the risk of the shrinkage estimators is strictly less than that of the USLSE in most of the parameter space. On the other hand, the risk of the pretest estimator depends on the validity of the restrictions of association parameters. A simulation study is conducted to evaluate the performance of the proposed estimators relative to that of the USLSE. A real data example is applied to illustrate the practical usefulness of the proposed estimation procedures.     

Probability Generating Function Based Jeffrey's Divergence for Statistical Inference

Statistical inference procedures based on transforms such as characteristic function and probability generating function have been examined by many researchers because they are much simpler than probability density functions. Here, a probability generating function based Jeffrey's divergence measure is proposed for parameter estimation and goodness-of-fit test. Being a member of the M-estimators, the proposed estimator is consistent. Also, the proposed goodness-of-fit test has good statistical power. The proposed divergence measure shows improved performance over existing probability generating function based measures. Real data examples are given to illustrate the proposed parameter estimation method and goodness-of-fit test.     

Penalized models to estimate customer survival

In this paper we propose a novel procedure, for the estimation of semiparametric survival functions. The proposed technique adapts penalized likelihood survival models to the context of lifetime value modeling. The method extends classical Cox model by introducing a smoothing parameter that can be estimated by means of penalized maximum likelihood procedures. Markov Chain Monte Carlo methods are employed to effectively estimate such smoothing parameter, using an algorithm which combines Metropolis–Hastings and Gibbs sampling. Our proposal is contextualized and compared with conventional models, with reference to a marketing application that involves the prediction of customer’s lifetime value estimation.     

Strong consistency in nonlinear regression with multipucative error

Sufficient conditions are given to ensure the existence of a sequence of strongly consistent estimators of the true parameter for a nonlinear regression model naving random regressors and a multiplicative disturbance term. Special cases of this result include the least absolute value and the least squares estimation procedures.     

On influence diagnostics in elliptical multivariate regression models with equicorrelated random errors

In this paper we discuss estimation and diagnostic procedures in elliptical multivariate regression models with equicorrelated random errors. Two procedures are proposed for the parameter estimation and the local influence curvatures are derived under some usual perturbation schemes to assess the sensitivity of the maximum likelihood estimates (MLEs). Two motivating examples preliminarily analyzed under normal errors are reanalyzed considering appropriate elliptical distributions. The local influence approach is used to compare the sensitivity of the model estimates.     

Modified weighted squared error estimation procedures with special emphasis on the stable laws

Two families of parameter estimation procedures for the stable laws based on a variant of the characteristic function are provided. The methodology which produces viable computational procedures for the stable laws is generally applicable to other families of distributions across a variety of settings. Both families of procedures may be described as a modified weighted chi-squared minimization procedure, and both explicitly take account of constraints on the parameter space. Influence func-tions for and efficiencies of the estimators are given. If x₁, x₂, …x_n random sample from an unknown distribution F , a method for determining the stable law to which F is attracted is developed. Procedures for regression and autoregres-sion with stable error structure are provided. A number of examples are given.     

Expressing Estimators of Expected Quality Adjusted Survival as Functions of Nelson-Aalen Estimators

Quality adjusted survival has been increasingly advocated in clinical trials to be assessed as a synthesis of survival and quality of life. We investigate nonparametric estimation of its expectation for a general multistate process with incomplete follow-up data. Upon establishing a representation of expected quality adjusted survival through marginal distributions of a set of defined events, we propose two estimators for expected quality adjusted survival. Expressed as functions of Nelson-Aalen estimators, the two estimators are strongly consistent and asymptotically normal. We derive their asymptotic variances and propose sample-based variance estimates, along with evaluation of asymptotic relative efficiency. Monte Carlo studies show that these estimation procedures perform well for practical sample sizes. We illustrate the methods using data from a national, multicenter AIDS clinical trial.     

Focused and Model Average Estimation for Regression Analysis of Panel Count Data

Panel count data arise in many fields and a number of estimation procedures have been developed along with two procedures for variable selection. In this paper, we discuss model selection and parameter estimation together. For the former, a focused information criterion (FIC) is presented and for the latter, a frequentist model average (FMA) estimation procedure is developed. A main advantage, also the difference from the existing model selection methods, of the FIC is that it emphasizes the accuracy of the estimation of the parameters of interest, rather than all parameters. Further efficiency gain can be achieved by the FMA estimation procedure as unlike existing methods, it takes into account the variability in the stage of model selection. Asymptotic properties of the proposed estimators are established, and a simulation study conducted suggests that the proposed methods work well for practical situations. An illustrative example is also provided. © 2014 Board of the Foundation of the Scandinavian Journal of Statistics     

Efficient Robust Estimation for Linear Models with Missing Response at Random

Coefficient estimation in linear regression models with missing data is routinely carried out in the mean regression framework. However, the mean regression theory breaks down if the error variance is infinite. In addition, correct specification of the likelihood function for existing imputation approach is often challenging in practice, especially for skewed data. In this paper, we develop a novel composite quantile regression and a weighted quantile average estimation procedure for parameter estimation in linear regression models when some responses are missing at random. Instead of imputing the missing response by randomly drawing from its conditional distribution, we propose to impute both missing and observed responses by their estimated conditional quantiles given the observed data and to use the parametrically estimated propensity scores to weigh check functions that define a regression parameter. Both estimation procedures are resistant to heavy‐tailed errors or outliers in the response and can achieve nice robustness and efficiency. Moreover, we propose adaptive penalization methods to simultaneously select significant variables and estimate unknown parameters. Asymptotic properties of the proposed estimators are carefully investigated. An efficient algorithm is developed for fast implementation of the proposed methodologies. We also discuss a model selection criterion, which is based on an IC_Q ‐type statistic, to select the penalty parameters. The performance of the proposed methods is illustrated via simulated and real data sets.     

Experience with a bayesian bootstrap method incorporating proper prior information

Although bootstrapping has become widely used in statistical analysis, there has been little reported concerning bootstrapped Bayesian analyses, especially when there is proper prior informa-tion concerning the parameter of interest. In this paper, we first propose an operationally implementable definition of a Bayesian bootstrap. Thereafter, in simulated studies of the estimation of means and variances, this Bayesian bootstrap is compared to various parametric procedures. It turns out that little information is lost in using the Bayesian bootstrap even when the sampling distribution is known. On the other hand, the parametric procedures are at times very sensitive to incorrectly specified sampling distributions, implying that the Bayesian bootstrap is a very robust procedure for determining the posterior distribution of the parameter.     

Regularized robust estimation in binary regression models

In this paper, we investigate robust parameter estimation and variable selection for binary regression models with grouped data. We investigate estimation procedures based on the minimum-distance approach. In particular, we employ minimum Hellinger and minimum symmetric chi-squared distances criteria and propose regularized minimum-distance estimators. These estimators appear to possess a certain degree of automatic robustness against model misspecification and/or for potential outliers. We show that the proposed non-penalized and penalized minimum-distance estimators are efficient under the model and simultaneously have excellent robustness properties. We study their asymptotic properties such as consistency, asymptotic normality and oracle properties. Using Monte Carlo studies, we examine the small-sample and robustness properties of the proposed estimators and compare them with traditional likelihood estimators. We also study two real-data applications to illustrate our methods. The numerical studies indicate the satisfactory finite-sample performance of our procedures.