Estimation in a linear regression model under the Kullback–Leibler loss and its application to model selection

This paper is concerned with the problem of constructing a good predictive distribution relative to the Kullback–Leibler information in a linear regression model. The problem is equivalent to the simultaneous estimation of regression coefficients and error variance in terms of a complicated risk, which yields a new challenging issue in a decision-theoretic framework. An estimator of the variance is incorporated here into a loss for estimating the regression coefficients. Several estimators of the variance and of the regression coefficients are proposed and shown to improve on usual benchmark estimators both analytically and numerically. Finally, the prediction problem of a distribution is noted to be related to an information criterion for model selection like the Akaike information criterion (AIC). Thus, several AIC variants are obtained based on proposed and improved estimators and are compared numerically with AIC as model selection procedures.     

Comparison of procedures for estimation of error rates in discriminant analysis under nonnormal population

The parametric and nonparametric methods for estimating the error rates in linear discriminant analysis are examined both in normal and in nonnormal situations. A Monte Carlo experiment was carried out under the assumption that two population distributions were characterized by a mixture of two multivariate normal distributions. The bootstrap bias-corrected apparent error rate compares favourably to other available estimators for nonnormal populations with small Mahalanobis distance. The methods for error estimation are also applied to a practical problem in medical diagnosis     

A comparison of parametric conditional error-rate estimators for the two-group linear discriminant function

Using mean absolute deviation, we compare the efficay of two new parametric conditional error rate estimators with six others, four of which are well known.The performance of both new estimators is found to be superior to the six competing estimators examined in this paper, especially when the ratio of the training sample size to the feature dimensionality is small.     

Gamma mixture of generalized error distribution

Abstract

A new symmetric heavy-tailed distribution, namely gamma mixture of generalized error distribution is defined by scaling generalized error distribution with gamma distribution, its probability density function, k-moment, skewness and kurtosis are derived. After tedious calculation, we also give the Fisher information matrix, moment estimators and maximum likelihood estimators for the parameters of gamma mixture of generalized error distribution. In order to evaluate the effectiveness of the point estimators and the stability of Fisher information matrix, extensive simulation experiments are carried out in three groups of parameters. Additionally, the new distribution is applied to Apple Inc. stock (AAPL) data and compared with normal distribution, F-S skewed standardized t distribution and generalized error distribution. It is found that the new distribution has better fitting effect on the data under the Akaike information criterion (AIC). To a certain extent, our results enrich the probability distribution theory and develop the scale mixture distribution, which will provide help and reference for financial data analysis.     

On preliminary test ridge regression estimators for linear restrictions in a regression model with non-normal disturbances

In this paper, we study the properties of the preliminary test, restricted and unrestricted ridge regression estimators of the linear regression model with non-normal disturbances. We present the estimators of the regression coefficients combining the idea of preliminary test and ridge regression methodology, when it is suspected that the regression coefficients may be restricted to a subspace and the regression error is distributed as multivariate t. Accordingly we consider three estimators, namely the Unrestricted Ridge Regression Estimator (URRRE), the Restricted Ridge Regression Estimator (RRRE) and finally the Preliminary test Ridge Regression Estimator (PTRRE). The biases and the mean square error (MSE) of the estimators are derived under the null and alternative hypotheses and compared with the usual estimators. By studying the MSE criterion, the regions of optimahty of the estimators are determined.     

Combining the unrestricted estimators into a single estimator and a simulation study on the unrestricted estimators

The purpose of this paper is to combine several regression estimators (ordinary least squares (OLS), ridge, contraction, principal components regression (PCR), Liu, r?k and r?d class estimators) into a single estimator. The conditions for the superiority of this new estimator over the PCR, the r?k class, the r?d class, β?(k, d), OLS, ridge, Liu and contraction estimators are derived by the scalar mean square error criterion and the estimators of the biasing parameters for this new estimator are examined. Also, a numerical example based on Hald data and a simulation study are used to illustrate the results.     

A Combined Nonlinear Programming Model and Kibria Method for Choosing Ridge Parameter Regression

Ridge regression solves multicollinearity problems by introducing a biasing parameter that is called ridge parameter; it shrinks the estimates as well as their standard errors in order to reach acceptable results. Many methods are available for estimating a ridge parameter. This article has considered some of these methods and also proposed a combined nonlinear programming model and Kibria method. A simulation study has been made to evaluate the performance of the proposed estimators based on the minimum mean squared error criterion. The simulation study indicates that under certain conditions the proposed estimators outperform the least squares (LS) estimators and other popular existing estimators. Moreover, the new proposed model is applied on dataset that suffers also from the presence of heteroscedastic errors.     

A new estimator combining the ridge regression and the restricted least squares methods of estimation

It is well-known in the literature on multicollinearity that one of the major consequences of multicollinearity on the ordinary least squares estimator is that the estimator produces large sampling variances, which in turn might inappropriately lead to exclusion of otherwise significant coefficients from the model. To circumvent this problem, two accepted estimation procedures which are often suggested are the restricted least squares method and the ridge regression method. While the former leads to a reduction in the sampling variance of the estimator, the later ensures a smaller mean square error value for the estimator. In this paper we have proposed a new estimator which is based on a criterion that combines the ideas underlying these two estimators. The standard properties of this new estimator have been studied in the paper. It has also been shown that this estimator is superior to both the restricted least squares as well as the ordinary ridge regression estimators by the criterion of mean sauare error of the estimator of the regression coefficients when the restrictions are indeed correct. The conditions for superiority of this estimator over the other two have also been derived for the situation when the restrictions are not correct.     

On shrinkage estimation of the exponential location parameter

Under the assumption that the exponential distribution is a reasonable model for a given population, some shrinkage estimators for the location parameter based on type 1 and type II censored samples have been derived. It is shown that these estimators dominate maximum likelihood estimators (MLE's) asymptotically under the mean squared error (MSE) criterion. A Monte Carlo study shows a significant improvement of our estimators over MLE's in terms of MSE for small samples.     

MISSING VALUES, IMPUTATION AND ERROR RATE ESTIMATORS IN DISCRIMINANT ANALYSIS

Error rate is a popular criterion for assessing the performance of an allocation rule in discriminant analysis. Training samples which involve missing values cause problems for those error rate estimators that require all variables to be observed at all data points. This paper explores imputation algorithms, their effects on, and problems of implementing them with, eight commonly used error rate estimators (three parametric and five non-parametric) in linear discriminant analysis. The results indicate that imputation should not be based on the way error rate estimators are calculated, and that imputed values may underestimate error rates.     

Estimating Mark Functions Through Spectral Analysis for Marked Point Patterns

Classical techniques for modeling numerical data associated to a regular grid have been widely developed in the literature. When a trigonometric model for the data is considered, it is possible to use the corresponding least squares (classical) estimators, but when the data are not observed on a regular grid, these estimators do not show appropriate properties. In this article we propose a novel way to model data that is not observed on a regular grid, and we establish a practical criterion, based on the mean squared error (MSE), to objectively decide which estimator should be used in each case: the inappropriate classical or the new unbiased estimator, which has greater variance. Jackknife and cross-validation techniques are used to follow a similar criterion in practice, when the MSE is not known. Finally, we present an application of the methodology to univariate and bivariate data.     

A class of Shrunken estimators for normal mean

This paper presents a class of estimators for the mean of a normal population and determines the conditions on characterizing scalars under which the class of estimators uniformly dominates over the conventional sample mean according to the mean-square-error criterion.     

Improved two-parameter estimators for the negative binomial and Poisson regression models

Negative binomial regression (NBR) and Poisson regression (PR) applications have become very popular in the analysis of count data in recent years. However, if there is a high degree of relationship between the independent variables, the problem of multicollinearity arises in these models. We introduce new two-parameter estimators (TPEs) for the NBR and the PR models by unifying the two-parameter estimator (TPE) of Özkale and Kaç?ranlar [The restricted and unrestricted two-parameter estimators. Commun Stat Theory Methods. 2007;36:2707–2725]. These new estimators are general estimators which include maximum likelihood (ML) estimator, ridge estimator (RE), Liu estimator (LE) and contraction estimator (CE) as special cases. Furthermore, biasing parameters of these estimators are given and a Monte Carlo simulation is done to evaluate the performance of these estimators using mean square error (MSE) criterion. The benefits of the new TPEs are also illustrated in an empirical application. The results show that the new proposed TPEs for the NBR and the PR models are better than the ML estimator, the RE and the LE.     

Empirical likelihood and general relative error criterion with divergent dimension

In estimation of multiplicative or accelerated failure time models, the relative error criterion has been recognized as an alternative to the squared or absolute error criterion. The general relative error criterion introduced by Chen et al. [Least product relative error estimation. J Multivariate Anal. 2016;144:91–98] is a unified framework for efficient estimation, which includes the least absolute relative error estimation and least product relative error estimation as special cases. In this paper, by combining the empirical likelihood and general relative error criterion in multiplicative model, we develop a new empirical likelihood method for inference on the unknown parameters under high-dimensional setting. Limiting theory is established for the proposed empirical likelihood statistic. We conduct some simulation studies and real data analysis to evaluate the effectiveness of the proposed method.     

Robust linear discriminant analysis using S‐estimators

The authors consider a robust linear discriminant function based on high breakdown location and covariance matrix estimators. They derive influence functions for the estimators of the parameters of the discriminant function and for the associated classification error. The most B‐robust estimator is determined within the class of multivariate S‐estimators. This estimator, which minimizes the maximal influence that an outlier can have on the classification error, is also the most B‐robust location S‐estimator. A comparison of the most B‐robust estimator with the more familiar biweight S‐estimator is made.     

Weighted nonparametric regression

In the fixed design regression model, additional weights are considered for the Nad a ray a-Watson and Gasser-Miiller kernel estimators. We study their asymptotic behavior and the relationships between new and classical estimators. For a simple family of weights, and considering the AIMSEAS global loss criterion, we show some possible theoretical advantages. An empirical study illustrates the performance of the weighted kernel estimators in theoretical ideal situations and in simulated data sets. Also some results concerning the use of weights for local polynomial estimators are given.     

Modified and Restricted r-k Class Estimators

In this article, we introduce the modified r-k class estimator and the restricted r-k class estimator. We compare the performances of the new estimators to the r-k class estimator with respect to the matrix mean square error (MSE) criterion. As a special case of the restricted r-k class estimator, we obtain the restricted principal components regression (RPCR) estimator. Finally, we conduct a Monte Carlo simulation study and a numerical example to investigate the performances of the proposed estimators by the scalar mean square error (mse) criterion.     

Estimation of two ordered normal means when a covariance matrix is known

Estimation of two normal means with an order restriction is considered when a covariance matrix is known. It is shown that restricted maximum likelihood estimator (MLE) stochastically dominates both estimators proposed by Hwang and Peddada [Confidence interval estimation subject to order restrictions. Ann Statist. 1994;22(1):67–93] and Peddada et al. [Estimation of order-restricted means from correlated data. Biometrika. 2005;92:703–715]. The estimators are also compared under the Pitman nearness criterion and it is shown that the MLE is closer to ordered means than the other two estimators. Estimation of linear functions of ordered means is also considered and a necessary and sufficient condition on the coefficients is given for the MLE to dominate the other estimators in terms of mean squared error.     

Modified ratio estimators using robust regression methods

When there is an outlier in the data set, the efficiency of traditional methods decreases. In order to solve this problem, Kadilar et al. (2007) adapted Huber-M method which is only one of robust regression methods to ratio-type estimators and decreased the effect of outlier problem. In this study, new ratio-type estimators are proposed by considering Tukey-M, Hampel M, Huber MM, LTS, LMS and LAD robust methods based on the Kadilar et al. (2007). Theoretically, we obtain the mean square error (MSE) for these estimators. We compared with MSE values of proposed estimators and MSE values of estimators based on Huber-M and OLS methods. As a result of these comparisons, we observed that our proposed estimators give more efficient results than both Huber M approach which was proposed by Kadilar et al. (2007) and OLS approach. Also, under all conditions, all of the other proposed estimators except Lad method are more efficient than robust estimators proposed by Kadilar et al. (2007). And, these theoretical results are supported with the aid of a numerical example and simulation by basing on data that includes an outlier.     

Bayesian and maximum likelihood estimations of the inverse Weibull parameters under progressive type-II censoring

In this paper, the statistical inference of the unknown parameters of a two-parameter inverse Weibull (IW) distribution based on the progressive type-II censored sample has been considered. The maximum likelihood estimators (MLEs) cannot be obtained in explicit forms, hence the approximate MLEs are proposed, which are in explicit forms. The Bayes and generalized Bayes estimators for the IW parameters and the reliability function based on the squared error and Linex loss functions are provided. The Bayes and generalized Bayes estimators cannot be obtained explicitly, hence Lindley's approximation is used to obtain the Bayes and generalized Bayes estimators. Furthermore, the highest posterior density credible intervals of the unknown parameters based on Gibbs sampling technique are computed, and using an optimality criterion the optimal censoring scheme has been suggested. Simulation experiments are performed to see the effectiveness of the different estimators. Finally, two data sets have been analysed for illustrative purposes.