Admissible Linear Predictors in the Superpopulation Model with Respect to Inequality Constraints

Admissibility of linear predictors of the linear quantity Qy is investigated under a general linear regression superpopulation model with some inequality constraints. The relation between admissible homogeneous and inhomogeneous linear predictors is characterized. Further, necessary and sufficient conditions for a linear predictor to be admissible in two cases of inequality constraints in the classes of homogeneous and inhomogeneous linear predictors are given, respectively.     

Admissibility of Linear Predictors in the Superpopulation Model with Respect to Inequality Constraints under Matrix Loss Function

Admissibility of linear predictors for the linear quantity Qy is investigated in a superpopulation model with respect to some inequality constraints. Necessary and sufficient conditions for a linear predictor to be admissible in the class of homogeneous linear predictors and the class of inhomogeneous linear predictors are obtained, respectively, under matrix loss function.     

Stable prediction in high-dimensional linear models

We propose a Random Splitting Model Averaging procedure, RSMA, to achieve stable predictions in high-dimensional linear models. The idea is to use split training data to construct and estimate candidate models and use test data to form a second-level data. The second-level data is used to estimate optimal weights for candidate models by quadratic optimization under non-negative constraints. This procedure has three appealing features: (1) RSMA avoids model overfitting, as a result, gives improved prediction accuracy. (2) By adaptively choosing optimal weights, we obtain more stable predictions via averaging over several candidate models. (3) Based on RSMA, a weighted importance index is proposed to rank the predictors to discriminate relevant predictors from irrelevant ones. Simulation studies and a real data analysis demonstrate that RSMA procedure has excellent predictive performance and the associated weighted importance index could well rank the predictors.     

A Further Study of Predictions in Linear Mixed Models

This article is concerned with the prediction problems in linear mixed models (LMM). Both biased predictors and restricted predictors are introduced. It was found that the mean square error matrix (MSEM) of a predictor strongly depends on the MSEM of corresponding estimator of the fixed effects and precise formulas are obtained. As an application, we propose three new predictors to improve the best linear unbiased predictor (BLUP). The performance of the new predictors can be examined easily with the help of vast literature on the linear regression models (LM). We also illustrate our findings with a Monte Carlo simulation and a numerical example.     

Pitman closeness for hierarchical bayes predictors in mixed linear models

The present article considers the Pitman Closeness (PC) criterion of certain hierarchical Bayes (HB) predictors derived under a normal mixed linear models for known ratios of variance components using a uniform prior for the vector of fixed effects and some proper or improper prior on the error variance. For a generalized Euclidean error, simultaneous HB predictors of several linear combinations of vector of effects are shown to be the Pitman-closest in the frequentist sense in the class of equivariant predictors for location group of transformations. The normality assumption can be relaxed to show that these HB predictors are the Pitman-closest for location-scale group of transformations for a wider family of elliptically symmetric distributions. Also for this family, the HB predictors turn out to be Pitman-closest in the class of all linear unbiased predictors (LUPs). All these results are extended for the HB predictor of finite population mean vector in the context of finite population sampling.     

Nonparametric Decomposition of Time Series Data with Inputs

The backfitting algorithm commonly used in estimating additive models is used to decompose the component shares explained by a set of predictors on a dependent variable in the presence of linear dependencies (multicollinearity) among the predictors. Simulated and actual data show that the backfitting methods are superior in terms of predictive ability as the degree of multicollinearity worsens. Furthermore, the additive smoothing splines are especially superior when the linear model yield inadequate fit to the data and the predictors exhibit extreme multicollinearity.     

Linear predictors of future order statistics

In this paper we establish an optimal asymptotic linear predictor which does not involve the finite-sample variance-covariance structure. Extensions to the problem of finding the best linear unbiased and simple linear unbiased predictors for k samples are given. Moreover, we obtain alternative linear predictors by modifying the covariance matrix by either an identity matrix or a diagonal matrix. For normal, logistic and Rayleigh samples of size 10, the alternative linear predictors with these modifications have high efficiency when compared with the best linear unbiased predictor.     

Improvement of mixed predictors in linear mixed models

In this paper, we introduce stochastic-restricted Liu predictors which will be defined by combining in a special way the two approaches followed in obtaining the mixed predictors and the Liu predictors in the linear mixed models. Superiorities of the linear combination of the new predictor to the Liu and mixed predictors are done in the sense of mean square error matrix criterion. Finally, numerical examples and a simulation study are done to illustrate the findings. In numerical examples, we took some arbitrary observations from the data as the prior information since we did not have historical data or additional information about the data sets. The results show that this case does the new estimator gain efficiency over the constituent estimators and provide accurate estimation and prediction of the data.     

On Optimal Point and Block Prediction in Log-Gaussian Random Fields

Abstract.  This work discusses the problems of point and block prediction in log-Gaussian random fields with unknown mean. New point and block predictors are derived that are optimal in mean squared error sense within certain families of predictors that contain the corresponding lognormal kriging point and block predictors, as well as a block predictor originally motivated under the assumption of 'preservation of lognormality', and hence improve upon them. A comparison between the optimal, lognormal kriging and best linear unbiased predictors is provided, as well as between the two new block predictors. Somewhat surprisingly, it is shown that the corresponding optimal and lognormal kriging predictors are almost identical under most scenarios. It is also shown that one of the new block predictors is uniformly better than the other.     

Adaptation of the jackknifed ridge methods to the linear mixed models

The purpose of this article is to obtain the jackknifed ridge predictors in the linear mixed models and to examine the superiorities, the linear combinations of the jackknifed ridge predictors over the ridge, principal components regression, r?k class and Henderson's predictors in terms of bias, covariance matrix and mean square error criteria. Numerical analyses are considered to illustrate the findings and a simulation study is conducted to see the performance of the jackknifed ridge predictors.     

Penalised spline estimation for generalised partially linear single-index models

Generalised linear models are frequently used in modeling the relationship of the response variable from the general exponential family with a set of predictor variables, where a linear combination of predictors is linked to the mean of the response variable. We propose a penalised spline (P-spline) estimation for generalised partially linear single-index models, which extend the generalised linear models to include nonlinear effect for some predictors. The proposed models can allow flexible dependence on some predictors while overcome the “curse of dimensionality”. We investigate the P-spline profile likelihood estimation using the readily available R package mgcv, leading to straightforward computation. Simulation studies are considered under various link functions. In addition, we examine different choices of smoothing parameters. Simulation results and real data applications show effectiveness of the proposed approach. Finally, some large sample properties are established.     

Pitman comparisons of predictors of censored observations from progressively censored samples for exponential distribution

On the basis of a progressively censored sample, Basak et al. [On some predictors of times to failure of censored items in progressively censored samples. Comput Statist Data Anal. 2006;50:1313 –1337] considered the problem of predicting the unobserved censored units at various stages of progressive censoring. They then discussed several different point predictors of these censored units and compared them with respect to mean square prediction error. In this work, we use the Pitman closeness (PC) criterion to compare the maximum likelihood, best linear unbiased, best linear equivariant, and conditional median predictors (CMPs) of these progressively censored units. Next, we compare all these with respect to the median unbiased predictor in terms of PC. Numerical computations are then performed to compare all these predictors. By comparing our results to those of Basak et al. (2006), we note that our findings in the sense of PC are similar to theirs in which the CMP competes well when compared to all other predictors.     

A bayesian predictive approach to the selection of variables in multiple regression

The squared error loss function applied to Bayesian predictive distributions is investigated as a variable selection criterion in linear regression equations. It is illustrated that “cost-free” variables may be eliminated if they are poor predictors. Regression models where the predictors are fixed and where they are stochastic are both considered. An empirical examination of the criterion and a comparison with other techniques are presented.     

Multiple Imputation of Missing or Faulty Values Under Linear Constraints

Many statistical agencies, survey organizations, and research centers collect data that suffer from item nonresponse and erroneous or inconsistent values. These data may be required to satisfy linear constraints, for example, bounds on individual variables and inequalities for ratios or sums of variables. Often these constraints are designed to identify faulty values, which then are blanked and imputed. The data also may exhibit complex distributional features, including nonlinear relationships and highly nonnormal distributions. We present a fully Bayesian, joint model for modeling or imputing data with missing/blanked values under linear constraints that (i) automatically incorporates the constraints in inferences and imputations, and (ii) uses a flexible Dirichlet process mixture of multivariate normal distributions to reflect complex distributional features. Our strategy for estimation is to augment the observed data with draws from a hypothetical population in which the constraints are not present, thereby taking advantage of computationally expedient methods for fitting mixture models. Missing/blanked items are sampled from their posterior distribution using the Hit-and-Run sampler, which guarantees that all imputations satisfy the constraints. We illustrate the approach using manufacturing data from Colombia, examining the potential to preserve joint distributions and a regression from the plant productivity literature. Supplementary materials for this article are available online.     

Functional variable selection via Gram–Schmidt orthogonalization for multiple functional linear regression

ABSTRACT

Functional linear model is of great practical importance, as exemplified by applications in high-throughput studies such as meteorological and biomedical research. In this paper, we propose a new functional variable selection procedure, called functional variable selection via Gram–Schmidt (FGS) orthogonalization, for a functional linear model with a scalar response and multiple functional predictors. Instead of the regularization methods, FGS takes into account the similarity between the functional predictors in a data-driven way and utilizes the technique of Gram–Schmidt orthogonalization to remove the irrelevant predictors. FGS can successfully discriminate between the relevant and the irrelevant functional predictors to achieve a high true positive ratio without including many irrelevant predictors, and yield explainable models, which offers a new perspective for the variable selection method in the functional linear model. Simulation studies are carried out to evaluate the finite sample performance of the proposed method, and also a weather data set is analysed.     

Variable selection for high-dimensional generalized linear models with the weighted elastic-net procedure

High-dimensional data arise frequently in modern applications such as biology, chemometrics, economics, neuroscience and other scientific fields. The common features of high-dimensional data are that many of predictors may not be significant, and there exists high correlation among predictors. Generalized linear models, as the generalization of linear models, also suffer from the collinearity problem. In this paper, combining the nonconvex penalty and ridge regression, we propose the weighted elastic-net to deal with the variable selection of generalized linear models on high dimension and give the theoretical properties of the proposed method with a diverging number of parameters. The finite sample behavior of the proposed method is illustrated with simulation studies and a real data example.     

Nonparametric estimation of the link function including variable selection

Nonparametric methods for the estimation of the link function in generalized linear models are able to avoid bias in the regression parameters. But for the estimation of the link typically the full model, which includes all predictors, has been used. When the number of predictors is large these methods fail since the full model cannot be estimated. In the present article a boosting type method is proposed that simultaneously selects predictors and estimates the link function. The method performs quite well in simulations and real data examples.     

Mean and quantile boosting for partially linear additive models

Additive models are often applied in statistical learning which allow linear and nonlinear predictors to coexist. In this article we adapt existing boosting methods for both mean regression and quantile regression in additive models which can simultaneously identify nonlinear, linear and zero predictors. We use gradient boosting in which simple linear regression and univariate penalized spline are used as base learners. Twin boosting is applied to achieve better variable selection accuracy. Simulation studies as well as real data applications illustrate the strength of our proposed methods.     

Predictions and estimations under a group of linear models with random coefficients

This article investigates the problem of establishing best linear unbiased predictors and best linear unbiased estimators of all unknown parameters in a group of linear models with random coefficients and correlated covariance matrix. We shall derive a variety of fundamental statistical properties of the predictors and estimators by using some matrix analysis tools. In particular, we shall establish necessary and sufficient conditions for the predictors and estimators to be equivalent under single and combined equations in the group of models by using the method of matrix equations, matrix rank formulas, and partitioned matrix calculations.     

Maximum Likelihood Inference for Log-linear Models Subject to Constraints of Double Monotone Dependence

To model an hypothesis of double monotone dependence between two ordinal categorical variables A and B usually a set of symmetric odds ratios defined on the joint probability function is subject to linear inequality constraints. Conversely in this paper two sets of asymmetric odds ratios defined, respectively, on the conditional distributions of A given B and on the conditional distributions of B given A are subject to linear inequality constraints. If the joint probabilities are parameterized by a saturated log-linear model, these constraints are nonlinear inequality constraints on the log-linear parameters. The problem here considered is a non-standard one both for the presence of nonlinear inequality constraints and for the fact that the number of these constraints is greater than the number of the parameters of the saturated log-linear model.This work has been supported by the COFIN 2002 project, references 2002133957_002, 2002133957_004. Preliminary findings have been presented at SIS (Società Italiana di Statistica) Annual Meeting, Bari, 2004.