Robust regression: an inferential method for determining which independent variables are most important

Consider the usual linear regression model consisting of two or more explanatory variables. There are many methods aimed at indicating the relative importance of the explanatory variables. But in general these methods do not address a fundamental issue: when all of the explanatory variables are included in the model, how strong is the empirical evidence that the first explanatory variable is more or less important than the second explanatory variable? How strong is the empirical evidence that the first two explanatory variables are more important than the third explanatory variable? The paper suggests a robust method for dealing with these issues. The proposed technique is based on a particular version of explanatory power used in conjunction with a modification of the basic percentile method.     

Nonparametric Independence Screening in Sparse Ultra-High Dimensional Additive Models

A variable screening procedure via correlation learning was proposed in Fan and Lv (2008) to reduce dimensionality in sparse ultra-high dimensional models. Even when the true model is linear, the marginal regression can be highly nonlinear. To address this issue, we further extend the correlation learning to marginal nonparametric learning. Our nonparametric independence screening is called NIS, a specific member of the sure independence screening. Several closely related variable screening procedures are proposed. Under general nonparametric models, it is shown that under some mild technical conditions, the proposed independence screening methods enjoy a sure screening property. The extent to which the dimensionality can be reduced by independence screening is also explicitly quantified. As a methodological extension, a data-driven thresholding and an iterative nonparametric independence screening (INIS) are also proposed to enhance the finite sample performance for fitting sparse additive models. The simulation results and a real data analysis demonstrate that the proposed procedure works well with moderate sample size and large dimension and performs better than competing methods.     

An empirical note concerning systematic measurement errors in regression analysis

The bias in the estimated coefficient of an explanatory variable in a regression equation because of a systematic measurement error in another explanatory variable is considered. A general expression for bias is set forth. An actual problem is used as a case study in which the magnitude of the bias in an estimated price coefficient is evaluated using real data.     

Order-restricted Dose-related Trend Phi-divergence Tests for Generalized Linear Models

In this paper a new family of test statistics is presented for testing the independence between the binary response Y and an ordered categorical explanatory variable X (doses) against the alternative hypothesis of an increase dose-response relationship between a response variable Y and X (doses). The properties of these test statistics are studied. This new family of test statistics is based on the family of φ-divergence measures and contains as a particular case the likelihood ratio test. We pay special attention to the family of test statistics associated with the power divergence family. A simulation study is included in order to analyze the behavior of the power divergence family of test statistics.     

The Finite-Sample Performance of White's Test for Heteroskedasticity Under Stochastic Regressors

We study the finite-sample properties of White's test for heteroskedasticity in stochastic regression models where explanatory variables are random and not given. We investigate by simulation the effect of non independence of explanatory variables and error term and heteroskedasticity on White's test. A standard bootstrap method in the computationally convenient form is found to work well with respect to the size and power.     

Kernel regression estimation in a Banach space

In this paper, we study a nonparametric regression estimator when the response variable is in a separable Banach space and the explanatory variable takes values in a separable semi-metric space. Under general conditions, we establish some asymptotic results and give upper bounds for the p$p$-mean and almost sure (pointwise and integrated) estimation errors. Finally, we present the case where the explanatory variable is the Wiener process.     

On the use of the Box-Cox transformation in a linear regression with a lagged dependent variable

In this paper, we show a maximum likelihood estimation procedure in the Box-Cox model when a lagged dependent variable is included among explanatory variables and the first observation of the dependent variable is random. It is shown in a numerical example that a test of a coefficientof the lagged dependent variable is sensitive to whether the first observation of the dependentvariable is random or not.     

A nominal association matrix with feature selection for categorical data

An intrinsic association matrix is introduced to measure category-to-variable association based on proportional reduction of prediction error by an explanatory variable. The normalization of the diagonal gives rise to the expected rates of error-reduction and the off-diagonal yields expected distributions of the rates of error for all response categories. A general framework of association measures based on the proposed matrix is established using an application-specific weight vector. A hierarchy of equivalence relations defined by the association matrix and vector is shown. Applications to financial and survey data together with simulation results are presented.     

Nonlinear models,residuals and transformations 1

A general review is given of statistical methods for nonlinear regression situations and some associated problems are mentioned. After general preliminary comments, a number of types of nonlinear model are outlined. Their parameterization is then examined. Analysis of residuals, with some emphasis on nonlinear models, is then considered and finally transformations of response and explanatory variable are discussed.     

Uniformly most powerful unbiased test in univariate linear calibration

In this paper, we deal with the hypothesis testing problems for the univariate linear calibration, where a normally distributed response variable and an explanatory variable are involved, and the observations of the response variable corresponding to known values of the explanatory variable are used for making inferences concerning a single unknown value of the explanatory variable. The uniformly most powerful unbiased tests for both one-sided and two-sided hypotheses are constructed and verified. The power behaviour of the proposed tests is numerically compared with that of the existing method, and simulations show that the proposed tests make the powers improved.     

Measurement error in the generalised linear model

This paper considers the problem of estimating the linear parameters of a Generalised Linear Model (GLM) when the explanatory variable is subject to measurement error. In this situation the induced model for dependence on the approximate explanatory variable is not usually of GLM form. However, when the distribution of measurement error is known or estimated from replicated measurements, application of the GLIM iteratively reweighted least squares algorithm with transformed data and weighting is shown to produce maximum quasi likelihood estimates in many cases. Details of this approach are given for two particular generalized linear models; simulation results illustrate the usefulness of the theory for these models.     

The SSR Plot: A Graphical Representation for Regression

An alternative graphical method, called the SSR plot, is proposed for use with a multiple regression model. The new method uses the fact that the sum of squares for regression (SSR) of two explanatory variables can be partitioned into the SSR of one variable and the increment in SSR due to the addition of the second variable. The SSR plot represents each explanatory variable as a vector in a half circle. Our proposed SSR plot explains that the explanatory variables corresponding to the vectors located closer to the horizontal axis have stronger effects on the response variable. Furthermore, for a regression model with two explanatory variables, the magnitude of the angle between two vectors can be used to identify suppression.     

Likelihood estimation for a longitudinal negative binomial regression model with missing outcomes

Joint damage in psoriatic arthritis can be measured by clinical and radiological methods, the former being done more frequently during longitudinal follow-up of patients. Motivated by the need to compare findings based on the different methods with different observation patterns, we consider longitudinal data where the outcome variable is a cumulative total of counts that can be unobserved when other, informative, explanatory variables are recorded. We demonstrate how to calculate the likelihood for such data when it is assumed that the increment in the cumulative total follows a discrete distribution with a location parameter that depends on a linear function of explanatory variables. An approach to the incorporation of informative observation is suggested. We present analyses based on an observational database from a psoriatic arthritis clinic. Although the use of the new statistical methodology has relatively little effect in this example, simulation studies indicate that the method can provide substantial improvements in bias and coverage in some situations where there is an important time varying explanatory variable.     

Joint confidence regions in the multivariate calibration problem

Consider a vector valued response variable related to a vector valued explanatory variable through a normal multivariate linear model. The multivariate calibration problem deals with statistical inference on unknown values of the explanatory variable. The problem addressed is the construction of joint confidence regions for several unknown values of the explanatory variable. The problem is investigated when the variance covariance matrix is a scalar multiple of the identity matrix and also when it is a completely unknown positive definite matrix. The problem is solved in only two cases: (i) the response and explanatory variables have the same dimensions, and (ii) the explanatory variable is a scalar. In the former case, exact joint confidence regions are derived based on a natural pivot statistic. In the latter case, the joint confidence regions are only conservative. Computational aspects and the practical implementation of the confidence regions are discussed and illustrated using an example.     

Diagnostic plots in beta-regression models

Two diagnostic plots for selecting explanatory variables are introduced to assess the accuracy of a generalized beta-linear model. The added variable plot is developed to examine the need for adding a new explanatory variable to the model. The constructed variable plot is developed to identify the nonlinearity of the explanatory variable in the model. The two diagnostic procedures are also useful for detecting unusual observations that may affect the regression much. Simulation studies and analysis of two practical examples are conducted to illustrate the performances of the proposed plots.     

Flexible Bayesian Modelling for Survival Data

The analysis of failure time data often involves two strong assumptions. The proportional hazards assumption postulates that hazard rates corresponding to different levels of explanatory variables are proportional. The additive effects assumption specifies that the effect associated with a particular explanatory variable does not depend on the levels of other explanatory variables. A hierarchical Bayes model is presented, under which both assumptions are relaxed. In particular, time-dependent covariate effects are explicitly modelled, and the additivity of effects is relaxed through the use of a modified neural network structure. The hierarchical nature of the model is useful in that it parsimoniously penalizes violations of the two assumptions, with the strength of the penalty being determined by the data.     

Review: Reversed low-rank ANOVA model for transforming high dimensional genetic data into low dimension

A general modeling procedure for analyzing genetic data is reviewed. We review ANOVA type model that can handle both the continuous and discrete genetic variables in one modeling framework. Unlike the regression type models which typically set the phenotype variable as a response, this ANOVA model treats the phenotype variable as an explanatory variable. By reversely treating the phenotype variable, usual high dimensional problem is turned into low dimension. Instead, the ANOVA model always includes interaction term between the genetic locations and phenotype variable to find potential association between them. The interaction term is designed to be low rank with the multiplication of bilinear terms so that the required number of parameters is kept in a manageable degree. We compare the performance of the reviewed ANOVA model to the other popular methods via microarray and SNP data sets.     

A distance based regression model for prediction with mixed data

A multiple regression method based on distance analysis and metric scaling is proposed and studied. This method allow us to predict a continuous response variable from several explanatory variables, is compatible with the general linear model and is found to be useful when the predictor variables are both continuous and categorical. Real data examples are given to illustrate the results obtained.     

互联网企业广告收入预测研究——基于低频数据的神经网络和时间序列组合模型

神经网络模型对大样本时间序列的拟合效果优于传统时间序列模型,但对于年度、月度、日度等低频时间序列的预测则难以发挥其优势。鉴于此,本文应用传统时间序列模型和神经网络模型,建立Holtwinters-BP组合模型,利用Holtwinters模型分别拟合各解释变量序列,利用BP模型拟合解释变量和自变量的非线性关系,基于某社交新闻类APP的日广告收入数据进行互联网企业广告收入预测研究。通过与循环神经网络(RNN)模型、长短期记忆神经网络(LSTM)模型等预测结果的对比发现:Holtwinters-BP组合模型的预测精度和稳定性更高;证明多维变量对于广告收入的显著影响,多变量模型的预测准确性高于单变量模型;构建的Holtwinters-BP组合模型对于低频数据预测有较好的有效性和适用性。     

On the Optimal Use of Suboptimal Forecasts of Explanatory Variables

Ashley (1983) gave a simple condition for determining when a forecast of an explanatory variable (X_t ) is sufficiently inaccurate that direct replacement of X_t by the forecast yields worse forecasts of the dependent variable than does respecification of the equation to omit X_t . Many available macroeconomic forecasts were shown to be of limited usefulness in direct replacement. Direct replacement, however, is not optimal if the forecast's distribution is known. Here optimal linear forms in commercial forecasts of several macroeconomic variables are obtained by using estimates of their distributions. Although they are an improvement on the raw forecasts (direct replacement), these optimal forms are still too inaccurate to be useful in replacing the actual explanatory variables in forecasting models. The results strongly indicate that optimal forms involving several commercial forecasts will not be very useful either. Thus Ashley's (1983) sufficient condition retains its value in gauging the usefulness of a forecast of an explanatory variable in a forecasting model, even though it focuses on direct replacement.