Estimation and variable selection for partially functional linear models

In this paper, a new estimation procedure based on composite quantile regression and functional principal component analysis (PCA) method is proposed for the partially functional linear regression models (PFLRMs). The proposed estimation method can simultaneously estimate both the parametric regression coefficients and functional coefficient components without specification of the error distributions. The proposed estimation method is shown to be more efficient empirically for non-normal random error, especially for Cauchy error, and almost as efficient for normal random errors. Furthermore, based on the proposed estimation procedure, we use the penalized composite quantile regression method to study variable selection for parametric part in the PFLRMs. Under certain regularity conditions, consistency, asymptotic normality, and Oracle property of the resulting estimators are derived. Simulation studies and a real data analysis are conducted to assess the finite sample performance of the proposed methods.     

A fuzzy robust regression approach applied to bedload transport data

Fuzzy least-square regression can be very sensitive to unusual data (e.g., outliers). In this article, we describe how to fit an alternative robust-regression estimator in fuzzy environment, which attempts to identify and ignore unusual data. The proposed approach concerns classical robust regression and estimation methods that are insensitive to outliers. In this regard, based on the least trimmed square estimation method, an estimation procedure is proposed for determining the coefficients of the fuzzy regression model for crisp input-fuzzy output data. The investigated fuzzy regression model is applied to bedload transport data forecasting suspended load by discharge based on a real world data. The accuracy of the proposed method is compared with the well-known fuzzy least-square regression model. The comparison results reveal that the fuzzy robust regression model performs better than the other models in suspended load estimation for the particular dataset. This comparison is done based on a similarity measure between fuzzy sets. The proposed model is general and can be used for modeling natural phenomena whose available observations are reported as imprecise rather than crisp.     

Preliminary test Liu-type estimators based on W,LR, and LM test statistics in a regression model

This article discusses the preliminary test approach for the regression parameter in multiple regression model. The preliminary test Liu-type estimators based on the Wald (W), Likelihood ratio (LR), and Lagrangian multiplier(LM) tests are presented, when it is supposed that the regression parameter may be restricted to a subspace. We also give the bias and mean squared error of the proposed estimators and the superior of the proposed estimators is also discussed.     

A class of finite mixture of quantile regressions with its applications

Mixture of linear regression models provide a popular treatment for modeling nonlinear regression relationship. The traditional estimation of mixture of regression models is based on Gaussian error assumption. It is well known that such assumption is sensitive to outliers and extreme values. To overcome this issue, a new class of finite mixture of quantile regressions (FMQR) is proposed in this article. Compared with the existing Gaussian mixture regression models, the proposed FMQR model can provide a complete specification on the conditional distribution of response variable for each component. From the likelihood point of view, the FMQR model is equivalent to the finite mixture of regression models based on errors following asymmetric Laplace distribution (ALD), which can be regarded as an extension to the traditional mixture of regression models with normal error terms. An EM algorithm is proposed to obtain the parameter estimates of the FMQR model by combining a hierarchical representation of the ALD. Finally, the iterated weighted least square estimation for each mixture component of the FMQR model is derived. Simulation studies are conducted to illustrate the finite sample performance of the estimation procedure. Analysis of an aphid data set is used to illustrate our methodologies.     

An alternative to model selection in ordinary regression

The weaknesses of established model selection procedures based on hypothesis testing and similar criteria are discussed and an alternative based on synthetic (composite) estimation is proposed. It is developed for the problem of prediction in ordinary regression and its properties are explored by simulations for the simple regression. Extensions to a general setting are described and an example with multiple regression is analysed. Arguments are presented against using a selected model for any inferences.     

Bayesian Tobit quantile regression using g-prior distribution with ridge parameter

A Bayesian approach is proposed for coefficient estimation in the Tobit quantile regression model. The proposed approach is based on placing a g-prior distribution depends on the quantile level on the regression coefficients. The prior is generalized by introducing a ridge parameter to address important challenges that may arise with censored data, such as multicollinearity and overfitting problems. Then, a stochastic search variable selection approach is proposed for Tobit quantile regression model based on g-prior. An expression for the hyperparameter g is proposed to calibrate the modified g-prior with a ridge parameter to the corresponding g-prior. Some possible extensions of the proposed approach are discussed, including the continuous and binary responses in quantile regression. The methods are illustrated using several simulation studies and a microarray study. The simulation studies and the microarray study indicate that the proposed approach performs well.     

Multivariate regression models for discrete and continuous repeated measurements

A general class of multivariate regression models is considered for repeated measurements with discrete and continuous outcome variables. The proposed model is based on the seemingly unrelated regression model (Zellner, 1962) and an extension of the model of Park and Woolson(1992). The regression parameters of the model are consistently estimated using the two-stage least squares method. When the out come variables are multivariate normal, the two-stage estimator reduces to Zellner’s two-stage estimator. As a special case, we consider the marginal distribution described by Liang and Zeger (1986). Under this this distributional assumption, we show that the two-stage estimator has similar asymptotic properties and comparable small sample properties to Liang and Zeger's estimator. Since the proposed approach is based on the least squares method, however, any distributional assumption is not required for variables outcome variables. As a result, the proposed estimator is more robust to the marginal distribution of outcomes.     

A note on multivariate linear regression

In this note, a hypothesis test based on relevant statistical differences is proposed for multivariate linear regression models whose design matrix rank does not equal the number of regression variables. A statistical example is also provided to illustrate the proposed hypothesis test.     

Assessing large sample bias in misspecified model scenarios with reference to exposure model misspecification in errors-in-variable regression: A new computational approach

In this paper, we develop a numerical method for evaluating the large sample bias in estimated regression coefficients arising due to exposure model misspecification while adjusting for measurement errors in errors-in-variable regression. The application of the proposed method has been demonstrated in the case of a logistic errors-in-variable regression model. The method is based on the combination of Monte-Carlo, numerical and, in some special cases, analytic integration techniques. The proposed method facilitates the investigation of the limiting bias in the estimated regression parameters based on a single data set rather than on repeated data sets as required by the conventional repeated sample method. Simulation studies demonstrate that the proposed method provides very similar estimates of bias in the estimated regression parameters under exposure model misspecification in logistic errors-in-variable regression with a higher degree of precision as compared to the conventional repeated sample method.     

A new estimator of entropy

Vasicek (1976) proposed an estimator of entropy based on spacings. A new estimator of entropy is proposed. This new estimator is based on local linear regression. Comparisons between this new estimator and Vasicek's estimator are made. The mean square error (MSE) of the new estimator is consistently smaller than the MSE of Vasicek's estimator.     

Back propagation neural networks and multiple regressions in the case of heteroskedasticity

This paper compares the performance between regression analysis and a clustering based neural network approach when the data deviates from the homoscedasticity assumption of regression. Heteroskedasticity is a problem that arises in linear regression due to the unequal error variances. One of the methods to deal heteroskedasticity in classical regression theory is weighted least-square regression (WLS). In order to deal the problem of heteroskedasticity, backpropagation neural network is applied. In this context, an algorithm is proposed which is based on robust estimates of location and dispersion matrix that helps in preserving the error assumption of the linear regression. Analysis is carried out with appropriate designs using simulated data and the results are presented.     

Feasible robust estimator in restricted semiparametric regression models based on the LTS approach

Under some nonstochastic linear restrictions based on either additional information or prior knowledge in a semiparametric regression model, a family of feasible generalized robust estimators for the regression parameter is proposed. The least trimmed squares (LTS) method proposed by Rousseeuw as a highly robust regression estimator is a statistical technique for fitting a regression model based on the subset of h observations (out of n) whose least-square fit possesses the smallest sum of squared residuals. The coverage h may be set between n/2 and n. The LTS estimator involves computing the hyperplane that minimizes the sum of the smallest h squared residuals. For practical purpose, it is assumed that the covariance matrix of the error term is unknown and thus feasible estimators are replaced. Then, we develop an algorithm for the LTS estimator based on feasible methods. Through the Monte Carlo simulation studies and a real data example, performance of the feasible type of robust estimators is compared with the classical ones in restricted semiparametric regression models.     

Estimation of regression coefficient from survey data based on test of significance

The ordinary least squares (OLS)estimator of regression coeffecient is implicitly based on I.I.D.assumption, which is rarely satisfied by survey data. Many approaches are proposed in the literature which can be classified in two broad categories as model based and design consistent.Du Mouchel and Duncan (1983) proposed a test statistic λwhich helps in testing the ignorability of sampling weights.In this article a preliminary test estimator based on λ is proposed. The model based properties of this estimator has been invetigated theoritically where as to study the design based properties simulation approach is adopted. It has been observed that the proposed estimator is a better cimpromise between model based and randomization based inferential frame work.     

Modeling veterans’ health benefit grants using the expectation maximization algorithm

A novel application of the expectation maximization (EM) algorithm is proposed for modeling right-censored multiple regression. Parameter estimates, variability assessment, and model selection are summarized in a multiple regression settings assuming a normal model. The performance of this method is assessed through a simulation study. New formulas for measuring model utility and diagnostics are derived based on the EM algorithm. They include reconstructed coefficient of determination and influence diagnostics based on a one-step deletion method. A real data set, provided by North Dakota Department of Veterans Affairs, is modeled using the proposed methodology. Empirical findings should be of benefit to government policy-makers.     

Methods of Statistical Inference for Median Regression Models with Doubly Censored Data

Recently, least absolute deviations (LAD) estimator for median regression models with doubly censored data was proposed and the asymptotic normality of the estimator was established. However, it is invalid to make inference on the regression parameter vectors, because the asymptotic covariance matrices are difficult to estimate reliably since they involve conditional densities of error terms. In this article, three methods, which are based on bootstrap, random weighting, and empirical likelihood, respectively, and do not require density estimation, are proposed for making inference for the doubly censored median regression models. Simulations are also done to assess the performance of the proposed methods.     

Bootstrap Bartlett correction in inflated beta regression

The inflated beta regression model aims to enable the modeling of responses in the intervals (0, 1], [0, 1), or [0, 1]. In this model, hypothesis testing is often performed based on the likelihood ratio statistic. The critical values are obtained from asymptotic approximations, which may lead to distortions of size in small samples. In this sense, this article proposes the bootstrap Bartlett correction to the statistic of likelihood ratio in the inflated beta regression model. The proposed adjustment only requires a simple Monte Carlo simulation. Through extensive Monte Carlo simulations the finite sample performance (size and power) of the proposed corrected test is compared to the usual likelihood ratio test and the Skovgaard adjustment already proposed in the literature. The numerical results evidence that inference based on the proposed correction is much more reliable than that based on the usual likelihood ratio statistics and the Skovgaard adjustment. At the end of the work, an application to real data is also presented.     

Bayesian spectral analysis models for quantile regression with Dirichlet process mixtures

This paper presents a Bayesian analysis of partially linear additive models for quantile regression. We develop a semiparametric Bayesian approach to quantile regression models using a spectral representation of the nonparametric regression functions and the Dirichlet process (DP) mixture for error distribution. We also consider Bayesian variable selection procedures for both parametric and nonparametric components in a partially linear additive model structure based on the Bayesian shrinkage priors via a stochastic search algorithm. Based on the proposed Bayesian semiparametric additive quantile regression model referred to as BSAQ, the Bayesian inference is considered for estimation and model selection. For the posterior computation, we design a simple and efficient Gibbs sampler based on a location-scale mixture of exponential and normal distributions for an asymmetric Laplace distribution, which facilitates the commonly used collapsed Gibbs sampling algorithms for the DP mixture models. Additionally, we discuss the asymptotic property of the sempiparametric quantile regression model in terms of consistency of posterior distribution. Simulation studies and real data application examples illustrate the proposed method and compare it with Bayesian quantile regression methods in the literature.     

Estimation of the Variance Function in Heteroscedastic Linear Regression Models

Heteroscedasticity generally exists when a linear regression model is applied to analyzing some real-world problems. Therefore, how to accurately estimate the variance functions of the error term in a heteroscedastic linear regression model is of great importance for obtaining efficient estimates of the regression parameters and making valid statistical inferences. A method for estimating the variance function of heteroscedastic linear regression models is proposed in this article based on the variance-reduced local linear smoothing technique. Some simulations and comparisons with other method are conducted to assess the performance of the proposed method. The results demonstrate that the proposed method can accurately estimate the variance functions and therefore produce more efficient estimates of the regression parameters.     

A ridge-type estimator and good prior means

Swindel (1976) introduced a modified ridge regression estimator based on prior information. A necessary and sufficient condition is derived for Swindel's proposed estimator to have lower risk than the conventional ordinary ridge regression estimator when both estimators are computed using the same value of k.     

A plug-in the number of knots selector for polynomial spline regression

A plug-in the number of interior knots (NIKs) selector is proposed for polynomial spline estimation in nonparametric regression. The existence and properties of the optimal NIKs for spline regression are established by minimising the weighted mean integrated squared error. We obtain plug-in formulae for the optimal NIKs based on the theoretical results of asymptotic optimality, and develop strategies for choosing the NIKs of the spline estimator. The proposed NIKs selection method is tested on our simulated data with quite satisfactory performance, and is illustrated by analysing a fossil data set.