Bayesian inferences related to shifting sequences and two-phase regression

The problem of estimating the switch point in a sequence of independent random variables is studied from a Bayesian viewpoint. Theoretical results and numerical examples are given for the normal sequence and two-phase regression.     

Two-phase regression and goodness of fit

Analysis of two-phase regression has traditionally been carried out using a variety of likelihood approaches. In this paper we present an alternative procedure based on a goodness of fit criterion.

Exact hypothesis tests for a known switch point are developed. Approximate (conservative) tests for an unknown switch point are also obtained     

Optimal sampling from concomitant variables for regression problems

Accounting for an auxiliary covariate in a two-phase sampling strategy in order to reduce the experimental costs was initially proposed by Cochran (Sampling Techniques, 2nd Edition, Wiley, New York, 1963, Sampling Techniques, 3rd Edition, Wiley, New York, 1977) in the context of sample surveys. Conniffe and Moran (Biometrics 28 (1972) 1011) have extended this methodology to the estimation of linear regression functions. More recently, Conniffe (J. Econometrics 27 (1985) 179) and Causeur and Dhorne (Biometrics 54 (4) (1998) 1591) have derived two-phase sampling estimators of the linear regression function in the situation where many auxiliary covariates are available. A detailed study of the distributional aspects of these estimators is provided by Causeur (Statistics 32 (1999) 297). In the same multivariate context, this paper aims at an extension of the double-sampling strategies to monotone designs accounting for differences between the costs of subsets of covariates. In particular, the maximum-likelihood estimators are provided and asymptotic solutions for the optimal designs are derived.     

Smooth piecewise linear regression splines with hyperbolic covariates

We-propose the use of hyperbolas as covariates in piecewise linear regression splines to fit data exhibiting a multi-phase linear response with smooth transitions between phases. The hyperbolic regression spline model, fitted by non-linear regression, provides an intuitive and easy way to extend to multiple phases the two-phase hyperbolic response model previously proposed by others. The small additional effort required to fit non-linear, as opposed to linear, regression models is particularly worthwhile when investigators are unwilling to assume that the slope of the response changes abruptly at the join points. Furthermore, undue influence on the join point and slope estimates, resulting from points in the transition region, may be avoided by using the hyperbolic regression spline. Two examples illustrate the use of this method.     

Heteroscedasticity diagnostics in two-phase linear regression models

In two-phase linear regression models, it is a standard assumption that the random errors of two phases have constant variances. However, this assumption is not necessarily appropriate. This paper is devoted to the tests for variance heterogeneity in these models. We initially discuss the simultaneous test for variance heterogeneity of two phases. When the simultaneous test shows that significant heteroscedasticity occurs in the whole model, we construct two individual tests to investigate whether or not both phases or one of them have/has significant heteroscedasticity. Several score statistics and their adjustments based on Cox and Reid [D. R. Cox and N. Reid, Parameter orthogonality and approximate conditional inference. J. Roy. Statist. Soc. Ser. B 49 (1987), pp. 1–39] are obtained and illustrated with Australian onion data. The simulated powers of test statistics are investigated through Monte Carlo methods.     

Variance estimation of DEE and regression estimator when a first-phase cluster sample is restratified

We consider a variance estimation when a stratified single stage cluster sample is selected in the first phase and a stratified simple random element sample is selected in the second phase. We propose explicit formulas of (asymptotically), we propose explicit formulas of (asymptotically) unbiased variance estimators for the double expansion estimator and regression estimator. We perform a small simulation study to investigate the performance of the proposed variance estimators. In our simulation study, the proposed variance estimator showed better or comparable performance to the Jackknife variance estimator. We also extend the results to a two-phase sampling design in which a stratified pps with replacement cluster sample is selected in the first phase.     

Nonstationary nonlinear quantile regression

This study examines estimation and inference based on quantile regression for parametric nonlinear models with an integrated time series covariate. We first derive the limiting distribution of the nonlinear quantile regression estimator and then consider testing for parameter restrictions, when the regression function is specified as an asymptotically homogeneous function. We also study linear-in-parameter regression models when the regression function is given by integrable regression functions as well as asymptotically homogeneous regression functions. We, furthermore, propose a fully modified estimator to reduce the bias in the original estimator under a certain set of conditions. Finally, simulation studies show that the estimators behave well, especially when the regression error term has a fat-tailed distribution.     

Robust Poisson regression

Count data are very often analyzed under the assumption of a Poisson model [(Agresti, A., 1996. An Introduction to Categorical Data Analysis. Wiley, New York; Generalized Linear Models, second ed. Chapman & Hall, New York)]. However, the derived inference is generally erroneous if the underlying distribution is not Poisson (Biometrika 70, 269–274).A parametric robust regression approach is proposed for the analysis of count data. More specifically it will be demonstrated that the Poisson regression model could be properly adjusted to become asymptotically valid for inference about regression parameters, even if the Poisson assumption fails. With large samples the novel robust methodology provides legitimate likelihood functions for regression parameters, so long as the true underlying distributions have finite second moments. Adjustments that robustify the Poisson regression will be given, respectively, under log link and identity link functions. Simulation studies will be used to demonstrate the efficacy of the robust Poisson regression model.     

Sliced inverse regression for multivariate response regression

We consider a regression analysis of multivariate response on a vector of predictors. In this article, we develop a sliced inverse regression-based method for reducing the dimension of predictors without requiring a prespecified parametric model. Our proposed method preserves as much regression information as possible. We derive the asymptotic weighted chi-squared test for dimension. Simulation results are reported and comparisons are made with three methods—most predictable variates, k-means inverse regression and canonical correlation approach.     

On nonparametric regression estimators based on regression quantiles

In the ciassical regression model Y_i=h(x_i) + ? _i, i=1,…,n, Cheng (1984) introduced linear combinations of regression quantiles as a new class of estimators for the unknown regression function h(x). The asymptotic properties studied in Cheng (1984) are reconsidered. We obtain a sharper scrong consistency rate and we improve on the conditions for asymptotic normality by proving a new result on the remainder term in the Bahadur representation for regression quantiles.     

Bayesian bridge quantile regression

Regularization methods for simultaneous variable selection and coefficient estimation have been shown to be effective in quantile regression in improving the prediction accuracy. In this article, we propose the Bayesian bridge for variable selection and coefficient estimation in quantile regression. A simple and efficient Gibbs sampling algorithm was developed for posterior inference using a scale mixture of uniform representation of the Bayesian bridge prior. This is the first work to discuss regularized quantile regression with the bridge penalty. Both simulated and real data examples show that the proposed method often outperforms quantile regression without regularization, lasso quantile regression, and Bayesian lasso quantile regression.     

Diagnostics for non-linear regression

Sensitivity analysis in regression is concerned with assessing the sensitivity of the results of a regression model (e.g., the objective function, the regression parameters, and the fitted values) to changes in the data. Sensitivity analysis in least squares linear regression has seen a great surge of research activities over the last three decades. By contrast, sensitivity analysis in non-linear regression has received very little attention. This paper deals with the problem of local sensitivity analysis in non-linear regression. Closed-form general formulas are provided for the sensitivities of three standard methods for the estimation of the parameters of a non-linear regression model based on a set of data. These methods are the least squares, the minimax, and the least absolute value methods. The effectiveness of the proposed measures is illustrated by application to several non-linear models including the ultrasonic data and the onion yield data. The proposed sensitivity measures are shown to deal effectively with the detection of influential observations in non-linear regression models.     

New regression model with four regression structures and computational aspects

A new general class of exponentiated sinh Cauchy regression models for location, scale, and shape parameters is introduced and studied. It may be applied to censored data and used more effectively in survival analysis when compared with the usual models. For censored data, we employ a frequentist analysis for the parameters of the proposed model. Further, for different parameter settings, sample sizes, and censoring percentages, various simulations are performed. The extended regression model is very useful for the analysis of real data and could give more adequate fits than other special regression models.     

Ridge regression estimators in the linear regression models with non-spherical errors

The present paper considers a family of ordinary ridge regression estimators in the linear regression model when the disturbances covariance matrix depends upon a few unknown parameters. An asymptotic expansion for the distribution of the ridge regression estimator is developed and under the quadratic loss function its asymptotic risk is compared with that of the feasible GLS estimator.     

Inference in segmented line regression: a simulation study

A segmented line regression model has been used to describe changes in cancer incidence and mortality trends [Kim, H.-J., Fay, M.P., Feuer, E.J. and Midthune, D.N., 2000, Permutation tests for joinpoint regression with applications to cancer rates. Statistics in Medicine, 19, 335–351. Kim, H.-J., Fay, M.P., Yu, B., Barrett., M.J. and Feuer, E.J., 2004, Comparability of segmented line regression models. Biometrics, 60, 1005–1014.]. The least squares fit can be obtained by using either the grid search method proposed by Lerman [Lerman, P.M., 1980, Fitting segmented regression models by grid search. Applied Statistics, 29, 77–84.] which is implemented in Joinpoint 3.0 available at http://srab.cancer.gov/joinpoint/index.html, or by using the continuous fitting algorithm proposed by Hudson [Hudson, D.J., 1966, Fitting segmented curves whose join points have to be estimated. Journal of the American Statistical Association, 61, 1097–1129.] which will be implemented in the next version of Joinpoint software. Following the least squares fitting of the model, inference on the parameters can be pursued by using the asymptotic results of Hinkley [Hinkley, D.V., 1971, Inference in two-phase regression. Journal of the American Statistical Association, 66, 736–743.] and Feder [Feder, P.I., 1975a, On asymptotic distribution theory in segmented regression Problems-Identified Case. The Annals of Statistics, 3, 49–83.] Feder [Feder, P.I., 1975b, The log likelihood ratio in segmented regression. The Annals of Statistics, 3, 84–97.] Via simulations, this paper empirically examines small sample behavior of these asymptotic results, studies how the two fitting methods, the grid search and the Hudson's algorithm affect these inferential procedures, and also assesses the robustness of the asymptotic inferential procedures.     

Local influence in multivariate regression

The method of local influence is generalized to the multivariate regression. The scheme of perturbations adopted in multivariate regression is similar in spirit to the perturbation of case-weights in univariate regression case. The method developed here is useful for identifying influential observations in multivariate regression as an exploratory or confirmatory data analysis. An illustrative example is given for the effectiveness of the local influence approach in multivariate regression.     

Design and relative efficiency in two-phase studies

Many two-phase sampling designs have been applied in practice to obtain efficient estimates of regression parameters while minimizing the cost of data collection. This research investigates two-phase sampling designs for so-called expensive variable problems, and compares them with one-phase designs. Closed form expressions for the asymptotic relative efficiency of maximum likelihood estimators from the two designs are derived for parametric normal models, providing insight into the available information for regression coefficients under the two designs. We further discuss when we should apply the two-phase design and how to choose the sample sizes for two-phase samples. Our numerical study indicates that the results can be applied to more general settings.     

Semi-non parametric smooth isotonic regression

We propose a new method for smooth isotonic regression analysis. Unlike most existing methods for isotonic regression, the proposed method is akin to parametric regression without order restriction. To account for smoothness and isotonicity simultaneously, we exploit the flexible class of semi-non parametric densities to model isotonic regression functions. Under this framework, the full range of inference techniques for parametric regression models become applicable for model estimation and model validation in isotonic regression.     

Kernel spline regression

The authors propose «kernel spline regression,» a method of combining spline regression and kernel smoothing by replacing the polynomial approximation for local polynomial kernel regression with the spline basis. The new approach retains the local weighting scheme and the use of a bandwidth to control the size of local neighborhood. The authors compute the bias and variance of the kernel linear spline estimator, which they compare with local linear regression. They show that kernel spline estimators can succeed in capturing the main features of the underlying curve more effectively than local polynomial regression when the curvature changes rapidly. They also show through simulation that kernel spline regression often performs better than ordinary spline regression and local polynomial regression.     

Asymptotic distribution of regression correlation coefficient for Poisson regression model

This study treats an asymptotic distribution for measures of predictive power for generalized linear models (GLMs). We focus on the regression correlation coefficient (RCC) that is one of the measures of predictive power. The RCC, proposed by Zheng and Agresti is a population value and a generalization of the population value for the coefficient of determination. Therefore, the RCC is easy to interpret and familiar. Recently, Takahashi and Kurosawa provided an explicit form of the RCC and proposed a new RCC estimator for a Poisson regression model. They also showed the validity of the new estimator compared with other estimators. This study discusses the new statistical properties of the RCC for the Poisson regression model. Furthermore, we show an asymptotic normality of the RCC estimator.