A semi-parametric approach to dual modeling when no replication exists

In many applications, it is of interest to simultaneously model the mean and variance of a response when no replication exists. Modeling the mean and variance simultaneously is commonly referred to as dual modeling. Parametric approaches to dual modeling are popular when the underlying mean and variance functions can be expressed explicitly. Quite often, however, nonparametric approaches are more appropriate due to the presence of unusual curvature in the underlying functions. In sparse data situations, nonparametric methods often fit the data too closely while parametric estimates exhibit problems with bias. We propose a semi-parametric dual modeling approach [Dual Model Robust Regression (DMRR)] for non-replicated data. DMRR combines parametric and nonparametric fits resulting in improved mean and variance estimation. The methodology is illustrated with a data set from the literature as well as via a simulation study.     

Interval robust design under contaminated and non normal data

Abstract

Robust parameter design (RPD) is an effective tool, which involves experimental design and strategic modeling to determine the optimal operating conditions of a system. The usual assumptions of RPD are that normally distributed experimental data and no contamination due to outliers. And generally the parameter uncertainties in response models are neglected. However, using normal theory modeling methods for a skewed data and ignoring parameter uncertainties can create a chain of degradation in optimization and production phases such that misleading fit, poor estimated optimal operating conditions, and poor quality products. This article presents a new approach based on confidence interval (CI) response modeling for the process mean. The proposed interval robust design makes the system median unbiased for the mean and uses midpoint of the interval as a measure of location performance response. As an alternative robust estimator for the process variance response modeling, using biweight midvariance is proposed which is both resistant and robust of efficiency where normality is not met. The results further show that the proposed interval robust design gives a robust solution to the skewed structure of the data and to contaminated data. The procedure and its advantages are illustrated using two experimental design studies.     

Generalised Variance Function Estimation for Binary Variables in Large‐Scale Sample Surveys

Generalised variance function (GVF) models are data analysis techniques often used in large‐scale sample surveys to approximate the design variance of point estimators for population means and proportions. Some potential advantages of the GVF approach include operational simplicity, more stable sampling errors estimates and providing a convenient method of summarising results when a high number of survey variables is considered. In this paper, several parametric and nonparametric methods for GVF estimation with binary variables are proposed and compared. The behavior of these estimators is analysed under heteroscedasticity and in the presence of outliers and influential observations. An empirical study based on the annual survey of living conditions in Galicia (a region in the northwest of Spain) illustrates the behaviour of the proposed estimators.     

Heteroscedastic regression models with non-normally distributed errors

Although regression estimates are quite robust to slight departure from normality, symmetric prediction intervals assuming normality can be highly unsatisfactory and problematic if the residuals have a skewed distribution. For data with distributions outside the class covered by the Generalized Linear Model, a common way to handle non-normality is to transform the response variable. Unfortunately, transforming the response variable often destroys the theoretical or empirical functional relationship connecting the mean of the response variable to the explanatory variables established on the original scale. Further complication arises if a single transformation cannot both stabilize variance and attain normality. Furthermore, practitioners also find the interpretation of highly transformed data not obvious and often prefer an analysis on the original scale. The present paper presents an alternative approach for handling simultaneously heteroscedasticity and non-normality without resorting to data transformation. Unlike classical approaches, the proposed modeling allows practitioners to formulate the mean and variance relationships directly on the original scale, making data interpretation considerably easier. The modeled variance relationship and form of non-normality in the proposed approach can be easily examined through a certain function of the standardized residuals. The proposed method is seen to remain consistent for estimating the regression parameters even if the variance function is misspecified. The method along with some model checking techniques is illustrated with a real example.     

Sequential design for nonparametric inference

The performance of nonparametric function estimates often depends on the choice of design points. Based on the mean integrated squared error criterion, we propose a sequential design procedure that updates the model knowledge and optimal design density sequentially. The methodology is developed under a general framework covering a wide range of nonparametric inference problems, such as conditional mean and variance functions, the conditional distribution function, the conditional quantile function in quantile regression, functional coefficients in varying coefficient models and semiparametric inferences. Based on our empirical studies, nonparametric inference based on the proposed sequential design is more efficient than the uniform design and its performance is close to the true but unknown optimal design. The Canadian Journal of Statistics 40: 362–377; 2012 © 2012 Statistical Society of Canada     

Estimating curves and derivatives with parametric penalized spline smoothing

Accurate estimation of an underlying function and its derivatives is one of the central problems in statistics. Parametric forms are often proposed based on the expert opinion or prior knowledge of the underlying function. However, these strict parametric assumptions may result in biased estimates when they are not completely accurate. Meanwhile, nonparametric smoothing methods, which do not impose any parametric form, are quite flexible. We propose a parametric penalized spline smoothing method, which has the same flexibility as the nonparametric smoothing methods. It also uses the prior knowledge of the underlying function by defining an additional penalty term using the distance of the fitted function to the assumed parametric function. Our simulation studies show that the parametric penalized spline smoothing method can obtain more accurate estimates of the function and its derivatives than the penalized spline smoothing method. The parametric penalized spline smoothing method is also demonstrated by estimating the human height function and its derivatives from the real data.     

The Performance of a Robust Multistage Estimator in Nonlinear Regression with Heteroscedastic Errors

In this article, a robust multistage parameter estimator is proposed for nonlinear regression with heteroscedastic variance, where the residual variances are considered as a general parametric function of predictors. The motivation is based on considering the chi-square distribution for the calculated sample variance of the data. It is shown that outliers that are influential in nonlinear regression parameter estimates are not necessarily influential in calculating the sample variance. This matter persuades us, not only to robustify the estimate of the parameters of the models for both the regression function and the variance, but also to replace the sample variance of the data by a robust scale estimate.     

Model misspecification in parametric dual modeling

In typical normal theory regression, the assumption of homogeneity of variances is often not appropriate. Instead of treating the variances as a nuisance and transforming away the heterogeneity, the structure of the variances may be of interest and it is desirable to model the variances. Simultaneous modeling of the mean and variance of a response is known as dual modeling. When parametric models for the mean and variance are prescribed, estimation of the mean and variance parameters are interrelated. One commonly used dual model assumes a linear model for the mean and a log-linear variance model (Aitkin, 1987). This paper considers the impact of model misspecification (mean and variance) on the dual model estimation procedure. Asymptotic expressions for the mean and variance estimates, graphical illustrations of the impact of model misspecification, and simulation results are presented.     

Likelihood inference for correlated binary data without any information about the joint distributions

We propose a universal robust likelihood that is able to accommodate correlated binary data without any information about the underlying joint distributions. This likelihood function is asymptotically valid for the regression parameter for any underlying correlation configurations, including varying under- or over-dispersion situations, which undermines one of the regularity conditions ensuring the validity of crucial large sample theories. This robust likelihood procedure can be easily implemented by using any statistical software that provides naïve and sandwich covariance matrices for regression parameter estimates. Simulations and real data analyses are used to demonstrate the efficacy of this parametric robust method.     

A semiparametric stochastic mixed effects model for bivariate cyclic longitudinal data

We propose a flexible semiparametric stochastic mixed effects model for bivariate cyclic longitudinal data. The model can handle either single cycle or, more generally, multiple consecutive cycle data. The approach models the mean of responses by parametric fixed effects and a smooth nonparametric function for the underlying time effects, and the relationship across the bivariate responses by a bivariate Gaussian random field and a joint distribution of random effects. The proposed model not only can model complicated individual profiles, but also allows for more flexible within-subject and between-response correlations. The fixed effects regression coefficients and the nonparametric time functions are estimated using maximum penalized likelihood, where the resulting estimator for the nonparametric time function is a cubic smoothing spline. The smoothing parameters and variance components are estimated simultaneously using restricted maximum likelihood. Simulation results show that the parameter estimates are close to the true values. The fit of the proposed model on a real bivariate longitudinal dataset of pre-menopausal women also performs well, both for a single cycle analysis and for a multiple consecutive cycle analysis. The Canadian Journal of Statistics 48: 471–498; 2020 © 2020 Statistical Society of Canada     

Automatic bandwidth selection for modified m-smoother ∗

Traditional parametric and nonparametric regression techniques encounter serious over smoothing problems when jump point discontinuities exist in the underlying mean function. Recently, Chu, Glad, Godtliebsen and Marron (1998) developed a method using a modified M-smoothing technique to preserve jumps and spikes while producing a smooth estimate of the mean function. The performance of Chu etal.'s (1998) method is quite sensitive to the choice of the required bandwidths g and h. Furthermore, it is not obvious how to extend certain commonly used automatic bandwidth selection procedures when jumps and spikes are present. In this paper we propose a rule of thumb method of choosing the smoothing parameters based on asymptotic optimal bandwidth formulas and robust estimates of unknown quantities. We also evaluate the proposed bandwidth selection method via a small simulation study.     

Simulation-Extrapolation via the Bezier Curve in Measurement Error Models

Simulation-extrapolation (SIMEX) is a method for correcting for bias in measurement error models, and parametric SIMEX estimates are often used. In this paper, we propose a nonparametric method for computing the SIMEX estimate via the Bezier curve, which is a popular smoothing technique in the computer graphics area. Comparisons are done for the bias of the limit values of parametric SIMEX estimates and the Bezier estimate in the various nonlinear measurement error models.     

On Semiparametric Mode Regression Estimation

It has been found that, for a variety of probability distributions, there is a surprising linear relation between mode, mean, and median. In this article, the relation between mode, mean, and median regression functions is assumed to follow a simple parametric model. We propose a semiparametric conditional mode (mode regression) estimation for an unknown (unimodal) conditional distribution function in the context of regression model, so that any m-step-ahead mean and median forecasts can then be substituted into the resultant model to deliver m-step-ahead mode prediction. In the semiparametric model, Least Squared Estimator (LSEs) for the model parameters and the simultaneous estimation of the unknown mean and median regression functions by the local linear kernel method are combined to infer about the parametric and nonparametric components of the proposed model. The asymptotic normality of these estimators is derived, and the asymptotic distribution of the parameter estimates is also given and is shown to follow usual parametric rates in spite of the presence of the nonparametric component in the model. These results are applied to obtain a data-based test for the dependence of mode regression over mean and median regression under a regression model.     

Sparse estimation and inference for censored median regression

Censored median regression has proved useful for analyzing survival data in complicated situations, say, when the variance is heteroscedastic or the data contain outliers. In this paper, we study the sparse estimation for censored median regression models, which is an important problem for high dimensional survival data analysis. In particular, a new procedure is proposed to minimize an inverse-censoring-probability weighted least absolute deviation loss subject to the adaptive LASSO penalty and result in a sparse and robust median estimator. We show that, with a proper choice of the tuning parameter, the procedure can identify the underlying sparse model consistently and has desired large-sample properties including root-n consistency and the asymptotic normality. The procedure also enjoys great advantages in computation, since its entire solution path can be obtained efficiently. Furthermore, we propose a resampling method to estimate the variance of the estimator. The performance of the procedure is illustrated by extensive simulations and two real data applications including one microarray gene expression survival data.     

Asymptotic results for model robust regression

Since the mid 1980's many statisticians have studied methods for combining parametric and nonparametric models to improve the quality of fits in a regression problem. Notably Einsporn (1987) proposed the Model Robust Regression 1 estimate (MRRl) in which the parametric function, f, and the nonparametric functiong were combined in a straightforward fashion via the use of a mixing parameter, λ This technique was studied extensively atsmall samples and was shown to be quite effective at modeling various unusual functions. In this paper we have asymptotic results for the MRRl estimate in the case where λ is theoretically optimal, is asymptotically optimal and data driven, and is chosen with the PRESS statistic (Allen, 1971) We demonstrate that the MRRl estimate with λchosen by the PRESS statistic is slightly inferior asymptotically to the other two estimates, but, nevertheless possesses positive asymptotic qualities.     

Heteroscedastic Weibull-Normal Mixture Models: A Bayesian Approach

In this article, we propose Bayesian methodology to obtain parameter estimates of the mixture of distributions belonging to the normal and biparametric Weibull families, modeling the mean and the variance parameters. Simulated studies and applications show the performance of the proposed models.     

Error variance function estimation in nonparametric regression models

In this article, a new class of variance function estimators is proposed in the setting of heteroscedastic nonparametric regression models. To obtain a variance function estimator, the main proposal is to smooth the product of the response variable and residuals as opposed to the squared residuals. The asymptotic properties of the proposed methodology are investigated in order to compare its asymptotic behavior with that of the existing methods. The finite sample performance of the proposed estimator is studied through simulation studies. The effect of the curvature of the mean function on its finite sample behavior is also discussed.     

Inferences of variance function - a parametric robust way

Tsou (2003a) proposed a parametric procedure for making robust inference for mean regression parameters in the context of generalized linear models. This robust procedure is extended to model variance heterogeneity. The normal working model is adjusted to become asymptotically robust for inference about regression parameters of the variance function for practically all continuous response variables. The connection between the novel robust variance regression model and the estimating equations approach is also provided.     

Minimum distance lack-of-fit tests for fixed design

This paper discusses a class of tests of lack-of-fit of a parametric regression model when design is non-random and uniform on [0,1]. These tests are based on certain minimized distances between a nonparametric regression function estimator and the parametric model being fitted. We investigate asymptotic null distributions of the proposed tests, their consistency and asymptotic power against a large class of fixed and sequences of local nonparametric alternatives, respectively. The best fitted parameter estimate is seen to be n^1/2-consistent and asymptotically normal. A crucial result needed for proving these results is a central limit lemma for weighted degenerate U statistics where the weights are arrays of some non-random real numbers. This result is of an independent interest and an extension of a result of Hall for non-weighted degenerate U statistics.