A weighted bootstrap approximation for comparing the error distributions in nonparametric regression

Several procedures have been proposed for testing the equality of error distributions in two or more nonparametric regression models. Here we deal with methods based on comparing estimators of the cumulative distribution function (CDF) of the errors in each population to an estimator of the common CDF under the null hypothesis. The null distribution of the associated test statistics has been approximated by means of a smooth bootstrap (SB) estimator. This paper proposes to approximate their null distribution through a weighted bootstrap. It is shown that it produces a consistent estimator. The finite sample performance of this approximation is assessed by means of a simulation study, where it is also compared to the SB. This study reveals that, from a computational point of view, the proposed approximation is more efficient than the one provided by the SB.     

Locally Efficient Semiparametric Estimators for Proportional Hazards Models with Measurement Error

We propose a new class of semiparametric estimators for proportional hazards models in the presence of measurement error in the covariates, where the baseline hazard function, the hazard function for the censoring time, and the distribution of the true covariates are considered as unknown infinite dimensional parameters. We estimate the model components by solving estimating equations based on the semiparametric efficient scores under a sequence of restricted models where the logarithm of the hazard functions are approximated by reduced rank regression splines. The proposed estimators are locally efficient in the sense that the estimators are semiparametrically efficient if the distribution of the error‐prone covariates is specified correctly and are still consistent and asymptotically normal if the distribution is misspecified. Our simulation studies show that the proposed estimators have smaller biases and variances than competing methods. We further illustrate the new method with a real application in an HIV clinical trial.     

Laplace based approximate posterior inference for differential equation models

Ordinary differential equations are arguably the most popular and useful mathematical tool for describing physical and biological processes in the real world. Often, these physical and biological processes are observed with errors, in which case the most natural way to model such data is via regression where the mean function is defined by an ordinary differential equation believed to provide an understanding of the underlying process. These regression based dynamical models are called differential equation models. Parameter inference from differential equation models poses computational challenges mainly due to the fact that analytic solutions to most differential equations are not available. In this paper, we propose an approximation method for obtaining the posterior distribution of parameters in differential equation models. The approximation is done in two steps. In the first step, the solution of a differential equation is approximated by the general one-step method which is a class of numerical numerical methods for ordinary differential equations including the Euler and the Runge-Kutta procedures; in the second step, nuisance parameters are marginalized using Laplace approximation. The proposed Laplace approximated posterior gives a computationally fast alternative to the full Bayesian computational scheme (such as Makov Chain Monte Carlo) and produces more accurate and stable estimators than the popular smoothing methods (called collocation methods) based on frequentist procedures. For a theoretical support of the proposed method, we prove that the Laplace approximated posterior converges to the actual posterior under certain conditions and analyze the relation between the order of numerical error and its Laplace approximation. The proposed method is tested on simulated data sets and compared with the other existing methods.     

Logistic regression in meta-analysis using aggregate data

We derived two methods to estimate the logistic regression coefficients in a meta-analysis when only the 'aggregate' data (mean values) from each study are available. The estimators we proposed are the discriminant function estimator and the reverse Taylor series approximation. These two methods of estimation gave similar estimators using an example of individual data. However, when aggregate data were used, the discriminant function estimators were quite different from the other two estimators. A simulation study was then performed to evaluate the performance of these two estimators as well as the estimator obtained from the model that simply uses the aggregate data in a logistic regression model. The simulation study showed that all three estimators are biased. The bias increases as the variance of the covariate increases. The distribution type of the covariates also affects the bias. In general, the estimator from the logistic regression using the aggregate data has less bias and better coverage probabilities than the other two estimators. We concluded that analysts should be cautious in using aggregate data to estimate the parameters of the logistic regression model for the underlying individual data.     

Estimation and hypothesis testing in multivariate linear regression models under non normality

This paper discusses the problem of statistical inference in multivariate linear regression models when the errors involved are non normally distributed. We consider multivariate t-distribution, a fat-tailed distribution, for the errors as alternative to normal distribution. Such non normality is commonly observed in working with many data sets, e.g., financial data that are usually having excess kurtosis. This distribution has a number of applications in many other areas of research as well. We use modified maximum likelihood estimation method that provides the estimator, called modified maximum likelihood estimator (MMLE), in closed form. These estimators are shown to be unbiased, efficient, and robust as compared to the widely used least square estimators (LSEs). Also, the tests based upon MMLEs are found to be more powerful than the similar tests based upon LSEs.     

Approximate estimations in capture–recapture model with heteroscedastic measurement errors

In a capture–recapture experiment, the number of measurements for individual covariates usually equals the number of captures. This creates a heteroscedastic measurement error problem and the usual surrogate condition does not hold in the context of a measurement error model. This study adopts a small measurement error assumption to approximate the conventional estimating functions and the population size estimator. This study also investigates the biases of the resulting estimators. In addition, modifications for two common approximation methods, regression calibration and simulation extrapolation, to accommodate heteroscedastic measurement error are also discussed. These estimation methods are examined through simulations and illustrated by analysing a capture–recapture data set.     

Estimation of Standard Errors of Empirical Bayes Estimators in Capm-Type Models

This paper is concerned with the problem of estimating the standard errors of the empirical Bayes estimators in linear regression models. The problem of deriving an exact expression for the standard error of this estimator is generally intractable. We suggest a procedure based on Efron’s bootstrap method as a way of estimating the standard error. It is shown, through simulations, that the bootstrap method provides a more accurate estimate of the standard error of the empirical Bayes estimator than the traditional large sample method.     

Consistent estimation of regression coefficients in ultrastructural measurement error model using stochastic prior information

The problem of consistent estimation of regression coefficients in a multivariate linear ultrastructural measurement error model is considered in this article when some additional information on regression coefficients is available a priori. Such additional information is expressible in the form of stochastic linear restrictions. Utilizing stochastic restrictions given a priori, some methodologies are presented to obtain the consistent estimators of regression coefficients under two types of additional information separately, viz., covariance matrix of measurement errors and reliability matrix associated with explanatory variables. The measurement errors are assumed to be not necessarily normally distributed. The asymptotic properties of the proposed estimators are derived and analyzed analytically as well as numerically through a Monte Carlo simulation experiment.     

Randomized quantile regression estimation for heteroskedastic non parametric model

In this paper, we propose robust randomized quantile regression estimators for the mean and (condition) variance functions of the popular heteroskedastic non parametric regression model. Unlike classical approaches which consider quantile as a fixed quantity, our method treats quantile as a uniformly distributed random variable. Our proposed method can be employed to estimate the error distribution, which could significantly improve prediction results. An automatic bandwidth selection scheme will be discussed. Asymptotic properties and relative efficiencies of the proposed estimators are investigated. Our empirical results show that the proposed estimators work well even for random errors with infinite variances. Various numerical simulations and two real data examples are used to demonstrate our methodologies.     

Estimation of population distribution function involving measurement error in the presence of non response

Abstract

This article addresses the problem of estimating population distribution function for simple random sampling in the presence of non response and measurement error together. We suggest a general class of estimators for estimating the cumulative distribution function using the auxiliary information. The expressions for the bias and mean squared error are derived up to the first order of approximation. The performance of the proposed class of estimators is compared with considered estimators both theoretically and numerically. A real data set is used to support the theoretical findings.     

Empirical distribution function under heteroscedasticity

Neglecting heteroscedasticity of error terms may imply the wrong identification of a regression model (see appendix). Employment of (heteroscedasticity resistent) White's estimator of covariance matrix of estimates of regression coefficients may lead to the correct decision about the significance of individual explanatory variables under heteroscedasticity. However, White's estimator of covariance matrix was established for least squares (LS)-regression analysis (in the case when error terms are normally distributed, LS- and maximum likelihood (ML)-analysis coincide and hence then White's estimate of covariance matrix is available for ML-regression analysis, tool). To establish White's-type estimate for another estimator of regression coefficients requires Bahadur representation of the estimator in question, under heteroscedasticity of error terms. The derivation of Bahadur representation for other (robust) estimators requires some tools. As the key too proved to be a tight approximation of the empirical distribution function (d.f.) of residuals by the theoretical d.f. of the error terms of the regression model. We need the approximation to be uniform in the argument of d.f. as well as in regression coefficients. The present paper offers this approximation for the situation when the error terms are heteroscedastic.     

A New Characterization and Estimation of the Zero–bias Bandwidth

It is well established that bandwidths exist that can yield an unbiased non–parametric kernel density estimate at points in particular regions (e.g. convex regions) of the underlying density. These zero–bias bandwidths have superior theoretical properties, including a 1/n convergence rate of the mean squared error. However, the explicit functional form of the zero–bias bandwidth has remained elusive. It is difficult to estimate these bandwidths and virtually impossible to achieve the higher–order rate in practice. This paper addresses these issues by taking a fundamentally different approach to the asymptotics of the kernel density estimator to derive a functional approximation to the zero–bias bandwidth. It develops a simple approximation algorithm that focuses on estimating these zero–bias bandwidths in the tails of densities where the convexity conditions favourable to the existence of the zerobias bandwidths are more natural. The estimated bandwidths yield density estimates with mean squared error that is O(n^–4/5), the same rate as the mean squared error of density estimates with other choices of local bandwidths. Simulation studies and an illustrative example with air pollution data show that these estimated zero–bias bandwidths outperform other global and local bandwidth estimators in estimating points in the tails of densities.     

A Combined Nonlinear Programming Model and Kibria Method for Choosing Ridge Parameter Regression

Ridge regression solves multicollinearity problems by introducing a biasing parameter that is called ridge parameter; it shrinks the estimates as well as their standard errors in order to reach acceptable results. Many methods are available for estimating a ridge parameter. This article has considered some of these methods and also proposed a combined nonlinear programming model and Kibria method. A simulation study has been made to evaluate the performance of the proposed estimators based on the minimum mean squared error criterion. The simulation study indicates that under certain conditions the proposed estimators outperform the least squares (LS) estimators and other popular existing estimators. Moreover, the new proposed model is applied on dataset that suffers also from the presence of heteroscedastic errors.     

Heteroskedasticity-consistent interval estimators

The linear regression model is commonly used in applications. One of the assumptions made is that the error variances are constant across all observations. This assumption, known as homoskedasticity, is frequently violated in practice. A commonly used strategy is to estimate the regression parameters by ordinary least squares and to compute standard errors that deliver asymptotically valid inference under both homoskedasticity and heteroskedasticity of an unknown form. Several consistent standard errors have been proposed in the literature, and evaluated in numerical experiments based on their point estimation performance and on the finite sample behaviour of associated hypothesis tests. We build upon the existing literature by constructing heteroskedasticity-consistent interval estimators and numerically evaluating their finite sample performance. Different bootstrap interval estimators are also considered. The numerical results favour the HC4 interval estimator.     

Linear calibration in functional regression models

This paper discusses calibration in functional regression models. Classical and inverse type estimators are considered. First order approximation to the bias and to the mean squared error (MSE) of the estimators are considered. Numerical comparisons seem to indicate that the classical estimator obtained via maximum likelihood estimation performs better than the other estimators considered.     

A robust and efficient estimation method for nonparametric models with jump points

Nonparametric models with jump points have been considered by many researchers. However, most existing methods based on least squares or likelihood are sensitive when there are outliers or the error distribution is heavy tailed. In this article, a local piecewise-modal method is proposed to estimate the regression function with jump points in nonparametric models, and a piecewise-modal EM algorithm is introduced to estimate the proposed estimator. Under some regular conditions, the large-sample theory is established for the proposed estimators. Several simulations are presented to evaluate the performances of the proposed method, which shows that the proposed estimator is more efficient than the local piecewise-polynomial regression estimator in the presence of outliers or heavy tail error distribution. What is more, the proposed procedure is asymptotically equivalent to the local piecewise-polynomial regression estimator under the assumption that the error distribution is a Gaussian distribution. The proposed method is further illustrated via the sea-level pressures.     

A new class of ratio-type estimators for improving mean estimation of nonsensitive and sensitive variables by using supplementary information

ABSTRACT

Motivated by some recent improvements for mean estimation in finite sampling theory, we propose, in a design-based approach, a new class of ratio-type estimators. The class is initially discussed on the assumption that the study variable has a nonsensitive nature, meaning that it deals with topics that do not generate embarrassment when respondents are directly questioned about them. Under this standard setting, some estimators belonging to the class are shown and the bias, mean square error and minimum mean square error are determined up to the first-order of approximation. The class is subsequently extended to the case where the study variable refers to sensitive issues which produce measurement errors due to nonresponses and/or untruthful reporting. These errors may be reduced by enhancing respondent cooperation through scrambled response methods that mask the true value of the sensitive variable. Hence, four methods (say the additive, multiplicative, mixed and combined additive-multiplicative methods) are discussed for the purposes of the article. Finally, a simulation study is carried out to assess the performance of the proposed class by comparing a number of competing estimators, both in the sensitive and the nonsensitive setting.     

Using difference-based methods for inference in nonparametric regression with time series errors

Summary. We show that difference-based methods can be used to construct simple and explicit estimators of error covariance and autoregressive parameters in nonparametric regression with time series errors. When the error process is Gaussian our estimators are efficient, but they are available well beyond the Gaussian case. As an illustration of their usefulness we show that difference-based estimators can be used to produce a simplified version of time series cross-validation. This new approach produces a bandwidth selector that is equivalent, to both first and second orders, to that given by the full time series cross-validation algorithm. Other applications of difference-based methods are to variance estimation and construction of confidence bands in nonparametric regression.     

Efficient estimation in a regression model with missing responses

This article examines methods to efficiently estimate the mean response in a linear model with an unknown error distribution under the assumption that the responses are missing at random. We show how the asymptotic variance is affected by the estimator of the regression parameter, and by the imputation method. To estimate the regression parameter, the ordinary least squares is efficient only if the error distribution happens to be normal. If the errors are not normal, then we propose a one step improvement estimator or a maximum empirical likelihood estimator to efficiently estimate the parameter.To investigate the imputation’s impact on the estimation of the mean response, we compare the listwise deletion method and the propensity score method (which do not use imputation at all), and two imputation methods. We demonstrate that listwise deletion and the propensity score method are inefficient. Partial imputation, where only the missing responses are imputed, is compared to full imputation, where both missing and non-missing responses are imputed. Our results reveal that, in general, full imputation is better than partial imputation. However, when the regression parameter is estimated very poorly, the partial imputation will outperform full imputation. The efficient estimator for the mean response is the full imputation estimator that utilizes an efficient estimator of the parameter.     

Applications of the Bootstrap in ROC Analysis

The problem of estimating standard errors for diagnostic accuracy measures might be challenging for many complicated models. We can address such a problem by using the Bootstrap methods to blunt its technical edge with resampled empirical distributions. We consider two cases where bootstrap methods can successfully improve our knowledge of the sampling variability of the diagnostic accuracy estimators. The first application is to make inference for the area under the ROC curve resulted from a functional logistic regression model which is a sophisticated modelling device to describe the relationship between a dichotomous response and multiple covariates. We consider using this regression method to model the predictive effects of multiple independent variables on the occurrence of a disease. The accuracy measures, such as the area under the ROC curve (AUC) are developed from the functional regression. Asymptotical results for the empirical estimators are provided to facilitate inferences. The second application is to test the difference of two weighted areas under the ROC curve (WAUC) from a paired two sample study. The correlation between the two WAUC complicates the asymptotic distribution of the test statistic. We then employ the bootstrap methods to gain satisfactory inference results. Simulations and examples are supplied in this article to confirm the merits of the bootstrap methods.