An Estimate of a Change Point in Variance of Measurement Errors and Its Convergence Rate

In this article, an estimate of a change point in variance of measurement errors (ME) is given in terms of characteristic functions when the variances are known. Its modification is also given for the case that the variances are unknown. In addition, the consistency and convergence rates of the estimator and its modification are investigated. The simulation study shows that the proposed estimators perform well.     

Outlier detection using difference-based variance estimators in multiple regression

In this article, we propose an outlier detection approach in a multiple regression model using the properties of a difference-based variance estimator. This type of a difference-based variance estimator was originally used to estimate error variance in a non parametric regression model without estimating a non parametric function. This article first employed a difference-based error variance estimator to study the outlier detection problem in a multiple regression model. Our approach uses the leave-one-out type method based on difference-based error variance. The existing outlier detection approaches using the leave-one-out approach are highly affected by other outliers, while ours is not because our approach does not use the regression coefficient estimator. We compared our approach with several existing methods using a simulation study, suggesting the outperformance of our approach. The advantages of our approach are demonstrated using a real data application. Our approach can be extended to the non parametric regression model for outlier detection.     

Finite sample distributions of statistics in change point analysis

This paper is concerned with derivation of finite sampling distributions of some statistics which appear frequently in change point analysis. The exact distribution of cusum test statistic is approximated by two methods. Approximations are presented and their accuracies are measured. We first consider the change point in mean problem and we study the exact distribution of change point estimator. Finally, we consider the change point in variance case.     

A weighted bootstrap approximation for comparing the error distributions in nonparametric regression

Several procedures have been proposed for testing the equality of error distributions in two or more nonparametric regression models. Here we deal with methods based on comparing estimators of the cumulative distribution function (CDF) of the errors in each population to an estimator of the common CDF under the null hypothesis. The null distribution of the associated test statistics has been approximated by means of a smooth bootstrap (SB) estimator. This paper proposes to approximate their null distribution through a weighted bootstrap. It is shown that it produces a consistent estimator. The finite sample performance of this approximation is assessed by means of a simulation study, where it is also compared to the SB. This study reveals that, from a computational point of view, the proposed approximation is more efficient than the one provided by the SB.     

ON THE USE OF THE STEIN VARIANCE ESTIMATOR IN THE DOUBLE k-CLASS ESTIMATOR IN REGRESSION

This paper investigates the predictive mean squared error performance of a modified double k-class estimator by incorporating the Stein variance estimator. Recent studies show that the performance of the Stein rule estimator can be improved by using the Stein variance estimator. However, as we demonstrate below, this conclusion does not hold in general for all members of the double k-class estimators. On the other hand, an estimator is found to have smaller predictive mean squared error than the Stein variance-Stein rule estimator, over quite large parts of the parameter space.     

Nonparametric kernel estimation of expected shortfall under negatively associated sequences

Abstract

In this article, Bahadur type expansions of a nonparametric kernel estimator for ES under NA sequences are given. The strong consistency and the uniformly asymptotic normality of the estimator are yielded from the Bahadur type expansions, while the convergence rates of the above asymptotic properties are also obtained. Moreover, the expectation, the variance and the mean squared error (MSE) of the estimator are given. Besides, the optimal bandwidth selection of this estimator is discussed. We point out that all above results are based on the NA sequences. Finally, we conduct numerical simulations and compare performances of some ES estimators.     

Reducing the mean squared error in kernel density estimation

In this article, we propose a version of a kernel density estimator which reduces the mean squared error of the existing kernel density estimator by combining bias reduction and variance reduction techniques. Its theoretical properties are investigated, and a Monte Carlo simulation study supporting theoretical results on the proposed estimator is given.     

Bayesian Estimation of the Change Point Using X¯ Control Chart

The process personnel always seek the opportunity to improve the processes. One of the essential steps for process improvement is to quickly recognize the starting time or the change point of a process disturbance. The proposed approach combines the X¯ control chart with the Bayesian estimation technique. We show that the control chart has some information about the change point and this information can be used to make an informative prior. Then two Bayes estimators corresponding to the informative and a non informative prior along with MLE are considered. Their efficiencies are compared through a series of simulations. The results show that the Bayes estimator with the informative prior is more accurate and more precise when the means of the process before and after the change point time are not too closed. In addition, the efficiency of the Bayes estimator with the informative prior increases as the change point goes away from the origin.     

On the use of the Stein variance estimator in the double <Emphasis Type="Italic">k</Emphasis>-class estimator when each individual regression coefficient is estimated

In this paper we consider the double k-class estimator which incorporates the Stein variance estimator. This estimator is called the SVKK estimator. We derive the explicit formula for the mean squared error (MSE) of the SVKK estimator for each individual regression coefficient. It is shown analytically that the MSE performance of the Stein-rule estimator for each individual regression coefficient can be improved by utilizing the Stein variance estimator. Also, MSE’s of several estimators included in a family of the SVKK estimators are compared by numerical evaluations.     

Variance Estimation and Asymptotic Confidence Bands for the Mean Estimator of Sampled Functional Data with High Entropy Unequal Probability Sampling Designs

For fixed size sampling designs with high entropy, it is well known that the variance of the Horvitz–Thompson estimator can be approximated by the Hájek formula. The interest of this asymptotic variance approximation is that it only involves the first order inclusion probabilities of the statistical units. We extend this variance formula when the variable under study is functional, and we prove, under general conditions on the regularity of the individual trajectories and the sampling design, that we can get a uniformly convergent estimator of the variance function of the Horvitz–Thompson estimator of the mean function. Rates of convergence to the true variance function are given for the rejective sampling. We deduce, under conditions on the entropy of the sampling design, that it is possible to build confidence bands whose coverage is asymptotically the desired one via simulation of Gaussian processes with variance function given by the Hájek formula. Finally, the accuracy of the proposed variance estimator is evaluated on samples of electricity consumption data measured every half an hour over a period of 1 week.     

A Note on Prediction Error with Collinearity

Abstract

Least squares (LS) estimator is the best linear unbiased estimator for linear models. It is well known that LS performs poorly in estimation when collinearity is present among regressors. However, it is not fully understood and is even controversial whether LS performs well in prediction. To address this controversy, we study the mean and variance of the prediction squared error (PSE) of LS estimator, and conclude theoretically that although the mean PSE remains invariant regardless of the collinearity, the variance of PSE increases with the collinearity. Thus the prediction error is sensitive to the location in the feature space.     

A conditional perspective of weighted variance estimation of the optimal regression estimator

The estimation of the variance for the GREG (general regression) estimator by weighted residuals is widely accepted as a method which yields estimators with good conditional properties. Since the optimal (regression) estimator shares the properties of GREG estimators which are used in the construction of weighted variance estimators, we introduce the weighting procedure also for estimating the variance of the optimal estimator. This method of variance estimation was originally presented in a seemingly ad hoc manner, and we shall discuss it from a conditional point of view and also look at an alternative way of utilizing the weights. Examples that stress conditional behaviour of estimators are then given for elementary sampling designs such as simple random sampling, stratified simple random sampling and Poisson sampling, where for the latter design we have conducted a small simulation study.     

Empirical Uncertain Bayes Methods in Area‐level Models

Random effects model can account for the lack of fitting a regression model and increase precision of estimating area‐level means. However, in case that the synthetic mean provides accurate estimates, the prior distribution may inflate an estimation error. Thus, it is desirable to consider the uncertain prior distribution, which is expressed as the mixture of a one‐point distribution and a proper prior distribution. In this paper, we develop an empirical Bayes approach for estimating area‐level means, using the uncertain prior distribution in the context of a natural exponential family, which we call the empirical uncertain Bayes (EUB) method. The regression model considered in this paper includes the Poisson‐gamma and the binomial‐beta, and the normal‐normal (Fay–Herriot) model, which are typically used in small area estimation. We obtain the estimators of hyperparameters based on the marginal likelihood by using a well‐known expectation‐maximization algorithm and propose the EUB estimators of area means. For risk evaluation of the EUB estimator, we derive a second‐order unbiased estimator of a conditional mean squared error by using some techniques of numerical calculation. Through simulation studies and real data applications, we evaluate a performance of the EUB estimator and compare it with the usual empirical Bayes estimator.     

The effects of the proxy information on the iterative Stein-rule estimator of the disturbance variance

In a regression model with proxy variables, we consider the iterative estimator of the disturbance variance to obtain more precise estimates. In the formula of the estimator of the disturbance variance, the estimator is obtained by using Stein-rule (SR) estimator instead of OLS (ordinary least squares) estimator is called Iterative estimator of the disturbance variance. It is shown that, in a regression model with proxy variables the mean square error (MSE) of the iterative estimator of the disturbance variance is greater than the MSE of the disturbance variance related to the OLS estimator under certain conditions.     

On the Estimation of the Probability Density by Trigonometric Series

We present in this article an estimator based on a new orthogonal trigonometric series. We give its statistical properties (bias, variance, mean square error, and mean integrated square error) and the asymptotic properties (convergence of variance, convergence of the mean square error, convergence of the mean integrated square error, uniform convergence in probability, and the rate of convergence of the mean integrated square error). The comparison by simulation on a test density between the estimator obtained from a new trigonometric series with Fejer estimator also based on orthogonal trigonometric series, shows that our estimator is more performant in the sense of the mean integrated square error.     

Estimation of a Change Point in a Hazard Function Based on Censored Data

The hazard function plays an important role in reliability or survival studies since it describes the instantaneous risk of failure of items at a time point, given that they have not failed before. In some real life applications, abrupt changes in the hazard function are observed due to overhauls, major operations or specific maintenance activities. In such situations it is of interest to detect the location where such a change occurs and estimate the size of the change. In this paper we consider the problem of estimating a single change point in a piecewise constant hazard function when the observed variables are subject to random censoring. We suggest an estimation procedure that is based on certain structural properties and on least squares ideas. A simulation study is carried out to compare the performance of this estimator with two estimators available in the literature: an estimator based on a functional of the Nelson-Aalen estimator and a maximum likelihood estimator. The proposed least squares estimator tums out to be less biased than the other two estimators, but has a larger variance. We illustrate the estimation method on some real data sets.     

A tobit model with garch errors

In the context of time series regression, we extend the standard Tobit model to allow for the possibility of conditional heteroskedastic error processes of the GARCH type. We discuss the likelihood function of the Tobit model in the presence of conditionally heteroskedastic errors. Expressing the exact likelihood function turns out to be infeasible, and we propose an approximation by treating the model as being conditionally Gaussian. The performance of the estimator is investigated by means of Monte Carlo simulations. We find that, when the error terms follow a GARCH process, the proposed estimator considerably outperforms the standard Tobit quasi maximum likelihood estimator. The efficiency loss due to the approximation of the likelihood is finally evaluated.     

Identifying the time of step change in the mean of autocorrelated processes

Control charts are used to detect changes in a process. Once a change is detected, knowledge of the change point would simplify the search for and identification of the special cause. Consequently, having an estimate of the process change point following a control chart signal would be useful to process analysts. Change-point methods for the uncorrelated process have been studied extensively in the literature; however, less attention has been given to change-point methods for autocorrelated processes. Autocorrelation is common in practice and is often modeled via the class of autoregressive moving average (ARMA) models. In this article, a maximum likelihood estimator for the time of step change in the mean of covariance-stationary processes that fall within the general ARMA framework is developed. The estimator is intended to be used as an “add-on” following a signal from a phase II control chart. Considering first-order pure and mixed ARMA processes, Monte Carlo simulation is used to evaluate the performance of the proposed change-point estimator across a range of step change magnitudes following a genuine signal from a control chart. Results indicate that the estimator provides process analysts with an accurate and useful estimate of the last sample obtained from the unchanged process. Additionally, results indicate that if a change-point estimator designed for the uncorrelated process is applied to an autocorrelated process, the performance of the estimator can suffer dramatically.     

Restricted ridge estimator in the logistic regression model

It is known that when the multicollinearity exists in the logistic regression model, variance of maximum likelihood estimator is unstable. As a remedy, Schaefer et al. presented a ridge estimator in the logistic regression model. Making use of the ridge estimator, when some linear restrictions are also present, we introduce a restricted ridge estimator in the logistic regression model. Statistical properties of this newly defined estimator will be studied and comparisons are done in the simulation study in the sense of mean squared error criterion. A real-data example and a simulation study are introduced to discuss the performance of this estimator.     

Two-parameter ridge estimator in the binary logistic regression

The binary logistic regression is a commonly used statistical method when the outcome variable is dichotomous or binary. The explanatory variables are correlated in some situations of the logit model. This problem is called multicollinearity. It is known that the variance of the maximum likelihood estimator (MLE) is inflated in the presence of multicollinearity. Therefore, in this study, we define a new two-parameter ridge estimator for the logistic regression model to decrease the variance and overcome multicollinearity problem. We compare the new estimator to the other well-known estimators by studying their mean squared error (MSE) properties. Moreover, a Monte Carlo simulation is designed to evaluate the performances of the estimators. Finally, a real data application is illustrated to show the applicability of the new method. According to the results of the simulation and real application, the new estimator outperforms the other estimators for all of the situations considered.