Bayesian inverse problems with heterogeneous variance

We consider inverse problems in Hilbert spaces under correlated Gaussian noise, and use a Bayesian approach to find their regularized solution. We focus on mildly ill-posed inverse problems with fractional noise, using a novel wavelet-based vaguelette–vaguelette approach. It allows us to apply sequence space methods without assuming that all operators are simultaneously diagonalizable. The results are proved for more general bases and covariance operators. Our primary aim is to study posterior contraction rate in such inverse problems over Sobolev classes and compare it to the derived minimax rate. Secondly, we study effect of plugging in a consistent estimator of variances in sequence space on the posterior contraction rate. This result is applied to the problem with error in forward operator. Thirdly, we show that empirical Bayes posterior distribution with a plugged-in maximum marginal likelihood estimator of the prior scale contracts at the optimal rate, adaptively, in the minimax sense.     

A comparison of estimators for the mean in the inverse gaussian distribution with a known coefficient of variation

In this paper, we discuss an estimation problem of the mean in the inverse Gaussian distribution with a known coefficient of variation. Two types of linear estimators for the mean, the linear minimum variance unbiased estimator and the linear minimum mean squared error estimator, are constructed by using the squared error loss function and their properties are examined. It is observed that, for small samples the performance of the proposed estimators is better than that of the maximum likelihood estimator, when the coefficient of variation is large.     

Mean likelihood estimators

The use of Mathematica in deriving mean likelihood estimators is discussed. Comparisons are made between the mean likelihood estimator, the maximum likelihood estimator, and the Bayes estimator based on a Jeffrey's noninformative prior. These estimators are compared using the mean-square error criterion and Pitman measure of closeness. In some cases it is possible, using Mathematica, to derive exact results for these criteria. Using Mathematica, simulation comparisons among the criteria can be made for any model for which we can readily obtain estimators.In the binomial and exponential distribution cases, these criteria are evaluated exactly. In the first-order moving-average model, analytical comparisons are possible only for n = 2. In general, we find that for the binomial distribution and the first-order moving-average time series model the mean likelihood estimator outperforms the maximum likelihood estimator and the Bayes estimator with a Jeffrey's noninformative prior. Mathematica was used for symbolic and numeric computations as well as for the graphical display of results. A Mathematica notebook which provides the Mathematica code used in this article is available: http://www.stats.uwo.ca/mcleod/epubs/mele. Our article concludes with our opinions and criticisms of the relative merits of some of the popular computing environments for statistics researchers.     

A New Regression Model: Modal Linear Regression

The mode of a distribution provides an important summary of data and is often estimated on the basis of some non‐parametric kernel density estimator. This article develops a new data analysis tool called modal linear regression in order to explore high‐dimensional data. Modal linear regression models the conditional mode of a response Y given a set of predictors x as a linear function of x . Modal linear regression differs from standard linear regression in that standard linear regression models the conditional mean (as opposed to mode) of Y as a linear function of x . We propose an expectation–maximization algorithm in order to estimate the regression coefficients of modal linear regression. We also provide asymptotic properties for the proposed estimator without the symmetric assumption of the error density. Our empirical studies with simulated data and real data demonstrate that the proposed modal regression gives shorter predictive intervals than mean linear regression, median linear regression and MM‐estimators.     

Goodness-of-fit test based on correcting moments of modified entropy estimator

In this paper, we first propose a new estimator of entropy for continuous random variables. Our estimator is obtained by correcting the coefficients of Vasicek's [A test for normality based on sample entropy, J. R. Statist. Soc. Ser. B 38 (1976), pp. 54–59] entropy estimator. We prove the consistency of our estimator. Monte Carlo studies show that our estimator is better than the entropy estimators proposed by Vasicek, Ebrahimi et al. [Two measures of sample entropy, Stat. Probab. Lett. 20 (1994), pp. 225–234] and Correa [A new estimator of entropy, Commun. Stat. Theory Methods 24 (1995), pp. 2439–2449] in terms of root mean square error. We then derive the non-parametric distribution function corresponding to our proposed entropy estimator as a piece-wise uniform distribution. We also introduce goodness-of-fit tests for testing exponentiality and normality based on the said distribution and compare its performance with their leading competitors.     

Penalized contrast estimation in functional linear models with circular data

Our aim is to estimate the unknown slope function in the functional linear model when the response Y is real and the random function X is a second-order stationary and periodic process. We obtain our estimator by minimizing a standard (and very simple) mean-square contrast on linear finite dimensional spaces spanned by trigonometric bases. Our approach provides a penalization procedure which allows to automatically select the adequate dimension, in a non-asymptotic point of view. In fact, we can show that our penalized estimator reaches the optimal (minimax) rate of convergence in the sense of the prediction error. We complete the theoretical results by a simulation study and a real example that illustrates how the procedure works in practice.     

Quasi-minimax estimation in the partial linear model

This article discusses the minimax estimator in partial linear model y = Zβ + f + ε under ellipsoidal restrictions on the parameter space and quadratic loss function. The superiority of the minimax estimator over the two-step estimator is studied in the mean squared error matrix criterion.     

Nonparametric local polynomial approximation of the time-varying frequency and amplitude

The problem of estimating the time-varying frequency, phase and amplitude of a real-valued harmonic signal is considered. It is assumed that the frequency and amplitude are unspecified rapidly time-varying functions of time. The technique is based on fitting a local polynomial approximation of the phase and amplitude which implements a high-order nonlinear nonparametric estimator. The estimator is shown to be strongly consistent and Gaussian. In particular, the convergence ratesO(h^-3/2 )and O(h^-5/2 ), where $i;h$ei; is the number of observations, are obtained for the frequency estimator when the amplitude is unknown constant or linear in time respectively. The orders of the bias and Gaussian distribution are obtained for a class of the time-varying frequency and amplitude with bounded second derivatives. The a priori amplitude information about the unknown time-varying frequency and amplitude and their derivatives can be incorporated to improve the accuracy of the estimation. Simulation results are given.     

PMSE dominance of the positive-part shrinkage estimator in a regression model with proxy variables

Consider a linear regression model with some relevant regressors are unobservable. In such a situation, we estimate the model by using the proxy variables as regressors or by simply omitting the relevant regressors. In this paper, we derive the explicit formula of predictive mean squared error (PMSE) of a general family of shrinkage estimators of regression coefficients. It is shown analytically that the positive-part shrinkage estimator dominates the ordinary shrinkage estimator even when proxy variables are used in place of the unobserved variables. Also, as an example, our result is applied to the double k-class estimator proposed by Ullah and Ullah (Double k-class estimators of coefficients in linear regression. Econometrica. 1978;46:705–722). Our numerical results show that the positive-part double k-class estimator with proxy variables has preferable PMSE performance.     

A Robust Variable Selection to t-type Joint Generalized Linear Models via Penalized t-type Pseudo-likelihood

Although the t-type estimator is a kind of M-estimator with scale optimization, it has some advantages over the M-estimator. In this article, we first propose a t-type joint generalized linear model as a robust extension to the classical joint generalized linear models for modeling data containing extreme or outlying observations. Next, we develop a t-type pseudo-likelihood (TPL) approach, which can be viewed as a robust version to the existing pseudo-likelihood (PL) approach. To determine which variables significantly affect the variance of the response variable, we then propose a unified penalized maximum TPL method to simultaneously select significant variables for the mean and dispersion models in t-type joint generalized linear models. Thus, the proposed variable selection method can simultaneously perform parameter estimation and variable selection in the mean and dispersion models. With appropriate selection of the tuning parameters, we establish the consistency and the oracle property of the regularized estimators. Simulation studies are conducted to illustrate the proposed methods.     

Value-at-risk forecasting based on Gaussian mixture ARMA–GARCH model

In this paper, we develop a new forecasting algorithm for value-at-risk (VaR) based on ARMA–GARCH (autoregressive moving average–generalized autoregressive conditional heteroskedastic) models whose innovations follow a Gaussian mixture distribution. For the parameter estimation, we employ the conditional least squares and quasi-maximum-likelihood estimator (QMLE) for ARMA and GARCH parameters, respectively. In particular, Gaussian mixture parameters are estimated based on the residuals obtained from the QMLE of GARCH parameters. Our algorithm provides a handy methodology, spending much less time in calculation than the existing resampling and bias-correction method developed in Hartz et al. [Accurate value-at-risk forecasting based on the normal-GARCH model, Comput. Stat. Data Anal. 50 (2006), pp. 3032–3052]. Through a simulation study and a real-data analysis, it is shown that our method provides an accurate VaR prediction.     

A Generalized Class of Ratio Type Exponential Estimators of Population Mean Under Linear Transformation of Auxiliary Variable

Recently, Shabbir and Gupta [Shabbir, J. and Gupta, S. (2011). On estimating finite population mean in simple and stratified random sampling. Communications in Statistics-Theory and Methods, 40(2), 199–212] defined a class of ratio type exponential estimators of population mean under a very specific linear transformation of auxiliary variable. In the present article, we propose a generalized class of ratio type exponential estimators of population mean in simple random sampling under a very general linear transformation of auxiliary variable. Shabbir and Gupta's [Shabbir, J. and Gupta, S. (2011). On estimating finite population mean in simple and stratified random sampling. Communications in Statistics-Theory and Methods, 40(2), 199–212] class of estimators is a particular member of our proposed class of estimators. It has been found that the optimal estimator of our proposed generalized class of estimators is always more efficient than almost all the existing estimators defined under the same situations. Moreover, in comparison to a few existing estimators, our proposed estimator becomes more efficient under some simple conditions. Theoretical results obtained in the article have been verified by taking a numerical illustration. Finally, a simulation study has been carried out to see the relative performance of our proposed estimator with respect to some existing estimators which are less efficient under certain conditions as compared to the proposed estimator.     

A stochastic restricted k–d class estimator

In this paper, we introduce a stochastic restricted k–d class estimator for the vector of parameters in a linear model when additional linear restrictions on the parameter vector are assumed to hold. The stochastic restricted k–d class estimator is a generalization of the ordinary mixed estimator and the k–d class estimator. We show that our new biased estimator is superior in the mean squared error matrix sense to the k–d class estimator [S. Sakall?o?lu and S. Kaçiranlar, A new biased estimator based on ridge estimation, Statist. Papers 49 (2008), pp. 669–689] and the stochastic restricted Liu estimator [H. Yang and J.W. Xu, An alternative stochastic restricted Liu estimator in linear regression, Statist. Papers 50 (2009), pp. 639–647]. Finally, a numerical example is given to show the theoretical results.     

Different estimators of the spectral matrix: an empirical comparison testing a new shrinkage estimator

ABSTRACT

In this paper we propose a new non parametric estimator of the spectral matrix of a multivariate stationary stochastic process, with the main goal to locally improve the deficiencies of the smoothed periodogram in terms of mean square error of the estimates. Our estimator is based on a convex linear combination of the frequency averaged periodogram and an estimate of the true mean spectral matrix across frequencies. In a wide simulation study we show that our estimator turns out to be able to markedly improve the frequency averaged periodogram especially at central frequencies.     

Robust ridge and robust Liu estimator for regression based on the LTS estimator

In the multiple linear regression analysis, the ridge regression estimator and the Liu estimator are often used to address multicollinearity. Besides multicollinearity, outliers are also a problem in the multiple linear regression analysis. We propose new biased estimators based on the least trimmed squares (LTS) ridge estimator and the LTS Liu estimator in the case of the presence of both outliers and multicollinearity. For this purpose, a simulation study is conducted in order to see the difference between the robust ridge estimator and the robust Liu estimator in terms of their effectiveness; the mean square error. In our simulations, the behavior of the new biased estimators is examined for types of outliers: X-space outlier, Y-space outlier, and X-and Y-space outlier. The results for a number of different illustrative cases are presented. This paper also provides the results for the robust ridge regression and robust Liu estimators based on a real-life data set combining the problem of multicollinearity and outliers.     

Objective Bayesian Analysis of Skew‐t Distributions

Abstract. We study the Jeffreys prior and its properties for the shape parameter of univariate skew‐t distributions with linear and nonlinear Student's t skewing functions. In both cases, we show that the resulting priors for the shape parameter are symmetric around zero and proper. Moreover, we propose a Student's t approximation of the Jeffreys prior that makes an objective Bayesian analysis easy to perform. We carry out a Monte Carlo simulation study that demonstrates an overall better behaviour of the maximum a posteriori estimator compared with the maximum likelihood estimator. We also compare the frequentist coverage of the credible intervals based on the Jeffreys prior and its approximation and show that they are similar. We further discuss location‐scale models under scale mixtures of skew‐normal distributions and show some conditions for the existence of the posterior distribution and its moments. Finally, we present three numerical examples to illustrate the implications of our results on inference for skew‐t distributions.     

Adaptive smoothing in associated kernel discrete functions estimation using Bayesian approach

This paper demonstrates that cross-validation (CV) and Bayesian adaptive bandwidth selection can be applied in the estimation of associated kernel discrete functions. This idea is originally proposed by Brewer [A Bayesian model for local smoothing in kernel density estimation, Stat. Comput. 10 (2000), pp. 299–309] to derive variable bandwidths in adaptive kernel density estimation. Our approach considers the adaptive binomial kernel estimator and treats the variable bandwidths as parameters with beta prior distribution. The best variable bandwidth selector is estimated by the posterior mean in the Bayesian sense under squared error loss. Monte Carlo simulations are conducted to examine the performance of the proposed Bayesian adaptive approach in comparison with the performance of the Asymptotic mean integrated squared error estimator and CV technique for selecting a global (fixed) bandwidth proposed in Kokonendji and Senga Kiessé [Discrete associated kernels method and extensions, Stat. Methodol. 8 (2011), pp. 497–516]. The Bayesian adaptive bandwidth estimator performs better than the global bandwidth, in particular for small and moderate sample sizes.     

Near Optimal Prediction from Relevant Components

Abstract. The random x regression model is approached through the group of rotations of the eigenvectors for the x ‐covariance matrix together with scale transformations for each of the corresponding regression coefficients. The partial least squares model can be constructed from the orbits of this group. A generalization of Pitman's Theorem says that the best equivariant estimator under a group is given by the Bayes estimator with the group's invariant measure as the prior. A straightforward application of this theorem turns out to be impossible since the relevant invariant prior leads to a non‐defined posterior. Nevertheless we can devise an approximate scale group with a proper invariant prior leading to a well‐defined posterior distribution with a finite mean. This Bayes estimator is explored using Markov chain Monte Carlo technique. The estimator seems to require heavy computations, but can be argued to have several nice properties. It is also a valid estimator when p>n.     

Improved robust Bayes estimators of the error variance in linear models

We consider the problem of estimating the error variance in a general linear model when the error distribution is assumed to be spherically symmetric, but not necessary Gaussian. In particular we study the case of a scale mixture of Gaussians including the particularly important case of the multivariate-t distribution. Under Stein's loss, we construct a class of estimators that improve on the usual best unbiased (and best equivariant) estimator. Our class has the interesting double robustness property of being simultaneously generalized Bayes (for the same generalized prior) and minimax over the entire class of scale mixture of Gaussian distributions.     

Two-sample high-dimensional empirical likelihood

In this paper, we apply empirical likelihood for two-sample problems with growing high dimensionality. Our results are demonstrated for constructing confidence regions for the difference of the means of two p-dimensional samples and the difference in value between coefficients of two p-dimensional sample linear model. We show that empirical likelihood based estimator has the efficient property. That is, as p → ∞ for high-dimensional data, the limit distribution of the EL ratio statistic for the difference of the means of two samples and the difference in value between coefficients of two-sample linear model is asymptotic normal distribution. Furthermore, empirical likelihood (EL) gives efficient estimator for regression coefficients in linear models, and can be as efficient as a parametric approach. The performance of the proposed method is illustrated via numerical simulations.