On the restricted almost unbiased estimators in linear regression

In this paper, the restricted almost unbiased ridge regression estimator and restricted almost unbiased Liu estimator are introduced for the vector of parameters in a multiple linear regression model with linear restrictions. The bias, variance matrices and mean square error (MSE) of the proposed estimators are derived and compared. It is shown that the proposed estimators will have smaller quadratic bias but larger variance than the corresponding competitors in literatures. However, they will respectively outperform the latter according to the MSE criterion under certain conditions. Finally, a simulation study and a numerical example are given to illustrate some of the theoretical results.     

Large Deviation Results for Statistics of Short- and Long-memory Gaussian Processes

This paper discusses the large deviation principle of several important statistics for short- and long-memory Gaussian processes. First, large deviation theorems for the log-likelihood ratio and quadratic forms for a short-memory Gaussian process with mean function are proved. Their asymptotics are described by the large deviation rate functions. Since they are complicated, they are numerically evaluated and illustrated using the Maple V system (Char et al ., 1991a,b). Second, the large deviation theorem of the log-likelihood ratio statistic for a long-memory Gaussian process with constant mean is proved. The asymptotics of the long-memory case differ greatly from those of the short-memory case. The maximum likelihood estimator of a spectral parameter for a short-memory Gaussian stationary process is asymptotically efficient in the sense of Bahadur.     

A Non‐Parametric Estimator of the Spectral Density of a Continuous‐Time Gaussian Process Observed at Random Times

Abstract. In numerous applications data are observed at random times and an estimated graph of the spectral density may be relevant for characterizing and explaining phenomena. By using a wavelet analysis, one derives a non‐parametric estimator of the spectral density of a Gaussian process with stationary increments (or a stationary Gaussian process) from the observation of one path at random discrete times. For every positive frequency, this estimator is proved to satisfy a central limit theorem with a convergence rate depending on the roughness of the process and the moment of random durations between successive observations. In the case of stationary Gaussian processes, one can compare this estimator with estimators based on the empirical periodogram. Both estimators reach the same optimal rate of convergence, but the estimator based on wavelet analysis converges for a different class of random times. Simulation examples and an application to biological data are also provided.     

Nonparametric local polynomial approximation of the time-varying frequency and amplitude

The problem of estimating the time-varying frequency, phase and amplitude of a real-valued harmonic signal is considered. It is assumed that the frequency and amplitude are unspecified rapidly time-varying functions of time. The technique is based on fitting a local polynomial approximation of the phase and amplitude which implements a high-order nonlinear nonparametric estimator. The estimator is shown to be strongly consistent and Gaussian. In particular, the convergence ratesO(h^-3/2 )and O(h^-5/2 ), where $i;h$ei; is the number of observations, are obtained for the frequency estimator when the amplitude is unknown constant or linear in time respectively. The orders of the bias and Gaussian distribution are obtained for a class of the time-varying frequency and amplitude with bounded second derivatives. The a priori amplitude information about the unknown time-varying frequency and amplitude and their derivatives can be incorporated to improve the accuracy of the estimation. Simulation results are given.     

M-estimators with non-standard rates of convergence and weakly dependent data

This paper analyzes M-estimators over general objective functions. We do not assume convexity and differentiability of the functions. A new result regarding M-estimators is derived. Unlike most of the former econometric literature, the rate of convergence is not square root n. The rate of convergence is non-standard and depends on the moment bounds of the objective function analyzed. We can actually connect the rate of convergence to the smoothness of the objective function in certain class of functions as described in van der Vaart and Wellner (Weak Convergence and Empirical Processes, Springer, Berlin, 1996). We also simplify this rate of convergence idea and extend to weakly dependent data from iid case. This rate is simple and usable in econometrics literature. We illustrate the techniques by deriving the rate of convergence for LAD estimator for censored regression and maximum score estimator with weakly dependent data.     

Performances of the Positive-Rule Stein-Type Ridge Estimator in a Linear Regression Model with Spherically Symmetric Error Distributions

In this article, the positive-rule Stein-type ridge estimator (PSRE) is introduced for the parameters in a multiple linear regression model with spherically symmetric error distributions when it is suspected that the parameter vector may be restricted to a linear manifold. The bias and quadratic risk functions of the PSRE are derived and compared with some related competing estimators in literatures. Particularly, some sufficient conditions are derived for superiority of the PSRE over the ordinary ridge estimator, the restricted ridge estimator and the preliminary test ridge estimator, respectively. Furthermore, some graphical results are provided to illustrate some of the theoretical results.     

A generalization of the Kaplan-Meier estimator to Harris-recurrent Markov chains

In this paper we study an extension of the Kaplan Meier estimator to the case of dependent failure times. We assume that the failure times follow a Harris recurrent Markov Chain. We prove strong convergence of the estimator and weak convergence to a Gaussian process.     

Sur un test d'égalité des autocovariances de deux séries chronologiques

We propose a test for the equality of the autocovariance functions of two independent and stationary time series. The test statistic is a quadratic form in the vector of differences of the first J + 1 autocovariances. Its asymptotic distribution is derived under the null hypothesis, and the finite-sample properties of the test, namely the bias and the power, are investigated by Monte Carlo methods. A by-product of this study is a new estimator of the covariance between two sample autocovariances which provides a positive definite covariance matrix. We establish the convergence of this estimator in the L₁ norm.     

Positive-rule stein-type almost unbiased ridge estimator in linear regression model

Abstract

In this article, when it is suspected that regression coefficients may be restricted to a subspace, we discuss the parameter estimation of regression coefficients in a multiple regression model. Then, in order to improve the preliminary test almost ridge estimator, we study the positive-rule Stein-type almost unbiased ridge estimator based on the positive-rule stein-type shrinkage estimator and almost unbiased ridge estimator. After that, quadratic bias and quadratic risk values of the new estimator are derived and compared with some relative estimators. And we also discuss the option of parameter k. Finally, we perform a real data example and a Monte Carlo study to illustrate theoretical results.     

Precise Asymptotics in Complete Moment Convergence of Parameter Estimator in the Gaussian Autoregressive Process

In this article, we study the precise asymptotic behaviors of the least-squares estimator in the Gaussian autoregressive process. Two kinds of complete moment convergence of this estimator can be obtained by the methods of deviation inequalities for this estimator and nonuniform Berry-Esseen bound for martingales.     

Minimax Linear Smoothers

We consider the problem of estimating the mean of a multivariate distribution. As a general alternative to penalized least squares estimators, we consider minimax estimators for squared error over a restricted parameter space where the restriction is determined by the penalization term. For a quadratic penalty term, the minimax estimator among linear estimators can be found explicitly. It is shown that all symmetric linear smoothers with eigenvalues in the unit interval can be characterized as minimax linear estimators over a certain parameter space where the bias is bounded. The minimax linear estimator depends on smoothing parameters that must be estimated in practice. Using results in Kneip (1994), this can be done using Mallows' C _L -statistic and the resulting adaptive estimator is now asymptotically minimax linear. The minimax estimator is compared to the penalized least squares estimator both in finite samples and asymptotically.     

Improved estimation for the parameters of an inverse gaussian distribution

James-Stein estimators are proposed for the #-parameter of an inverse Gaussian #G# distribution. The estimators of this class have smaller expected quadratic loss than the maximum likelihood estimator usually employed when analysing real sets of data. This problem is also studied for the case of an unknown nuisance parameter. Finally, improved estimators are considered for # in the two sample problem.     

Exact simulation of Gaussian Time Series from Nonparametric Spectral Estimates with Application to Bootstrapping

The circulant embedding method for generating statistically exact simulations of time series from certain Gaussian distributed stationary processes is attractive because of its advantage in computational speed over a competitive method based upon the modified Cholesky decomposition. We demonstrate that the circulant embedding method can be used to generate simulations from stationary processes whose spectral density functions are dictated by a number of popular nonparametric estimators, including all direct spectral estimators (a special case being the periodogram), certain lag window spectral estimators, all forms of Welch's overlapped segment averaging spectral estimator and all basic multitaper spectral estimators. One application for this technique is to generate time series for bootstrapping various statistics. When used with bootstrapping, our proposed technique avoids some – but not all – of the pitfalls of previously proposed frequency domain methods for simulating time series.     

Two-Step Estimation of Incomplete Information Social Interaction Models With Sample Selection

ABSTRACT

This article considers linear social interaction models under incomplete information that allow for missing outcome data due to sample selection. For model estimation, assuming that each individual forms his/her belief about the other members’ outcomes based on rational expectations, we propose a two-step series nonlinear least squares estimator. Both the consistency and asymptotic normality of the estimator are established. As an empirical illustration, we apply the proposed model and method to National Longitudinal Study of Adolescent Health (Add Health) data to examine the impacts of friendship interactions on adolescents’ academic achievements. We provide empirical evidence that the interaction effects are important determinants of grade point average and that controlling for sample selection bias has certain impacts on the estimation results. Supplementary materials for this article are available online.     

Bias reduction of the maximum-likelihood estimator for a conditional Gaussian MA(1) model

In this paper, we consider an estimation for the unknown parameters of a conditional Gaussian MA(1) model. In the majority of cases, a maximum-likelihood estimator is chosen because the estimator is consistent. However, for small sample sizes the error is large, because the estimator has a bias of O(n^{? 1}). Therefore, we provide a bias of O(n^{? 1}) for the maximum-likelihood estimator for the conditional Gaussian MA(1) model. Moreover, we propose new estimators for the unknown parameters of the conditional Gaussian MA(1) model based on the bias of O(n^{? 1}). We investigate the properties of the bias, as well as the asymptotical variance of the maximum-likelihood estimators for the unknown parameters, by performing some simulations. Finally, we demonstrate the validity of the new estimators through this simulation study.     

A new non parametric estimator for Pdf based on inverse gamma distribution

ABSTRACT

The non parametric approach is considered to estimate probability density function (Pdf) which is supported on(0, ∞). This approach is the inverse gamma kernel. We show that it has same properties as gamma, reciprocal inverse Gaussian, and inverse Gaussian kernels such that it is free of the boundary bias, non negative, and it achieves the optimal rate of convergence for the mean integrated squared error. Also some properties of the estimator were established such as bias and variance. Comparison of the bandwidth selection methods for inverse gamma kernel estimation of Pdf is done.     

Efficient bias-Robust M-Estimators of location in Kolmogorov normal neighbourhoods

We consider minimax-bias M-estimation of a location parameter in a Kolmogorov neighbourhood K() of a normal distribution. The maximum asymptotic bias of M-estimators for the Kolmogorov normal neighbourhood is derived, and its relation with the gross-error sensitivity of the estimator at the nominal model (the Gaussian case) is found. In addition, efficient bias-robust M-estimators T_i are constructed. Numerical results are also obtained to show the percentage of increase in maximum asymptotic bias and the efficiency we can achieve for some well-known -functions.     

Estimating the variance in nonparametric regression—what is a reasonable choice?

The exact mean-squared error (MSE) of estimators of the variance in nonparametric regression based on quadratic forms is investigated. In particular, two classes of estimators are compared: Hall, Kay and Titterington's optimal difference-based estimators and a class of ordinary difference-based estimators which generalize methods proposed by Rice and Gasser, Sroka and Jennen-Steinmetz. For small sample sizes the MSE of the first estimator is essentially increased by the magnitude of the integrated first two squared derivatives of the regression function. It is shown that in many situations ordinary difference-based estimators are more appropriate for estimating the variance, because they control the bias much better and hence have a much better overall performance. It is also demonstrated that Rice's estimator does not always behave well. Data-driven guidelines are given to select the estimator with the smallest MSE.     

Expectation of quadratic forms in normal and nonnormal variables with applications

We derive some new results on the expectation of quadratic forms in normal and nonnormal variables. Using a nonstochastic operator, we show that the expectation of the product of an arbitrary number of quadratic forms in noncentral normal variables follows a recurrence formula. This formula includes the existing result for central normal variables as a special case. For nonnormal variables, while the existing results are available only for quadratic forms of limited order (up to 3), we derive analytical results to a higher order 4. We use the nonnormal results to study the effects of nonnormality on the finite sample mean squared error of the OLS estimator in an AR(1) model and the QMLE in an MA(1) model.     

On the Estimation of the Probability Density by Trigonometric Series

We present in this article an estimator based on a new orthogonal trigonometric series. We give its statistical properties (bias, variance, mean square error, and mean integrated square error) and the asymptotic properties (convergence of variance, convergence of the mean square error, convergence of the mean integrated square error, uniform convergence in probability, and the rate of convergence of the mean integrated square error). The comparison by simulation on a test density between the estimator obtained from a new trigonometric series with Fejer estimator also based on orthogonal trigonometric series, shows that our estimator is more performant in the sense of the mean integrated square error.