Statistical Analysis of Non Linear Least Squares Estimation for Harmonic Signals in Multiplicative and Additive Noise

Abstract

This article concerns the stochastically constrained linear model under a biased assumption. We propose a quasi-stochastically constrained least squares estimator. Furthermore, we provide the expectation of this estimator, demonstrate its consistency and asymptotic normality. In the end of the article, the simulation study of the new estimator shows that it is superior to the least squares estimator, ridge estimator, and the linear constrained estimators under certain conditions by comparing the mean squared errors of these estimators.     

Empirical study of robust estimation methods for PAR models with application to the air quality area

Abstract

This paper compares three estimators for periodic autoregressive (PAR) models. The first is the classical periodic Yule-Walker estimator (YWE). The second is a robust version of YWE (RYWE) which uses the robust autocovariance function in the periodic Yule-Walker equations, and the third is the robust least squares estimator (RLSE) based on iterative least squares with robust versions of the original time series. The daily mean particulate matter concentration (PM10) data is used to illustrate the methodologies in a real application, that is, in the Air Quality area.     

Slow-explosive AR(1) processes converging to random walk

Abstract

This article investigates slow-explosive AR(1) processes, which converge to a random walk (RW) process with logarithm rates, to fill the gap between nearly non-stationary AR(1) and moderately deviated AR(1) processes, and derives the asymptotics of the least squares estimator using central limit theorems for (reduced) U-statistic. We successfully establish the smooth link between the nearly non-stationary AR(1) and the moderately deviated AR(1) processes. Some novel results are reported, which include the convergence of the least squares estimator to a biased fractional Brownian motion.     

Robust alternatives of the least squares estimator

The purpose of this paper consists in deriving estimators which are less sensitive than the least squares estimator, when the assumption that the expectation vector lies in a certain linear subspace is violated. The obtained robust estimators are convex combinations of the least squares estimator and of the random vector Y.     

Estimating the Intercept in an Orthogonally Blocked Experiment when the Block Effects are Random

Abstract

For an orthogonally blocked experiment, Khuri [Khuri, A. I. (1992). Response surface models with random block effects. Technometrics 34:26–37] has shown that the ordinary least squares estimator, the generalized least squares estimator and the intra-block estimator of the factor effects in a response surface model with random block effects coincide. The ordinary least squares estimator ignores the blocks, whereas the generalized least squares and the intra-block estimators treat the block effects as random and fixed, respectively. As shown in this paper, the equivalence does not hold for the estimation of the intercept when the block sizes are heterogeneous. Practical examples are given to illustrate the theoretical results.     

Nonparametric density estimation based on beta prime kernel

Abstract

In this work, we propose beta prime kernel estimator for estimation of a probability density functions defined with nonnegative support. For the proposed estimator, beta prime probability density function used as a kernel. It is free of boundary bias and nonnegative with a natural varying shape. We obtained the optimal rate of convergence for the mean squared error (MSE) and the mean integrated squared error (MISE). Also, we use adaptive Bayesian bandwidth selection method with Lindley approximation for heavy tailed distributions and compare its performance with the global least squares cross-validation bandwidth selection method. Simulation studies are performed to evaluate the average integrated squared error (ISE) of the proposed kernel estimator against some asymmetric competitors using Monte Carlo simulations. Moreover, real data sets are presented to illustrate the findings.     

Using Liu-Type Estimator to Combat Collinearity

Abstract

Linear regression model and least squares method are widely used in many fields of natural and social sciences. In the presence of collinearity, the least squares estimator is unstable and often gives misleading information. Ridge regression is the most common method to overcome this problem. We find that when there exists severe collinearity, the shrinkage parameter selected by existing methods for ridge regression may not fully address the ill conditioning problem. To solve this problem, we propose a new two-parameter estimator. We show using both theoretic results and simulation that our new estimator has two advantages over ridge regression. First, our estimator has less mean squared error (MSE). Second, our estimator can fully address the ill conditioning problem. A numerical example from literature is used to illustrate the results.     

A note on the equality of the OLSE and the BLUE of the parametric function in the general Gauss–Markov model

In this note we consider the equality of the ordinary least squares estimator (OLSE) and the best linear unbiased estimator (BLUE) of the estimable parametric function in the general Gauss–Markov model. Especially we consider the structures of the covariance matrix V for which the OLSE equals the BLUE. Our results are based on the properties of a particular reparametrized version of the original Gauss–Markov model.      

The Equality of the Ordinary Least Squares Estimator and the Best Linear Unbiased Estimator

It is well known that the ordinary least squares estimator of Xβ in the general linear model E _y = Xβ, cov y = σ² V, can be the best linear unbiased estimator even if V is not a multiple of the identity matrix. This article presents, in a historical perspective, the development of the several conditions for the ordinary least squares estimator to be best linear unbiased. Various characterizations of these conditions, using generalized inverses and orthogonal projectors, along with several examples, are also given. In addition, a complete set of references is provided.     

Estimation and Simulation of the Riesz-Bessel Distribution

ABSTRACT

In this article, further properties of the Riesz-Bessel distribution are provided. These properties allow for the simulation of random variables from the Riesz-Bessel distribution. Estimation is addressed by nonlinear generalized least squares regression on the empirical characteristic function. The estimator is seen to approximate the maximum likelihood estimator. The distribution is illustrated with financial data.     

Estimation of dynamic panel data models with both individual and time-specific effects

This paper proposes a generalized least squares and a generalized method of moment estimators for dynamic panel data models with both individual-specific and time-specific effects. We also demonstrate that the common estimators ignoring the presence of time-specific effects are inconsistent when N→∞$N\to \infty$ but T is finite if the time-specific effects are indeed present. Monte Carlo studies are also conducted to investigate the finite sample properties of various estimators. It is found that the generalized least squares estimator has the smallest bias and root mean square error, and also has nominal size close to the empirical size. It is also found that even when there is no presence of time-specific effects, there is hardly any efficiency loss of the generalized least squares estimator assuming its presence compared to the generalized least squares estimator allowing only the presence of individual-specific effects.     

Shrinkage and LASSO strategies in high-dimensional heteroscedastic models

ABSTRACT

In this paper, we consider the estimation problem of the parameter vector in the linear regression model with heteroscedastic errors. First, under heteroscedastic errors, we study the performance of shrinkage-type estimators and their performance as compared to theunrestricted and restricted least squares estimators. In order to accommodate the heteroscedastic structure, we generalize an identity which is useful in deriving the risk function. Thanks to the established identity, we prove that shrinkage estimators dominate the unrestricted estimator. Finally, we explore the performance of high-dimensional heteroscedastic regression estimator as compared to classical LASSO and shrinkage estimators.     

Adaptive LASSO estimation for ARDL models with GARCH innovations

ABSTRACT

In this paper, we show the validity of the adaptive least absolute shrinkage and selection operator (LASSO) procedure in estimating stationary autoregressive distributed lag(p,q) models with innovations in a broad class of conditionally heteroskedastic models. We show that the adaptive LASSO selects the relevant variables with probability converging to one and that the estimator is oracle efficient, meaning that its distribution converges to the same distribution of the oracle-assisted least squares, i.e., the least square estimator calculated as if we knew the set of relevant variables beforehand. Finally, we show that the LASSO estimator can be used to construct the initial weights. The performance of the method in finite samples is illustrated using Monte Carlo simulation.     

Semiparametric additive isotonic regression

We consider the efficient estimation in the semiparametric additive isotonic regression model where each additive nonparametric component is assumed to be a monotone function. We show that the least-square estimator of the finite-dimensional regression coefficient is root-n$n$ consistent and asymptotically normal. Moreover, the isotonic estimator of each additive functional component is proved to have the oracle property, which means the additive component can be estimated with the highest asymptotic accuracy as if the other components were known. A fast algorithm is developed by iterating between a cyclic pool adjacent violators procedure and solving a standard ordinary least squares problem. Simulations are used to illustrate the performance of the proposed procedure and verify the oracle property.     

Superiority of the r–k Class Estimator Over Some Estimators In A Linear Model

In regression analysis, to overcome the problem of multicollinearity, the r ? k class estimator is proposed as an alternative to the ordinary least squares estimator which is a general estimator including the ordinary ridge regression estimator, the principal components regression estimator and the ordinary least squares estimator. In this article, we derive the necessary and sufficient conditions for the superiority of the r ? k class estimator over each of these estimators under the Mahalanobis loss function by the average loss criterion. Then, we compare these estimators with each other using the same criterion. Also, we suggest to test to verify if these conditions are indeed satisfied. Finally, a numerical example and a Monte Carlo simulation are done to illustrate the theoretical results.     

SHRINKAGE, PRETEST AND ABSOLUTE PENALTY ESTIMATORS IN PARTIALLY LINEAR MODELS

We consider a partially linear model in which the vector of coefficients β in the linear part can be partitioned as ( β ₁, β ₂) , where β ₁ is the coefficient vector for main effects (e.g. treatment effect, genetic effects) and β ₂ is a vector for ‘nuisance’ effects (e.g. age, laboratory). In this situation, inference about β ₁ may benefit from moving the least squares estimate for the full model in the direction of the least squares estimate without the nuisance variables (Steinian shrinkage), or from dropping the nuisance variables if there is evidence that they do not provide useful information (pretesting). We investigate the asymptotic properties of Stein‐type and pretest semiparametric estimators under quadratic loss and show that, under general conditions, a Stein‐type semiparametric estimator improves on the full model conventional semiparametric least squares estimator. The relative performance of the estimators is examined using asymptotic analysis of quadratic risk functions and it is found that the Stein‐type estimator outperforms the full model estimator uniformly. By contrast, the pretest estimator dominates the least squares estimator only in a small part of the parameter space, which is consistent with the theory. We also consider an absolute penalty‐type estimator for partially linear models and give a Monte Carlo simulation comparison of shrinkage, pretest and the absolute penalty‐type estimators. The comparison shows that the shrinkage method performs better than the absolute penalty‐type estimation method when the dimension of the β ₂ parameter space is large.     

Estimation of a probability density function using interval aggregated data

ABSTRACT

In economics and government statistics, aggregated data instead of individual level data are usually reported for data confidentiality and for simplicity. In this paper we develop a method of flexibly estimating the probability density function of the population using aggregated data obtained as group averages when individual level data are grouped according to quantile limits. The kernel density estimator has been commonly applied to such data without taking into account the data aggregation process and has been shown to perform poorly. Our method models the quantile function as an integral of the exponential of a spline function and deduces the density function from the quantile function. We match the aggregated data to their theoretical counterpart using least squares, and regularize the estimation by using the squared second derivatives of the density function as the penalty function. A computational algorithm is developed to implement the method. Application to simulated data and US household income survey data show that our penalized spline estimator can accurately recover the density function of the underlying population while the common use of kernel density estimation is severely biased. The method is applied to study the dynamic of China's urban income distribution using published interval aggregated data of 1985–2010.     

Mean square error and efficiency of the least squares estimator over interval constraints

We derive the mean square error of an interval constrained least squares estimator (INCLS) for a regression model. We then show that the INCLS estimator dominates, in mean square error, the unconstrained least squares estimator provided the regression residuais are normally distri'iiuted and Ynat Yrie imposed coii-

straint is satisfied or nearly satisfied.     

Shape-Constrained Kernel-Weighted Least Squares: Estimating Production Functions for Chilean Manufacturing Industries

ABSTRACT

In this article, we examine a novel way of imposing shape constraints on a local polynomial kernel estimator. The proposed approach is referred to as shape constrained kernel-weighted least squares (SCKLS). We prove uniform consistency of the SCKLS estimator with monotonicity and convexity/concavity constraints and establish its convergence rate. In addition, we propose a test to validate whether shape constraints are correctly specified. The competitiveness of SCKLS is shown in a comprehensive simulation study. Finally, we analyze Chilean manufacturing data using the SCKLS estimator and quantify production in the plastics and wood industries. The results show that exporting firms have significantly higher productivity.     

A study on the least squares estimator of multivariate isotonic regression function

Consider the problem of pointwise estimation of f in a multivariate isotonic regression model Z=f(X₁,…,X_d)+ϵ, where Z is the response variable, f is an unknown nonparametric regression function, which is isotonic with respect to each component, and ϵ is the error term. In this article, we investigate the behavior of the least squares estimator of f. We generalize the greatest convex minorant characterization of isotonic regression estimator for the multivariate case and use it to establish the asymptotic distribution of properly normalized version of the estimator. Moreover, we test whether the multivariate isotonic regression function at a fixed point is larger (or smaller) than a specified value or not based on this estimator, and the consistency of the test is established. The practicability of the estimator and the test are shown on simulated and real data as well.