Performance of Kibria's Method for the Heteroscedastic Ridge Regression Model: Some Monte Carlo Evidence

It is common for a linear regression model that the error terms display some form of heteroscedasticity and at the same time, the regressors are also linearly correlated. Both of these problems have serious impact on the ordinary least squares (OLS) estimates. In the presence of heteroscedasticity, the OLS estimator becomes inefficient and the similar adverse impact can also be found on the ridge regression estimator that is alternatively used to cope with the problem of multicollinearity. In the available literature, the adaptive estimator has been established to be more efficient than the OLS estimator when there is heteroscedasticity of unknown form. The present article proposes the similar adaptation for the ridge regression setting with an attempt to have more efficient estimator. Our numerical results, based on the Monte Carlo simulations, provide very attractive performance of the proposed estimator in terms of efficiency. Three different existing methods have been used for the selection of biasing parameter. Moreover, three different distributions of the error term have been studied to evaluate the proposed estimator and these are normal, Student's t and F distribution.     

Stochastic volatility models for exchange rates and their estimation using quasi-maximum-likelihood methods: an application to the South African Rand

This paper is concerned with the volatility modeling of a set of South African Rand (ZAR) exchange rates. We investigate the quasi-maximum-likelihood (QML) estimator based on the Kalman filter and explore how well a choice of stochastic volatility (SV) models fits the data. We note that a data set from a developing country is used. The main results are: (1) the SV model parameter estimates are in line with those reported from the analysis of high-frequency data for developed countries; (2) the SV models we considered, along with their corresponding QML estimators, fit the data well; (3) using the range return instead of the absolute return as a volatility proxy produces QML estimates that are both less biased and less variable; (4) although the log range of the ZAR exchange rates has a distribution that is quite far from normal, the corresponding QML estimator has a superior performance when compared with the log absolute return.     

Estimation and Asymptotic Properties in Periodic GARCH(1, 1) Models

This article studies the probabilistic structure and asymptotic inference of the first-order periodic generalized autoregressive conditional heteroscedasticity (PGARCH(1, 1)) models in which the parameters in volatility process are allowed to switch between different regimes. First, we establish necessary and sufficient conditions for a PGARCH(1, 1) process to have a unique stationary solution (in periodic sense) and for the existence of moments of any order. Second, using the representation of squared PGARCH(1, 1) model as a PARMA(1, 1) model, we then consider Yule-Walker type estimators for the parameters in PGARCH(1, 1) model and derives their consistency and asymptotic normality. The estimator can be surprisingly efficient for quite small numbers of autocorrelations and, in some cases can be more efficient than the least squares estimate (LSE). We use a residual bootstrap to define bootstrap estimators for the Yule-Walker estimates and prove the consistency of this bootstrap method. A set of numerical experiments illustrates the practical relevance of our theoretical results.     

Simultaneous robust estimation of location and scale parameters: A minimum-distance approach

Simultaneous robust estimates of location and scale parameters are derived from minimizing a minimum-distance criterion function. The criterion function measures the squared distance between the pth power (p > 0) of the empirical distribution function and the pth power of the imperfectly determined model distribution function over the real line. We show that the estimator is uniquely defined, is asymptotically bivariate normal and for p > 0.3 has positive breakdown. If the scale parameter is known, when p = 0.9 the asymptotic variance (1.0436) of the location estimator for the normal model is smaller than the asymptotic variance of the Hodges-Lehmann (HL)estimator (1.0472). Efficiencies with respect to HL and maximum-likelihood estimators (MLE) are 1.0034 and 0.9582, respectively. Similarly, if the location parameter is known, when p = 0.97 the asymptotic variance (0.6158) of the scale estimator is minimum. The efficiency with respect to the MLE is 0.8119. We show that the estimator can tolerate more corrupted observations at oo than at – for p < 1, and vice versa for p > 1.     

Test for the mean matrix in a Growth Curve model for high dimensions

We consider the problem of estimating and testing a general linear hypothesis in a general multivariate linear model, the so-called Growth Curve model, when the p × N observation matrix is normally distributed.

The maximum likelihood estimator (MLE) for the mean is a weighted estimator with the inverse of the sample covariance matrix which is unstable for large p close to N and singular for p larger than N. We modify the MLE to an unweighted estimator and propose new tests which we compare with the previous likelihood ratio test (LRT) based on the weighted estimator, i.e., the MLE. We show that the performance of these new tests based on the unweighted estimator is better than the LRT based on the MLE.     

Quasi maximum likelihood estimation of dynamic panel data models

This article establishes the almost sure convergence and asymptotic normality of levels and differenced quasi maximum likelihood (QML) estimators of dynamic panel data models. The QML estimators are robust with respect to initial conditions, conditional and time-series heteroskedasticity, and misspecification of the log-likelihood. The article also provides an ECME algorithm for calculating levels QML estimates. Finally, it compares the finite-sample performance of levels and differenced QML estimators, the differenced generalized method of moments (GMM) estimator, and the system GMM estimator. The QML estimators usually have smaller— typically substantially smaller—bias and root mean squared errors than the panel data GMM estimators.     

Testing sphericity and intraclass covariance structures under a growth curve model in high dimension

In this article, we consider the problem of testing (a) sphericity and (b) intraclass covariance structure under a growth curve model. The maximum likelihood estimator (MLE) for the mean in a growth curve model is a weighted estimator with the inverse of the sample covariance matrix which is unstable for large p close to N and singular for p larger than N. The MLE for the covariance matrix is based on the MLE for the mean, which can be very poor for p close to N. For both structures (a) and (b), we modify the MLE for the mean to an unweighted estimator and based on this estimator we propose a new estimator for the covariance matrix. This new estimator leads to new tests for (a) and (b). We also propose two other tests for each structure, which are just based on the sample covariance matrix.

To compare the performance of all four tests we compute for each structure (a) and (b) the attained significance level and the empirical power. We show that one of the tests based on the sample covariance matrix is better than the likelihood ratio test based on the MLE.     

GMM estimation of spatial autoregressive models in a system of simultaneous equations with heteroskedasticity

This paper proposes a GMM estimation framework for the SAR model in a system of simultaneous equations with heteroskedastic disturbances. Besides linear moment conditions, the proposed GMM estimator also utilizes quadratic moment conditions based on the covariance structure of model disturbances within and across equations. Compared with the QML approach, the GMM estimator is easier to implement and robust under heteroskedasticity of unknown form. We derive the heteroskedasticity-robust standard error for the GMM estimator. Monte Carlo experiments show that the proposed GMM estimator performs well in finite samples.     

Order-preserving Estimators and an Inequality on the Integration of Zonal Polynomial

ABSTRACT

Suppose X , p × p p.d. random matrix, has the distribution which depends on a p × p p.d. parameter matrix Σ and this distribution is orthogonally invariant. The orthogonally invariant estimator of Σ which has the eigenvalues of the same order as the eigenvalues of X is called order-preserving. We conjecture that a non-order-preserving estimator is dominated by modified order-preserving estimators with respect to the entropy (Stein's) loss function. We show that an inequality on the integration of zonal polynomial is sufficient for this conjecture. We also prove this inequality for the case p = 2.     

Beyond tail median and conditional tail expectation: Extreme risk estimation using tail Lp-optimization

The conditional tail expectation (CTE) is an indicator of tail behavior that takes into account both the frequency and magnitude of a tail event. However, the asymptotic normality of its empirical estimator requires that the underlying distribution possess a finite variance; this can be a strong restriction in actuarial and financial applications. A valuable alternative is the median shortfall (MS), although it only gives information about the frequency of a tail event. We construct a class of tail L^p-medians encompassing the MS and CTE. For p in (1,2), a tail L^p-median depends on both the frequency and magnitude of tail events, and its empirical estimator is, within the range of the data, asymptotically normal under a condition weaker than a finite variance. We extrapolate this estimator and another technique to extreme levels using the heavy-tailed framework. The estimators are showcased on a simulation study and on real fire insurance data.     

Asymptotics for L1-estimators of regression parameters under heteroscedasticityY

We consider the asymptotic behaviour of L₁ -estimators in a linear regression under a very general form of heteroscedasticity. The limiting distributions of the estimators are derived under standard conditions on the design. We also consider the asymptotic behaviour of the bootstrap in the heteroscedastic model and show that it is consistent to first order only if the limiting distribution is normal.     

Value-at-risk forecasting based on Gaussian mixture ARMA–GARCH model

In this paper, we develop a new forecasting algorithm for value-at-risk (VaR) based on ARMA–GARCH (autoregressive moving average–generalized autoregressive conditional heteroskedastic) models whose innovations follow a Gaussian mixture distribution. For the parameter estimation, we employ the conditional least squares and quasi-maximum-likelihood estimator (QMLE) for ARMA and GARCH parameters, respectively. In particular, Gaussian mixture parameters are estimated based on the residuals obtained from the QMLE of GARCH parameters. Our algorithm provides a handy methodology, spending much less time in calculation than the existing resampling and bias-correction method developed in Hartz et al. [Accurate value-at-risk forecasting based on the normal-GARCH model, Comput. Stat. Data Anal. 50 (2006), pp. 3032–3052]. Through a simulation study and a real-data analysis, it is shown that our method provides an accurate VaR prediction.     

Variable selection in the high-dimensional continuous generalized linear model with current status data

In survival studies, current status data are frequently encountered when some individuals in a study are not successively observed. This paper considers the problem of simultaneous variable selection and parameter estimation in the high-dimensional continuous generalized linear model with current status data. We apply the penalized likelihood procedure with the smoothly clipped absolute deviation penalty to select significant variables and estimate the corresponding regression coefficients. With a proper choice of tuning parameters, the resulting estimator is shown to be a root n/p_n-consistent estimator under some mild conditions. In addition, we show that the resulting estimator has the same asymptotic distribution as the estimator obtained when the true model is known. The finite sample behavior of the proposed estimator is evaluated through simulation studies and a real example.     

A MORE GENERAL CRITERION FOR SUBSET SELECTION IN MULTIPLE LINEAR REGRESSION

ABSTRACT

In this article, we propose a more general criterion called S_p -criterion, for subset selection in the multiple linear regression Model. Many subset selection methods are based on the Least Squares (LS) estimator of β, but whenever the data contain an influential observation or the distribution of the error variable deviates from normality, the LS estimator performs ‘poorly’ and hence a method based on this estimator (for example, Mallows’ C_p -criterion) tends to select a ‘wrong’ subset. The proposed method overcomes this drawback and its main feature is that it can be used with any type of estimator (either the LS estimator or any robust estimator) of β without any need for modification of the proposed criterion. Moreover, this technique is operationally simple to implement as compared to other existing criteria. The method is illustrated with examples.     

Empirical distribution function under heteroscedasticity

Neglecting heteroscedasticity of error terms may imply the wrong identification of a regression model (see appendix). Employment of (heteroscedasticity resistent) White's estimator of covariance matrix of estimates of regression coefficients may lead to the correct decision about the significance of individual explanatory variables under heteroscedasticity. However, White's estimator of covariance matrix was established for least squares (LS)-regression analysis (in the case when error terms are normally distributed, LS- and maximum likelihood (ML)-analysis coincide and hence then White's estimate of covariance matrix is available for ML-regression analysis, tool). To establish White's-type estimate for another estimator of regression coefficients requires Bahadur representation of the estimator in question, under heteroscedasticity of error terms. The derivation of Bahadur representation for other (robust) estimators requires some tools. As the key too proved to be a tight approximation of the empirical distribution function (d.f.) of residuals by the theoretical d.f. of the error terms of the regression model. We need the approximation to be uniform in the argument of d.f. as well as in regression coefficients. The present paper offers this approximation for the situation when the error terms are heteroscedastic.     

On the invertibility of EGARCH(p,q)

Of the two most widely estimated univariate asymmetric conditional volatility models, the exponential GARCH (or EGARCH) specification is said to be able to capture asymmetry, which refers to the different effects on conditional volatility of positive and negative effects of equal magnitude, and leverage, which refers to the negative correlation between the returns shocks and subsequent shocks to volatility. However, the statistical properties of the (quasi-)maximum likelihood estimator (QMLE) of the EGARCH(p, q) parameters are not available under general conditions, but only for special cases under highly restrictive and unverifiable sufficient conditions, such as EGARCH(1,0) or EGARCH(1,1), and possibly only under simulation. A limitation in the development of asymptotic properties of the QMLE for the EGARCH(p, q) model is the lack of an invertibility condition for the returns shocks underlying the model. It is shown in this article that the EGARCH(p, q) model can be derived from a stochastic process, for which sufficient invertibility conditions can be stated simply and explicitly when the parameters respect a simple condition.¹¹Using the notation introduced in part 2, this refers to the cases where α ≥ |γ| or α ≤ ? |γ|. The first inequality is generally assumed in the literature related to the invertibility of EGARCH. This article provides (in the Appendix) an argument for the possible lack of invertibility when these conditions are not met. This will be useful in reinterpreting the existing properties of the QMLE of the EGARCH(p, q) parameters.     

Improved minimax estimation of a normal precision matrix

Let S_{p × p} have a Wishart distribution with parameter matrix Σ and n degrees of freedom. We consider here the problem of estimating the precision matrix Σ^?1 under the loss functions L₁(σ) tr (σ) - log |σ| and L₂(σ) = tr (σ). James-Stein-type estimators have been derived for an arbitrary p. We also obtain an orthogonal invariant and a diagonal invariant minimax estimator under both loss functions. A Monte-Carlo simulation study indicates that the risk improvement of the orthogonal invariant estimators over the James-Stein type estimators, the Haff (1979) estimator, and the “testimator” given by Sinha and Ghosh (1987) is substantial.     

Estimation of a multivariate normal covariance matrix under a certain structure

We consider the estimation of Σ of the p-dimensional normal distribution N_p (0, Σ) when Σ?=?θ₀ I_p ?+?θ₁ aa′, where a is an unknown p-dimensional normalized vector and θ₀?>?0, θ₁?≥?0 are also unknown. First, we derive the restricted maximum likelihood (REML) estimator. Second, we propose a new estimator, which dominates the REML estimator with respect to Stein's loss function. Finally, we carry out Monte Carlo simulation to investigate the magnitude of the new estimator's superiority.     

Optimal sequential estimation procedures of a function of a probability of success under LINEX loss

In this paper, we investigate the problem of estimating a function g(p), where p is the probability of success in a sequential sample of independent identically Bernoulli distributed random variables. As a loss associated with estimation we introduce a generalized LINEX loss function. We construct a sequential procedure possessing some asymptotically optimal properties in the case when p tends to zero. In this approach to the problem, the conditions are given, under which the stopping time is asymptotically efficient and normal, and the corresponding sequential estimator is asymptotically normal. The procedure constructed guarantees that its sequential risk is asymptotically equal to a prescribed constant.     

On the stability of the jackknife variance estimator in ratio estimation

The stability of a slightly modified version of the usual jackknife variance estimator is evaluated exactly in small samples under a suitable linear regression model and compared with that of two different linearization variance estimators. Depending on the degree of heteroscedasticity of the error variance in the model, the stability of the jackknife variance estimator is found to be somewhat comparable to that of one or the other of the linearization variance estimators under conditions especially favorable to ratio estimation (i.e., regression approximately through the origin with a relatively small coefficient of variation in the x population). When these conditions do not hold, however, the jackknife variance estimator is found to be less stable than either of the linearization variance estimators.