Efficient M-estimators with auxiliary information

This paper introduces a new class of M-estimators based on generalised empirical likelihood (GEL) estimation with some auxiliary information available in the sample. The resulting class of estimators is efficient in the sense that it achieves the same asymptotic lower bound as that of the efficient generalised method of moment (GMM) estimator with the same auxiliary information. The paper also shows that in case of smooth estimating equations the proposed estimators enjoy a small second order bias property compared to both efficient GMM and full GEL estimators. Analytical formulae to obtain bias corrected estimators are also provided. Simulations show that with correctly specified auxiliary information the proposed estimators and in particular those based on empirical likelihood outperform standard M and efficient GMM estimators both in terms of finite sample bias and efficiency. On the other hand with moderately misspecified auxiliary information estimators based on the nonparametric tilting method are typically characterised by the best finite sample properties.     

Jackknife empirical likelihood for the error variance in linear models

Variance estimation is a fundamental yet important problem in statistical modelling. In this paper, we propose jackknife empirical likelihood (JEL) methods for the error variance in a linear regression model. We prove that the JEL ratio converges to the standard chi-squared distribution. The asymptotic chi-squared properties for the adjusted JEL and extended JEL estimators are also established. Extensive simulation studies to compare the new JEL methods with the standard method in terms of coverage probability and interval length are conducted, and the simulation results show that our proposed JEL methods perform better than the standard method. We also illustrate the proposed methods using two real data sets.     

Optimal estimation for the Pareto distribution

This paper proposes an optimal estimation method for the shape parameter, probability density function and upper tail probability of the Pareto distribution. The new method is based on a weighted empirical distribution function. The exact efficiency functions of the estimators relative to the existing estimators are derived. The paper gives L ₁-optimal and L ₂-optimal weights for the new weighted estimator. Monte Carlo simulation results confirm the theoretical conclusions. Both theoretical and simulation results show that the new estimation method is more efficient relative to several existing methods in many situations.     

An example of applying the asymptotic quasi-likelihood to dimension estimation for random spatial patterns

With reference to a specific example of a random spatial fractal and the modified box-counting method of dimension estimation, this paper aims to examine firstly the estimation of pointwise dimension via modification of the box-counting procedure, secondly the regression inspired estimation procedure, including generalised least squares and, finally, to develop a new estimation procedure – the asymptotic quasi-likelihood method – for the estimation of pointwise dimension. The main focus is on practicality – to arrive at an estimation method which is easy to use and robust.     

Optimality of Equal vs. Unequal Cluster Sizes in Multilevel Intervention Studies: A Monte Carlo Study for Small Sample Sizes

Optimality of equal versus unequal cluster sizes in the context of multilevel intervention studies is examined. A Monte Carlo study is done to examine to what degree asymptotic results on the optimality hold for realistic sample sizes and for different estimation methods. The relative D-criterion, comparing equal versus unequal cluster sizes, almost always exceeded 85%, implying that loss of information due to unequal cluster sizes can be compensated for by increasing the number of clusters by 18%. The simulation results are in line with asymptotic results, showing that, for realistic sample sizes and various estimation methods, the asymptotic results can be used in planning multilevel intervention studies.     

Model selection consistency of U-statistics with convex loss and weighted lasso penalty

In the paper we consider minimisation of U-statistics with the weighted Lasso penalty and investigate their asymptotic properties in model selection and estimation. We prove that the use of appropriate weights in the penalty leads to the procedure that behaves like the oracle that knows the true model in advance, i.e. it is model selection consistent and estimates nonzero parameters with the standard rate. For the unweighted Lasso penalty, we obtain sufficient and necessary conditions for model selection consistency of estimators. The obtained results strongly based on the convexity of the loss function that is the main assumption of the paper. Our theorems can be applied to the ranking problem as well as generalised regression models. Thus, using U-statistics we can study more complex models (better describing real problems) than usually investigated linear or generalised linear models.     

On the asymptotic non‐equivalence of efficient‐GMM and MEL estimators in models with missing data

The generalized method of moments (GMM) and empirical likelihood (EL) are popular methods for combining sample and auxiliary information. These methods are used in very diverse fields of research, where competing theories often suggest variables satisfying different moment conditions. Results in the literature have shown that the efficient‐GMM (GMM_E) and maximum empirical likelihood (MEL) estimators have the same asymptotic distribution to order n^?1/2 and that both estimators are asymptotically semiparametric efficient. In this paper, we demonstrate that when data are missing at random from the sample, the utilization of some well‐known missing‐data handling approaches proposed in the literature can yield GMM_E and MEL estimators with nonidentical properties; in particular, it is shown that the GMM_E estimator is semiparametric efficient under all the missing‐data handling approaches considered but that the MEL estimator is not always efficient. A thorough examination of the reason for the nonequivalence of the two estimators is presented. A particularly strong feature of our analysis is that we do not assume smoothness in the underlying moment conditions. Our results are thus relevant to situations involving nonsmooth estimating functions, including quantile and rank regressions, robust estimation, the estimation of receiver operating characteristic (ROC) curves, and so on.     

The influence function of penalized regression estimators

To perform regression analysis in high dimensions, lasso or ridge estimation are a common choice. However, it has been shown that these methods are not robust to outliers. Therefore, alternatives as penalized M-estimation or the sparse least trimmed squares (LTS) estimator have been proposed. The robustness of these regression methods can be measured with the influence function. It quantifies the effect of infinitesimal perturbations in the data. Furthermore, it can be used to compute the asymptotic variance and the mean-squared error (MSE). In this paper we compute the influence function, the asymptotic variance and the MSE for penalized M-estimators and the sparse LTS estimator. The asymptotic biasedness of the estimators make the calculations non-standard. We show that only M-estimators with a loss function with a bounded derivative are robust against regression outliers. In particular, the lasso has an unbounded influence function.     

Inference for random coefficient INAR(k) with the occasional level shift random noise based on dual empirical likelihood

The INAR(k) model has been widely used in various kinds of fields. However, there are little discussions about the INAR(k) model with the occasional level shift random noise. In this paper, the maximum likelihood estimation of parameter based on martingale difference sequence is given, the log empirical likelihood ratio test statistic is obtained and the test statistic converges to chi-square distribution, we prove that the confidence region of the parameter is convex. Furthermore, the numerical simulation of the proposed INAR(k) model is given, which illustrates the effectiveness of the model. Then, the proofs of asymptotic results are given in the Appendix.     

Inference of R=P[X<Y] for generalized logistic distribution

This paper deals with the estimation of R=P[X<Y] when X and Y come from two independent generalized logistic distributions with different parameters. The maximum-likelihood estimator (MLE) and its asymptotic distribution are proposed. The asymptotic distribution is used to construct an asymptotic confidence interval of R. Assuming that the common scale parameter is known, the MLE, uniformly minimum variance unbiased estimator, Bayes estimation and confidence interval of R are obtained. The MLE of R, asymptotic distribution of R in the general case, is also discussed. Monte Carlo simulations are performed to compare the different proposed methods. Analysis of a real data set has also been presented for illustrative purposes.     

Asymptotics of M-estimator in multivariate linear regression models for a class of random errors

It is known that linear regression models have immense applications in various areas such as engineering technology, economics and social sciences. In this paper, we investigate the asymptotic properties of M-estimator in multivariate linear regression model based on a class of random errors satisfying a generalised Bernstein-type inequality. By using the generalised Bernstein-type inequality, we obtain a general result on almost sure convergence for a class of random variables and then obtain the strong consistency for the M-estimator in multivariate linear regression models under some mild conditions. The result extends or improves some existing ones in the literature. Moreover, we also consider the case when the dimension $p$ tends to infinity by establishing the rate of almost sure convergence for a class of random variables satisfying generalised Bernstein-type inequality. Some numerical simulations are also provided to verify the validity of the theoretical results.     

On Nonsmooth Estimating Functions via Jackknife Empirical Likelihood

In many applications, the parameters of interest are estimated by solving non‐smooth estimating functions with U‐statistic structure. Because the asymptotic covariances matrix of the estimator generally involves the underlying density function, resampling methods are often used to bypass the difficulty of non‐parametric density estimation. Despite its simplicity, the resultant‐covariance matrix estimator depends on the nature of resampling, and the method can be time‐consuming when the number of replications is large. Furthermore, the inferences are based on the normal approximation that may not be accurate for practical sample sizes. In this paper, we propose a jackknife empirical likelihood‐based inferential procedure for non‐smooth estimating functions. Standard chi‐square distributions are used to calculate the p‐value and to construct confidence intervals. Extensive simulation studies and two real examples are provided to illustrate its practical utilities.     

Estimating the parameters of Poisson-exponential models

This paper proposes two methods of estimation for the parameters in a Poisson-exponential model. The proposed methods combine the method of moments with a regression method based on the empirical moment generating function. One of the methods is an adaptation of the mixed-moments procedure of Koutrouvelis & Canavos (1999). The asymptotic distribution of the estimator obtained with this method is derived. Finite-sample comparisons are made with the maximum likelihood estimator and the method of moments. The paper concludes with an exploratory-type analysis of real data based on the empirical moment generating function.     

Classical and Bayesian estimation of P(Y < X) for Kumaraswamy's distribution

The aim of this paper is to study the estimation of the reliability R=P(Y<X) when X and Y are independent random variables that follow Kumaraswamy's distribution with different parameters. If we assume that the first shape parameter is common and known, the maximum-likelihood estimator (MLE), the exact confidence interval and the uniformly minimum variance unbiased estimator of R are obtained. Moreover, when the first parameter is common but unknown, MLEs, Bayes estimators, asymptotic distributions and confidence intervals for R are derived. Furthermore, Bayes and empirical Bayes estimators for R are obtained when the first parameter is common and known. Finally, when all four parameters are different and unknown, the MLE of R is obtained. Monte Carlo simulations are performed to compare the different proposed methods and conclusions on the findings are given.     

Generalized empirical likelihood inference in partial linear regression model for longitudinal data

In this paper, empirical likelihood inference for longitudinal data within the framework of partial linear regression models are investigated. The proposed procedures take into consideration the correlation within groups without involving direct estimation of nuisance parameters in the correlation matrix. The empirical likelihood method is used to estimate the regression coefficients and the baseline function, and to construct confidence intervals. A nonparametric version of Wilk's theorem for the limiting distribution of the empirical likelihood ratio is derived. Compared with methods based on normal approximations, the empirical likelihood does not require consistent estimators for the asymptotic variance and bias. The finite sample behaviour of the proposed method is evaluated with simulation and illustrated with an AIDS clinical trial data set.     

Correlation analysis with additive distortion measurement errors

This paper studies the estimation of correlation coefficient between unobserved variables of interest. These unobservable variables are distorted in a additive fashion by an observed confounding variable. Two estimators, a direct-plug-in estimator and a residual-based estimator, are proposed. Their asymptotical results are obtained, and the residual-based estimator is shown asymptotically efficient. Moreover, we suggest an asymptotic normal approximation and an empirical likelihood-based statistic to construct the confidence interval. The empirical likelihood statistic is shown to be asymptotically chi-squared. Simulation studies are conducted to examine the performance of the proposed estimators. These methods are applied to analyse the Boston housing price data for an illustration.     

Testing for structural breaks in the presence of data perturbations: impacts and wavelet-based improvements

This paper investigates how classical measurement error and additive outliers (AO) influence tests for structural change based on F-statistics. We derive theoretically the impact of general additive disturbances in the regressors on the asymptotic distribution of these tests for structural change. The small sample properties in the case of classical measurement error and AO are investigated via Monte Carlo simulations, revealing that sizes are biased upwards and that powers are reduced. Two-wavelet-based denoising methods are used to reduce these distortions. We show that these two methods can significantly improve the performance of structural break tests.     

Generalised regression estimation via the bootstrap

A generalised regression estimation procedure is proposed that can lead to much improved estimation of population characteristics, such as quantiles, variances and coefficients of variation. The method involves conditioning on the discrepancy between an estimate of an auxiliary parameter and its known population value. The key distributional assumption is joint asymptotic normality of the estimates of the target and auxiliary parameters. This assumption implies that the relationship between the estimated target and the estimated auxiliary parameters is approximately linear with coefficients determined by their asymptotic covariance matrix. The main contribution of this paper is the use of the bootstrap to estimate these coefficients, which avoids the need for parametric distributional assumptions. First‐order correct conditional confidence intervals based on asymptotic normality can be improved upon using quantiles of a conditional double bootstrap approximation to the distribution of the studentised target parameter estimate.     

Inference of stress-strength for the Type-II generalized logistic distribution under progressively Type-II censored samples

This article studies the estimation of the reliability R = P[Y < X] when X and Y come from two independent generalized logistic distributions of Type-II with different parameters, based on progressively Type-II censored samples. When the common scale parameter is unknown, the maximum likelihood estimator and its asymptotic distribution are proposed. The asymptotic distribution is used to construct an asymptotic confidence interval of R. Bayes estimator of R and the corresponding credible interval using the Gibbs sampling technique have been proposed too. Assuming that the common scale parameter is known, the maximum likelihood estimator, uniformly minimum variance unbiased estimator, Bayes estimation, and confidence interval of R are extracted. Monte Carlo simulations are performed to compare the different proposed methods. Analysis of a real dataset is given for illustrative purposes. Finally, methods are extended for proportional hazard rate models.     

Semiparametric Whittle estimation of a cyclical long-memory time series based on generalised exponential models

This paper considers a semiparametric estimation of the memory parameter in a cyclical long-memory time series, which exhibits a strong dependence on cyclical behaviour, using the Whittle likelihood based on generalised exponential (GEXP) models. The proposed estimation is included in the so-called broadband or global method and uses information from the spectral density at all frequencies. We establish the consistency and the asymptotic normality of the estimated memory parameter for a linear process and thus do not require Gaussianity. A simulation study conducted using Monte Carlo experiments shows that the proposed estimation works well compared to other existing semiparametric estimations. Moreover, we provide an empirical application of the proposed estimation, applying it to the growth rate of Japan's industrial production index and detecting its cyclical persistence.