On the calculations of the maximum likelihood estimates for the polynomial spline regression model with unknown knots and AR(1) errors

Asymptotic distributions of the maximum likelihood estimators of the regression coefficients and knot points for the polynomial spline regression models with unknown knots and AR(1) errors have been derived by Chan (1989). Chan showed that under some mild conditions the maximum likelihood estimators, after suitable standardization, asymptotically follow normal distributions as n diverges to infinity. For the calculations of the maximum likelihood estimators, iterative methods must be applied. But this is not easy to implement for the model considered. In this paper, we suggested an alternative method to compute the estimates of the regression parameters and knots. It is shown that the estimates obtained by this method are asymptotically equivalent to the maximum likelihood estimates considered by Chan.     

Modelling a non-stationary BINAR(1) Poisson process

ABSTRACT

Non-stationarity in bivariate time series of counts may be induced by a number of time-varying covariates affecting the bivariate responses due to which the innovation terms of the individual series as well as the bivariate dependence structure becomes non-stationary. So far, in the existing models, the innovation terms of individual INAR(1) series and the dependence structure are assumed to be constant even though the individual time series are non-stationary. Under this assumption, the reliability of the regression and correlation estimates is questionable. Besides, the existing estimation methodologies such as the conditional maximum likelihood (CMLE) and the composite likelihood estimation are computationally intensive. To address these issues, this paper proposes a BINAR(1) model where the innovation series follow a bivariate Poisson distribution under some non-stationary distributional assumptions. The method of generalized quasi-likelihood (GQL) is used to estimate the regression effects while the serial and bivariate correlations are estimated using a robust moment estimation technique. The application of model and estimation method is made in the simulated data. The GQL method is also compared with the CMLE, generalized method of moments (GMM) and generalized estimating equation (GEE) approaches where through simulation studies, it is shown that GQL yields more efficient estimates than GMM and equally or slightly more efficient estimates than CMLE and GEE.     

The beta distribution as a latent response model for ordinal data (I): Estimation of location and dispersion parameters

Ordinal data are often modeled using a continuous latent response distribution, which is partially observed through windows of adjacent intervals defined by cutpoints. In this paper we propose the beta distribution as a model for the latent response. The beta distribution has several advantages over the other common distributions used, e.g. , normal and logistic. In particular, it enables separate modeling of location and dispersion effects which is essential in the Taguchi method of robust design. First, we study the problem of estimating the location and dispersion parameters of a single beta distribution (representing a single treatment) from ordinal data assuming known equispaced cutpoints. Two methods of estimation are compared: the maximum likelihood method and the method of moments. Two methods of treating the data are considered: in raw discrete form and in smoothed continuousized form. A large scale simulation study is carried out to compare the different methods. The mean square errors of the estimates are obtained under a variety of parameter configurations. Comparisons are made based on the ratios of the mean square errors (called the relative efficiencies). No method is universally the best, but the maximum likelihood method using continuousized data is found to perform generally well, especially for estimating the dispersion parameter. This method is also computationally much faster than the other methods and does not experience convergence difficulties in case of sparse or empty cells. Next, the problem of estimating unknown cutpoints is addressed. Here the multiple treatments setup is considered since in an actual application, cutpoints are common to all treatments, and must be estimated from all the data. A two-step iterative algorithm is proposed for estimating the location and dispersion parameters of the treatments, and the cutpoints. The proposed beta model and McCullagh's (1980) proportional odds model are compared by fitting them to two real data sets.     

Periodic integer-valued GARCH(1, 1) model

This article deals with some probabilistic and statistical properties of a periodic integer-valued GARCH(1,1) model. Necessary and sufficient conditions for the periodical stationary, both in mean and second order, are established. The closed-forms of the mean and the second moment are, under these conditions, obtained. The condition of the existence of higher moment orders and their explicit formula in terms of the parameters are established. The autocovariance structure is studied, while providing the closed-form of the periodic autocorrelation function. The Yule–Walker and the likelihood estimations of the underlying parameters are obtained. A simulation study and an application on real dataset are provided.     

Communication in Statistics-Theory and methods improved GQL estimation method for the generalised BINMA(1) model

In a recent research, the quasi-likelihood estimation methodology was developed to estimate the regression effects in the Generalized BINMA(1) (GBINMA(1)) process. The method provides consistent parameter estimates but, in the intermediate computations, moment estimating equations were used to estimate the serial- and cross-correlation parameters. This procedure may not result optimal parameter estimates, in particular, for the regression effects. This paper provides an alternative simpler GBINMA(1) process based on multivariate thinning properties where the main effects are estimated via a robust generalized quasi-likelihood (GQL) estimation approach. The two techniques are compared through some simulation experiments. A real-life data application is studied.     

Bootstrapping a time series model: some empirical results

The bootstrap is a methodology for estimating standard errors. The idea is to use a Monte Carlo simulation experiment based on a nonparametric estimate of the error distribution. The main objective of this article is to demonstrate the use of the bootstrap to attach standard errors to coefficient estimates in a second-order autoregressive model fitted by least squares and maximum likelihood estimation. Additionally, a comparison of the bootstrap and the conventional methodology is made. As it turns out, the conventional asymptotic formulae (both the least squares and maximum likelihood estimates) for estimating standard errors appear to overestimate the true standard errors. But there are two problems:i. The first two observations y₁ and y₂ have been fixed, and ii. The residuals have not been inflated. After these two factors are considered in the trial and bootstrap experiment, both the conventional maximum likelihood and bootstrap estimates of the standard errors appear to be performing quite well.     

Empirical likelihood inference for parameters in a partially linear errors-in-variables model

In this paper, we consider the application of the empirical likelihood method to a partially linear model with measurement errors in the non-parametric part. It is shown that the empirical log-likelihood ratio at the true parameters converges to the standard chi-square distribution. Furthermore, we obtain the maximum empirical likelihood estimate of the unknown parameter by using the empirical log-likelihood ratio function, and the resulting estimator is shown to be asymptotically normal. Some simulations and an application are conducted to illustrate the proposed method.     

A class of observation-driven random coefficient INAR(1) processes based on negative binomial thinning

Integer-valued time series models and their applications have attracted a lot of attention over the last years. In this paper, we introduce a class of observation-driven random coefficient integer-valued autoregressive processes based on negative binomial thinning, where the autoregressive parameter depends on the observed values of the previous moment. Basic probability and statistics properties of the process are established. The unknown parameters are estimated by the conditional least squares and empirical likelihood methods. Specially, we consider three aspects of the empirical likelihood method: maximum empirical likelihood estimate, confidence region and EL test. The performance of the two estimation methods is compared through simulation studies. Finally, an application to a real data example is provided.     

Le problème d'Anscombe pour les lois binomiales négatives généralisées

Consider a finite sample from a generalized negative-binomial distribution where both (canonical and index) parameters are unknown. This note proves that both the maximum-likelihood estimate and the moment estimate of the index parameter exist if and only if the sample variance is greater than the sample mean. This extends a result for the negative-binomial distribution that had been conjectured by Anscombe (1950) and later shown by Levin and Reeds (1977).     

A robust closed-form estimator for the GARCH(1,1) model

In this paper we extend the closed-form estimator for the generalized autoregressive conditional heteroscedastic (GARCH(1,1)) proposed by Kristensen and Linton [A closed-form estimator for the GARCH(1,1) model. Econom Theory. 2006;22:323–337] to deal with additive outliers. It has the advantage that is per se more robust that the maximum likelihood estimator (ML) often used to estimate this model, it is easy to implement and does not require the use of any numerical optimization procedure. The robustification of the closed-form estimator is done by replacing the sample autocorrelations by a robust estimator of these correlations and by estimating the volatility using robust filters. The performance of our proposal in estimating the parameters and the volatility of the GARCH(1,1) model is compared with the proposals existing in the literature via intensive Monte Carlo experiments and the results of these experiments show that our proposal outperforms the ML and quasi-maximum likelihood estimators-based procedures. Finally, we fit the robust closed-form estimator and the benchmarks to one series of financial returns and analyse their performances in estimating and forecasting the volatility and the value-at-risk.     

Importance sampling for signal extraction model in time series analysis

In time series analysis, signal extraction model (SEM) is used to estimate unobserved signal component from observed time series data. Since parameters of the components in SEM are often unknown in practice, a commonly used method is to estimate unobserved signal component using the maximum likelihood estimates (MLEs) of parameters of the components. This paper explores an alternative way to estimate unobserved signal component when parameters of the components are unknown. The suggested method makes use of importance sampling (IS) with Bayesian inference. The basic idea is to treat parameters of the components in SEM as a random vector and compute a posterior probability density function of the parameters using Bayesian inference. Then IS method is applied to integrate out the parameters and thus estimates of unobserved signal component, unconditional to the parameters, can be obtained. This method is illustrated with a real time series data. Then a Monte Carlo study with four different types of time series models is carried out to compare a performance of this method with that of a commonly used method. The study shows that IS method with Bayesian inference is computationally feasible and robust, and more efficient in terms of mean square errors (MSEs) than a commonly used method.     

Kernel Density-Based Linear Regression Estimate

For linear regression models with non normally distributed errors, the least squares estimate (LSE) will lose some efficiency compared to the maximum likelihood estimate (MLE). In this article, we propose a kernel density-based regression estimate (KDRE) that is adaptive to the unknown error distribution. The key idea is to approximate the likelihood function by using a nonparametric kernel density estimate of the error density based on some initial parameter estimate. The proposed estimate is shown to be asymptotically as efficient as the oracle MLE which assumes the error density were known. In addition, we propose an EM type algorithm to maximize the estimated likelihood function and show that the KDRE can be considered as an iterated weighted least squares estimate, which provides us some insights on the adaptiveness of KDRE to the unknown error distribution. Our Monte Carlo simulation studies show that, while comparable to the traditional LSE for normal errors, the proposed estimation procedure can have substantial efficiency gain for non normal errors. Moreover, the efficiency gain can be achieved even for a small sample size.     

A comparison of LS/ML and GMM estimation in a simple AR(1) model

This paper compares least squares (LS)/maximum likelihood (ML) and generalised method of moments (GMM) estimation in a simple. Gaussian autoregressive of order one (AR(1)) model. First, we show that the usual LS/ML estimator is a corner solution to a general minimisation problem that involves two moment conditions, while the new GMM we devise is not. Secondly, we examine asymptotic and finite sample properties of the new GMM estimator in comparison to the usual LS/ML estimator in a simple AR(1) model. For both stable and unstable (unit root) specifications, we show asymptotic equivalence of the distributions of the two estimators. However, in finite samples, the new GMM estimator performs better.     

Bias reduction for high quantiles

High quantile estimation is of importance in risk management. For a heavy-tailed distribution, estimating a high quantile is done via estimating the tail index. Reducing the bias in a tail index estimator can be achieved by using either the same order or a larger order of number of the upper order statistics in comparison with the theoretical optimal one in the classical tail index estimator. For the second approach, one can either estimate all parameters simultaneously or estimate the first and second order parameters separately. Recently, the first method and the second method via external estimators for the second order parameter have been applied to reduce the bias in high quantile estimation. Theoretically, the second method obviously gives rise to a smaller order of asymptotic mean squared error than the first one. In this paper we study the second method with simultaneous estimation of all parameters for reducing bias in high quantile estimation.     

Semiparametric estimation of the single-index varying-coefficient model

In this paper, we consider the choice of pilot estimators for the single-index varying-coefficient model, which may result in radically different estimators, and develop the method for estimating the unknown parameter in this model. To estimate the unknown parameters efficiently, we use the outer product of gradient method to find the consistent initial estimators for interest parameters, and then adopt the refined estimation method to improve the efficiency, which is similar to the refined minimum average variance estimation method. An algorithm is proposed to estimate the model directly. Asymptotic properties for the proposed estimation procedure have been established. The bandwidth selection problem is also considered. Simulation studies are carried out to assess the finite sample performance of the proposed estimators, and efficiency comparisons between the estimation methods are made.     

Empirical likelihood for nonlinear regression models with nonignorable missing responses

This article develops three empirical likelihood (EL) approaches to estimate parameters in nonlinear regression models in the presence of nonignorable missing responses. These are based on the inverse probability weighted (IPW) method, the augmented IPW (AIPW) method and the imputation technique. A logistic regression model is adopted to specify the propensity score. Maximum likelihood estimation is used to estimate parameters in the propensity score by combining the idea of importance sampling and imputing estimating equations. Under some regularity conditions, we obtain the asymptotic properties of the maximum EL estimators of these unknown parameters. Simulation studies are conducted to investigate the finite sample performance of our proposed estimation procedures. Empirical results provide evidence that the AIPW procedure exhibits better performance than the other two procedures. Data from a survey conducted in 2002 are used to illustrate the proposed estimation procedure. The Canadian Journal of Statistics 48: 386–416; 2020 © 2020 Statistical Society of Canada     

A non‐stationary bivariate INAR(1) process with a simple cross‐dependence: Estimation with some properties

This paper considers modelling of a non‐stationary bivariate integer‐valued autoregressive process of order 1 (BINAR(1)) where the cross‐dependence between the counting series is formed through the relationship of the current series with the previous‐lagged count series observations while the pair of innovations is independent and marginally Poisson. In addition, this paper proposes a generalised quasi‐likelihood (GQL) estimating equation based on the exact specification of the mean score and the auto‐covariance structure. The proposed approach is also compared with other popular techniques such as conditional maximum likelihood (CML), generalised least squares (GLS) and generalised method of moment (GMM) based on simulated data from the proposed BINAR(1). Moreover, the model is applied to weekly series of day and night road accidents arising in some regions of Mauritius and is compared with other existing BINAR(1) models.     

Extreme value index estimator using maximum likelihood and moment estimation

ABSTRACT

When a distribution function is in the max domain of attraction of an extreme value distribution, its tail can be well approximated by a generalized Pareto distribution. Based on this fact we use a moment estimation idea to propose an adapted maximum likelihood estimator for the extreme value index, which can be understood as a combination of the maximum likelihood estimation and moment estimation. Under certain regularity conditions, we derive the asymptotic normality of the new estimator and investigate its finite sample behavior by comparing with several classical or competitive estimators. A simulation study shows that the new estimator is competitive with other estimators in view of average bias, average MSE, and coefficient of variance of the new device for the optimal selection of the threshold.     

Estimating Moments in Linear Mixed Models

In this article, we investigate estimating moments, up to fourth order, in linear mixed models. For this estimation, we only assume the existence of moments. The obtained estimators of the model parameters and the third and fourth moments of the errors and random effects are proved to be consistent or asymptotically normal. The estimation provides a base for further statistical inference such as confidence region construction and hypothesis testing for the parameters of interest. Moreover, the method is readily extended to estimate higher moments. A simulation is carried out to examine the performance of this estimating method.     

Mean-squared errors of small-area estimators under a unit-level multivariate model

This work deals with estimating the vector of means of certain characteristics of small areas. In this context, a unit level multivariate model with correlated sampling errors is considered. An approximation is obtained for the mean-squared and cross-product errors of the empirical best linear unbiased predictors of the means, when model parameters are estimated either by maximum likelihood (ML) or by restricted ML. This approach has been implemented on a Monte Carlo study using social and labour data from the Spanish Labour Force Survey.