Optimal estimating function for estimation and prediction in semi-parametric models

In a seminal paper, Godambe [1985. The foundations of finite sample estimation in stochastic processes. Biometrika 72, 419–428.] introduced the ‘estimating function’ approach to estimation of parameters in semi-parametric models under a filtering associated with a martingale structure. Later, Godambe [1987. The foundations of finite sample estimation in stochastic processes II. Bernoulli, Vol. 2. V.N.V. Science Press, 49–54.] and Godambe and Thompson [1989. An extension of quasi-likelihood Estimation. J. Statist. Plann. Inference 22, 137–172.] replaced this filtering by a more flexible conditioning. Abraham et al. [1997. On the prediction for some nonlinear time-series models using estimating functions. In: Basawa, I.V., et al. (Eds.), IMS Selected Proceedings of the Symposium on Estimating Functions, Vol. 32. pp. 259–268.] and Thavaneswaran and Heyde [1999. Prediction via estimating functions. J. Statist. Plann. Inference 77, 89–101.] invoked the theory of estimating functions for one-step ahead prediction in time-series models. This paper addresses the problem of simultaneous estimation of parameters and multi-step ahead prediction of a vector of future random variables in semi-parametric models by extending the inimitable approach of 13 and 14. The proposed technique is in conformity with the paradigm of the modern theory of estimating functions leading to finite sample optimality within a chosen class of estimating functions, which in turn are used to get the predictors. Particular applications of the technique give predictors that enjoy optimality properties with respect to other well-known criteria.     

Optimal estimating functions and wedderburn's quasi-likelihood

There are two inference methods which can be considered as developed from the classical least squares and maximum likelihood methods. One was put forward by Wedderburn (1974) and is called the quasi-likelihood method. Another was introduced by Godambe and others from the viewpoint of the estimating functions. This method is also called quasi-likelihood although there x are some differences between these two methods. In order to clarify the relationship, this paper provides a unified discussion of the two methods from the viewpoint of estimating functions.     

Quasi Bayesian likelihood

If the form of the distribution of data is unknown, the Bayesian method fails in the parametric inference because there is no posterior distribution of the parameter. In this paper, a theoretical framework of Bayesian likelihood is introduced via the Hilbert space method, which is free of the distributions of data and the parameter. The posterior distribution and posterior score function based on given inner products are defined and, consequently, the quasi posterior distribution and quasi posterior score function are derived, respectively, as the projections of the posterior distribution and posterior score function onto the space spanned by given estimating functions. In the space spanned by data, particularly, an explicit representation for the quasi posterior score function is obtained, which can be derived as a projection of the true posterior score function onto this space. The methods of constructing conservative quasi posterior score and quasi posterior log-likelihood are proposed. Some examples are given to illustrate the theoretical results. As an application, the quasi posterior distribution functions are used to select variables for generalized linear models. It is proved that, for linear models, the variable selections via quasi posterior distribution functions are equivalent to the variable selections via the penalized residual sum of squares or regression sum of squares.     

Residuals in the Extended Growth Curve Model

Abstract.  The Extended Growth Curve model is considered. It turns out that the estimated mean of the model is the projection of the observations on the space generated by the design matrices which turns out to be the sum of two tensor product spaces. The orthogonal complement of this space is decomposed into four orthogonal spaces and residuals are defined by projecting the observation matrix on the resulting components. The residuals are interpreted and some remarks are given as to why we should not use ordinary residuals, what kind of information our residuals give and how this information might be used to validate model assumptions and detect outliers and influential observations. It is shown that the residuals are symmetrically distributed around zero and are uncorrelated with each other. The covariance between the residuals and the estimated model as well as the dispersion matrices for the residuals are also given.     

Independence in Contingency Tables Using Simplicial Geometry

Frequently, contingency tables are generated in a multinomial sampling. Multinomial probabilities are then organized in a table assigning probabilities to each cell. A probability table can be viewed as an element in the simplex. The Aitchison geometry of the simplex identifies independent probability tables as a linear subspace. An important consequence is that, given a probability table, the nearest independent table is obtained by orthogonal projection onto the independent subspace. The nearest independent table is identified as that obtained by the product of geometric marginals, which do not coincide with the standard marginals, except in the independent case. The original probability table is decomposed into orthogonal tables, the independent and the interaction tables. The underlying model is log-linear, and a procedure to test independence of a contingency table, based on a multinomial simulation, is developed. Its performance is studied on an illustrative example.     

Characterization of distributions by conditional expectation of generalized order statistics

In this work, general forms of many well-known continuous probability distributions are characterized by conditional expectation of some functions of generalized order statistics. These results are the generalization of the characterization results based on conditional expectation of the functions of order statistics given by Khan and Abu-Salih (1989).     

A profile Godambe information of power transformations for ARCH time series

Due to Godambe (1985 Godambe, V.P. (1985). The foundation of finite sample estimation in stochastic processes. Biometrika 72:419–428.[Crossref], [Web of Science ®] , [Google Scholar]), one can obtain the Godambe optimum estimating functions (EFs) each of which is optimum (in the sense of maximizing the Godambe information) within a linear class of EFs. Quasi-likelihood scores can be viewed as special cases of the Godambe optimum EFs (see, for instance, Hwang and Basawa, 2011 Hwang, S.Y., Basawa, I.V. (2011). Godambe estimating functions and asymptotic optimal inference. Stat. Probab. Lett. 81:1121–1127.[Crossref], [Web of Science ®] , [Google Scholar]). The paper concerns conditionally heteroscedastic time series with unknown likelihood. Power transformations are introduced in innovations to construct a class of Godambe optimum EFs. A “best” power transformation for Godambe innovation is then obtained via maximizing the “profile” Godambe information. To illustrate, the KOrea Stock Prices Index is analyzed for which absolute value transformation and square transformation are recommended according to the ARCH(1) and GARCH(1,1) models, respectively.     

Generalized quasi-likelihood

The quasi-score function, as defined by Wedderburn (1974) and McCullagh (1983) and so on, is a linear function of observations. The generalized quasi-score function introduced in this paper is a linear function of some unbiased basis functions, where the unbiased basis functions may be some linear functions of the observations or not, and can be easily constructed by the meaning of the parameters such as mean and median and so on. The generalized quasi-likelihood estimate obtained by such a generalized quasi-score function is consistent and has an asymptotically normal distribution. As a result, the optimum generalized quasi-score is obtained and a method to construct the optimum unbiased basis function is introduced. In order to construct the potential function, a conservative generalized estimating function is defined. By conservative, a potential function for the projected score has many properties of a log-likelihood function. Finally, some examples are given to illustrate the theoretical results. This paper is supported by NNSF project (10371059) of China and Youth Teacher Foundation of Nankai University.     

Invariant tests based on M-estimators,estimating functions,and the generalized method of moments

We study the invariance properties of various test criteria which have been proposed for hypothesis testing in the context of incompletely specified models, such as models which are formulated in terms of estimating functions (Godambe, 1960) or moment conditions and are estimated by generalized method of moments (GMM) procedures (Hansen, 1982), and models estimated by pseudo-likelihood (Gouriéroux, Monfort, and Trognon, 1984b,c) and M-estimation methods. The invariance properties considered include invariance to (possibly nonlinear) hypothesis reformulations and reparameterizations. The test statistics examined include Wald-type, LR-type, LM-type, score-type, and C(α)?type criteria. Extending the approach used in Dagenais and Dufour (1991), we show first that all these test statistics except the Wald-type ones are invariant to equivalent hypothesis reformulations (under usual regularity conditions), but all five of them are not generally invariant to model reparameterizations, including measurement unit changes in nonlinear models. In other words, testing two equivalent hypotheses in the context of equivalent models may lead to completely different inferences. For example, this may occur after an apparently innocuous rescaling of some model variables. Then, in view of avoiding such undesirable properties, we study restrictions that can be imposed on the objective functions used for pseudo-likelihood (or M-estimation) as well as the structure of the test criteria used with estimating functions and generalized method of moments (GMM) procedures to obtain invariant tests. In particular, we show that using linear exponential pseudo-likelihood functions allows one to obtain invariant score-type and C(α)?type test criteria, while in the context of estimating function (or GMM) procedures it is possible to modify a LR-type statistic proposed by Newey and West (1987) to obtain a test statistic that is invariant to general reparameterizations. The invariance associated with linear exponential pseudo-likelihood functions is interpreted as a strong argument for using such pseudo-likelihood functions in empirical work.     

Nonlinear recursive estimation of volatility via estimating functions

For certain volatility models, the conditional moments that depend on the parameter are of interest. Following Godambe and Heyde (1987), the combined estimating function method has been used to study inference when the conditional mean and conditional variance are functions of the parameter of interest (See Ghahramani and Thavaneswaran [Combining Estimating Functions for Volatility. Journal of Statistical Planning and Inference, 2009, 139, 1449-1461] for details). However, for application purposes, the resulting estimates are nonlinear functions of the observations and no closed form expressions of the estimates are available. As an alternative, in this paper, a recursive estimation approach based on the combined estimating function is proposed and applied to various classes of time series models, including certain volatility models.     

Projection Measures And Estimation Variances Of Intensities

The paper investigates random processes of geometrical objects in Euclidean spaces. General properties of the measure of total projections are derived by means of Palm distribution. Explicit formulas for variances of the projection measure are obtained for Poisson point processes of compact sets.

Intensity estimators of fibre (surface) processes are then studied by means of projection measures. Classification of direct and indirect probes is introduced. The indirect sampling design of vertical sections and projections is generalized and its statistical properties derived.     

Estimation suroptimale de la densité par projection

Superefficiency of a projection density estimator The author constructs a projection density estimator with a data‐driven truncation index. This estimator reaches the superoptimal rates 1/n in mean integrated square error and {In ln(n/n}^1/2 in uniform almost sure convergence over a given subspace which is dense in the class of all possible densities; the rate of the estimator is quasi‐optimal everywhere else. The subspace in question may be chosen a priori by the statistician.     

Some criteria for projection pursuit

The idea of searching for orthogonal projections, from a multidimensional space into a linear subspace, as an aid to detecting non-linear structure has been named exploratory projection pursuit.Most approaches are tied to the idea of searching for interesting projections. Typically, an interesting projection is one where the distribution of the projected data differs from the normal distribution. In this paper we define two projection indices which are aimed specifically at finding projections that best show grouped structure in the plane, if this exists in the multi-dimensional space. These involve a numerical optimization problem which is tackled in two stages, the projection and the pursuit; the first is based on a procedure to generate pseudo-random rotation matrices in the sense of the grand tour by D. Asimov (1985), and the second is a local numerical optimization procedure. One artificial and one real example illustrate the performance of the suggested indices.     

Efficient estimation for incomplete multivariate data

We review the Fisher scoring and EM algorithms for incomplete multivariate data from an estimating function point of view, and examine the corresponding quasi-score functions under second-moment assumptions. A bias-corrected REML-type estimator for the covariance matrix is derived, and the Fisher, Godambe and empirical sandwich information matrices are compared. We make a numerical investigation of the two algorithms, and compare with a hybrid algorithm, where Fisher scoring is used for the mean vector and the EM algorithm for the covariance matrix.     

Indicator function based on complex contrasts and its application in general factorial designs

A factorial design can be uniquely determined by an indicator function which is constructed by means of orthogonal contrasts. Since the orthogonal contrasts are not unique, invariant measures are preferred. However, some particular orthogonal contrasts may express more information about designs than the others and be worth our attention. In this paper, a kind of indicator function based on orthogonal complex contrasts is introduced to represent general factorial designs and its significance on projection designs is presented. Based on this function, a generalized resolution and a new aberration criterion are developed to rank combinatorially non-isomorphic designs with prime levels. Some results and comparison are provided by means of examples.     

Estimating Mixing Densities in Exponential Family Models for Discrete Variables

This paper is concerned with estimating a mixing density g using a random sample from the mixture distribution f(x)=∫f x | θ)g(θ)dθ where f(· | θ) is a known discrete exponen tial family of density functions. Recently two techniques for estimating g have been proposed. The first uses Fourier analysis and the method of kernels and the second uses orthogonal polynomials. It is known that the first technique is capable of yielding estimators that achieve (or almost achieve) the minimax convergence rate. We show that this is true for the technique based on orthogonal polynomials as well. The practical implementation of these estimators is also addressed. Computer experiments indicate that the kernel estimators give somewhat disappoint ing finite sample results. However, the orthogonal polynomial estimators appear to do much better. To improve on the finite sample performance of the orthogonal polynomial estimators, a way of estimating the optimal truncation parameter is proposed. The resultant estimators retain the convergence rates of the previous estimators and a Monte Carlo finite sample study reveals that they perform well relative to the ones based on the optimal truncation parameter.     

A note on empirical likelihoods derived from pairwise score functions

Pairwise likelihood functions are convenient surrogates for the ordinary likelihood, useful when the latter is too difficult or even impractical to compute. One drawback of pairwise likelihood inference is that, for a multidimensional parameter of interest, the pairwise likelihood analogue of the likelihood ratio statistic does not have the standard chi-square asymptotic distribution. Invoking the theory of unbiased estimating functions, this paper proposes and discusses a computationally and theoretically attractive approach based on the derivation of empirical likelihood functions from the pairwise scores. This approach produces alternatives to the pairwise likelihood ratio statistic, which allow reference to the usual asymptotic chi-square distribution and which are useful when the elements of the Godambe information are troublesome to evaluate or in the presence of large data sets with relative small sample sizes. Two Monte Carlo studies are performed in order to assess the finite-sample performance of the proposed empirical pairwise likelihoods.     

A note on the inequalities for tail probability of the multivariate normal distribution

In this paper inequalities given by Harkness Godambe (1976) for the rail probabilities of the multivariate normal distribution in the equicorrelated case are improved by using the properties of the characteristic roots of a matrix and of the convex function.     

Multivariate control charts based on a projection approach

Summary: In this paper the projection approach of Runger (1996) is applied to construct control charts for a multivariate process. It is assumed that a shift in the mean might only occur in a known subspace of the parameter space. The projection method permits a reduction of the dimensionality of the control problem.Several control schemes based on projections are introduced. We consider CUSUM type charts as well as EWMA schemes. The underlying variables are assumed to be independent and normally distributed. Using the average run length all control charts are compared with each other. Moreover, it is analyzed how sensitive the charts react on a false choice of the subspace.     

Interaction balance for symmetrical factorial designs with generalized minimum aberration

There are two different systems of contrast parameterization when analyzing the interaction effects among the factors with more than two levels, i.e., linear-quadratic system and orthogonal components system. Based on the former system and an ANOVA model, Xu and Wu (2001) introduced the generalized wordlength pattern for general factorial designs. This paper shows that the generalized wordlength pattern exactly measures the balance pattern of interaction columns of a symmetrical design ground on the orthogonal components system, and thus an alternative angle to look at the generalized minimum aberration criterion is given. This work is partially supported by NNSF of China grant No. 10231030.