Developing a restricted two-parameter Liu-type estimator: A comparison of restricted estimators in the binary logistic regression model

In the context of estimating regression coefficients of an ill-conditioned binary logistic regression model, we develop a new biased estimator having two parameters for estimating the regression vector parameter β when it is subjected to lie in the linear subspace restriction Hβ = h. The matrix mean squared error and mean squared error (MSE) functions of these newly defined estimators are derived. Moreover, a method to choose the two parameters is proposed. Then, the performance of the proposed estimator is compared to that of the restricted maximum likelihood estimator and some other existing estimators in the sense of MSE via a Monte Carlo simulation study. According to the simulation results, the performance of the estimators depends on the sample size, number of explanatory variables, and degree of correlation. The superiority region of our proposed estimator is identified based on the biasing parameters, numerically. It is concluded that the new estimator is superior to the others in most of the situations considered and it is recommended to the researchers.     

Estimating the Cost of a Warranty

The cost of certain types of warranties is closely related to functions that arise in renewal theory. The problem of estimating the warranty cost for a random sample of size n can be reduced to estimating these functions. In an earlier paper, I gave several methods of estimating the expected number of renewals, called the renewal function. This answered an important accounting question of how to arrive at a good approximation of the expected warranty cost. In this article, estimation of the renewal function is reviewed and several extensions are given. In particular, a resampling estimator of the renewal function is introduced. Further, I argue that managers may wish to examine other summary measures of the warranty cost, in particular the variability. To estimate this variability, I introduce estimators, both parametric and nonparametric, of the variance associated with the number of renewals. Several numerical examples are provided.     

Efficient estimation for incomplete multivariate data

We review the Fisher scoring and EM algorithms for incomplete multivariate data from an estimating function point of view, and examine the corresponding quasi-score functions under second-moment assumptions. A bias-corrected REML-type estimator for the covariance matrix is derived, and the Fisher, Godambe and empirical sandwich information matrices are compared. We make a numerical investigation of the two algorithms, and compare with a hybrid algorithm, where Fisher scoring is used for the mean vector and the EM algorithm for the covariance matrix.     

Characterization of an optimal matrix estimator under convex loss function

Characterization of an optimal vector estimator and an optimal matrix estimator are obtained. In each case appropriate convex loss functions are considered. The results are illustrated through the problems of simultaneous unbiased estimation, simultaneous equivariant estimation and simultaneous unbiased prediction. Further an optimality criterion is proposed for matrix unbiased estimation and it is shown that the matrix unbiased estimation of a matrix parametric function and the minimum variance unbiased estimation of its components are equivalent.     

Investigating the sensitivity of Gaussian processes to the choice of their correlation function and prior specifications

A Gaussian process (GP) can be thought of as an infinite collection of random variables with the property that any subset, say of dimension n, of these variables have a multivariate normal distribution of dimension n, mean vector β and covariance matrix Σ [O'Hagan, A., 1994, Kendall's Advanced Theory of Statistics, Vol. 2B, Bayesian Inference (John Wiley & Sons, Inc.)]. The elements of the covariance matrix are routinely specified through the multiplication of a common variance by a correlation function. It is important to use a correlation function that provides a valid covariance matrix (positive definite). Further, it is well known that the smoothness of a GP is directly related to the specification of its correlation function. Also, from a Bayesian point of view, a prior distribution must be assigned to the unknowns of the model. Therefore, when using a GP to model a phenomenon, the researcher faces two challenges: the need of specifying a correlation function and a prior distribution for its parameters. In the literature there are many classes of correlation functions which provide a valid covariance structure. Also, there are many suggestions of prior distributions to be used for the parameters involved in these functions. We aim to investigate how sensitive the GPs are to the (sometimes arbitrary) choices of their correlation functions. For this, we have simulated 25 sets of data each of size 64 over the square [0, 5]×[0, 5] with a specific correlation function and fixed values of the GP's parameters. We then fit different correlation structures to these data, with different prior specifications and check the performance of the adjusted models using different model comparison criteria.     

Parameter estimation for multivariate diffusion processes with the time inhomogeneously positive semidefinite diffusion matrix

Statistical inference for the diffusion coefficients of multivariate diffusion processes has been well established in recent years; however, it is not the case for the drift coefficients. Furthermore, most existing estimation methods for the drift coefficients are proposed under the assumption that the diffusion matrix is positive definite and time homogeneous. In this article, we put forward two estimation approaches for estimating the drift coefficients of the multivariate diffusion models with the time inhomogeneously positive semidefinite diffusion matrix. They are maximum likelihood estimation methods based on both the martingale representation theorem and conditional characteristic functions and the generalized method of moments based on conditional characteristic functions, respectively. Consistency and asymptotic normality of the generalized method of moments estimation are also proved in this article. Simulation results demonstrate that these methods work well.     

Bootstrap estimation of the variance of the error term in monotonic regression models

The variance of the error term in ordinary regression models and linear smoothers is usually estimated by adjusting the average squared residual for the trace of the smoothing matrix (the degrees of freedom of the predicted response). However, other types of variance estimators are needed when using monotonic regression (MR) models, which are particularly suitable for estimating response functions with pronounced thresholds. Here, we propose a simple bootstrap estimator to compensate for the over-fitting that occurs when MR models are estimated from empirical data. Furthermore, we show that, in the case of one or two predictors, the performance of this estimator can be enhanced by introducing adjustment factors that take into account the slope of the response function and characteristics of the distribution of the explanatory variables. Extensive simulations show that our estimators perform satisfactorily for a great variety of monotonic functions and error distributions.     

Use of vector divisions in solving quasi-likelihood equations for a Poisson model

Maximum-likelihood estimation technique is known to provide consistent and most efficient regression estimates but often this technique is tedious to implement, particularly in the modelling of correlated count responses. To overcome this limitation, researchers have developed semi- or quasi-likelihood functions that depend only on the correct specification of the mean and variance of the responses rather than on the distribution function. Moreover, quasi-likelihood estimation provides consistent and equally efficient estimates as the maximum-likelihood approach. Basically, the quasi-likelihood estimating function is a non-linear equation constituting of the gradient, Hessian and basic score matrices. Henceforth, to obtain estimates of the regression parameters, the quasi-likelihood equation is solved iteratively using the Newton–Raphson technique. However, the inverse of the Jacobian matrix involved in the Newton–Raphson method may not be easy to compute since the matrix is very close to singularity. In this paper, we consider the use of vector divisions in solving quasi-likelihood equations. The vector divisions are implemented to form secant method formulas. To assess the performance of the use of vector divisions with the secant method, we generate cross-sectional Poisson counts using different sets of mean parameters. We compute the estimates of the regression parameters using the Newton–Raphson technique and vector divisions and compare the number of non-convergent simulations under both algorithms.     

Variance function in regression analysis of longitudinal data using the generalized estimating equation approach

Longitudinal or clustered response data arise in many applications such as biostatistics, epidemiology and environmental studies. The repeated responses cannot in general be assumed to be independent. One method of analysing such data is by using the generalized estimating equations (GEE) approach. The current GEE method for estimating regression effects in longitudinal data focuses on the modelling of the working correlation matrix assuming a known variance function. However, correct choice of the correlation structure may not necessarily improve estimation efficiency for the regression parameters if the variance function is misspecified [Wang YG, Lin X. Effects of variance-function misspecification in analysis of longitudinal data. Biometrics. 2005;61:413–421]. In this connection two problems arise: finding a correct variance function and estimating the parameters of the chosen variance function. In this paper, we study the problem of estimating the parameters of the variance function assuming that the form of the variance function is known and then the effect of a misspecified variance function on the estimates of the regression parameters. We propose a GEE approach to estimate the parameters of the variance function. This estimation approach borrows the idea of Davidian and Carroll [Variance function estimation. J Amer Statist Assoc. 1987;82:1079–1091] by solving a nonlinear regression problem where residuals are regarded as the responses and the variance function is regarded as the regression function. A limited simulation study shows that the proposed method performs at least as well as the modified pseudo-likelihood approach developed by Wang and Zhao [A modified pseudolikelihood approach for analysis of longitudinal data. Biometrics. 2007;63:681–689]. Both these methods perform better than the GEE approach.     

A semiparametric approach for modelling multivariate nonlinear time series

In this article, a semiparametric time‐varying nonlinear vector autoregressive (NVAR) model is proposed to model nonlinear vector time series data. We consider a combination of parametric and nonparametric estimation approaches to estimate the NVAR function for both independent and dependent errors. We use the multivariate Taylor series expansion of the link function up to the second order which has a parametric framework as a representation of the nonlinear vector regression function. After the unknown parameters are estimated by the maximum likelihood estimation procedure, the obtained NVAR function is adjusted by a nonparametric diagonal matrix, where the proposed adjusted matrix is estimated by the nonparametric kernel estimator. The asymptotic consistency properties of the proposed estimators are established. Simulation studies are conducted to evaluate the performance of the proposed semiparametric method. A real data example on short‐run interest rates and long‐run interest rates of United States Treasury securities is analyzed to demonstrate the application of the proposed approach. The Canadian Journal of Statistics 47: 668–687; 2019 © 2019 Statistical Society of Canada     

The rank transformation as a method of discrimination with some examples

The procedure of statistical discrimination Is simple in theory but so simple in practice. An observation x₀possibly uiultivariate, is to be classified into one of several populations π₁,…,π_k which have respectively, the density functions f₁(x), ? ? ? , f_k(x). The decision procedure is to evaluate each density function at X₀ to see which function gives the largest value f_i(X₀) , and then to declare that X₀ belongs to the population corresponding to the largest value. If these den-sities can be assumed to be normal with equal covariance matricesthen the decision procedure is known as Fisher’s linear discrimi-nant function (LDF) method. In the case of unequal covariance matrices the procedure is called the quadratic discriminant func-tion (QDF) method. If the densities cannot be assumed to be nor-mal then the LDF and QDF might not perform well. Several different procedures have appeared in the literature which offer discriminant procedures for nonnormal data. However, these pro-cedures are generally difficult to use and are not readily available as canned statistical programs.

Another approach to discriminant analysis is to use some sortof mathematical trans format ion on the samples so that their distribution function is approximately normal, and then use the convenient LDF and QDF methods. One transformation that:applies to all distributions equally well is the rank transformation. The result of this transformation is that a very simple and easy to use procedure is made available. This procedure is quite robust as is evidenced by comparisons of the rank transform results with several published simulation studies.     

Covariance Matrix Estimation via Network Structure

In this article, we employ a regression formulation to estimate the high-dimensional covariance matrix for a given network structure. Using prior information contained in the network relationships, we model the covariance as a polynomial function of the symmetric adjacency matrix. Accordingly, the problem of estimating a high-dimensional covariance matrix is converted to one of estimating low dimensional coefficients of the polynomial regression function, which we can accomplish using ordinary least squares or maximum likelihood. The resulting covariance matrix estimator based on the maximum likelihood approach is guaranteed to be positive definite even in finite samples. Under mild conditions, we obtain the theoretical properties of the resulting estimators. A Bayesian information criterion is also developed to select the order of the polynomial function. Simulation studies and empirical examples illustrate the usefulness of the proposed methods.     

Estimation and inference for varying coefficient partially nonlinear models

In this paper, we extend the varying coefficient partially linear model to the varying coefficient partially nonlinear model in which the linear part of the varying coefficient partially linear model is replaced by a nonlinear function of the covariates. A profile nonlinear least squares estimation procedure for the parameter vector and the coefficient function vector of the varying coefficient partially nonlinear model is proposed and the asymptotic properties of the resulting estimators are established. We further propose a generalized likelihood ratio (GLR) test to check whether or not the varying coefficients in the model are constant. The asymptotic null distribution of the GLR statistic is derived and a residual-based bootstrap procedure is also suggested to derive the p-value of the GLR test. Some simulations are conducted to assess the performance of the proposed estimating and testing procedures and the results show that both the procedures perform well in finite samples. Furthermore, a real data example is given to demonstrate the application of the proposed model and its estimating and testing procedures.     

A two-way classification of regression estimation strategies in probability sampling

This paper examines strategies for estimating the mean of a finite population in the following situation: A linear regression model is assumed to describe the population scatter. Various estimators β for the vector of regression parameters β are considered. Several ways of transforming each estimator β into a model-based estimator for the population mean are considered. Some estimators constructed in this way become sensitive to correctness of the assumed model. The estimators favoured in this paper are the ones in which the observations are weighted to reflect the sampling design, so that asymptotic design unbiasedness is achieved. For these estimators, the randomization distribution gives protection against model breakdown.     

Linear discrimination for multi-level multivariate data with separable means and jointly equicorrelated covariance structure

In this article we study a linear discriminant function of multiple m-variate observations at u-sites and over v-time points under the assumption of multivariate normality. We assume that the m-variate observations have a separable mean vector structure and a “jointly equicorrelated covariance” structure. The new discriminant function is very effective in discriminating individuals in a small sample scenario. No closed-form expression exists for the maximum likelihood estimates of the unknown population parameters, and their direct computation is nontrivial. An iterative algorithm is proposed to calculate the maximum likelihood estimates of these unknown parameters. A discriminant function is also developed for unstructured mean vectors. The new discriminant functions are applied to simulated data sets as well as to a real data set. Results illustrating the benefits of the new classification methods over the traditional one are presented.     

Efficient Estimation of the Seemingly Unrelated Regression Cointegration Model and Testing for Purchasing Power Parity

Abstract

This paper studies the efficient estimation of seemingly unrelated linear models with integrated regressors and stationary errors. We consider two cases. The first one has no common regressor among the equations. In this case, we show that by adding leads and lags of the first differences of the regressors and estimating this augmented dynamic regression model by generalized least squares using the long-run covariance matrix, we obtain an efficient estimator of the cointegrating vector that has a limiting mixed normal distribution. In the second case we consider, there is a common regressor to all equations, and we discuss efficient minimum distance estimation in this context. Simulation results suggests that our new estimator compares favorably with others already proposed in the literature. We apply these new estimators to the testing of the proportionality and symmetry conditions implied by purchasing power parity (PPP) among the G-7 countries. The tests based on the efficient estimates easily reject the joint hypotheses of proportionality and symmetry for all countries with either the United States or Germany as numeraire. Based on individual tests, our results suggest that Canada and Germany are the most likely countries for which the proportionality condition holds, and that Italy and Japan for the symmetry condition relative to the United States.     

Wavelet threshold based on Stein's unbiased risk estimators of restricted location parameter in multivariate normal

In this paper, the problem of estimating the mean vector under non-negative constraints on location vector of the multivariate normal distribution is investigated. The value of the wavelet threshold based on Stein''s unbiased risk estimators is calculated for the shrinkage estimator in restricted parameter space. We suppose that covariance matrix is unknown and we find the dominant class of shrinkage estimators under Balance loss function. The performance evaluation of the proposed class of estimators is checked through a simulation study by using risk and average mean square error values.     

On Nonsmooth Estimating Functions via Jackknife Empirical Likelihood

In many applications, the parameters of interest are estimated by solving non‐smooth estimating functions with U‐statistic structure. Because the asymptotic covariances matrix of the estimator generally involves the underlying density function, resampling methods are often used to bypass the difficulty of non‐parametric density estimation. Despite its simplicity, the resultant‐covariance matrix estimator depends on the nature of resampling, and the method can be time‐consuming when the number of replications is large. Furthermore, the inferences are based on the normal approximation that may not be accurate for practical sample sizes. In this paper, we propose a jackknife empirical likelihood‐based inferential procedure for non‐smooth estimating functions. Standard chi‐square distributions are used to calculate the p‐value and to construct confidence intervals. Extensive simulation studies and two real examples are provided to illustrate its practical utilities.     

Efficient Estimation of the Seemingly Unrelated Regression Cointegration Model and Testing for Purchasing Power Parity

This paper studies the efficient estimation of seemingly unrelated linear models with integrated regressors and stationary errors. We consider two cases. The first one has no common regressor among the equations. In this case, we show that by adding leads and lags of the first differences of the regressors and estimating this augmented dynamic regression model by generalized least squares using the long-run covariance matrix, we obtain an efficient estimator of the cointegrating vector that has a limiting mixed normal distribution. In the second case we consider, there is a common regressor to all equations, and we discuss efficient minimum distance estimation in this context. Simulation results suggests that our new estimator compares favorably with others already proposed in the literature. We apply these new estimators to the testing of the proportionality and symmetry conditions implied by purchasing power parity (PPP) among the G-7 countries. The tests based on the efficient estimates easily reject the joint hypotheses of proportionality and symmetry for all countries with either the United States or Germany as numeraire. Based on individual tests, our results suggest that Canada and Germany are the most likely countries for which the proportionality condition holds, and that Italy and Japan for the symmetry condition relative to the United States.     

Unbiased statistical estimation functions for parameters in presence of nuisance parameters

We consider the problem of estimating a vector interesting parameter in the presence of nuisance parameters through vector unbiased statistical estimation functions (USEFs). An extension of the Cramer—Rao inequality relevant to the present problem is obtained. Three possible optimality criteria in the class of regular vector USEFs are those based on (i) the non-negative definiteness of the difference of dispersion matrices (ii) the trace of the dispersion matrix and (iii) the determinant of the dispersion matrix. We refer to these three criteria as M-optimality, T- optimality and D-optimality respectively. The equivalence of these three optimality criteria is established. By restricting the class of regular USEFs considered by Ferreira (1982), we study some interesting properties of the standardized USEFs and establish essential uniqueness of standardized M-optimal USEF in this restricted class. Finally some illustrative examples are included.