Changepoint Estimation in a Segmented Linear Regression via Empirical Likelihood

For a segmented regression system with an unknown changepoint over two domains of a predictor, a new empirical likelihood ratio statistic is proposed to test the null hypothesis of no change. Under the null hypothesis of no change, the proposed test statistic is shown empirically to be Gumbel distributed with robust location and scale estimators against various parameter settings and error distributions. A power analysis is conducted to illustrate the performance of the test. Under the alternative hypothesis with a changepoint, the test statistic is utilized to estimate the changepoint between the two domains. A comparison of the frequency distributions between the proposed estimator and two parametric methods indicates that the proposed method is effective in capturing the true changepoint.     

Variance Estimation in Spatial Regression Using a Non-parametric Semivariogram Based on Residuals

Abstract.  The empirical semivariogram of residuals from a regression model with stationary errors may be used to estimate the covariance structure of the underlying process. For prediction (kriging) the bias of the semivariogram estimate induced by using residuals instead of errors has only a minor effect because the bias is small for small lags. However, for estimating the variance of estimated regression coefficients and of predictions, the bias due to using residuals can be quite substantial. Thus we propose a method for reducing this bias. The adjusted empirical semivariogram is then isotonized and made conditionally negative-definite and used to estimate the variance of estimated regression coefficients in a general estimating equations setup. Simulation results for least squares and robust regression show that the proposed method works well in linear models with stationary correlated errors.     

On Semiparametric Mode Regression Estimation

It has been found that, for a variety of probability distributions, there is a surprising linear relation between mode, mean, and median. In this article, the relation between mode, mean, and median regression functions is assumed to follow a simple parametric model. We propose a semiparametric conditional mode (mode regression) estimation for an unknown (unimodal) conditional distribution function in the context of regression model, so that any m-step-ahead mean and median forecasts can then be substituted into the resultant model to deliver m-step-ahead mode prediction. In the semiparametric model, Least Squared Estimator (LSEs) for the model parameters and the simultaneous estimation of the unknown mean and median regression functions by the local linear kernel method are combined to infer about the parametric and nonparametric components of the proposed model. The asymptotic normality of these estimators is derived, and the asymptotic distribution of the parameter estimates is also given and is shown to follow usual parametric rates in spite of the presence of the nonparametric component in the model. These results are applied to obtain a data-based test for the dependence of mode regression over mean and median regression under a regression model.     

Local Linear Estimation for Time-Dependent Coefficients in Cox's Regression Models

This article develops a local partial likelihood technique to estimate the time-dependent coefficients in Cox's regression model. The basic idea is a simple extension of the local linear fitting technique used in the scatterplot smoothing. The coefficients are estimated locally based on the partial likelihood in a window around each time point. Multiple time-dependent covariates are incorporated in the local partial likelihood procedure. The procedure is useful as a diagnostic tool and can be used in uncovering time-dependencies or departure from the proportional hazards model. The programming involved in the local partial likelihood estimation is relatively simple and it can be modified with few efforts from the existing programs for the proportional hazards model. The asymptotic properties of the resulting estimator are established and compared with those from the local constant fitting. A consistent estimator of the asymptotic variance is also proposed. The approach is illustrated by a real data set from the study of gastric cancer patients and a simulation study is also presented.     

Model Order Estimation of a Multivariable Stochastic Process

ABSTRACT

This article presents a procedure allowing us to estimate the minimal order of a state-space representation, for a multivariable stochastic process, from a sequence of observations. The method proposes a statistical rule for testing the rank of a block Hankel matrix of data, since this rank is related to the order of the process. A new information criterion is then developed and used to decide upon the order of the model. In this article we generalize the Aoki C-test. Using two representative data sets as the basis for a Monte Carlo experiment and real data based on Danish economy, we estimate the order of multivariable stochastic processes.     

Singly and Doubly Censored Current Status Data: Estimation, Asymptotics and Regression

In biostatistical applications interest often focuses on the estimation of the distribution of time between two consecutive events. If the initial event time is observed and the subsequent event time is only known to be larger or smaller than an observed point in time, then the data is described by the well-understood singly censored current status model, also known as interval censored data, case I. Jewell et al. (1994) extended this current status model by allowing the initial time to be unobserved, with its distribution over an observed interval [A, B] known; the data is referred to as doubly censored current status data. This model has applications in AIDS partner studies. If the initial time is known to be uniformly distribute d, the model reduces to a submodel of the current status model with the same asymptotic information bounds as in the current status model, but the distribution of interest is essentially the derivative of the distribution of interest in the current status model. As a consequence the non-parametric maximum likelihood estimator is inconsistent. Moreover, this submodel contains only smooth heavy tailed distributions for which no moments exist. In this paper, we discuss the connection between the singly censored current status model and the doubly censored current status model (for the uniform initial time) in detail and explain the difficulties in estimation which arise in the doubly censored case. We propose a regularized MLE corresponding with the current status model. We prove rate results, efficiency of smooth functionals of the regularized MLE, and present a generally applicable efficient method for estimation of regression parameters, which does not rely on the existence of moments. We also discuss extending these ideas to a non-uniform distribution for the initial time.     

Estimation of tail parameters under type i censoring

This paper is a continuation of previous work concerning the estimation of tail-parameters under Type II censoring (Weissman 1978). The same estimation problem is considered here, this truip under Type I censoring. A sample of size n is censored below aE a given level x₀it is assumed that che underlying distriibution .function (df)belogs to the domain of attraction of a known extreme-value distribution and that K - K(x_o) , the number of observed values, remains finite as on - ∞ . We offer here estimators, which are asymptotically maximum likelihood estimators (MLE's), for quantiles associated with the tail of F such as location and scale parameters, quantiles and F(x) itself (for x in the tail). The results are applied to two illustrative examples.     

Change-Point Detection in Two-Phase Regression with Inequality Constraints on the Regression Parameters

Two-phase regression models with inequality constraints on the regression coefficients and with a small number of measurements is considered. A new test based on the likelihood ratio in linear model with inequality constraints for the presence of a change-point is proposed. Numerical approximations to the powers against various alternatives are given and compared with the powers of the likelihood ratio test in the two-phase regression models without inequality constraints, the backwards CUSUM test, and the k-linear-r-ahead recursive residuals tests. Performance of related likelihood based estimators of the change-point is briefly studied in a Monte Carlo experiment.     

Prediction Error Estimation Under Bregman Divergence for Non-Parametric Regression and Classification

Abstract.  Prediction error is critical to assess model fit and evaluate model prediction. We propose the cross-validation (CV) and approximated CV methods for estimating prediction error under the Bregman divergence (BD), which embeds nearly all of the commonly used loss functions in the regression, classification procedures and machine learning literature. The approximated CV formulas are analytically derived, which facilitate fast estimation of prediction error under BD. We then study a data-driven optimal bandwidth selector for local-likelihood estimation that minimizes the overall prediction error or equivalently the covariance penalty. It is shown that the covariance penalty and CV methods converge to the same mean-prediction-error-criterion. We also propose a lower-bound scheme for computing the local logistic regression estimates and demonstrate that the algorithm monotonically enhances the target local likelihood and converges. The idea and methods are extended to the generalized varying-coefficient models and additive models.     

Estimating the Conditional Error Distribution in Non‐parametric Regression

Abstract. We consider a general non‐parametric regression model, where the distribution of the error, given the covariate, is modelled by a conditional distribution function. For the estimation, a kernel approach as well as the (kernel based) empirical likelihood method are discussed. The latter method allows for incorporation of additional information on the error distribution into the estimation. We show weak convergence of the corresponding empirical processes to Gaussian processes and compare both approaches in asymptotic theory and by means of a simulation study.     

Semiparametric Efficient Estimation in the Generalized Odds-Rate Class of Regression Models for Right-Censored Time-to-Event Data

The generalized odds-rate class of regression models for time to event data is indexed by a non-negative constant and assumes thatg(S(t|Z)) = (t) + Zwhere g(s) = log(^-1(s-) for > 0, g₀(s) = log(- log s), S(t|Z) is the survival function of the time to event for an individual with qx1 covariate vector Z, is a qx1 vector of unknown regression parameters, and (t) is some arbitrary increasing function of t. When =0, this model is equivalent to the proportional hazards model and when =1, this model reduces to the proportional odds model. In the presence of right censoring, we construct estimators for and exp((t)) and show that they are consistent and asymptotically normal. In addition, we show that the estimator for is semiparametric efficient in the sense that it attains the semiparametric variance bound.     

Empirical Likelihood Ratio Test for a Change-Point in Linear Regression Model

A nonparametric method based on the empirical likelihood is proposed to detect the change-point in the coefficient of linear regression models. The empirical likelihood ratio test statistic is proved to have the same asymptotic null distribution as that with classical parametric likelihood. Under some mild conditions, the maximum empirical likelihood change-point estimator is also shown to be consistent. The simulation results show the sensitivity and robustness of the proposed approach. The method is applied to some real datasets to illustrate the effectiveness.     

Multiple Linear Regression Model Under Nonnormality

Abstract

We consider multiple linear regression models under nonnormality. We derive modified maximum likelihood estimators (MMLEs) of the parameters and show that they are efficient and robust. We show that the least squares esimators are considerably less efficient. We compare the efficiencies of the MMLEs and the M estimators for symmetric distributions and show that, for plausible alternatives to an assumed distribution, the former are more efficient. We provide real-life examples.     

Simulation-based Inference in a Zero-inflated Bernoulli Regression Model

The logistic regression model has become a standard tool to investigate the relationship between a binary outcome and a set of potential predictors. When analyzing binary data, it often arises that the observed proportion of zeros is greater than expected under the postulated logistic model. Zero-inflated binomial (ZIB) models have been developed to fit binary data that contain too many zeros. Maximum likelihood estimators in these models have been proposed and their asymptotic properties established. Several aspects of ZIB models still deserve attention however, such as the estimation of odds-ratios and event probabilities. In this article, we propose estimators of these quantities and we investigate their properties both theoretically and via simulations. Based on these results, we provide recommendations about the range of conditions (minimum sample size, maximum proportion of zeros in excess) under which a reliable statistical inference on the odds-ratios and event probabilities can be obtained in a ZIB regression model. A real-data example illustrates the proposed estimators.     

Bayesian Analysis in Regression Models Using Pseudo-Likelihoods

This article deals with the issue of using a suitable pseudo-likelihood, instead of an integrated likelihood, when performing Bayesian inference about a scalar parameter of interest in the presence of nuisance parameters. The proposed approach has the advantages of avoiding the elicitation on the nuisance parameters and the computation of multidimensional integrals. Moreover, it is particularly useful when it is difficult, or even impractical, to write the full likelihood function.

We focus on Bayesian inference about a scalar regression coefficient in various regression models. First, in the context of non-normal regression-scale models, we give a theroetical result showing that there is no loss of information about the parameter of interest when using a posterior distribution derived from a pseudo-likelihood instead of the correct posterior distribution. Second, we present non trivial applications with high-dimensional, or even infinite-dimensional, nuisance parameters in the context of nonlinear normal heteroscedastic regression models, and of models for binary outcomes and count data, accounting also for possibile overdispersion. In all these situtations, we show that non Bayesian methods for eliminating nuisance parameters can be usefully incorporated into a one-parameter Bayesian analysis.     

Iterated Partial Sum Sequences of Regression Residuals and Tests for Changepoints with Continuity Constraints

Iterated partial sum sequences of regression least squares residuals are defined and large sample properties of sequences of stochastic processes defined by these iterated partial sums are discussed. Also, finite sample properties of the iterated partial sum sequences are obtained. These include a property of least squares residuals of polynomial fits to equispaced data, namely the iterated partial sums sum to 0 provided that the order of iteration is not greater than the order of the polynomial, thus extending the well-known result that residuals sum to 0. Iterated partial sums are shown to play an important role in testing regression parameters for changes at unknown times under the constraint of continuity.     

Estimation in a truncated bivariate poisson distribution

Moment estimators for parameters in a truncated bivariate Poisson distribution are derived in Hamdan (1972) for the special case of λ₁ = λ₂, Where λ₁, λ₂ are the marginal means. Here we derive the maximum likelihood estimators for this special case. The information matrix is also obtained which provides asymptotic covariance matrix of the maximum likelihood estimators. The asymptotic covariance matrix of moment estimators is also derived. The asymptotic efficiency of moment estimators is computed and found to be very low.     

Estimation of the multi-criteria worths of the alternatives in a hierarchical structure of comparisons

The aggregated worths of the alternatives, when compared with respect to several criteria, are estimated in a hierarchical comparisons model introduced by Saaty (1980). A multiplicative model is used for the paired comparisons data which are collected in a ratio scale in this set-up in any level of this hierarchy. An iterative scheme is found for the maximum likelihood estimation of the worth parameters in this multiplicative model. The iterative values are shown to be convergent monotonically to the estimates. We also obtain the asymptotic dispersion matrix of the maximum likelihood estimates of the relative worths of the alternatives according to a single criterion as well as those according to the over-all suitability when compared under several criteria. A numerical example is presented to illustrate the method developed in this paper. Simulation techniques are employed to find the average number of iterations required for the convergence of the above iterative scheme.     

Estimation of the parameters of a truncated gamma distribution

This paper deals with the estimation of the parameters of a truncated gamma distribution over (0,τ), where τ is assumed to be a real number. We obtain a necessary and sufficient condition for the existence of the maximum likelihood estimator(MLE). The probability of nonexistence of MLE is observed to be positive. A simulation study indicates that the modified maximum likelihood estimator and the mixed estimator, which exist with probability one,are to be preferred over MLE. The bias, the mean square error, and the probability of nearness form a basis of our simulation study.     

Regression analysis with two mixed forms of heteroscedasticity

Regression models are here considered in which disturbances are related to both the expectation of the dependent variable and a linear conbination of certain auxiliary variables. The maximum likelihood and weighted least squares estimators are compared in estimating the form of heteroscedasticity and regression coefficients. Also a test for heteroscedasticity is discussed. Finally an example is worked out for the purpose of illustration.