Nonlinear Autoregression with Positive Innovations

Use of nonlinear models in analyzing time series data is becoming increasingly popular. This paper considers a broad class of nonlinear autoregressive models where the autoregressive part is additive and the terms are nonlinear functions of the past data. Also, the innovation distribution is supported on the non-negative reals and satisfies a tail regularity condition. The linear parameters of the autoregression are estimated using a linear programming recipe which yields much more accurate estimates than traditional methods such as conditional least squares. Limiting distribution of the linear programming estimators is obtained. Simulation studies validate the asymptotic results and reveal excellent small sample properties of the LPE estimator.     

Statistical inference on partial linear additive models with distortion measurement errors

We consider statistical inference for partial linear additive models (PLAMs) when the linear covariates are measured with errors and distorted by unknown functions of commonly observable confounding variables. A semiparametric profile least squares estimation procedure is proposed to estimate unknown parameter under unrestricted and restricted conditions. Asymptotic properties for the estimators are established. To test a hypothesis on the parametric components, a test statistic based on the difference between the residual sums of squares under the null and alternative hypotheses is proposed, and we further show that its limiting distribution is a weighted sum of independent standard chi-squared distributions. A bootstrap procedure is further proposed to calculate critical values. Simulation studies are conducted to demonstrate the performance of the proposed procedure and a real example is analyzed for an illustration.     

Linear or Nonlinear? Automatic Structure Discovery for Partially Linear Models

Partially linear models provide a useful class of tools for modeling complex data by naturally incorporating a combination of linear and nonlinear effects within one framework. One key question in partially linear models is the choice of model structure, that is, how to decide which covariates are linear and which are nonlinear. This is a fundamental, yet largely unsolved problem for partially linear models. In practice, one often assumes that the model structure is given or known and then makes estimation and inference based on that structure. Alternatively, there are two methods in common use for tackling the problem: hypotheses testing and visual screening based on the marginal fits. Both methods are quite useful in practice but have their drawbacks. First, it is difficult to construct a powerful procedure for testing multiple hypotheses of linear against nonlinear fits. Second, the screening procedure based on the scatterplots of individual covariate fits may provide an educated guess on the regression function form, but the procedure is ad hoc and lacks theoretical justifications. In this article, we propose a new approach to structure selection for partially linear models, called the LAND (Linear And Nonlinear Discoverer). The procedure is developed in an elegant mathematical framework and possesses desired theoretical and computational properties. Under certain regularity conditions, we show that the LAND estimator is able to identify the underlying true model structure correctly and at the same time estimate the multivariate regression function consistently. The convergence rate of the new estimator is established as well. We further propose an iterative algorithm to implement the procedure and illustrate its performance by simulated and real examples. Supplementary materials for this article are available online.     

Weighted Wilcoxon Estimators in Nonlinear Regression

In this paper, we consider the estimation of parameters of a general near regression model. An estimator that minimises the weighted Wilcoxon dispersion function is considered and its asymptotic properties established under mild regularity conditions similar to those used in least squares and least absolute deviations estimation. As in linear models, the procedure provides estimators that are robust and highly efficient. The estimates depend on the choice of a weight function and diagnostics which differentiate between nonlinear fits are provided along with appropriate benchmarks. The behavior of these estimates is discussed on a real data set. A simulation study verifies the robustness, efficiency and validity of these estimates over several error distributions including the normal and a family of contaminated normal distributions.     

Interpretation of the four types of analysis of variance tables in sas

Various computational methods exist for generating sums of squares in an analysis of variance table. When the ANOVA design is balanced, most of these computational methods will produce equivalent sums of squares for testing the significance of the ANOVA model parameters. However, when the design is unbalanced, as is frequently the case in practice, these sums of squares depend on the computational method used.- The basic reason for the difference in these sums of squares is that different hypotheses are being tested. The purpose of this paper is to describe these hypotheses in terms of population or cell means. A numerical example is given for the two factor model with interaction. The hypotheses that are tested by the four computational methods of the SAS general linear model procedure are specified.

Although the ultimate choice of hypotheses should be made by the researcher before conducting the experiment, this paper

PENDLETON,VON TRESS,AND BREMER

presents the following guidelines in selecting these hypotheses:

When the design is balanced, all of the SAS procedures will agree.

In unbalanced ANOVA designs when there are no missing cells. SAS Type III should be used. SAS Type III tests an unweighted hypothesis about cell means. SAS Types I and II test hypotheses that are functions of the ceil frequencies. These frequencies are often merely arti¬facts of the experimental process and not reflective of any underlying frequencies in the population.

When there are missing cells, i.e. no observations for some factor level combinations. Type IV should be used with caution. SAS Type IV tests hypotheses which depend     

Partially adaptive estimation of nonlinear models via a normal mixture

This paper extends the partially adaptive method Phillips (1994) provided for linear models to nonlinear models. Asymptotic results are established under conditions general enough they cover both cross-sectional and time series applications. The sampling efficiency of the new estimator is illustrated in a small Monte Carlo study in which the parameters of an autoregressive moving average are estimated. The study indicates that, for non-normal distributions, the new estimator improves on the nonlinear least squares estimator in terms of efficiency.     

Partially adaptive estimation of nonlinear models via a normal mixture

This paper extends the partially adaptive method Phillips (1994) provided for linear models to nonlinear models. Asymptotic results are established under conditions general enough they cover both cross-sectional and time series applications. The sampling efficiency of the new estimator is illustrated in a small Monte Carlo study in which the parameters of an autoregressive moving average are estimated. The study indicates that, for non-normal distributions, the new estimator improves on the nonlinear least squares estimator in terms of efficiency.     

Least absolute deviations estimation via the EM algorithm

This paper derives EM and generalized EM (GEM) algorithms for calculating least absolute deviations (LAD) estimates of the parameters of linear and nonlinear regression models. It shows that Schlossmacher's iterative reweighted least squares algorithm for calculating LAD estimates (E.J. Schlossmacher, Journal of the American Statistical Association 68: 857–859, 1973) is an EM algorithm. A GEM algorithm for computing LAD estimates of the parameters of nonlinear regression models is also provided and is applied in some examples.     

On the role of minque in testing of hypotheses under hiked linear models

Testing of hypotheses under balanced ANOVA models is fairly simple and generally based on the usual ANOVA sums of squares. Difficulties may arise in special cases when these sums of squares do not form a complete sufficient statistic. There is a huge literature on this subject which was recently surveyed in Seifert's contribution to the book of Mumak (1904). But there are only a few results about unbalanced models. In such models the consideration of likelihood ratios leads to more complex sums of squares known from MINQUE theory.

Uniform optimality of testsusually reduces to local optimality. Here we prespnt a small review of methods proposed for testing of hypotheses in unbalanced models. where MINQUEI playb a major role. We discuss the use of iterated MINQUE for the construction of asymptotically optimal tests described in Humak (1984) and approximate tests based on locally uncorrelated linear combinations of MINQUE estimators by Seifert (1985), We show that the latter tests coincide with robust locally optimal invariant tests proposeci by Kariya and Sinha and Das and Sinha, if the number of variance components is two. Explicit expressions for corresponding tests are given for the unbalanced two-way cross classification random model, which covers some other models as special cases. A simulation study under lines the relevance of MINQUE for testing of hypotheses problems.     

Semiparametric Sieve-Type Generalized Least Squares Inference

This article considers the problem of statistical inference in linear regression models with dependent errors. A sieve-type generalized least squares (GLS) procedure is proposed based on an autoregressive approximation to the generating mechanism of the errors. The asymptotic properties of the sieve-type GLS estimator are established under general conditions, including mixingale-type conditions as well as conditions which allow for long-range dependence in the stochastic regressors and/or the errors. A Monte Carlo study examines the finite-sample properties of the method for testing regression hypotheses.     

Autoregressive Model with Partial Forgetting within Rao-Blackwellized Particle Filter

The authors are concerned with Bayesian identification and prediction of a nonlinear discrete stochastic process. The fact that a nonlinear process can be approximated by a piecewise linear function advocates the use of adaptive linear models. They propose a linear regression model within Rao-Blackwellized particle filter. The parameters of the linear model are adaptively estimated using a finite mixture, where the weights of components are tuned with a particle filter. The mixture reflects a priori given hypotheses on different scenarios of (expected) parameters' evolution. The resulting hybrid filter locally optimizes the weights to achieve the best fit of a nonlinear signal with a single linear model.     

Applying skovgaard's modified directed likelihood statistic to mixed linear models

The introduction of software to calculate maximum likelihood estimates for mixed linear models has made likelihood estimation a practical alternative to methods based on sums of squares. Likelihood based tests and confidence intervals, however, may be misleading in problems with small sample sizes. This paper discusses an adjusted version of the directed log-likelihood statistic for mixed models that is highly accurate for testing one parameter hypotheses. Indroduced by Skovgaard (1996, Journal of the Bernoulli Society,2,145-165), we show in mixed models that the statistic has a simple conpact from that may be obtained from standard software. Simulation studies indicate that this statistic is more accurate than many of the specialized procedure that have been advocated.     

Variable Selection for Partially Linear Models with Randomly Censored Data

This article proposes a variable selection procedure for partially linear models with right-censored data via penalized least squares. We apply the SCAD penalty to select significant variables and estimate unknown parameters simultaneously. The sampling properties for the proposed procedure are investigated. The rate of convergence and the asymptotic normality of the proposed estimators are established. Furthermore, the SCAD-penalized estimators of the nonzero coefficients are shown to have the asymptotic oracle property. In addition, an iterative algorithm is proposed to find the solution of the penalized least squares. Simulation studies are conducted to examine the finite sample performance of the proposed method.     

Estimable functions in the nonlinear model

When the method of least squares is used to estimate the parameters in a general model and the generated system of normal equations is linearly dependent, the estimate of the vector of parameters which satisfies the criterion is not unique. However, there exist certain functions of the estimated vector of parameters which are invariant to the least squares solution obtained from the normal equations. We define those invariant functions to be estimable, and present a technique to determine the functions of the parameters which are estimable for the general model. The method results in solving either a linear first order partial differential equation or a system of linear first order partial differential equations corresponding, respectively, to a single or multiple dependency between columns of the Jacobian matrix of the mean of the model. The usual results concerning estimability for linear models are a special case of the general results developed.     

A SIMPLE VARIABLE SELECTION TECHNIQUE FOR NONLINEAR MODELS

Applying nonparametric variable selection criteria in nonlinear regression models generally requires a substantial computational effort if the data set is large. In this paper we present a selection technique that is computationally much less demanding and performs well in comparison with methods currently available. It is based on a polynomial approximation of the nonlinear model. Performing the selection only requires repeated least squares estimation of models that are linear in parameters. The main limitation of the method is that the number of variables among which to select cannot be very large if the sample is small and the order of an adequate polynomial at the same time is high. Large samples can be handled without problems.     

Estimating consumer acceptance limits

A methodology is developed for estimating consumer acceptance limits on a sensory attribute of a manufactured product. In concept these limits are analogous to engineering tolerances. The method is based on a generalization of Stevens' Power Law. This generalized law is expressed as a nonlinear statistical model. Instead of restricting the analysis to this particular case, a strategy is discussed for evaluating nonlinear models in general since scientific models are frequently of nonlinear form. The strategy focuses on understanding the geometrical contrasts between linear and nonlinear model estimation and assessing the bias in estimation and the departures from a Gaussian sampling distribution. Computer simulation is employed to examine the behavior of nonlinear least squares estimation. In addition to the usual Gaussian assumption, a bootstrap sample reuse procedure and a general triangular distribution are introduced for evaluating the effects of a non-Gaussian or asymmetrical error structure. Recommendations are given for further model analysis based on the simulation results. In the case of a model for which estimation bias is not a serious issue, estimating functions of the model are considered. Application of these functions to the generalization of Stevens’ Power Law leads to a means for defining and estimating consumer acceptance limits, The statistical form of the law and the model evaluation strategy are applied to consumer research data. Estimation of consumer acceptance limits is illustrated and discussed.     

Total least squares solution for compositional data using linear models

The restrictive properties of compositional data, that is multivariate data with positive parts that carry only relative information in their components, call for special care to be taken while performing standard statistical methods, for example, regression analysis. Among the special methods suitable for handling this problem is the total least squares procedure (TLS, orthogonal regression, regression with errors in variables, calibration problem), performed after an appropriate log-ratio transformation. The difficulty or even impossibility of deeper statistical analysis (confidence regions, hypotheses testing) using the standard TLS techniques can be overcome by calibration solution based on linear regression. This approach can be combined with standard statistical inference, for example, confidence and prediction regions and bounds, hypotheses testing, etc., suitable for interpretation of results. Here, we deal with the simplest TLS problem where we assume a linear relationship between two errorless measurements of the same object (substance, quantity). We propose an iterative algorithm for estimating the calibration line and also give confidence ellipses for the location of unknown errorless results of measurement. Moreover, illustrative examples from the fields of geology, geochemistry and medicine are included. It is shown that the iterative algorithm converges to the same values as those obtained using the standard TLS techniques. Fitted lines and confidence regions are presented for both original and transformed compositional data. The paper contains basic principles of linear models and addresses many related problems.     

A Simple Approach to Inference in Random Coefficient Models

Random coefficient regression models have been used to analyze cross-sectional and longitudinal data in economics and growth-curve data from biological and agricultural experiments. In the literature several estimators, including the ordinary least squares and the estimated generalized least squares (EGLS), have been considered for estimating the parameters of the mean model. Based on the asymptotic properties of the EGLS estimators, test statistics have been proposed for testing linear hypotheses involving the parameters of the mean model. An alternative estimator, the simple mean of the individual regression coefficients, provides estimation and hypothesis-testing procedures that are simple to compute and teach. The large sample properties of this simple estimator are shown to be similar to that of the EGLS estimator. The performance of the proposed estimator is compared with that of the existing estimators by Monte Carlo simulation.     

Testing hypotheses in truncated samples by means of divergence statistics

In this paper we consider the problem of testing hypotheses in parametric models, when only the first r (of n) ordered observations are known.Using divergence measures, a procedure to test statistical hypotheses is proposed, Replacing the parameters by suitable estimators in the expresion of the divergence measure, the test statistics are obtained.Asymptotic distributions for these statistics are given in several cases when maximum likelihood estimators for truncated samples are considered.Applications of these results in testing statistical hypotheses, on the basis of truncated data, are presented.The small sample behavior of the proposed test statistics is analyzed in particular cases.A comparative study of power values is carried out by computer simulation.     

Ridge estimation in generalized linear models and proportional hazards regressions

Conventionally, a ridge parameter is estimated as a function of regression parameters based on ordinary least squares. In this article, we proposed an iterative procedure instead of the one-step or conventional ridge method. Additionally, we construct an indicator that measures the potential degree of improvement in mean squared error when ridge estimates are employed. Simulations show that our methods are appropriate for a wide class of non linear models including generalized linear models and proportional hazards (PHs) regressions. The method is applied to a PH regression with highly collinear covariates in a cancer recurrence study.