Treatments of non-metric variables in partial least squares and principal component analysis

This paper reviews various treatments of non-metric variables in partial least squares (PLS) and principal component analysis (PCA) algorithms. The performance of different treatments is compared in an extensive simulation study under several typical data generating processes and associated recommendations are made. Moreover, we find that PLS-based methods are to prefer in practice, since, independent of the data generating process, PLS performs either as good as PCA or significantly outperforms it. As an application of PLS and PCA algorithms with non-metric variables we consider construction of a wealth index to predict household expenditures. Consistent with our simulation study, we find that a PLS-based wealth index with dummy coding outperforms PCA-based ones.     

Using the Hybrid Two-Step estimation approach for the identification of second-order latent variable models

The aim of this paper is to define a new approach, called Hybrid Two-Step, to estimate the parameters of a second-order latent variable (LV) model in the case of formative relationships between the first-order and the second-order LVs. In this respect, we introduce the two main approaches to the estimation of second-order constructs through the partial least squares-path modelling: the so-called Repeated Indicators approach and the Two-Step approach. Some criticisms of these methodologies are highlighted and a solution to the issue of the identification of formative second-order constructs is suggested through the adoption of a Hybrid Two-Step approach. A Monte Carlo simulation study aimed at comparing the approach proposed with the traditional ones was performed. Finally, a case study about the passenger satisfaction is presented to show the implementation of the method and to give some comparative empirical results.     

The peculiar shrinkage properties of partial least squares regression

Partial least squares regression has been widely adopted within some areas as a useful alternative to ordinary least squares regression in the manner of other shrinkage methods such as principal components regression and ridge regression. In this paper we examine the nature of this shrinkage and demonstrate that partial least squares regression exhibits some undesirable properties.     

On distribution-weighted partial least squares with diverging number of highly correlated predictors

Summary.  Because highly correlated data arise from many scientific fields, we investigate parameter estimation in a semiparametric regression model with diverging number of predictors that are highly correlated. For this, we first develop a distribution-weighted least squares estimator that can recover directions in the central subspace, then use the distribution-weighted least squares estimator as a seed vector and project it onto a Krylov space by partial least squares to avoid computing the inverse of the covariance of predictors. Thus, distrbution-weighted partial least squares can handle the cases with high dimensional and highly correlated predictors. Furthermore, we also suggest an iterative algorithm for obtaining a better initial value before implementing partial least squares. For theoretical investigation, we obtain strong consistency and asymptotic normality when the dimension p of predictors is of convergence rate O { n ^1/2/ log ( n )} and o ( n ^1/3) respectively where n is the sample size. When there are no other constraints on the covariance of predictors, the rates n ^1/2 and n ^1/3 are optimal. We also propose a Bayesian information criterion type of criterion to estimate the dimension of the Krylov space in the partial least squares procedure. Illustrative examples with a real data set and comprehensive simulations demonstrate that the method is robust to non-ellipticity and works well even in 'small n –large p ' problems.     

Statistical inference in nonstationary markov models with embedded explanatory variables

In this paper, we develop an operational nonstationary Markov process model for use with macro aggregate frequency data. Independent, time-variant factors assumed to affect the process of interest are embedded in the model. Transition probabilities are estimated indirectly from the coefficients on the embedded variables. We previously concluded that either the Marquardt or the simplex, derivative-free nonlinear programming algorithm could be used to estimate such a model. Here, we propose a test for parameter stationarity. By means of designed simulation experiments for the two-state model, we find that our test has acceptable Type I error probabilities, and that power rises with the degree of departure from the null hypothesis. Both validity and power performance can be improved by longer time records of data and a greater number of entities observed.     

Estimation of the linear-linear segmented regression model in the presence of measurement error

A simple segmented regression model in which the independent variable is measured with error is considered. The method of moments is used to obtain parameter estimates and the joint asymptotic distribution of the estimators is presented. The small sample properties of the inference procedures based on the asymptotic distribution of the estimators are studied numerically.     

Estimation of nonseparable models with censored dependent variables and endogenous regressors

In this article we develop a nonparametric estimator for the local average response of a censored dependent variable to endogenous regressors in a nonseparable model where the unobservable error term is not restricted to be scalar and where the nonseparable function need not be monotone in the unobservables. We formalize the identification argument put forward in Altonji, Ichimura, and Otsu (2012 Altonji, J. G., Ichimura, H., Otsu, T. (2012). Estimating derivatives in nonseparable models with limited dependent variables. Econometrica 80:1701–1719.[Crossref], [Web of Science ®] , [Google Scholar]), construct a nonparametric estimator, characterize its asymptotic property, and conduct a Monte Carlo investigation to study its small sample properties. Identification is constructive and is achieved through a control function approach. We show that the estimator is consistent and asymptotically normally distributed. The Monte Carlo results are encouraging.     

The asymptotic behaviour of the residual sum of squares in models with multiple break points

ABSTRACT

Models with multiple discrete breaks in parameters are usually estimated via least squares. This paper, first, derives the asymptotic expectation of the residual sum of squares and shows that the number of estimated break points and the number of regression parameters affect the expectation differently. Second, we propose a statistic for testing the joint hypothesis that the breaks occur at specified points in the sample. Our analytical results cover models estimated by the ordinary, nonlinear, and two-stage least squares. An application to U.S. monetary policy rejects the assumption that breaks are associated with changes in the chair of the Fed.     

Strong consistency of least squares estimates with i.i.d. errors with mean values not necessarily defined

We establish strong consistency of the least squares estimates in multiple regression models discarding the usual assumption of the errors having null mean value. Thus, we required them to be i.i.d. with absolute moment of order r, 0<r<2, and null mean value when r>1. Only moderately restrictive conditions are imposed on the model matrix. In our treatment, we use an extension of the Marcinkiewicz–Zygmund strong law to overcome the errors mean value not being defined. In this way, we get a unified treatment for the case of i.i.d. errors extending the results of some previous papers.     

Analysis of structural equation models with Incomplete Polytomous Data

A two-stage estimation procedure is developed to analyze structural equation models of polytomous variables based on incomplete data. At the first stage, the partition maximum likelihood approach is used to obtain the estimates of the elements in the correlation matrix. It will be shown that the asymptotic distribution of these estimates is jointly multivariate normal. The second stage estimates the structural parameters in the correlation matrix by the generalized least squared approach with a correctly specified weight matrix. Asymptotic properties of the second stage estimates are also provided. Extension of the theory to multisample models, and some illustrative examples are also included.     

The multiplier method in constrained estimation of covariance structure models

Based on the multiplier method of constrained minimization, an algorithm is developed to handle the constrained estimation problem in covariance structure analysis. In the context of a general model which has wide applicability in multivariate medical and behavioural researches, computer programs are implemented to produce the weighted least squares estimates and the maximum likelihood estimates. The multiplier method is compared with the penalty function method in terms of computer time, number of iterations and number of unconstrained minimizations. The indication is that the multiplier method is substantially better.     

Extensions of multiple linear regression

Many statistical methods are linked together through their connection with weighted least squares and hence regression. This article reviews these connections, emphasising the iteratively weighted least squares algorithm.