On O.L.S. estimation in singular linear models with general positive definite dispersion matrix

A condition in Graybill [1976] for the O.L.S.estimator to be B.L.U.E. in a linear model with positive definite dispersion not necessarily proportional to the identity matrix is extended to cover the case of a singular linear model.     

Mixed inverse problems arising in the estimation of calibration factors in positron emission tomography

A penalized likelihood approach to the estimation of calibration factors in positron emission tomography (PET) is considered, in particular the problem of estimating the efficiency of PET detectors. Varying efficiencies among the detectors create a non-uniform performance and failure to account for the non-uniformities would lead to streaks in the image, so efficient estimation of the non-uniformities is desirable to reduce the propagation of noise to the final image. The relevant data set is provided by a blank scan, where a model may be derived that depends only on the sources affecting non-uniformities: inherent variation among the detector crystals and geometric effects. Physical considerations suggest a novel mixed inverse model with random crystal effects and smooth geometric effects. Using appropriate penalty terms, the penalized maximum likelihood estimates are derived and an efficient computational algorithm utilizing the fast Fourier transform is developed. Data-driven shrinkage and smoothing parameters are chosen to minimize an estimate of the predictive loss function. Various examples indicate that the approach proposed works well computationally and compares well with the standard method.     

Least squates estimation of missing values in time series

The minimum mean square error linear interpolator for missing values in time series is extended to handle any pattern of nonconsecutive observations. The paper then develops evidence with simple ARMA models that the usefulness of either the"nonparametric"or the parametric form of the least squares interpolator depends on the time series model, the arrangement of the missing data and the objective for completing the series.     

Modelling association between two irregularly observed spatiotemporal processes by using maximum covariance analysis

Summary.  Climatic phenomena such as the El-Niño–southern oscillation and the north Atlantic oscillation are results of complex interactions between atmospheric and oceanic processes. Understanding the interactions has enabled scientists to give early warning of the forthcoming phenomena, thereby reducing damage caused by them. Statistical methods have played an important role in revealing effects of these phenomena on different regions of the world. One such method is maximum covariance analysis (MCA). Two apparent weaknesses are associated with MCA. Firstly, it tends to produce estimates with a low signal-to-noise ratio, especially when the sample size is small. Secondly, there has been no objective way of incorporating incomplete records, which are frequently encountered in climatology and oceanographic data-bases. We introduce an MCA which incorporates a smoothing procedure on the estimates. The introduction of the smoothing procedure is shown to improve the signal-to-noise ratio on the estimates. The estimation of smoothing parameters is carried out by using a penalized likelihood approach, which makes the inclusion of incomplete records quite straightforward. The methodology is applied to investigate the association between Irish winter precipitation and sea surface temperature anomalies around the world. The results show relationships between Irish precipitation anomalies and the El-Niño–southern oscillation and the north Atlantic oscillation phenomena.     

A small sample estimator for a polynomial regression with errors in the variables

An adjusted least squares estimator, introduced by Cheng and Schneeweiss for consistently estimating a polynomial regression of any degree with errors in the variables, is modified such that it shows good results in small samples without losing its asymptotic properties for large samples. Simulation studies corroborate the theoretical findings.     

Score estimation in the monotone single‐index model

We consider estimation in the single‐index model where the link function is monotone. For this model, a profile least‐squares estimator has been proposed to estimate the unknown link function and index. Although it is natural to propose this procedure, it is still unknown whether it produces index estimates that converge at the parametric rate. We show that this holds if we solve a score equation corresponding to this least‐squares problem. Using a Lagrangian formulation, we show how one can solve this score equation without any reparametrization. This makes it easy to solve the score equations in high dimensions. We also compare our method with the effective dimension reduction and the penalized least‐squares estimator methods, both available on CRAN as R packages, and compare with link‐free methods, where the covariates are elliptically symmetric.     

The asymptotic behaviour of the residual sum of squares in models with multiple break points

ABSTRACT

Models with multiple discrete breaks in parameters are usually estimated via least squares. This paper, first, derives the asymptotic expectation of the residual sum of squares and shows that the number of estimated break points and the number of regression parameters affect the expectation differently. Second, we propose a statistic for testing the joint hypothesis that the breaks occur at specified points in the sample. Our analytical results cover models estimated by the ordinary, nonlinear, and two-stage least squares. An application to U.S. monetary policy rejects the assumption that breaks are associated with changes in the chair of the Fed.     

The multiplier method in constrained estimation of covariance structure models

Based on the multiplier method of constrained minimization, an algorithm is developed to handle the constrained estimation problem in covariance structure analysis. In the context of a general model which has wide applicability in multivariate medical and behavioural researches, computer programs are implemented to produce the weighted least squares estimates and the maximum likelihood estimates. The multiplier method is compared with the penalty function method in terms of computer time, number of iterations and number of unconstrained minimizations. The indication is that the multiplier method is substantially better.     

Efficient estimation of the PDF and the CDF of the inverse Rayleigh distribution

In this paper, we consider the estimation of the probability density function and the cumulative distribution function of the inverse Rayleigh distribution. In this regard, the following estimators are considered: uniformly minimum variance unbiased estimator, maximum likelihood (ML) estimator, percentile estimator, least squares estimator and weighted least squares estimator. To do so, analytical expressions are derived for the mean integrated squared error. As the result of simulation studies and real data applications indicate, when the sample size is not very small the ML estimator performs better than the others.     

Estimation of the Magnitude of Victory in One-day Cricket RMIT University, Mayo Clinic Rochester and Simon Fraser University

The Duckworth–Lewis method is steadily becoming the standard approach for resetting targets in interrupted one-day cricket matches. In this paper we show that a modification of the Duckworth–Lewis resource table can be used to quantify the magnitude of a victory in one-day matches. This simple and direct application is particularly useful in breaking ties in tournament standings and in quantifying team strength.     

Exact expressions for the weights used in least-squares regression estimation for the log-logistic and Weibull distribution

Estimation for the log-logistic and Weibull distributions can be performed by using the equations used for probability plotting, and this technique outperforms the maximum likelihood (ML) estimation often in small samples. This leads to a highly heteroskedastic regression problem. Exact expressions for the variances of the residuals are derived which can be used to perform weighted regression. In large samples, the ML performs best, but it is shown that in smaller samples, the weighted regression outperforms the ML estimation with respect to bias and mean square error.     

An estimation in a linear model with response-related error

A maximum likelihood solution is presented for analyzing data which arise from a linear model whose error term is assumed to have variance proportional to some unknown power of the response. An efficient iterative method for solving the likelihood equations is obtained which incoporates use of a transfomation to orthogonalize the two variance paramaters. Assessments of the method are made through simulations study and the results are compared with those of the ordinary least squares. Examples from the literature are included to illustrate the method and also to compare the results with a weighted least squares estimates.     

Likelihood estimation of missing cell means in the fixed model analysis of variance

This paper examines the formation of maximum likelihood estimates of cell means in analysis of variance problems for cells with missing observations. Methods of estimating the means for missing cells has a long history which includes iterative maximum likelihood techniques, approximation techniques and ad hoc techniques. The use of the EM algorithm to form maximum likelihood estimates has resolved most of the issues associated with this problem. Implementation of the EM algorithm entails specification of a reduced model. As demonstrated in this paper, when there are several missing cells, it is possible to specify a reduced model that results in an unidentifiable likelihood. The EM algorithm in this case does not converge, although the slow divergence may often be mistaken by the unwary as convergence. This paper presents a simple matrix method of determining whether or not the reduced model results in an identifiable likelihood, and consequently in an EM algorithm that converges. We also show the EM algorithm in this case to be equivalent to a method which yields a closed form solution.     

Estimation in the simple linear regression model when there is heteroscedasticity of unknown form

A well-known problem is that ordinary least squares estimation of the parameters in the usual linear model can be highly ineficient when the error term has a heavy-tailed distribution. Inefficiency is also associated with situations where the error term is heteroscedastic, and standard confidence intervals can have probability coverage substantially different from the nominal level. This paper compares the small-sample efficiency of six methods that address this problem, three of which model the variance heterogeneity nonparametrically. Three methods were found to be relatively ineffective, but the other three perform relatively well. One of the six (M-regression with a Huber φ function and Schweppe weights) was found to have the highest efficiency for most of the situations considered in the simulations, but there might be situations where one of two other methods gives better results. One of these is a new method that uses a running interval smoother to estimate the optimal weights in weighted least squares, and the other is a method recently proposed by Cohen, Dalal, and Tukey. Computing a confidence interval for the slope using a bootstrap technique is also considered.     

Robust estimation in certain heteroscedastic linear models when there are many parameters

We study estimation of regression parameters in heteroscedastic linear models when the number of parameters is large. The results generalize work of Huber (1973), Yohai and Maronna (1979), and Carroll and Rupert (1982a).     

Alternative beta estimation for the market model using partially adaptive techniques

The leptokurtosls of many security market return distributions can contaminate ordinary least squares estimates of the β coefficient of the market model. Partially adaptive estimation techniques accommodate the possibility of fat tailed distributions. this methodology limits the influence of extremely large residuals and yields estimates which are both statistically and practically different from ordinary least squares.     

Application of 'delete = replace' to deletion diagnostics for variance component estimation in the linear mixed model

Summary.  'Delete = replace' is a powerful and intuitive modelling identity. This paper extends previous work by stating and proving the identity in more general terms and extending its application to deletion diagnostics for estimates of variance components obtained by restricted maximum likelihood estimation for the linear mixed model. We present a new, fast, transparent and approximate computational procedure, arising as a by-product of the fitting process. We illustrate the effect of the deletion of individual observations, of 'subjects' and of arbitrary subsets. Central to the identity and its application is the conditional residual.     

An iterative procedure for the estimation of parameters in a dose-response model

The least squares estimates of the parameters in the multistage dose-response model are unduly affected by outliers in a data set whereas the minimum sum of absolute errors, MSAE estimates are more resistant to outliers. Algorithms to compute the MSAE estimates can be tedious and computationally burdensome. We propose a linear approximation for the dose-response model that can be used to find the MSAE estimates by a simple and computationally less intensive algorithm. A few illustrative ex-amples and a Monte Carlo study show that we get comparable values of the MSAE estimates of the parameters in a dose-response model using the exact model and the linear approximation.     

The unconditional distributions of the OLS,TSLS and LIML estimators in a simple structural equations model

The exact distributions of the standard estimators of the structural coe?cients in a linear structural equations model conditional on the exogenous variables have been shown to have some unexpected and quirky features. Since the argument for conditioning on exogenous (ancillary) variables has been weakened over the past 20 years by the discovery of an “ancillarity paradox,” it is natural to wonder whether such finite sample properties are in fact due to conditioning on the exogenous variables. This article studies the exact distributions of the ordinary least squares (OLS), two-stage least squares (TSLS), and limited information maximum likelihood (LIML) estimators of the structural coe?cients in a linear structural equation without conditioning on the exogenous variables.     

On the use of cross-validation to assess performance in multivariate prediction

We describe a Monte Carlo investigation of a number of variants of cross-validation for the assessment of performance of predictive models, including different values of k in leave-k-out cross-validation, and implementation either in a one-deep or a two-deep fashion. We assume an underlying linear model that is being fitted using either ridge regression or partial least squares, and vary a number of design factors such as sample size n relative to number of variables p, and error variance. The investigation encompasses both the non-singular (i.e. n > p) and the singular (i.e. n p) cases. The latter is now common in areas such as chemometrics but has as yet received little rigorous investigation. Results of the experiments enable us to reach some definite conclusions and to make some practical recommendations.