Sparse alternatives to ridge regression: a random effects approach

In a calibration of near-infrared (NIR) instrument, we regress some chemical compositions of interest as a function of their NIR spectra. In this process, we have two immediate challenges: first, the number of variables exceeds the number of observations and, second, the multicollinearity between variables are extremely high. To deal with the challenges, prediction models that produce sparse solutions have recently been proposed. The term ‘sparse’ means that some model parameters are zero estimated and the other parameters are estimated naturally away from zero. In effect, a variable selection is embedded in the model to potentially achieve a better prediction. Many studies have investigated sparse solutions for latent variable models, such as partial least squares and principal component regression, and for direct regression models such as ridge regression (RR). However, in the latter, it mainly involves an L₁ norm penalty to the objective function such as lasso regression. In this study, we investigate new sparse alternative models for RR within a random effects model framework, where we consider Cauchy and mixture-of-normals distributions on the random effects. The results indicate that the mixture-of-normals model produces a sparse solution with good prediction and better interpretation. We illustrate the methods using NIR spectra datasets from milk and corn specimens.     

Modelling association between two irregularly observed spatiotemporal processes by using maximum covariance analysis

Summary.  Climatic phenomena such as the El-Niño–southern oscillation and the north Atlantic oscillation are results of complex interactions between atmospheric and oceanic processes. Understanding the interactions has enabled scientists to give early warning of the forthcoming phenomena, thereby reducing damage caused by them. Statistical methods have played an important role in revealing effects of these phenomena on different regions of the world. One such method is maximum covariance analysis (MCA). Two apparent weaknesses are associated with MCA. Firstly, it tends to produce estimates with a low signal-to-noise ratio, especially when the sample size is small. Secondly, there has been no objective way of incorporating incomplete records, which are frequently encountered in climatology and oceanographic data-bases. We introduce an MCA which incorporates a smoothing procedure on the estimates. The introduction of the smoothing procedure is shown to improve the signal-to-noise ratio on the estimates. The estimation of smoothing parameters is carried out by using a penalized likelihood approach, which makes the inclusion of incomplete records quite straightforward. The methodology is applied to investigate the association between Irish winter precipitation and sea surface temperature anomalies around the world. The results show relationships between Irish precipitation anomalies and the El-Niño–southern oscillation and the north Atlantic oscillation phenomena.     

Resolving Zeckhauser’s paradox

Theory and Decision - Zeckhauser’s paradox has puzzled and entertained many rationality enthusiasts for almost half a century. You are forced to play a Russian Roulette with a 6-chamber...     

Quasi-likelihood Estimation of Non-invertible Moving Average Processes

Classical methods based on Gaussian likelihood or least-squares cannot identify non-invertible moving average processes, while recent non-Gaussian results are based on full likelihood consideration. Since the error distribution is rarely known a quasi-likelihood approach is desirable, but its consistency properties are yet unknown. In this paper we study the quasi-likelihood associated with the Laplacian model, a convenient non-Gaussian model that yields a modified L ₁ procedure. We show that consistency holds for all standard heavy tailed errors, but not for light tailed errors, showing that a quasi-likelihood procedure cannot be applied blindly to estimate non-invertible models. This is an interesting contrast to the standard results of the quasi-likelihood in regression models, where consistency usually holds much more generally. Similar results hold for estimation of non-causal non-invertible ARMA processes. Various simulation studies are presented to validate the theory and to show the effect of the error distribution, and an analysis of the US unemployment series is given as an illustration.     

Wallet Game: Probability,Likelihood, and Extended Likelihood

We propose a likelihood explanation to the two-person wallet game, a probability-related paradox, where an obviously fair game may appear favorable to both players. Yet a small variation of the game, without changing its fairness, turns it to seem unfavorable. The extended likelihood concept seems logically necessary if we want to allow the sense of uncertainty associated with a realized but still unobserved random outcome, while at the same time avoid potential probability-related paradoxes.     

Mixed inverse problems arising in the estimation of calibration factors in positron emission tomography

A penalized likelihood approach to the estimation of calibration factors in positron emission tomography (PET) is considered, in particular the problem of estimating the efficiency of PET detectors. Varying efficiencies among the detectors create a non-uniform performance and failure to account for the non-uniformities would lead to streaks in the image, so efficient estimation of the non-uniformities is desirable to reduce the propagation of noise to the final image. The relevant data set is provided by a blank scan, where a model may be derived that depends only on the sources affecting non-uniformities: inherent variation among the detector crystals and geometric effects. Physical considerations suggest a novel mixed inverse model with random crystal effects and smooth geometric effects. Using appropriate penalty terms, the penalized maximum likelihood estimates are derived and an efficient computational algorithm utilizing the fast Fourier transform is developed. Data-driven shrinkage and smoothing parameters are chosen to minimize an estimate of the predictive loss function. Various examples indicate that the approach proposed works well computationally and compares well with the standard method.