Regression and correlation for 3 × 3 rotation matrices

This paper investigates a regression model for orthogonal matrices introduced by Prentice (1989). It focuses on the special case of 3 × 3 rotation matrices. The model under study expresses the dependent rotation matrix V as A₁UA^t₂ perturbed by experimental errors, where A₁ and A₂ are unknown 3 × 3 rotation matrices and U is an explanatory 3 × 3 rotation matrix. Several specifications for the errors in this regression model are proposed. The asymptotic distributions, as the sample size n becomes large or as the experimental errors become small, of the least squares estimators for A₁ and A₂ are derived. A new algorithm for calculating the least squares estimates of A₁ and A₂ is presented. The independence model is not a submodel of Prentice's regression model, thus the independence between the U and the V sample cannot be tested when fitting Prentice's model. To overcome this difficulty, permutation tests of independence are investigated. Examples dealing with postural variations of subjects performing a drilling task and with the calibration of a camera system for motion analysis using a magnetic tracking device illustrate the methodology of this paper.