Use of weighted least squares regression jn simulation studies

When a simulation or Monte Carlo analysis uses the same set of N samples as input for the comparison of the power of tests, the resulting estimates of power are highly correlated. As a result, the statistical analysis of these results should use weighted least squares or other equivalent procedures. A reanalysis of one simulation study (Thode et al., 1983) found that the weighted least squares estimates had much smaller standard errors than the ordinary least squares estimates. The reduction in the standard errors of the parameters was a factor between 4 and 9 for the tests found to be more powerful. The necessary calculations are described.