Reliability and Measurement Error in the Presence of Homogeneity

ABSTRACT

This paper describes a limitation of using Cronbach's Alpha to estimate reliability when using a sample with homogeneous responses in the measured construct. More specifically, it describes the risk of falsely concluding that a new instrument may have poor reliability and demonstrates the use of an alternate statistic that may serve as a cushion against such errors. Data from two validation studies are used to illustrate the utility of the new statistic, referred to as R-Alpha or Relative Alpha. Included is a discussion of the limitations and appropriate use of the statistic in validating multi-item tests, assessment scales, and inventories.