Getting the story straight: evaluating the test-retest reliability of a university health history questionnaire.

This study was designed to establish the reliability of a health history questionnaire used as a screening tool for incoming university students. The authors used a test-retest design, with a test interval of 6 months, on a sample of medical and nursing students. The analysis focused on overall reliability of the questionnaire and reproducibility of specific items, based on question format. Questionnaire items of specific interest were those with dichotomous yes/no response options versus open-ended format questions, those using the words frequently or recently, or those that asked multiple questions. Demographic characteristics of the subjects were considered in the evaluation of reliability. Overall reliability of the questionnaire (93.6%) was above the anticipated level of 90%, and subject sex or program of study did not show any significant differences in reproducibility of responses. Although wording of questions did not affect item reliability, dichotomous format questions demonstrated a higher degree of reliability (96.4%) than the overall reliability of the questionnaire. Recommendations for enhancing the reliability of the questionnaire are based on item analysis and information gathered from interviews with subjects.     

Reliability of the instrument DOA: dialogue about ability related to work

The aim of this study was to examine the reliability of the new instrument "DOA--a dialogue about ability related to work" [6]. DOA was developed using the Model of Human of Occupation as a theoretical base [12,13]. The instrument is divided into two sections; client self-assessment and professional assessment focusing on the individual's ability to perform work-related activities. Assessments are followed by a dialogue to distinguish goals for the return-to-work process based on the client's own preferences [6,14]. 34 outpatients' clients and 14 professionals from four clinics within psychiatric work rehabilitation in Sweden participated in the study. The Spearman Rank Correlation test generally showed acceptable significant correlations in test-retest (n=34). A greater part of items correlated "moderate to good" (r=0.51-0.75). In the raters' test (n=14) Percentage of agreement (PA) was good ranging between 93.1%-96.5%, which is above the acceptable criteria of 80%. There were no significance differences between the raters. The study has indicated that items in DOA work with generally good reliability to measure ability related to work. The instrument was demonstrated to be a reliable therapeutic tool in goal setting for return-to-work in a position outside of the clients' preferences.     

Assessing the Validity and Reliability of the Perceptions of the Consent to Sex Scale

Although sexual assault prevention education tends to focus on consent promotion as a means to reduce rates of sexual assault, little is known about how college students consent to sexual activity. The current study aimed to better understand college students' consent via the systematic development of the Consent to Sex Scale (CSS), utilizing mixed methods via three phases and two waves of data collection. In Phase 1, qualitative data were collected from college students (n = 185) to provide a foundation for item writing. In Phase 2, closed-ended items were written for a quantitative instrument and reviewed by a team of experts. In Phase 3, a quantitative survey, including items written in Phase 2, was administered to college students (n = 685); the measure was assessed for its psychometric properties. Exploratory factor analysis was utilized, resulting in a five-factor solution. The CSS and corresponding factors demonstrated high internal consistency reliability and expected gender differences, supporting the construct validity of the measure. The CSS assesses college students' cues for indicating consent to sex, a construct not addressed by previous measures. The validated scale may be useful in future research to better understand how consent relates to other behaviors or constructs.     

Cross-Cultural Adaptation of the Female Genital Self-Image Scale (FGSIS) in Iranian Female College Students

The aim of the present study was to investigate the psychometric properties of a translated and culturally adapted Iranian version of the Female Genital Self-Image Scale (FGSIS-I) in a sample of college women. Further, the relationship between women's self-image, body appreciation, sexual functioning, and gynecological exam behavior was explored. A sample of 1,877 female students from five different universities across Qazvin and Tehran completed the Female Sexual Function Index (FSFI), the Body Appreciation Scale (BAS), the Rosenberg Self-Esteem Scale (RSES), the FGSIS-I, and a gynecological exam behavior questionnaire. Good to excellent internal consistency reliability, test-retest reliability, and convergent and construct validity were found. Exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) both provided a two-factor structure for the FGSIS-I. The validity of the FGSIS-I in predicting gynecological exam behavior of college women was tested using structural equation modeling (SEM). The final model accounted for 33% of the variance in gynecological exam behavior (p < 0.01). In conclusion, the FGSIS-I was found to be a highly valid and reliable instrument to assess female genital self-image in Iranian women.     

A model of substance abuse risk: adapting to the Sri Lankan context

This study translated and validated the Substance Use Risk Profile Scale (SURPS) among 13 to 18 year old Sri Lankan adolescents attending school. A standard systematic translation procedure was followed to translate the original SURPS into Sinhala language. A Delphi process was conducted to determine judgmental validity of Sinhala SURPS. Confirmatory factor analysis was performed to test the translated version against the original theoretical model of the SURPS. Test-retest and internal consistency were used to ensure reliability. A few terms in one of the items posed some difficulty in translating into Sinhala. Adequate judgmental validity was determined except for one item, which was modified to suit the Sri Lankan setting. The four subscales introversion/ hopelessness, impulsivity, sensation seeking and anxiety sensitivity showed satisfactory test-retest reliability estimates of .74, .68, .76, and .88, respectively. The Sinhala SURPS is applicable in Sri Lankan context. Several implications based on the results are discussed.     

The sexuality scale: An instrument to measure sexual‐esteem,sexual‐depression,and sexual‐preoccupation

This article describes the development of the Sexuality Scale, an instrument designed to measure three aspects of human sexuality: sexual‐esteem, defined as positive regard for and confidence in the capacity to experience one's sexuality in a satisfying and enjoyable way; sexual‐depression, defined as the experience of feelings of depression regarding one's sex life; and sexual‐preoccupation, defined as the tendency to think about sex to an excessive degree. The procedure involved (a) item construction, selection and subsequent validation through item analysis; and (b) a factor analysis of the items on the Sexuality Scale and the establishment of factorial validity. The results indicated that the three subscales were psychometrically sound, that males reported more sexual‐preoccupation than did females, and that the three subscales have unique intercorrelation patterns. The exploratory nature of these findings are discussed.     

The South Oaks Gambling Screen Revised for Adolescents (SOGS-RA): Further Psychometric Findings from a Community Sample

The broad expansion of gambling across North America during the last two decades has generated concern about the extent of gambling and problem gambling in youth, and the need to more accurately monitor it. The South Oaks Gambling Screen-Revised for Adolescents (SOGS-RA) is a promising instrument for screening problem gambling (Winters, Stinchfield, & Fulkerson, 1993) that requires more evaluation. Accordingly, further psychometric analysis of the instrument was conducted as part of a community survey of gambling in a sample of 1,000 male and female youth, aged 12 to 17 years. The analyses extended previous focus by including females, young adolescents, and an evaluation of youth classified as at-risk. Consistent with preliminary findings obtained during scale development, the distribution of item endorsement revealed trends of over-endorsement for some items (e.g., gambled more than intended, felt bad about the amount bet), and under-endorsement for others (e.g., criticized or told you had a gambling problem). These results suggest consideration of some form of weighting procedure, item deletion or re-wording. A factor analysis of the SOGS-RA items suggested a two-factor solution, with one factor interpreted as Control over Gambling and the other Gambling Consequences. It is proposed that the two factors may represent early versus more severe levels of gambling problems, respectively. The results highlight the need for further psychometric evaluation and refinement of instruments used to identify gambling problems in young people.     

Validation of a French Canadian version of the Organizational Policies and Practices (OPP) questionnaire

Organizational factors are potentially powerful in accounting for work-related chronic disability following a musculoskeletal disorder. This study documents the psychometric qualities of the French Canadian version of the Organizational Policies and Practices questionnaire (OPP) [1] on a population of nurses (N=124). By excluding the two items composing the ergonomic practices factor, a factorial structure identical to that obtained by the OPP's authors is obtained for the disability management policies and practices factor, the people-oriented culture factor and the safety climate factor. The internal consistency coefficients (Cronbach's alpha) are satisfactory while the coefficients intraclass are less than those obtained by the authors in the test-retest. However, the test-retest interval is greater in this study. Consistent relationships are observed between the dimensions of the OPP and three job-related psychosocial indicators: perceived stress, social support and satisfaction. This suggests a good construct validity for the OPP. Although additional validation efforts are recommended, all of the results obtained support the validity and reliability of the French Canadian version of the OPP. This version can be used to examine the importance of organizational aspects in studies on the prevention of chronic disability.     

Defining and measuring patient satisfaction with medical care

This paper describes the development of Form II of the Patient Satisfaction Questionnaire (PSQ), a self-administered survey instrument designed for use in general population studies. The PSQ contains 55 Likert-type items that measure attitudes toward the more salient characteristics of doctors and medical care services (technical and interpersonal skills of providers, waiting time for appointments, office waits, emergency care, costs of care, insurance coverage, availability of hospitals, and other resources) and satisfaction with care in general. Scales are balanced to control for acquiescent response set. Scoring rules for 18 multi-item subscales and eight global scales were standardized following replication of item analyses in four field tests. Internal-consistency and test-retest estimates indicate satisfactory reliability for studies involving group comparisons. The PSQ well represents the content of characteristics of providers and services described most often in the literature and in response to open-ended questions. Empirical tests of validity have also produced generally favorable results.     

Development of a short version of the Aging Males’ Symptoms scale: Mokken scaling analysis and Rasch analysis

The aim of this study was to develop a psychometrically sound short version of the 17-item Aging Males’ Symptoms (AMS) scale using Mokken scale analysis (MSA) and Rasch analysis. We recruited a convenient sample of 1787 men (age: mean (SD)?=?43.8 (11.5) years) who visited a men’s health polyclinic in Taiwan and completed the AMS scale. The scale was first assessed using MSA. The remaining items were assessed using Rasch analysis. We used a stepwise approach to remove items with χ² item statistics and mean square values while monitoring unidimensionality. The item reduction process resulted in a 6-item version of the AMS scale (AMS-6). The AMS-6 scale included a 5-item psychosomatic subscale (original items 1, 4, 5, 8, and 9) and a 1-item sexual subscale (original item 16). Analyses confirmed that the 5-item psychosomatic subscale was a Rasch scale. The AMS-6 correlated well with the AMS scales: the 5-item psychosomatic subscale correlated with the AMS scale (r between 0.50 and 0.92); the 1-item sexual subscale correlated with the sexual subscale of the AMS scale (r?=?0.81). A 6-item short form of the AMS scale had satisfactory measurement properties. This version may be useful for estimating psychosomatic and sexual symptoms as well as health-related quality of life with a minimal burden on respondents.     

The sexual abuse questionnaire: a preliminary examination of a time and cost efficient method in evaluating the presence of childhood sexual abuse in adult patients

This paper provides the results of two studies designed to evaluate a newly constructed self-report instrument, the Sexual Abuse Questionnaire (SAQ). The SAQ was designed as a brief screening device to aid in the identification of a childhood sexual abuse history. A "unique" feature of the SAQ is the inclusion of a number of non-face valid questions derived from clinical experience. Both studies used an undergraduate population of self-reported abused and non-abused participants. Based upon the combined results of the two studies, the final version of the SAQ was developed, which is comprised of 45 items that manifest good test-retest reliability, internal consistency, and convergent and discriminative validity. The SAQ can discriminate between abused and non-abused male and female participants.     

Assessing Family Outcomes: Psychometric Evaluation of the Beach Center Family Quality of Life Scale

There is currently a lack of reliable scales with which to assess the construct of family quality of life, particularly for families who have children with disabilities. The current work presents 2 studies, including a total of 488 families with children with disabilities, which were conducted to complete the development of a scale to assess family quality of life. The measure was refined through confirmatory factor analyses into 25 items that assess 5 domains of Family Quality of Life: Family Interaction, Parenting, Emotional Well‐Being, Physical/Material Well‐Being, and Disability‐Related Support. Each subscale was found to be unidimensional and internally consistent. An initial examination of test‐retest reliability and convergent validity is also presented. Implications for future research, scale use, and policy are discussed.     

20 Questions of Gamblers Anonymous: A Psychometric Study with Population of Spain

The authors present a psychometric study of the questionnaire proposed by the Gamblers Anonymous organization for the self-assessment of problem gambling. The study was carried out in Spain with two samples: one of 127 problem gamblers (mean age: 41.67; sex: 78.7% men; mean duration of the problem: 5 years) and the other of 142 social gamblers (mean age: 38.46; sex: 60% men). The questionnaire was self-applied, in the presence of one of the investigators in case any doubts arose. The results indicate that it is a good screening instrument, with high reliability (Cronbach's alpha = 0.94), good covergent validity (the correlation with the SOGS questionnaire is high (r = 0.94; p < 0.001)), good factorial validity (the questionnaire's structure is unidimensional and coherent (just one factor explains more than 50% of the variance)), and its discriminative power is high (diagnostic efficacy = 98.88%). The study offers a comparative analysis of this instrument with other measures of problem gambling, emphasizing its extraordinary performance.     

The Four Es of Problem Gambling: A Psychological Measure of Risk

A focus group of Reno area Gamblers Anonymous members identified four psychological traits contributing to risk for problem gambling, including: Escape, Esteem, Excess and Excitement. A panel of four experts authored 240 Likert-type items to measure these traits. By design, none of the items explicitly referred to gambling activities. Study 1 narrowed the field of useful items by employing a quasi-experimental design which compared the answers of Reno area Gamblers Anonymous members (N = 39) to a control sample (N = 34). Study 2 submitted successful items, plus new items authored with the knowledge gained from Study 1, to validation in a random sample telephone survey across Queensland, Australia (N=2577). The final 40 item Four Es scale (4Es) was reliable (α=.90); predicted gambling problems as measured by the Canadian Problem Gambling Index of Severity (PGSI, Ferris & Wynne (2001). The Canadian Problem Gambling Index: Final Report: Canadian Centre on Substance Abuse); and distinguished problem gamblers from persons with alcohol abuse problems. The new scale can provide a basis for further study in harm minimization, treatment, and theory development.     

Toward a reliable measure of breast self-examination. Attitudes and performance barriers

Breast self-examination (BSE) attitude scales are typically administered as part of face-to-face interviews or telephone surveys. The purpose of this study is to test the reliability (i.e., internal consistency) of a newly developed BSE scale that can be self-administered. The scale is made up of 24 items, each rated on a 5-point Likert-type scale. The scale items cover such issues as perceived seriousness and susceptibility to breast cancer, perceived effectiveness of breast self-examination, and personal barriers. Preliminary results were encouraging. Results of an exploratory factor analysis suggest that six factors may underlie this scale (common variance = 53%). Areas of item and scale improvements and implications for the use of this scale in health evaluation research are discussed.     

An item response theory investigation of the Family Assessment Form

Drawing from a sample obtained from a large scale, community-based child maltreatment prevention initiative, factor analysis and item response theory (IRT) were used to examine item discrimination and to identify items on the Family Assessment Form (FAF) that statistically do not add value for predictive modeling. Results suggest a consolidated, six-factor structure in general accordance with the current configuration of the instrument. In addition, 13 of the 59 items are identified as low information, indicating that they do not contribute unique measurement information regarding the underlying dimensions of the FAF. Findings suggest that from a statistical measurement perspective, the FAF can be shortened and retain its internal validity, although from a functional assessment perspective, certain items may need to be retained for comprehensive family assessment and case planning.     

Test-retest reliability on nine tasks of the Physical Work Performance Evaluation

The Physical Work Performance Evaluation (PWPE) is one of many functional capacity evaluations (FCEs) currently available to assist with determining injured workers' physical potential to return to work. Previous research has explored interrater reliability, construct and predictive validity of the PWPE. This research examined test-retest reliability on a sample of 24 clients with stable physical injuries who were participating in vocational rehabilitation. Nine of the 21 main tasks of the PWPE were evaluated: lifting floor to waist, bilateral carrying, pushing, sitting, standing, kneeling, stair climbing, repetitive squatting and walking. Kappa scores ranged from 0.19 (error) to 0.77 and percent agreement from 66.7% to 87.5%. The results for kneeling, lifting floor to waist, bilateral carrying and pushing tasks suggest substantial test-retest reliability with moderate reliability also suggested for the standing and repetitive squatting tasks. Self-limiting behavior and alterations in pain scores, position adjustments and movement deviations are seen to be the main contributors to affect scoring between the first and second tests.     

Men Sexually Interested in Transwomen (MSTW): Gendered Embodiment and the Construction of Sexual Desire

Unreliable self‐reporting of sexual behavior has been suggested as a major cause of variance in survey research. In addition, differing gender‐related biases are thought to account for discrepancies in reports of oral and anal sex. Insufficient attention, however, has been given to assessing the reliability of the sexual partner's reports. Data obtained from 70 couples concerning 68 sexual behaviors was reviewed. Four different reliability indices yielded moderate to substantial agreement. These included a simple percentage criterion of 87%, a Kappa of .67, and conditional probabilities of .77 and.91. Previously reported discrepancies in the reporting of oral and anal sex were not found. Kappa was found to be superior to simple percentage agreement, but produced overly conservative reliability estimates when derived from behaviors that occur at either high or low base‐rates. The two conditional probability indices are proposed for use in conjunction with Kappa to evaluate the reliability of partner data.     

Measuring Romantic Love: Psychometric Properties of the Infatuation and Attachment Scales

Romantic love is ubiquitous and has major influences on people's lives. Because romantic love consists of infatuation and attachment, researchers need to be able to differentiate between these constructs when examining the behavioral, affective, cognitive, and physiological correlates of this intriguing phenomenon. Existing love questionnaires appear less suitable for measuring the two-dimensional construct of romantic love. We present here the new 20-item Infatuation and Attachment Scales (IAS) questionnaire. In Study 1, exploratory factor analyses in a Dutch-speaking sample (n = 162) revealed a clear-cut two-factor structure, with 10 infatuation and 10 attachment items loading on separate components. This two-factor structure was confirmed in a new Dutch-speaking sample (n = 214, Study 2), and in an English-speaking sample (n = 183, Study 3). In all studies, it was additionally shown that both scales possessed good convergent and discriminant validity, as well as excellent internal consistency and test-retest reliability. We argue that the IAS is a widely applicable, psychometrically sound instrument that will be useful in future research exploring the effects of infatuation and attachment on behavior, emotion, cognition, peripheral physiology, and brain functioning.     

Construct validity of the instrument DOA: a dialogue about ability related to work

This study examined the construct validity of the newly developed instrument A Dialogue About Ability Related To Work (DOA). DOA [18] was developed using concepts from the Model of Human Occupation [20] as theoretical framework. The instrument assumes the assessment of work ability by both the client and the occupational therapist, and is followed by a dialogue which is expected to distinguish possible goal-setting for further rehabilitation [18,19]. Twenty-one professionals and 126 clients from three clinics in Sweden participated in the study. All clients were involved as outpatients in psychiatric work rehabilitation in county councils or community services. Results of a Rasch analysis test indicated that items are well separated and generally work together in five unidimensional continuums to measure work ability. Twenty-nine of 34 items showed goodness-of-fit statistics, which means acceptable infit MnSq values from > or = 0.6 to < or = 1.4 in association with Z values from -2 to 2. The five misfit items have to be revised. The study has indicated that DOA is a valid instrument in psychiatric work rehabilitation and ought to be useful in the dialogue between occupational therapists and clients about goalsetting and treatment planning.