A Psychometric Analysis of Field Evaluation Instruments Designed to Measure Students’ Generalist-Level Social Work Competencies

ABSTRACT

There is a dearth of research on the evaluation of the psychometric performance of instruments designed to measure students’ generalist-level social work competencies. There is also uncertainty on the performance of various response option formats used to measure students’ competencies in assessment instruments. Using a sample of 198 master of social work students and 198 field instructors, the current study employs confirmatory factor analysis to assess the psychometric performance of a field evaluation instrument designed to measure students’ generalist-level social work competencies. The results provide evidence to support the validity and reliability of the instrument. The results also highlight nuances associated with a 5-point response option format for items used to measure competency indicators. Implications, limitations, and directions for future research are discussed.     

Initial Validation of the Open Adoption Scale: Measuring the Influence of Adoption Myths on Attitudes Toward Open Adoption

All adoptions, including those facilitated by public agencies, should be evaluated for openness. Although there are no national figures related to openness in public welfare, there are indications that it is not considered in all cases. Adoption myths may be one of the reasons that adoption workers do not always assess a family's ability to maintain a more open adoption arrangement. The following study is the initial validation of a multidimensional psychometric instrument developed to measure myths as they relate to open adoption. Based on a sample of 547 university students, the results indicate very good reliability and acceptable factorial and construct validity. The instrument was designed as a training and research tool to inform child welfare personnel and policy makers of the influence of adoption-related myths on child welfare adoption practice.     

Toward the development of an adolescent gambling problem severity scale

The development and initial psychometric properties of an adolescent gambling problem severity measure are described. The scale, based on a revision of the South Oaks Gambling Screen (Lesieur & Blume, 1987), was administered to 1,101 older (ages 15 to 18) Minnesota adolescents as part of a state-wide gambling survey. Study results indicated that the scale had moderate internal consistency reliability and was significantly related to alternate measures of problem severity for male subjects. Because the rate and severity of gambling among females was very low, the psychometric adequacy of the scale for females is not known at this time. The implications of study findings to the conceptual understanding of adolescent gambling problem severity as well as study limitations and future research needs, are discussed.Support for the study was provided by the Minnesota Department of Human Services and the Center for Addiction Studies, University of Minnesota, Duluth. The authors gratefully acknowledge the assistance from Robert Blum, Linda Harris, Durand Jacobs, J. Laundergan, Alan Mathiason, Brenda Miller, James Schaefer, school officials from the participating high schools, and interviewers who assisted with the telephone survey.     

Adaptation and validation of the Japanese version of the Gambling Urge Scale

In Japan, accurately assessing and targeting gambling urges is important for the treatment of gambling disorder because people are constantly exposed to external triggers that elicit gambling urges. However, a valid measure that assesses gambling urges has not been adequately established. Therefore, the purpose of this study was to examine the psychometric properties of the Japanese version of the Gambling Urge Scale (GUS-J), which is a well-known valid 6-item instrument in English-speaking countries. Two hundred and thirty-nine participants completed questionnaires, 86 of which met the criteria for a probable gambling disorder. As with the original GUS, the GUS-J was demonstrated to be a one-factor model that accounted for 63.82% of the total item variance, with all items demonstrating loadings of .40 or higher. With regard to reliability, the GUS-J showed good internal consistency (α = .88). Furthermore, validity based on evidence of relationships with other variables, especially convergent and concurrent validity of the GUS-J, were supported by several analyses that examined hypotheses based on previous findings. In conclusion, this study showed that the GUS-J has good psychometric properties as an assessment tool for gambling urges in Japanese people and it will be useful for future clinical practice and research.     

Assessing behavior in children aged 12–24 months using the Strengths and Difficulties Questionnaire

Behavioral and emotional problems in infants and toddlers are common, often persist and put children at risk of later mental health problems. Reliable, efficient, and sensitive tools are needed to identify young children who may benefit from further assessment and support. The Strengths and Difficulties Questionnaire (SDQ), offers a brief, convenient means of screening for early problems, however, it lacks psychometric validation in infants. The aim of this study was to assess the validity and reliability of the SDQ in children aged 12–24 months. Ninety-three participants, with children aged 12–24 months, completed the SDQ and Child Behavior Checklist (CBCL) online. Concurrent validity of the SDQ was assessed through comparison with the CBCL. The results demonstrated that key subscales of the SDQ and CBCL were significantly correlated (r range= −.19 to –.57). Key SDQ subscales showed moderate reliability (Cronbach's alpha range = .38–.79, mean inter-item correlation range = .06–.43). The SDQ shows promising reliability and validity as a measure for rating the behavior of 12–24-months-old children, particularly for externalizing symptoms. Further research is needed to assess its predictive utility.     

Construction and validation of Rejection Sensitive Expectation,Perception, and Reaction Questionnaire—Partner (RSEPR-P)

The Rejection Sensitive Expectation, Perception, and Reaction Questionnaire—Partner (RSEPR-P) was developed to assess rejection expectation, perception of rejection, and reactions to perceived rejection among intimate partners. This article details the construction and validation procedures, including item pool generation, factor analysis, and the estimation of reliability and validity. The researchers examined the psychometric properties of RSEPR-P in a community sample of 151 participants. Confirmatory factor analysis (CFA) with the final 24 items suggested four-factor loading with each subscale loading on a separate factor. The subscales had good to excellent internal consistency. In support of the convergent validity, RSEPR-P exhibited robust correlations with rejection sensitivity-personal, mindfulness, marital adjustment, and self-esteem measures. A 6-month test–retest reliability was established. RSEPR-P is proposed to have potential therapeutic and research utility.     

Psychometric Properties of the Family Inventory of Resources and Stressors in Families of Children with Emotional/Behavior Disorders: Ethnic/Racial and Developmental Differences

The Family Inventory of Resources and Stressors (FIRST) has been shown to have adequate psychometric properties in the assessment of families of children with emotional and behavior disorders (EBD). However, the extent to which ethnic/racial and developmental differences moderate the psychometric properties of this instrument is unknown. The study sample consisted of 150 families with an EBD child consecutively admitted to the Astor Transitions Program. This study seeks (1) to conduct a psychometric analysis of the FIRST subscales among African American and Hispanic/Latino families with older students versus early childhood students with EBD, and (2) to determine whether profiles will show greater stress for families of older students due to longer exposure to their EBD. Alpha coefficients were computed with a cutoff of 0.70 (or higher) indicating acceptable internal consistency reliability. Criterion variables reflecting family environment, mental health, and level of service correlated with relevant FIRST subscales assessed validity. Ethnic/racial and developmental effects were found to moderate the psychometric properties of the FIRST. No statistically significant between-group mean differences on the FIRST were found for families of early childhood versus older EBD students. Despite evidence of response bias on the part of families and lack of standardization during test administration by caseworkers, the current findings still reveal adequate reliability and limited validity of FIRST subscales.     

Development and validation of the Emotional Self-Awareness Questionnaire: a measure of emotional intelligence

This study examined the psychometric characteristics of the Emotional Self-Awareness Questionnaire (ESQ), a self-report measure of emotional intelligence. The ESQ, Emotional Intelligence Scale, and measures of alexithymia, positive negative affect, personality, cognitive ability, life satisfaction, and leadership aspirations were administered to 1,406 undergraduate psychology students. The ESQ was reduced from 118 to 60 items via factor and reliability analyses, retaining 11 subscales and a normal score distribution with a reliability of .92. The ESQ had significant positive correlations with the Emotional Intelligence Test and positive affect, significant negative correlations with alexithymia and negative affect, and an insignificant correlation with cognitive ability. The ESQ accounted for 35% of the variance in life satisfaction over and above the Big Five, cognitive ability, and self-esteem, and demonstrated incremental validity in explaining GPA and leadership aspirations. The significance of emotional intelligence as a unique contributor to psychological well-being and performance, and applications for the ESQ in assessment and outcome research in couple and family therapy are discussed.     

Assessing Family Caregiving: A Comparison of Three Retrospective Parentification Measures

Despite the frequent use of the Parentification Questionnaire (PQ) and the Parentification Scale (PS) in research studies, scant attention has been directed toward the psychometric properties of the scores derived from these measures. Moreover, given the importance of parentification as a clinical topic for treatment, it is surprising that more attention has not been paid to the accuracy and psychometric properties of parentification instruments. The current study was the first to compare the psychometric properties of these two commonly used parentification instruments (i.e., PQ and PS). This study also examined another parentification instrument, newly developed: the Parentification Inventory (PI). More specifically, the psychometric properties of the PI were compared comprehensively with those of the PQ and PS in a sample of college students (N = 787). Scores derived from the three measures were found to be comparable, valid, and reliable. Implications for couple and family system practice, and directions for future research are proffered.     

An Examination of the Reliability of the “What If” Situations Test: A Brief Report

ABSTRACT

One of the primary barriers to evaluating child sexual abuse prevention programs is the limited availability of measurement tools with demonstrated psychometric properties. The purpose of this study was to investigate the internal and test-retest reliability of the “What If” Situations Test (WIST) to determine its utility as an evaluation tool for young children. The WIST contains six scales designed to assess children's abilities to recognize, resist, and report inappropriate touching. The WIST was administered to 406 preschoolers involved in five previous studies. Internal and test-retest reliabilities of the six scales met research standards, supporting the internal consistency and temporal stability of the WIST. Advantages and limitations of the measure are presented and validation issues are then discussed.     

Migration Motives of University Students: An Empirical Research

The main aim of this paper is to analyse the motives affecting the migration decisions of young people, particularly university students. Two scales were developed for measuring the perception of the importance of these motives. The data used in the research were collected via a survey of the opinions and attitudes of university students in Osijek, in June 2010. The paper also analyses psychometric properties of the scales – their dimensionality and reliability. The results of a confirmatory factor analysis undoubtedly indicate that both scales are multidimensional constructs. A combination of the results of t‐tests for an independent sample, factor analysis (exploratory and confirmatory) and reliability analysis suggest that emigration and stay motives are two sides of the same migration decision, and that they can be classified into several factors: the economic situation, social networks, insider advantages (that can be divided into inherited amenities and public‐safety conditions) and the wealth of opportunities. Depending on the power of the initial and target destination, the factors can function as ‘push’ or ‘pull’ factors. The results of the study show social networks as being the only ‘pull’ factor for the city of Osijek, whereas the other factors, especially the economic ones, proved to demonstrate the ‘push’ effect. However, the effects of all factors were very mild.     

Psychometric Evaluation of the Problem Gambling Severity Index-Chinese Version (PGSI-C)

The purpose of this study was to examine the psychometric properties of the Chinese 9-item Problem Gambling Severity Index (PGSI) derived from the 31-item Canadian Problem Gambling Index (CPGI) originally developed by Ferris and Wynne (2001). Exploratory factor analysis (EFA; n = 386; Group A data) and confirmatory factor analysis (CFA; n = 387; Group B data) on the Chinese student and community data (Mean age = 25.36 years) showed that a unifactorial model fitted the data with good reliability score (Cronbach’s alpha = 0.77). The concurrent validity of the PGSI-C was good in terms of the Chinese data matching the expected correlation between PGSI-C and other variables or scales such as SOGS, gambling frequency, gambling urge, gambling cognitions, depression, anxiety, and stress. The scale also reported good discriminant and predictive validity. In sum, the PGSI-C has good psychometric properties and can be used among Chinese communities to identify at-risk problem gamblers. Implications and suggestions for future research are discussed.     

Interpersonal and Societal Mattering in Work: A Review and Critique

The need exists for a comprehensive literature review of the concept of mattering with regard to work. In response, this review examined a theoretical framework of societal and interpersonal mattering, mattering within the work context, and existing psychometric measures of mattering. The author analyzed the research on the current state of mattering at work, identified core frameworks and psychometric issues that have been investigated to date, and suggests the use of a more comprehensive framework for work mattering that illuminates both the relational and social aspects of the work experience. Several promising directions for future research and potential benefits of incorporating work mattering in career counseling practice emerged and are discussed.     

Multidimensional Demand Strength of Recreational Activity Programming in Assisted Living: A Conceptual Model and Prototypical Rating Scale

ABSTRACT

This article describes a framework for measuring activity demands and the psychometric properties of a related measurement tool, the Multidimensional Activities Demand Strength for Assisted Living (MADS-AL). Nine experts—three in physical, cognitive, and social domains, respectively—rated demands of 37 common assisted-living activities. Interrater reliability and internal consistency for physical, intraclass correlation coefficient (ICC)(2, 3) = .96, Cronbach's α = .97, cognitive, ICC(2, 3) = .71, α = .93, and social, ICC(2, 3) = .80, α = .86, scales give preliminary support for reliability of the MADS-AL tool. Overall, this article moves the field toward considering task demands as a relevant variable to inform future activities research and practice.     

The Development of the Problematic Pornography Consumption Scale (PPCS)

To date, no short scale exists with strong psychometric properties that can assess problematic pornography consumption based on an overarching theoretical background. The goal of the present study was to develop a brief scale, the Problematic Pornography Consumption Scale (PPCS), based on Griffiths’s (2005) six-component addiction model that can distinguish between nonproblematic and problematic pornography use. The PPCS was developed using an online sample of 772 respondents (390 females, 382 males; M_age = 22.56, SD = 4.98 years). Creation of items was based on previous problematic pornography use instruments and on the definitions of factors in Griffiths’s model. A confirmatory factor analysis (CFA) was carried out—because the scale is based on a well-established theoretical model—leading to an 18-item second-order factor structure. The reliability of the PPCS was excellent, and measurement invariance was established. In the current sample, 3.6% of the users belonged to the at-risk group. Based on sensitivity and specificity analyses, we identified an optimal cutoff to distinguish between problematic and nonproblematic pornography users. The PPCS is a multidimensional scale of problematic pornography use with a strong theoretical basis that also has strong psychometric properties in terms of factor structure and reliability.     

A Preliminary Investigation of a Telephone Administration of the Quality of Marriage Index

Follow‐up data collection presents many challenges for longitudinal research. Validating paper‐and‐pencil assessments for use via telephone may alleviate these issues. This study evaluates psychometric properties of the Quality of Marriage Index (QMI; Norton, 1983, Journal of Marriage and the Family, 45, 141) when administered by telephone and compares scores with the paper‐and‐pencil version. Participants included women with major depressive disorder and dyadic discord and their male spouses (n = 24 couples). Results suggest excellent scale reliability for the verbal QMI. However, a significant mean difference was found: participants scored on average 3 points higher on the telephone‐administered QMI than on the written QMI. Recommendations for future research and clinical implications are discussed.     

An Assessment of the Psychometric Properties of Italian Version of CPGI

The aim of this study was to adapt to the Italian context a very commonly used international instrument to detect problem gambling, the canadian problem gambling index (CPGI), and assess its psychometric properties. Cross-cultural adaptation of CPGI was performed in several steps and the questionnaire was administered as a survey among Italian general population (n = 5,292). Cronbach’s alpha reliability coefficient was 0.87 and can be considered to be highly reliable. Construct validity was assessed first by means of a principal component analysis and then by means of confirmatory factor analysis, showing that only one factor, problem gambling, was extracted from the CPGI questionnaire (an eigenvalues of 4,684 with percentage of variance 52 %). As far as convergent validity is concerned, CPGI was compared with Lie/Bet questionnaire, a two-item screening tool for detecting problem gamblers, and with both depression and stress scales. A short form DSM-IV CIDI questionnaire was used for depression and VRS scale, a rating scale, was used for rapid stress evaluation. A strong convergent validity with these instruments was found and these findings are consistent with past research on problem gambling, where another way to confirm the validity is to determine the extent to which it correlates with other qualities or measures known to be directly related to problem gambling. In sum, despite the lack of a direct comparison with a classic gold-standard such as DSM-IV, the Italian version of CPGI exhibits good psychometric properties and can be used among the Italian general population to identify at-risk problem gamblers.     

Preenrollment Immunization Policies of American Colleges: An Assessment of the Need for Policy Implementation

Abstract

Alcohol abuse among college students is prevalent, yet few instruments with sound reliability and validity are available to assess these problems in this population. As part of a large, baseline assessment battery for a prospective study of offspring of alcoholics, the 27-item Young Adult Alcohol Problems Screening Test (YAAPST) was given to 490 freshmen at a large mid western university; approximately 9 months later, 482 subjects completed the scale again. In addition to asking about such traditional problems as experiencing blackouts and driving while intoxicated, the YAAPST included specific items relating to college experiences (eg, getting into sexual situations that were later regretted, missing classes, and receiving lower grades than usual). The YAAPST was designed to assess these drinking consequences over two different time frames, lifetime and past year, and also to indicate the frequency of occurrence during the past year. Results indicated that the YAAPST is a unidimensional scale with good psychometric properties (good internal consistency and test-retest reliability). Three different approaches were used to demonstrate the validity of the YAAPST. Findings supported criterion validity (with interview-based alcohol abuse/dependence diagnoses as the criterion), concurrent validity (comparing the YAAPST with other drinking measures), and construct validity (correlating the YAAPST with etiologically relevant personality, motivational, and peer influence variables). The YAAPST is a promising screening instrument for alcohol problems in college students. It has excellent psychometric properties and the potential to provide a range of useful information to the clinician or researcher.     

The construction and psychometric properties of an instrument measuring support for sexual rights

The purpose of this paper is to report the findings of an attempt to construct and examine the psychometric properties of an instrument that measures support for the 11 sexual rights formulated by the World Association of Sexology. Since international conferences held in the mid-1990s, a new approach to sexual issues has been promoted by health professionals, sex educators, and human rights advocates. This approach emphasizes the empowerment of individuals and examines sexual health concerns within a human rights framework. Goals of a sexual rights approach have been to promote awareness and advocacy of sexual issues and especially to facilitate the work toward better sex education and sexual health services. Analysis of reliability and validity data from two university samples (n = 388 and n = 175) indicated that the majority of the sexual rights scales have acceptable psychometric properties. This paper concludes with a discussion of ways to improve the scales for future use and the need to validate the scales using samples that allow generalizability of findings.     

THE FAMILY TASK INTERVIEW: A TOOL FOR CLINICAL RESEARCH IN FAMILY INTERACTION

The Family Task Interview (FTI) is a multiple task interview administered by tape recorder and designed to elicit clinically relevant family interaction. The FTI has been designed to improve on previous similar instruments and a series of studies to evaluate its validity, reliability, and acceptability have been carried out. Further psychometric and practical evaluation is necessary, but results so far suggest that the FTI is a well standardized and easily administered instrument which is useful for research where assessment of family interaction using direct observation is required.