Rating Methodological Quality: Toward Improved Assessment and Investigation

Assessing methodological quality is considered essential in deciding what investigations to include in research syntheses and in detecting potential sources of bias in meta-analytic results. Quality assessment is also useful in characterizing the strengths and limitations of the research in an area of study. Although numerous instruments to measure research quality have been developed, they have lacked empirically-supported components. In addition, different summary quality scales have yielded different findings when they were used to weight treatment effect estimates for the same body of research. Suggestions for developing improved quality instruments include: distinguishing distinct domains of quality, such as internal validity, external validity, the completeness of the study report, and adherence to ethical practices; focusing on individual aspects, rather than domains of quality; and focusing on empirically-verified criteria. Other ways to facilitate the constructive use of quality assessment are to improve and standardize the reporting of research investigations, so that the quality of studies can be more equitably and thoroughly compared, and to identify optimal methods for incorporating study quality ratings into meta-analyses.     

Comprehensive audits for use in evaluating scientific research

Calls for new arrangements to deal with problems involving the integrity of results reported from scientific research have generally taken the form of investigations into data quality and the research methods utilized. Alternatives to proposed government regulation of scientific research have included suggestions for using accounting type audits of the kind used for attesting to the representations of management in corporate financial reports. Here a different type of audit is suggested for the advantages it offers in dealing with the situations in science—which are much less structured and not restricted to situations like those to which financial accounting audits are directed. A broadening from “accounting”; to “accountability”; is thereby achieved. Experimentation with the use of such audits is suggested which can simultaneously help to improve scientific processes and accommodate the interests of the public in responding to the results of scientific research. A beginning is also suggested in the form of “GAO type”; (U.S. General Accounting Office) audits of research activities undertaken (or to be undertaken) at laboratories such as Los Alamos, Livermore and Sandia as they are being redirected from military to civilian type research. It is also suggested that GAO type audits be extended from audits of management activities to examination and evaluation of the validity and significance of the research that has been, or is to be, undertaken.     

Bilingual Validation of the Family Responsibility Scale

Abstract

In the context of the HIV pandemic, infected persons in the United States are increasingly poor women of color, many of whom are single parent heads of household with responsibilities for caring for another ill family member. A significant proportion of such persons are recent immigrants, and many do not speak English as a primary language. Responding to the lack of cross-cultural instruments suitable for assessment and research in such populations, the present study describes the bilingual (Spanish and English) validation of the Family Responsibility Scale. Based on a sample of 161 persons receiving HIV-related service in family agencies in two major urban centers, the findings support a unidimensional rapid assessment instrument with substantial equivalence across language groups. Results indicate very good reliability, and evidence of strong content, factorial, and construct validity. Implications for bilingual validation and applications in health and social service settings are discussed.     

Graduate Social Work Education

This paper reviews the current state of empirical research on graduate social work education, focusing upon the major curriculum content areas of research, practice skills, HBSE, policy, values, and field instruction. The authors address what has been accomplished to date, and which areas are especially in necd of quality research investigations.     

Spiritual Assessment and Latter-Day Saints: Establishing the Preliminary Validity of Spiritual Eco-Maps

ABSTRACT

Few spiritual assessment approaches have been validated with Latter-Day Saints (LDS), in spite of the importance of religion to this culturally distinct population and its burgeoning growth. Drawing on the concept of social validity, this mixed-method study validates a qualitative spiritual assessment approach—spiritual eco-maps—using a geographically diverse sample of social workers (N = 100), all of whom had at least 5 years of direct practice experience with LDS clients. These individuals were asked to identify: a) the degree of cultural consistency, strengths, and limitations of the spiritual eco-map at a conceptual level; and 2) how the questions designed to operationalize the approach might be improved to be more valid, relevant, and consistent with LDS culture. The results suggest that spiritual eco-maps are consistent with LDS culture as long as they are operationalized appropriately. Toward this end, a culturally valid question protocol is provided along with suggestions for administering the instrument in a culturally competent manner. Spiritual eco-maps may be particularly useful in settings where a quick, visually oriented assessment that focuses on clients’ present relationships is indicated. Future research might build upon the present study by assessing the validity of spiritual eco-maps with both community and client samples.     

Assessment of the psychometric properties of the Graded Care Profile version 2 (GCP2) tool for measuring child neglect

Child neglect is highly prevalent, with extensive effects on children's development and often long‐lasting consequences. It is a complex issue, and the identification and assessment of neglect presents particular difficulties for professionals. The Graded Care Profile (GCP) is a tool that can help practitioners to assess neglect, but the updated version of this tool (GCP2) requires psychometric testing. We sought to test the GCP2 with qualified social work practitioners in the United Kingdom. Interrater reliability was assessed by comparing GCP2 scores between 3 pairs of practitioners following assessments of 30 children. Concurrent validity was assessed by comparing GCP2 scores to scores obtained using 2 other validated measures with 15 children. Professionals were surveyed concerning face validity. There was a substantial level of interrater agreement for 2 domains and a moderate level of agreement for the other 2 domains. There were strong correlations between the GCP2 and the other tools across all domains, and face validity results were positive. In conclusion, although this was a small sample, our findings provide preliminary evidence that the GCP2 has adequate psychometric properties pending further rigorous psychometric testing. It is thus likely to be a useful tool in aiding professionals in the assessment of child neglect.     

The Parent Concerns Questionnaire and Parenting Stress Index: comparison of two Common Assessment Framework‐compatible assessment instruments

There is a long‐standing interest in finding ways to develop systematic and consistent means for conducting assessments. Behind this lies a concern with issues such as adequate responses to need, prevention, early intervention and evidence‐based practice. The Common Assessment Framework (CAF) has been developed for wide use for families with additional needs, for various contexts, including children's centres. This, however, has been subject to some criticism. Two instruments with considerable potential relevance to areas consistently important in child and family assessment, focusing on parents (and hence likely to be of generic use with CAF), are the Parent Concerns Questionnaire (PCQ), developed specifically in relation to children's social care populations, and Parenting Stress Index (PSI). The way forward is to investigate empirically what they offer to practice. This paper examines their potential for practice in children's centres through a comparative methodology, an approach particularly suitable to identifying the distinctive qualities and contribution of each. The findings demonstrate both have considerable potential relevance for practice, outlines what each offers, with the detailed individual assessments and its ecological framework characteristic of the PCQ providing benefits additional to the PSI.     

End-of-Life Preparations among Lesbian,Gay, Bisexual,and Transgender People: Integrative Review of Prevalent Behaviors

Proactively making end-of-life (EOL) preparations is important to ensure high quality EOL care. Critical to preparation is the discussion of preferences with one’s primary health care providers. Lesbian, gay, bisexual, and transgender (LGBT) people often experience discrimination from health care providers that will detrimentally affect their ability to communicate their care preferences. Structural barriers, such as those based on sexual orientation and gender identity, may impede timely and quality care when one is most in need. The aim of this study was to examine the prevalence of EOL preparatory behaviors among LGBT people, with particular focus on transgender individuals. Eight survey instruments with 30 prevalence estimates found in the literature were analyzed. EOL discussions between LGBT people and their primary health care providers were rare (10%). Transgender people were found to be even less prepared for EOL; they were 50–70% less likely than their LGB counterparts to have a will, a living will or to have appointed a healthcare proxy. A need exists for future mixed-methods research focused on LGBT populations accompanied by the cultural sensitivity needed to ensure their wishes are honored at the EOL.     

Further Evidence of Racial Disparities in Drug Court Outcomes: Enhancing Service-Delivery to Reduce Criminal Recidivism Rates for Non-White Participants

Abstract

The first drug court began in 1989, and since their inception, they have expanded to over 3,000 in the United States and United States territories. The long-term goal of drug courts is to reduce criminal recidivism rates for nonviolent offenders who have substance use disorders. This study adds to the literature by using secondary data to compare criminal recidivism rates between drug court participants (n?=?163) and probationers who had diagnosed substance use disorders and arrests that were eligible for drug court but they did probation instead (n?=?185). Criminal recidivism was measured up to 36?months post drug court/probation discharge, which provides a more accurate assessment of the long-term effectiveness of drug court. Furthermore, this study identified which drug court participants were most likely to recidivate. Drug court participants were less likely to recidivate than the probation group. However, differences between the two groups may have contributed to the difference in criminal recidivism rates and also suggest that screening criteria may exclude some non-White participants from drug court. Non-white participants were more likely to recidivate than their White counterparts. Implications for future research and drug court practice are discussed, focusing on enhancing the service-delivery of education and employment opportunities to non-White drug court participants.     

Cognitive Pretesting and the Developmental Validity of Child Self-Report Instruments: Theory and Applications

OBJECTIVE: In the context of the importance of valid self-report measures to research and evidence-based practice in social work, an argument-based approach to validity is presented and the concept of developmental validity introduced. Cognitive development theories are applied to the self-report process of children and cognitive pretesting is reviewed as a methodology to advance the validity of self-report instruments for children. An application of cognitive pretesting is presented in the development of the Elementary School Success Profile. METHOD: Two phases of cognitive pretesting were completed to gather data about how children read, interpret and answer self-report items. RESULTS: Cognitive pretesting procedures identified validity problems with numerous items leading to modifications. CONCLUSIONS: Cognitive pretesting framed by an argument-based approach to validity holds significant potential to improve the developmental validity of child self-report instruments.     

Policy integration: mapping the different concepts

A growing number of studies have examined the collaboration of actors from two or more policy domains in order to integrate aims and concerns derived from one policy domain into another. In our literature review, we refer to this empirical phenomenon as ‘policy integration’, exemplified by the Health in All Policies approach. Despite the wealth of literature on the subject, the scientific community only has access to a portion of the insights that have come out of this field of research, due primarily to the fact that policy integration is discussed using a variety of different terms, which tend to be specific to the policy domain under investigation. To facilitate a more inclusive scientific debate on policy integration, we provide a comprehensive overview of the different terminologies associated with policy integration and analyse the recurring themes in the respective literature strands. What is the motivation for policy-makers to promote policy integration? What is the design of the instruments used for policy integration? How does policy integration affect the policy-making process? And how well does policy integration perform? These are the four questions guiding our study.     

Researching Quality of Life in Early Old Age: The Importance of the Sociological Dimension

Measurement of quality of life has become a major feature of much social and epidemiological research in health and social care settings. It is seen as an important alternative to more process‐based outcome measures but remains poorly defined. A major weakness is the absence of any coherent theoretical underpinning whether sociological, psychological or philosophical. Into this conceptual vacuum proxies for quality of life have been introduced. Quality of life [QoL] research into older populations has focused on measures of health and illness as equivalents of QoL. This paper argues that this response is inadequate as it reduces old age to a dimension of health, disability and disease. Instead, we argue that it is necessary to create a theoretically based measure of QoL in early old age which relates to those aspects of later life that are not defined by health. We present a model of QoL that is derived from aspects of contemporary social theory as they relate to the ontology of late modernity. In particular, we utilize a model based upon needs satisfaction. The model contains four domains: Control, Autonomy, Pleasure and Self‐realization. The measure consists of a 19‐item scale. The four domains load on to a single latent QoL factor. We argue that the CASP 19 scale offers an approach to QoL that integrates a sociologically based model of quality of life with a meaningful and valid research instrument.     

Research in spirituality, religion and aging

Research on the concept of "spirituality" demonstrates its overlap with "religion," so for many purposes they need to be considered together as "spirituality/religion." Investigations of age differences point to the likelihood that spirituality tends to increase during later adulthood. It has important positive relationships with various measures of life satisfaction, psychosocial well-being, and both physical and mental health. It benefits therapy for recovery from illness and is a source of meaning and purpose in life. Spiritual interventions help to relieve psychological distress and death anxiety, as well as the stresses of caregiving. Because of its therapeutic value, prayer is an important resource for coping with problems experienced during the life course. The Spiritual Well-Being Scale and many other instruments have been developed to measure spirituality and related concepts. As in all other domains of research on people, they all have limits and must be applied with caution for both technical and ethical reasons. Nevertheless, applications of the research findings, which overwhelmingly demonstrate the importance of spirituality to human well-being, already are improving the effectiveness of clinical work and social services in all of the health and human service professions. As scientific knowledge of spirituality expands, so does awareness of the need for further research, including the refinement of methodological procedures, expansion to new topics, and extension to international cultures and diverse religions. The outlook for research on spirituality and the consequent practical applications to benefit humanity is very promising.     

Addressing procedural challenges of ethical review system: Towards a better ethical quality of clinical trials review in Malaysia

The review of clinical trials with human participants in Malaysia is governed by a decentralized control system. The clinical trial protocols are reviewed by 13 registered research ethics committees (RECs) in Malaysia. A governmental body, the National Pharmaceutical Control Bureau, is responsible for the inspection and oversight of these registered RECs to ensure that they comply with the regulatory requirements. However, this study highlights that each REC in Malaysia has a different standard operating procedure and ethical review process. Other procedural challenges identified include inadequate membership, poor mechanism for research monitoring after ethical approval, and insufficient resources. Establishments of a national standard of REC and a central ethics committee are suggested to ensure procedural compliance in the oversight of clinical trials in Malaysia. While there is a growing concern that procedural compliance may not have a direct impact on the protection of human subjects, our key point is that an ethical review system compliant with the national standards could serve as a strong framework to support and enhance the ethical quality of decision-making and judgement. We believe that being aware of how influential procedural compliance can be would help committees improve the ethical quality of their research review.     

Beyond Recovery From Trauma: Implications for Clinical Practice and Research

This article draws implications for clinicians working with survivors of major life crises in four general areas: the relation of psychological well-being, distress, and posttraumatic growth; conceptual issues in this type of clinical work; the process of encouraging growth in clients following traumatic events; and suggestions for additional research. Posttraumatic growth can be accompanied by an increase in well-being, but distress and growth may also coexist. Positive changes can occur in several domains, but many are likely to be phenomenological. Degree of change produced by clinical intervention may be limited in scope, but there clearly are some ways in which the clinician may make growth more likely for the client. Suggestions for future research include the call for longitudinal investigations, studies of rumination and responses of the social network, and the examination of potential gender differences in posttraumatic growth.     

Validating a Korean Version of the Drug Abuse Screening Test-10 (DAST-10)

ABSTRACT

There are no screening instruments for adults in the Korean language that are specific to drugs other than alcohol and tobacco. The purpose of this study is to validate the Korean version of the Drug Abuse Screening Test-10 (DAST-10). The Korean version of the DAST-10 was validated with 1,000 adults including general adults and at-risk adults. The results indicated that the Korean version of the DAST-10 was unidimensional similar to its English-language counterpart and showed satisfactory internal consistency reliability and convergent validity. The results also showed that the Korean version demonstrated optimal performance at a cutoff score of 2, which is the same as that of its English-language counterpart. It is expected that as the development of Korean-language instruments measuring alcohol problems contributes to the expansion of related research, the development of the Korean version of the DAST-10 will facilitate research on various topics related to substance abuse in Korea. Validation with Korean adults identified as substance abusers is needed to improve the ability of the Korean version to correctly identify substance-abusing adults.     

大学生主观生活质量及其影响因素研究

大学生的主观生活质量是目前学术领域研究比较薄弱的一个环节。文章通过实证调查所得资料的分析,认为当今大学生的主观生活质量总体上处于比较理想的水平,并通过因子分析和多元回归等统计方法,发现影响大学生主观生活质量的因素首先是学习满意度,其次包括人际关系满意度、精神满意度等10个方面。性别、家庭经济状况和年级对大学生的主观生活质量也有不同程度的影响。     

Validating a Behavioral Health Instrument for Adults: Exploratory Factor Analysis

ABSTRACT

Increasingly, social workers and behavioral health practitioners use assessment instruments to support service planning and to monitor progress. Following statewide implementation of the Adult Needs and Strengths Assessment (ANSA) to identify behavioral health symptoms, related functional challenges, risks, and strengths, this validation study explored the underlying structure of the instrument. An exploratory factor analysis used routinely collected information for Midwestern adults with diagnosed behavioral health disorders who participated in community-based services (N = 46,013). Five factors with adequate to good internal consistency (α = 0.733?0.880) emerged: personal recovery, trauma and stress related problems, substance use risks, self-sufficiency, and cultural-linguistic considerations. Validation of the ANSA supports use of the instrument to engage individuals and families, to plan services, to monitor progress, and to conduct research. Implications for social work education, supervision, and practice include the importance of understanding culture, holistic assessment, and services supporting personal recovery for individuals living with mental illness or substance use disorders. Confirmation of findings requires additional research.     

Judgmental Discounting and Environmental Risk Perception: Dimensional Similarities, Domain Differences, and Implications for Sustainability

Environmental risks constitute a special category of risks because they often involve consequences that are highly uncertain, strongly delayed, occurring at distant places, and—therefore—mostly borne by others. Economic, decision–theoretic, and psychological research about the way people deal with such consequences is reviewed. Two major findings are presented: first, there is evidence that discounting mechanisms are stable across different preference dimensions (uncertainty, temporal, spatial, and social distance). Second, discount rates tend to vary across different problem domains (e.g., environmental vs. health vs. financial risks). In particular, it appears that temporal discounting is less pronounced for environmental risks than in other domains. Several factors are identified that affect the nature of the risk evaluation process, and it is argued that environmental risks differ from other risks on such factors. These environmental-risk characteristics may have important implications for policy strategies to promote environmental sustainability. Contrary to other domains, appealing to the public's long-term preferences may be successful. Also in policy making, insights from standard economic decision theory to environmental decision making should be applied with caution.     

Carers of People with Learning Disabilities, and their Experience of the 1995 Carers Act

Summary This paper seeks to highlight the impact of the Carers (Recognitionand Services) Act of 1995 on carers of people with learningdisabilities. It draws on research conducted in the south westbetween 1997 and 1999, which examined the views and experiencesof carers who had an assessment of their needs, together withthose of the person they care for and the professional conductingthe assessment. It concludes that the Carers Act is not widelyused, or understood, by this group of carers. Recommendationsfor improving practice include changes to the current terminology,new triggers for a carer's needs assessment, and a greater inputfrom other agencies, especially health services. Although thecurrent paper focuses on the views and experiences of carers,an exploration of the Carers Act, and its effect on potentialconflicts of interest with the person with learning disabilities,is available elsewhere (Williams and Robinson, 2001).