Portfolios for learning and assessment: laying the foundations for continuing professional development

Portfolios are widely used in a range of education programmes at different levels in a number of countries. They are also well established at all stages in social work education. However, in the literature there is surprisingly little critical analysis of the application of the portfolio in social work. In this paper the authors' objective is to contribute to professional knowledge about the use of portfolios in social work. The authors are all members of the Higher Education Funding Council Self-Assessment in Professional and Higher Education Project and are concerned to develop self-assessment in social work and enhance the development of life-long learning. They begin by examining the literature about portfolios from the field of education. Findings are then analysed from a small exploratory study of the use of portfolios in practice learning on a Diploma in Social Work course. Five different but related conceptual themes which emerge from the analysis are discussed: compiling the portfolio; issues of evidence; reflection in a context of assessment; the practice teacher and the learning dialogue; and the challenge of assessment. Suggestions are offered for effective portfolio practice.     

Dimensions of conflict for separated families index: An index for family courts

Many services and interventions focus on assisting separated families experiencing high conflict. Yet, little research has focussed on identifying and distinguishing levels of conflict. This research tested reliability and validity of an Index based on assessing 15 dimensions of conflict typical of high conflict in separation. Seventy-seven mothers and 47 fathers receiving services from family courts or social service agencies were assessed by two raters for inter-rater reliability. Internal consistency was high, with Cronbach's alpha = 0.94 (mothers) and 0.96 (fathers). Inter-rater reliability was high, with ICC = 0.85 (mothers) and 0.83 (fathers). Preliminary evidence of validity was found comparing scores on the index to scores on the Conflict Tactics Scale and the Ahron's Quality of Coparenting Communication Scale.     

The Portfolio as Artifact and Actor

This article proposes a tentative framework to support empirical analysis of portfolios as pedagogical tools for formative and summative assessment in higher education. It aims to get a deeper understanding of the role of the portfolio in student learning both as a tool and as representations of this tool. To that end, we use three sets of theoretical ideas: Wartofsky's conceptualization of perception through the creation of artifacts, actor-network theory, and Wertch's notions of internalization, appropriation and mastery. This article is meant to complement current portfolio research with an original approach that addresses explicitly epistemological questions and concepts like learning and knowledge and explores possible avenues for analyzing and understanding portfolios as artifacts and tools in learning activities.     

ASSESSING CHANGE IN FAMILY FUNCTIONING AS A RESULT OF TREATMENT: THE STRUCTURAL FAMILY SYSTEMS RATING SCALE (SFSR)

This article describes a theoretically based structural family assessment procedure designed for use in evaluating therapy outcome. The standardized procedure for eliciting family interaction consists of the administration of three family tasks. The family's interactional patterns on the tasks are evaluated along six dimensions of structural family functioning: Structure, Flexibility, Resonance, Developmental Stage, Identified Patienthood, and Conflict Resolution. A manual has been developed for standardizing ratings for research applications. The measure demonstrated good interrater and internal consistency reliability. Available evidence for the content, factor, and construct validity of the measure is presented. Clinical outcome research studies conducted with the measure provide evidence that the measure is sensitive to different types of structural family intervention modalities and that it discriminates between a structural family intervention versus a no treatment/wait list control condition as well as between a structural family intervention versus an individual psychotherapy treatment modality. Finally, the measure has been demonstrated to be nonreactive under both treatment and wait list conditions.     

Case study as a research method to study life histories of elderly people: Some ideas and a case study of a case study

The article explores conditions under which life history interviewing, based on case study research, may become a valuable research instrument. Validity and reliability of case study as a form of qualitative research are discussed. It is shown that neither triangulation nor internal consistency methods can guarantee a single interpretation. As an example a case study of Zina, an immigrant from Russia, is used, and two explanations of her behavior are proposed. The choice between them is made through construction of a holistic image of Zina, which essentially means introduction of a new dimension—theory, worldview—as a criterion of validity and reliability of case studies. Relations of a case study to a theory are discussed, and the conclusion is made that a case study enables a researcher to utilize the most the diversity of life histories of elderly people.     

Using client exit interviews to illuminate outcomes in program logic models: a case example

This paper describes how client exit interviews can assist human service administrators and workers to better understand the outcomes their programs are designed to accomplish. Specifically, the qualitative component of a demonstration family literacy program evaluation is used to illustrate how client input can be used to fine-tune the outcomes component of a program's logic model. An analysis of semi-structured exit interviews with 35 clients, who were randomly selected from all 89 served in the first year of the program, resulted in revision to the program's original logic model, creating explicit ‘testable’ pathways to achieving intended outcomes.     

The Phone-FITT: a brief physical activity interview for older adults

The purpose of this study was to develop a brief physical activity interview for older adults (Phone-FITT) and evaluate its test-retest reliability and validity. Summary scores were derived for household, recreational, and total PA. Reliability was evaluated in a convenience sample from a fall-prevention study (N = 43, 79.4 +/- 2.9 years, 51% male), and validity, in a random sample of individuals in older adult exercise programs (N = 48, 77.4 +/- 4.7 years, 25% male). Mean time to complete the Phone-FITT was 10 min for participants sampled from exercise programs. Evaluation of test-retest reliability indicated substantial to almost perfect agreement for all scores, with intraclass correlation coefficients (95% confidence intervals) ranging from .74 (.58-.85) to .88 (.8-.94). For validity, Spearman's rho correlations of Phone-FITT scores with accelerometer counts ranged from .29 (.01-.53) to .57 (.34-.73). Correlations of Phone-FITT recreational scores with age and seconds to complete a self-paced step test ranged from -.29 (-.53 to -.01) to -.45 (-.68 to -.14). This study contributes preliminary evidence of the reliability and validity of the Phone-FITT.     

The Use of Owner Resources in Small and Family Owned Businesses: Literature Review and Future Research Directions

This paper integrates relevant literature and the Sustainable Family Business Model regarding interchange of financial resources between family and business. Two distinct literatures on the use of owner resources in small businesses are examined: the intermingling of business and household resources from the family firm literature and financial bootstrapping studies from the small business finance literature. What has not been addressed in both literatures about the use of owner resources is discussed and the risks that owner resource bootstrapping and intermingling may place on the household and the business are considered. Recommendations and propositions for future research are suggested. To fully understand the makeup and success of household financial portfolios and family businesses, it is important to understand the use of owner resources in a holistic manner.     

The Family Unpredictability Scale: Reliability and Validity

We developed the Family Unpredictability Scale (FUS) to better understand the factors that underlie the presence and consequences of family dysfunction. Familial unpredictability is defined as a lack of consistency in family behaviors and regulatory systems. Family researchers and therapists generated and evaluated items, which were self‐administered by married parents who had children aged 2 to 18 (N= 314 ). Principal‐components analysis yielded four subscales (discipline, nurturance, meals, and money; 22 items total) that were internally consistent; confirmatory factor analysis based on a separate subsample yielded an adequate preliminary representation of this factor structure. The FUS related significantly to known measures of family and individual functioning but not to social desirability. The present study provides preliminary reliability, construct validity, and concurrent validity for this new multidimensional, self‐report assessment of family unpredictability. Research and clinical implications are discussed.     

Learning to be interprofessional through the use of reflective portfolios?

Learning portfolios are used extensively in health and social care. There is evidence of their value as a tool to consolidate learning although limited information is available regarding their use in social work education. This study explores the use of a portfolio to encourage social work students to reflect on their interprofessional learning (IPL). The portfolio offers a means of demonstrating professionalism concerning knowledge, skills and attitudes in the context of collaborative practice. The findings of this research suggest that qualifying students were largely capable of reflecting on their development of interprofessional competence after attending specific IPL events, which were part of a wider programme. Social work students found reflective writing difficult, although this improved over time. In particular students struggled to write about skills and attitudes. Students perceived the portfolio to be a good way to assess their progress and believed this helped them to engage with their learning and make meaning through reflection and analysis. They appreciated that IPL and feedback from their assessments had advanced their abilities for self-analysis; despite needing help with reflective writing, they felt better prepared for ongoing use of reflection in their professional development, and in ‘working together’.     

The Structured Interview of Family Assessment Risk: Convergent Validity,Inter-rater Reliability and Structural Relations

The quality of parenting, composition and socio-economic status are family risk or protective factors on juvenile delinquency, however, there is not an instrument to help psychologists and social workers to specifically assess the adolescent offender’s family. This study set out to accomplish an additional validation study of the structured interview of family assessment risk (SIFAR), a structured professional judgment tool for the assessment of family risk and protective factors of juvenile delinquents. The statistical analyses included inter-rater reliability, convergent validity with YLS/CMI, the Partial Least Squares approach to structural equation modeling and receiving operator characteristics (ROC) analysis. A sample of 130 male adolescent delinquents detained in Portuguese forensic facilities and their parents, were paired analyzed. The YLS/CMI was used to analyze the convergent validity with SIFAR. SIFAR shows a strong correlation with the YLS/CMI family context, moderate to high values of inter-rater reliability; SIFAR factors show that they are predictive determinants of the Moderate Four risk factors. ROC analysis shows adequate accuracy power. The findings show that SIFAR it is useful as an additional assessment tool for structured risk assessment instruments since it allows understanding the vulnerabilities and strengths of the delinquent’s family.     

Student portfolios as windows into intercultural knowledge and knowing

This research paper deals with intercultural knowledge and knowing as displayed in higher education student portfolios. The portfolios were written by student pairs taking a global education course at Centria University of Applied Sciences, Finland, during seven academic years. Conceptual metaphor theory and metaphor analysis were utilised to explore intercultural knowledge and knowing in the portfolio texts. The study is one of the few metaphor analyses conducted in the context of intercultural/global education. For the analysis, we selected a sample of student portfolios that represented both degree-taking students and exchange students. In this study, interculturality was defined as interaction of people representing various cultural backgrounds, world-views and identities. The research questions were the following: What conceptual metaphors and metonymies do the students utilise when writing about culture and interculturality? What main conceptions of culture and interculturality do the metaphors and metonymies reflect? In the result section of this paper, we provide citations of portfolio text to discuss the four most frequently occurring metaphors and metonymies. We also discuss our interpretations about the underlying conceptions of culture and their significance for the practice of intercultural education.     

Investigating the factor structure and validity of the family empowerment scale for parents of children with emotional disturbance in middle school

Parent empowerment involves the ability of caregivers to meet the needs of their family while maintaining feelings of control and is particularly important for families of children at-risk. It is necessary to establish reliable and valid tools to measure parent empowerment. The purpose of this study was to examine the internal factor structure, score reliability, and convergent validity of the FES scores with caregivers of middle school youth who had an Individualized Education Plan for Emotional Disturbance (ED) or Other Health Impairment (OHI) due to emotional or behavioral needs. A confirmatory factor analysis (CFA) was used to examine the internal structure the FES. Score reliability was examined by computing coefficient alpha (Cronbach, 1951) for each subscale score and computing Coefficient omega and coefficient omega hierarchical for good-fitting factor models and bifactor models. Convergent validity was examined by generating composite scores for each subscale, followed by computing Pearson correlations between FES subscale scores and scores from the PAM, CGSQ and the SDQ. Results indicated that the hypothesized three-factor model fit the data adequately. FES scores were reliable based on coefficient alpha and omega, and evidence of convergent validity with measures of parent activation, caregiver strain, and child behavior were moderate to strong. The results support the use of the FES with parents of middle school youth who have ED and further validate the three factor structure identified in the initial measure development. Practical and clinical implications of these findings include support for the use of the FES with this particular group of parents.     

The Adaptive Success Identity Plan (ASIP): A Career Intervention for College Students

The Adaptive Success Identity Plan (ASIP) uses the construct of success identity as the core of a career intervention program. Three empirical studies (N > 2,000) explored the success identity construct and its application in a structured career intervention. Study 1 provided reliability and validity evidence of the measures. Study 2 indicated that programming efforts targeting family involvement, self-efficacy, and stress and time management should enhance a number of college outcomes. Study 3 found that students who received the ASIP intervention were more likely to use the success identity construct in defining their role in college success.     

Reliability and validity of the Vulnerability Index-Service Prioritization Decision Assistance Tool (VI-SPDAT) in real-world implementation

This study examined the reliability and validity of the Vulnerability Index-Service Prioritization Decision Assistance Tool (VI-SPDAT), a widely-used assessment of the health and social vulnerabilities and housing needs of individuals experiencing homelessness. Homeless Management Information System data were obtained for 1495 individuals who were administered the VI-SPDAT between 2014 and 2016. Subsamples were selected for reliability and validity assessments. Results suggest there are challenges to the reliability and validity of the VI-SPDAT in practical use. VI-SPDAT total scores did not significantly predict risk of return to homeless services, while type of housing was a significant predictor. Vulnerability assessment instruments have important implications for communities working to end homelessness by facilitating prioritization of scarce housing resources. Findings suggest that further testing and development of the VI-SPDAT is necessary.     

Assessment of the complex effects of trauma across child serving settings: Measurement properties of the CANS-Trauma Comprehensive

This study examined the measurement properties of the Child and Adolescent Needs and Strengths-Trauma Comprehensive Version (CANS-Trauma). This comprehensive, trauma-focused, and strengths-based assessment strategy is designed to assess the complex needs of traumatized children and adolescents across child-serving settings and support the treatment/service delivery process. The current study assessed 257 children and adolescents from three mental health agencies across the country that were partners in the National Child Traumatic Stress Network (NCTSN). Internal consistency and inter-rater reliability, convergent and discriminant validity, and scale properties of the CANS-Trauma were examined. Overall, measurement properties of the CANS-Trauma showed promising results. Internal consistency varied from marginally acceptable to excellent across most domain scores (α = 0.62–0.90); the traumatic stress symptoms domain demonstrated very good internal consistency (α = 0.82). Inter-rater reliability for the CANS-Trauma was good overall, while the traumatic stress symptoms domain had marginally acceptable reliability. There was evidence for convergent validity with more limited evidence for discriminant validity, based on the association between domains on the CANS-Trauma and subscales of the Trauma Symptom Checklist for Children-Alternate (TSCC-A) and the Child Behavioral Checklist for Children (CBCL). Rasch modeling analyses indicated that the CANS-Trauma had good scale properties overall, including item fit within each of the domains. These preliminary results also suggested that the various mental health and trauma symptom domains on the CANS-Trauma were more sensitive to populations with significant trauma symptoms and mental health difficulties compared to populations with lower to moderate difficulties in these areas. Implications of these findings are discussed in terms of the utility of a comprehensive, trauma-informed assessment strategy in practice settings.     

Youth Program Quality Survey: Youth Assessment of Program Quality

The Youth Program Quality survey, a 24-item survey of youth participant perceptions of program quality, based on program elements identified by the National Research Council (NRC) and Institute of Medicine, was developed and piloted with 614 younger teens (ages 10–13 years) and 486 older teens (ages 14–17 years) who attended 4-H camps and conferences. Evidence is presented for content and construct validity. In addition, the overall instrument demonstrated high reliability, as measured by Cronbach's alpha, generally ranging from .70 to .96, and moderate subscale reliability of .60 or higher on four factors in the younger sample (α=.60 or higher on four factors in the older sample). Results are discussed in terms of goodness of fit to the National Research Council model, the significance of youth voice in assessment of program quality, developmental differences in perceptions of quality, and programming design. Recommendations are offered for practice, research, and policy.     

Measuring the willingness to foster children with emotional and behavioral problems

Many children in foster care have emotional or behavioral problems or are at risk for these problems. It is important to identify parents willing to foster children with these problems in order to ensure placement, care, stability and well-being of such children. This study presents a new 40-item self-report measure of the willingness of parents to foster children with emotional and behavioral problems, and two 20-item parallel forms of this measure. In addition, this study presents evidence of reliability and validity of scores derived from these measures with a national sample of 297 foster mothers. Coefficient alpha for these measures was .92 or greater, indicating excellent internal consistency reliability. Scores from these measures were unrelated to demographic characteristics, providing evidence of discriminant validity. In addition, scores from these measures were higher for foster mothers licensed to provide treatment foster care than for mothers only licensed to provide regular foster care, providing support for known groups validity. Finally, support for construct validity is provided by the fact that foster mothers with higher scores on these measures had fostered longer, were fostering more children at the time of this study, and had fewer children removed from their home at their request.     

Developing a Family Risk Assessment Tool in Adolescent Offenders: An Exploratory Study

The quality of family relations it is a relevant risk or protective factor in the adolescent criminal involvement, being the interplay between family and adolescent individual features, fundamental issues in the understanding of the adolescent risk and strengths assessment. This study aims to contribute to the validation of the Structured Interview of Family Assessment Risk (SIFAR) tool for the adolescent offenders’ family, based in nine family living items as risk factors and four protective factors. In an exploratory design, the structural consistency of SIFAR was analyzed using Cronbach’s alpha, mean inter-item correlation and exploratory factor analysis over risk items, because of the protective items dichotomous scale. The participants were a sample of 130 male adolescent delinquents detained in Portuguese facilities of the Directorate-general of Social Rehabilitation and Imprisonment, and their parents, paired analyzed. The SIFAR presents a structured professional judgment design for adolescent offender’s family protective and risk assessment. A reliability value of .75 was obtained for the risk items education, employment, housing/transport, legal problems, social dissonance, poverty, social net, social security and parenting, and an .79 was registered to the protective items family involvement, high discipline, low physical punishment and low parental stress. The risk items revealed a two-factor structure (social-economics and social conformity factors) explaining 48.15 % of the total variance. Results show the potential usefulness of SIFAR in the assessment the family protective and risk factors of adolescent offenders, presenting adequate structural reliability and construct validity. However further investigation it is necessary to the validation process of this tool.