Phenomenography: the development and application of an innovative research design in higher education research

This article considers the application of phenomenography, as arguably the only research design (so far) to have been developed substantially within higher education research by higher education researchers. It considers its origins, meaning and application, and the issues and critiques that it has raised. It concludes that, while phenomenography has both acknowledged limitations and undoubted achievements, it still appears to have much to offer to higher education research, and its usage might be extended to other fields.     

A participatory evaluation framework in the establishment and implementation of transdisciplinary collaborative centers for health disparities research

ObjectiveWe describe the formulation and implementation of a participatory evaluation plan for three Transdisciplinary Collaborative Centers for Health Disparities Research funded by the National Institute of Minority Health and Health Disparities.MethodsAlthough different in scope of work, all three centers share a common goal of establishing sustainable centers in health disparities science in three priority areas – social determinants of health, men’s health research, and health policy research.ResultsThe logic model guides the process, impact, and outcome evaluation. Emphasis is placed on process evaluation in order to establish a “blue print” that can guide other efforts as well as assure that activities are being implemented as planned.ConclusionWe have learned three major lessons in this process: (1) Significant engagement, participation, and commitment of all involved is critical for the evaluation process; (2) Having a “roadmap” (logic model) and “directions” (evaluation worksheets) are instrumental in getting members from different backgrounds to follow the same path; and (3) Participation of the evaluator in the leadership and core meetings facilitates continuous feedback.     

Examination of the empirical research environment of program evaluation: methodology and application

The study dealt with a research environment in which the treatment effect is heterogeneous, and in which individuals use their assessments of the treatment effect to decide whether or not to enroll in an intervention program. In this article, a new methodology is proposed for examining the validity of the specified research environment in a given intervention program and database. The methodology is based on direct examination of the assumptions underlying that research environment. Comparison of the commonly used verification method with the one proposed here revealed that these methods are complementary rather than substitutional.     

Women's erotic rape fantasies: an evaluation of theory and research.

This article is the first systematic review of the research literature on women's rape fantasies. Current research indicates that between 31% and 57% of women have fantasies in which they are forced into sex against their will, and for 9% to 17% of women these are a frequent or favorite fantasy experience. Erotic rape fantasies are paradoxical: they do not appear to make sense. Why would a person have an erotic and pleasurable fantasy about an event that, in real life, would be abhorrent and traumatic? In this article, the major theories of women's rape fantasies are evaluated both rationally and empirically. These theories explain rape fantasies in terms of masochism, sexual blame avoidance, openness to sexuality, sexual desirability, male rape culture, biological predisposition to surrender, sympathetic physiological activation, and adversary transformation. This article evaluates theory and research, makes provisional judgments as to which theories appear to be most viable, and begins the task of theoretical integration to arrive at a more complete and internally consistent explanation for why many women engage in erotic rape fantasies. Methodological critiques and programs for future research are presented throughout.     

Measuring the lifetime experience of domestic violence: application of the life history calendar method

In the absence of a "gold standard," research on domestic violence relies primarily on self-report, the quality of which is known to decline as the length of the recall period increases. Eliciting valid and reliable self-report data is crucial to the development of prevention and intervention policies and services. Nevertheless, existing measures typically do not incorporate devices to facilitate respondents' recall of the lifetime experience of domestic violence. This article describes the application of the Life History Calendar (LHC) method (Freedman, Thornton, Camburn, Alwin, & Young-DeMarco, 1988) to increase a respondent's recall of domestic violence victimization over the lifecourse. The LHC method elicits memorable information of a personal nature (e.g., children's birth dates) and uses this information to facilitate the recall of less memorable events. A recent field test of this LHC measure indicates its utility in assessing domestic violence victimization, which takes place in a complex sequence of episodes and often involves multiple perpetrators over the lifecourse.     

Incorporating intersectionality into research design: an example using qualitative interviews

As intersectionality gains more prominence, scholars still face difficulties of incorporating principles of intersectionality into empirical research. Key concepts of intersectionality theory include moving away from additive thinking, relationality, and social constructionism. An important challenge is how to incorporate these concepts into research design. While existing scholarship examines intersectional methodology, most of the focus has been on issues of analysis not data collection. I argue that some of the difficulties in intersectionality scholarship are not just issues of analysis but issues of data collection. In particular, I discuss how scholars can incorporate intersectionality concepts into research design by offering examples from my own research that used qualitative interviews to examine how race and gender influenced neighborhood experiences. I also present the obstacles I encountered in conveying the key concepts of intersectionality into language that is relatable to study participants.     

Applicability of standards for evaluations of educational programs, projects and materials in an international setting: Qualitative research

Qualitative studies are much more the practice in Europe and increasingly elsewhere than in the United States. For an international audience, therefore, there needs to be much greater emphasis on qualitative techniques than there is in the standards report. The most difficult ethical problem arises from the conflict between the right to privacy of research subjects and the right to know of audiences. By its nature, qualitative research focuses on the particular, the detailed analysis of the specific incident. It is therefore often difficult to conceal the identity of the individual organisations and people concerned. Even a confidential report might work to the detriment of the subjects studied. Published reports pose the same problem. Have subjects the right to have material they believe to their disadvantage excised from a report even if that renders the report valueless? There are two technical points in the standards document that should be amplified. The first concerns the verification of data. We need to specify procedures to ensure credibility for qualitative studies. The second concern is the specification of quantitative techniques for the analysis of qualitative data. The statement in the standards document is too weak. We need to define much more clearly the steps that should be taken to ensure that qualitative data can be tested rigorously. Finally, there is the problem of collusion, the danger that the qualitative evaluators will select from their material the data which are in conformity with their own predilictions or presuppositions, the expectations of their audience, or the requirements of their sponsors. This problem may be acute in some parts of the world. To be of maximum international value, the standards report needs to be extended and to pay much greater attention to problems arising with qualitative evaluation.     

Measuring relative efficiency of government-sponsored R&D projects: A three-stage approach

Without considering differences in operating environment, traditional methods of efficiency evaluation are suffering from external environmental influences. This study presents an alternative approach for assessing the relative efficiency of government-sponsored research and development projects (GSP). A three-stage approach employing data envelopment analysis to evaluate efficiency and Tobit regression to control external variables was applied to 110 projects over 9 years. This study finds that firm size, industry, and ratio of public subsidy on research and development (R&D) budget of recipient firm significantly influences the technical efficiency of GSP in Taiwan. After controlling these external variables, the mean value of technical efficiency in the third stage increases and becomes significantly different to that in the first stage. Most GSPs increase their returns when their projects are scaled up. Furthermore, government policy makers must establish the upper-limit ratio of subsidies on R&D budgets of recipient firms to avoid inefficient use of public funds.     

Collaborative evaluation and market research converge: an innovative model agricultural development program evaluation in Southern Sudan

In June and July 2006 a team of outside experts arrived in Yei, Southern Sudan through an AID project to provide support to a local agricultural development project. The team brought evaluation, agricultural marketing and financial management expertise to the in-country partners looking at steps to rebuild the economy of the war ravaged region. A partnership of local officials, agricultural development staff, and students worked with the outside team to craft a survey of agricultural traders working between northern Uganda and Southern Sudan the steps approach of a collaborative model. The goal was to create a market directory of use to producers, government officials and others interested in stimulating agricultural trade. The directory of agricultural producers and distributors served as an agricultural development and promotion tool as did the collaborative process itself.     

Early truancy intervention: Results of an evaluation using a regression discontinuity design

This study evaluates the effectiveness of a truancy reduction program. A Regression Discontinuity design was used to assess attendance outcomes for 700 children. Approximately half received a case management intervention, while the other half received a warning letter, only. Truancy rates in the control group remained at the pre-intervention levels, while truancy in the treatment group significantly declined (p < .01). Successful case closure was associated with a reduction in truancy (p < .001). The intervention was less effective with non-White children and with those who had been previously retained. Findings substantiate continued support for the intervention; however, additional resources should include culturally-responsive treatment approaches and academic enrichment programs.     

Measuring the impacts of community development initiatives: a new application of the adjusted interrupted time-series method

The authors contribute to the development of empirical methods for measuring the impacts of place-based local development strategies by introducing the adjusted interrupted time-series (AITS) approach. It estimates a more precise counterfactual scenario, thus offering a stronger basis for drawing causal inferences about impacts. The authors applied the AITS approach to three community development initiatives using single-family home prices as the outcome indicator and found that it could measure impacts on both the base level of prices and the rate of price appreciation. The authors also found a situation in which the method appears unreliable, however. The AITS approach benefits from more recurrent data on outcomes during the pre-and post-intervention periods, with an intertemporal pattern that avoids great volatility. The AITS approach to measuring effects of community development initiatives holds strong promise, with caveats.     

Applied behavioral intervention in rural school settings: Problems in research design,implementation, and evaluation

This article discusses a number of theoretical, practical, and methodological difficulties encountered in the design, implementation, and evaluation of treatment programs conducted in the natural environment. In order to illustrate how these difficulties might be overcome, a case example is provided. Contingency contracts were used to modify the assaultive and disruptive behaviors of a 6-year-old child attending a preschool program for emotionally and physically handicapped children. Treatment effectiveness was evaluated through the use of a multiple baseline design and maintenance of behavior change was assessed in periodic follow ups of the child for 1 year after his admittance to public school. The importance of, planning for the generalization of newly acquired behaviors to different environments is stressed and suggestions for facilitating this process are discussed.     

标志设计中平面构成基本形式的应用研究

平面构成的基本形式是传统标志设计的创作方法之一,在现代标志设计的视觉语汇环境下,平面构成基本形式在标志设计中传统的运用也面临着挑战。对此,本文主要从现代标志设计中平面构成基本形式的发展方面进行详细阐述。     

Development of an evaluation framework for publicly funded R&D projects: The case of Korea's Next Generation Network

For decades, efforts have been made globally to measure the performance of large-scale public projects and to develop a framework to perform such measurements due to the complexity and dynamics of R&D and stakeholder interests. Still, limitations such as the use of a simply modified model and the lack of a comprehensive viewpoint are prevalent in existing approaches. In light of these research gaps, this study suggests a practical model to evaluate the performance of large-scale and publicly funded projects. The proposed model suggests a standard matrix framework of indices that evaluates the performance of particular elements in an industrial ecosystem in vertical categories and the economic and technological outcomes of those elements in horizontal categories. Based on the application of a balanced scorecard, this study uses mixed methodologies such as social network analysis, inter-industry analysis, and the analytic hierarchy process. Finally, the model evaluates the performance of Korea's Next Generation Network project as a case study.     

Measuring university-based research in Europe: competing indicators and underlying assumptions

Modern societies have a growing need for information and numbers for governing social life. Numbers have the ability to represent a complex reality in a simplified and linear form, easily communicated. Far from being the product of a mere technical process, numbers are the result of a process that “is fundamentally social – an artifact of human action, imagination, ambition, accomplishment” (Espeland, W.N., and Stevens, M.L., 2008. A sociology of quantification. European journal of sociology, 49 (3), 401–436, p. 431). In the modern policy-making climate, numbers become key mechanisms for simplifying, classifying, comparing and evaluating. Along with this, the fields of visibility of evaluative objects, meanings and understandings (Dean, M., 2010. Governmentality. Power and rule in modern societies. 2nd ed. London: Sage) are re-framed consistently with what Clarke, J. (2004. Changing welfare, changing states. New direction in social policy. London: Sage) terms a ‘performance/evaluation nexus’ that links effort, values, purposes and self-understanding to measures and comparisons of outputs (Ball, S.J., 2012. Performativity, commodification and commitment: an I-Spy guide to the neoliberal university. British journal of educational studies, 601, 17–28). In this paper, we focus on the field of higher education (HE), where numbers, in the form of performance indicators, benchmarks and headline targets, are frequently used to strategically orient the sector towards the objectives and goals of the Bologna Process and of the overall Europe 2020 agenda (Waldow, F., 2014. From taylor to tyler to no child left behind: legitimating educational standards. Prospects, 45 (1), 49–62). We aim to offer a comparative overview of the complex spectrum of metrics, provided at the supranational level, within the field of higher education by focusing on the European Research Area (ERA) in order to map and analyse some of the crucial issues in play. A second ambition of this paper is to move from a mapping and analytical perspective to a deconstruction of a specific subset of research metrics, with the aim of challenging the ‘self-evident truths’ and the dominant conventional wisdom that define current European metrics in order to bring into question whether they contribute to restructuring the universities’ research environments, affecting research policies and procedures. Performance indicators are posited to be‘conceptual technologies’, encompassing theoretical and normative assumptions that shape the objects they aspire to measure.