Evaluation and communication: using a communication audit to evaluate organizational communication

This article identifies a surprising dearth of studies that explicitly link communication and evaluation at substantive, theoretical, and methodological levels. A three-fold typology of evaluation studies referring to communication is proposed and examples given. The importance of organizational communication in program delivery is stressed and illustrative studies reviewed. It is proposed that organizational communication should be considered in all program evaluations and that this should be approached through communication audit. Communication audits are described with particular reference to established survey questionnaire instruments. Two case studies exemplify the use of such instruments in the evaluation of educational and social programs.     

The social practice of sustainable agriculture under audit discipline: Initial insights from the ARGOS project in New Zealand

One of the most interesting recent developments in global agri-food systems has been the rapid emergence and elaboration of market audit systems claiming environmental qualities or sustainability. In New Zealand, as a strongly export-oriented, high-value food producer, these environmental market audit systems have emerged as an important pathway for producers to potentially move towards more sustainable production. There have, however, been only sporadic and fractured attempts to study the emerging social practice of sustainable agriculture - particularly in terms of the emergence of new audit disciplines in farming. The ARGOS project in New Zealand was established in 2003 as a longitudinal matched panel study of over 100 farms and orchards using different market audit systems (e.g., organic, integrated or GLOBALG.A.P.). This article reports on the results of social research into the social practice of sustainable agriculture in farm households within the ARGOS projects between 2003 and 2009. Results drawn from multiple social research instruments deployed over six years provide an unparalleled level of empirical data on the social practice of sustainable agriculture under audit disciplines. Using 12 criteria identified in prior literature as contributing a significant social dynamic around sustainable agriculture practices in other contexts, the analysis demonstrated that 9 of these 12 dimensions did demonstrate differences in social practices emerging between (or co-constituting) organic, integrated, or conventional audit disciplines. These differences clustered into three main areas: 1) social and learning/knowledge networks and expertise, 2) key elements of farmer subjectivity - particularly in relation to subjective positioning towards the environment and nature, and 3) the role and importance of environmental dynamics within farm management practices and systems. The findings of the project provide a strong challenge to some older framings of the social practice of sustainable agriculture: particularly those that rely on paradigm-driven evaluation of social motivations, strong determinism of sustainable practice driven by coherent farmer identity, or deploying overly categorical interpretations of what it means to be ’organic’ or ’conventional’. The complex patterning of the ARGOS data can only be understood if the social practice of organic, integrated or (even more loosely) conventional production is understood as being co-produced by four dynamics: subjectivity/identity, audit disciplines, industry cultures/structure and time. This reframing of how we might research the social practice of sustainable agriculture opens up important new opportunities for understanding the emergence and impact of new audit disciplines in agriculture.     

Gender bias in performance evaluations: The impact of gender quotas

The intention behind imposing quotas for women on corporate boards is to close the gender gap in economic participation and help women to be promoted within organizations. However, the broader social psychological literature lends support to ideas that affirmative action policies, such as quotas, may do more harm than good for the beneficiaries. We extend this idea beyond the affected beneficiaries and ask whether this unintended negative effect spills over to women who are not immediate targets of the quota, by signaling incompetence. We develop an experimental design to investigate whether the announcement (study 1) and the implementation (study 2) of a quota for women have a direct negative effect on performance evaluations and hence reinforce the existing gender bias in evaluation. We observed that the performance of women was evaluated significantly lower than that of men. However, this gender bias was limited to sequential (rating) evaluations and was not evident in joint (ranking) evaluations. The quota did not significantly influence the amount of this bias. In addition, we observed more pronounced sexism in males compared to females. Results of study 2 gave a hint to an association between higher sexism and lower evaluations of women’s performance. We also found some evidence for a stronger evaluation bias in females when controlling for sexist attitudes. Hence, our results imply that the bias, which is overall quite robust and strongly pronounced, is still affected by individual gender-related characteristics.     

Evaluations of Social Work Education: A Critical Review

Since the introduction in England of the social work degree in 2002, there have been a number of significant changes to social work education culminating with the implementation of the Social Work Reform Board's innovations in 2013. This article critically explores the role of evaluations of changes in social work education and their implications for social work pedagogy. Evaluations can be linked to wider trends in society such as modernity, reflexive modernity, and the audit society. These wider influences affect the use of evaluations. A way forward for social work is proposed. Social work education needs to develop a critical pedagogy of hope to transform the profession's relationship not only to evaluations but to practise as well. The key message of this article is that evaluation studies of social work education mirror the underlying tensions of late modernity. Therefore, social work education needs to adopt a critical pedagogic approach to the use of evaluation research.     

Corporate social responsibility attribute rankings

This research presents the findings of a rank order evaluation of corporate social responsibility attributes by a target public (N = 72) of a contemporary corporate social responsibility campaign. The results indicate that publics value corporate honesty above other corporate social responsibility behaviors.     

It's Not What You Do,But Who You Are: Informal Social Control,Social Status,and Normative Seriousness in Organizations

This paper analyzes the impact of managerial social status on the normative evaluation of managerial acts in organizational contexts. We test several propositions on the relationship between social status and normative evaluation derived from Donald Black's theoretical framework on social control. The research design consists of a factorial survey of 200 managers. Each respondent evaluated the seriousness of a normatively questionable managerial act. In each vignette, the perpetrator's social status was systematically manipulated in either a high or a low condition. The results generally support the argument that the higher a manager's social status the less vulnerable that individual is to unfavorable normative evaluations, holding constant the act. The paper closes with discussion of our findings in light of social structural and rational choice perspectives on informal social control in organizations. Additionally, we discuss methodological issues related to experimental research on informal social control in organizations, the consistency of our findings with those from previous studies of social control across diverse settings, potential theoretical applications and extensions of Black's framework in organizational contexts, and practical implications for the implementation of corporate codes of conduct and corporate dispute resolution systems.     

Designing useful evaluations: The stakeholder survey

The failure of program evaluations to provide policy decision-makers with useful information is a familiar complaint. This paper discusses an approach towards designing and implementing more useful evaluations — the stakeholder survey. Stakeholders are individuals, or groups, who have a direct interest in the program being evaluated. The central premise of the paper is that the information needs of stakeholders should be incorporated directly into the formulation of evaluation objectives, and the design of the evaluation to achieve those objectives. The paper presents a rationale for the stakeholder survey, describes its major components, and reports on the application of the approach in a large program evaluation. Key issues involved in applying the approach are also considered. The evidence from this study suggests that such surveys are not only feasible, but also of considerable value to the evaluation team.     

The conduct of program evaluation reviews in the general accounting office

Professional behavior during the General Accounting Office's (GAO's) present era of oversight can best be understood by reviewing the audit types that populate village life in the GAO and the repertoire the office uses in conducting program evaluation audits. Four types of auditors are present, each of which has a role to play in the audit repertoire. This repertoire focuses on finding faults in government programs in order to catch or sustain congressional interest in GAO's work. The “audit trail” of each report may last up to 3 years and involves considerable organizational strife and conflict with outside groups. To sustain themselves through this arduous auditing process, auditors have developed a creed through which they conceptualize themselves as an elite with a mission to insure good government.     

Welfare and capital-theoretic foundations of corporate social responsibility and corporate sustainability

Corporate social responsibility (CSR) and corporate sustainability (CS) are variously defined in the literature. Providing formal definitions for the two concepts, we show in this article that CS and CSR are distinct but interrelated concepts that can be usefully formalized with capital-theoretic and welfare economic approaches. CSR can particularly constitute a strategy to cope with externalities and serve as insurance against reputational risks that harm profit prospects and corporate value. Moreover, we present a formal approach for integrating the societal perspective of sustainable development with the corporate perspective of CSR and CS. This approach is complementary to a company's internal CSR and CS evaluation.     

Attenuating public skepticism: Effects of pre-crisis corporate engagement and post-crisis CSR initiatives on corporate evaluations

This study examined the effects of stated motives in post-crisis corporate social responsibility (CSR) messages, CSR-crisis issue congruence, and pre-crisis corporate engagement on public skepticism toward CSR and subsequent corporate evaluations. We conducted a 3 (motive of post-crisis CSR initiative: no statement vs. firm-serving vs. public-serving) x 2 (issue congruence: CSR initiative incongruent vs. congruent with crisis) x 2 (pre-crisis corporate engagement: absence vs. presence) between-subjects online experiment via Qualtrics panel (N = 378). The findings showed that stating a public-serving motive or launching a CSR initiative incongruent with a crisis heightened some aspects of CSR skepticism. We further found that pre-crisis corporate engagement interacted with stated motive and issue congruence to influence corporate evaluations. Implications of the findings and directions for future research are discussed.     

An evaluation paradox in social work? An empirical study of evaluation use in connection with temporary programmes in Swedish social work

This article contributes to the debate on evaluation use by analysing temporary national programmes in Swedish social work. Previous empirical research shows a gloomy picture of evaluation use, thus supporting an evaluation paradox: evaluations are not used for learning and development, but are continually prioritised. The aim of this article is to study, through interviews and document analysis, how evaluations of temporary programmes in social work are designed and used. The results support the paradox and we found that it has two dimensions. First, evaluations are prioritised, but they function as a ritual, characterised by routine and unclear intentions, with limited relevance for professional learning and improvement. The second dimension is about limited use by those commissioning the evaluations, but the evaluators are using data for new purposes and in new contexts. This is, at best, a form of conceptual use, hopefully contributing indirectly to social work practice. The article ends in a discussion on how evaluations could be designed for learning and improvement by focusing on three normative models: utilisation-focused evaluation, responsive evaluation and evidence-based practice approach (EBP).     

Subjectivity in Feminist Science and Technology Studies: Implications and Applications for Sociological Research

Feminist science and technology studies calls the researcher to reconsider subjectivity in three ways. First, who or what has subjectivity? Second, is subjectivity a property of an individual being with sentience, or is it a more diffuse process? Third, who or what acts in meaningful ways to impact social relations (and are thus worthy of sociological study)? According to feminist STS, the conferral of subjectivity has been nationalized, racialized, and sexualized, and the influence of non‐human life and non‐living matter has been underemphasized. We suggest that sociological research could benefit from a more expansive understanding of subjectivity and a more interactive (or “entangled”) notion of social–material relations. Human relations and action need not just be considered in the context of the human and social but can also be considered in relation to the non‐human and material. To make the implications of feminist STS more concrete, we offer specific applications of feminist STS methodologies across a range of sociological methods and topics.     

Governing Global Supply Chain Sustainability through the Ethical Audit Regime

Over the past two decades multinational corporations have been expanding ‘ethical’ audit programs with the stated aim of reducing the risk of sourcing from suppliers with poor practices. A wave of government regulation—such as the California Transparency in Supply Chains Act (2012) and the UK Modern Slavery Act (2015)—has enhanced the legitimacy of auditing as a tool to govern labor and environmental standards in global supply chains, backed by a broad range of civil society actors championing audits as a way of promoting corporate accountability. The growing adoption of auditing as a governance tool is a puzzling trend, given two decades of evidence that audit programs generally fail to detect or correct labor and environmental problems in global supply chains. Drawing on original field research, this article shows that in spite of its growing legitimacy and traction among government and civil society actors, the audit regime continues to respond to and protect industry commercial interests. Conceptually, the article challenges prevailing characterizations of the audit regime as a technical, neutral, and benign tool of supply chain governance, and highlights its embeddedness in struggles over the legitimacy and effectiveness of the industry-led privatization of global governance.     

Evaluation of auditor decisions: Hindsight bias effects and the expectation gap

Hindsight bias is said to exist when individuals overestimate the extent to which an outcome could have been anticipated prior to its occurrence. Ex post, litigants tend to blame auditors in hindsight for failing to foresee and anticipate subsequent financial problems of their audit clients. However, it is not known whether the blame attributed in hindsight is due to hindsight bias or other factors such as the generally hypothesized user-preparer expectation gap. A behavioral experiment was conducted with 65 federal and state general jurisdiction judges and 58 auditors from one of the Big-Six public accounting firms. Results indicate that judges provided significantly lower evaluations of auditors' performance than did auditors, reflective of an expectation gap. Of greater importance, our results reveal that evaluative judgments of auditors' performance are subject to hindsight bias. Additional analysis revealed that our results are consistent with a cognitive interpretation of hindsight bias.     

Evaluating a private foundation's health program

Public policy decisions in health are increasingly difficult and expensive. Although there will never be enough information available, private foundations can help to bridge the most important gaps in knowledge. Larger foundations may also wish to respond to those who doubt the value of foundation activities. This article reviews the experiences over the past eight years of The Robert Wood Johnson Foundation in employing evaluation and related social research procedures in the planning and implementation of a major philanthropic effort to improve the health and medical care of Americans. Discussed are the still evolving Foundation evaluation framework, the unanticipated problems in undertaking specific evaluations, and the substantive findings of some of the studies.     

Introduction to the Volume on Relocation Issues in Child Custody Cases

ABSTRACT

Recent JCC articles on the topics of gender bias in domestic violence research and practice and parental alienations syndrome are critiqued, with a brief preview of related articles on these topics scheduled for the upcoming special issue on domestic violence. The problem of false negative and false positive errors in evaluations of abuse allegations are discussed and the social psychological factors that contribute to systems failure in family courts are examined.     

An examination of methods in the evaluation of volunteer programs.

Although many volunteer programs have been called "successful" and the use of paraprofessionals urged, few program evaluations with a benefit-cost analysis have been reported. Several major studies of volunteer utilization are reviewed, and particular needs with reference to program evaluation are discussed. Three alternative evaluation methodologies are proposed. The importance of utilizing a comprehensive evaluation incorporating all three methodologies is discussed. An ongoing volunteer program in a community mental health center is used as an illustration.     

The Influence of Maternal Socialization on Infants’ Social Evaluation in Two Cultures

Although a growing body of research has explored the early development of social evaluation, no research has directly compared social evaluations of infants between different cultures. In addition, there has been little understanding regarding socialization's effects on this ability. The goal of this study was to expand on earlier findings on social evaluation in infants by investigating a broader sample from two cultures, and to explore the influence of maternal socialization on infants’ social evaluation. Using the violation of expectations and the preferential reaching paradigm, four groups aged 6‐, 9‐, 12‐, and 15–18 months and their mothers from Japan and the United States (159 dyads) were compared in terms of spontaneous social evaluations. Japanese and European American infants showed similar performance in dishabituation to the inconsistent behavior and in their reaching preference for prosocial over antisocial agents, indicating that the emergence of spontaneous social evaluation is not culture‐specific. Furthermore, our study provides a novel finding regarding the relationship between mothers’ socially evaluative speech and infants’ preference for prosocial over antisocial agents. These results suggest that the development of sociomoral understanding results from complicated interactions among evolutionary, cognitive, and social factors.     

Performance Management and the Stifling of Academic Freedom and Knowledge Production

In an era of neoliberal reforms, academics in UK universities have become increasingly enmeshed in audit, particularly of research ‘outputs’. Using the data of performance management and training documents, this paper analyses the role of discourse in redefining the meaning of research, and in colonizing a new kind of entrepreneurial, corporate academic. The new regime in universities is characterized by slippage between the audit and disciplinary functions of performance management. We conclude that academic freedom is unlikely to emerge from a system which demands compliance with a regime of unattainable targets and constant surveillance.