Evaluating qualitative management research: Towards a contingent criteriology

The term qualitative management research embraces an array of non‐statistical research practices. Here it is argued that this diversity is an outcome of competing philosophical assumptions which produce distinctive research perspectives and legitimate the appropriation of different sets of evaluation criteria. Some confusion can arise when evaluation criteria constituted by particular philosophical conventions are universally applied to this heterogeneous management field. In order to avoid such misappropriation, this paper presents a first step towards a contingent criteriology located in a metatheoretical analysis of three modes of qualitative management research which are compared with the positivist mainstream to elaborate different forms of evaluation. It is argued that once armed with criteria that vary accordingly, evaluation can reflexively focus upon the extent to which any management research consistently embraces the particular methodological principles that are sanctioned by its a priori philosophical commitments.     

Are Some Risk Comparisons More Effective Under Conflict?: A Replication and Extension of Roth et al.

Despite many claims for and against the use of risk comparisons in risk communication, few empirical studies have explored their effect. Only one study, published by Roth et al. in this journal in 1990, has tested the 1988 predictions by Covello et al. as to the public's relative preferences for 14 kinds of risk comparisons as they might be used by a factory manager to explain risks of his ethylene oxide plant. That study found no correlations between the Covello predictions and seven different measures of "acceptability" of Covello's examples of each type of comparison. However, two critics of the Roth study, as well as its own authors, suggested that a scenario involving local risks, a conflict-ridden situation, and a plant manager unknown to the townspeople might better evoke Covello-like preferences than the distant, calm, friends-involving scenario used by Roth. The research reported here replicated the Roth study using the same scenario, risk comparison examples, and evaluation measures, and added a second scenario intended to replicate the conditions suggested by critics. Over 200 New Jersey residents answered the study questionnaire. The replication scenario reproduced Roth's results, and the conflict scenario also evoked no rankings correlated with Covello's predictions. Furthermore, neither agreement nor disagreement with five statements representing "conflict"--respondents' reports that the industrial-plant scenario made them angry, they lived near industry, they were concerned about industrial risks, people in their home town were angry about industrial pollution, and they worried "frequently" about long-term effects of pollution--correlated with Covello's predictions. Over half of all ratings ascribed to the comparisons in aggregate were positive, and most detailed comments offered by respondents also were positive, despite many criticisms and suggestions for their improvement. The wide variability in individuals' rankings also undermines the notion of any single ranking of preferred comparisons. These findings have implications for use of risk comparisons, but also reveal the inaccuracy of the field's assumptions about public reaction to industrial risk information, including risk comparison.     

Communicating Worst-Case Scenarios: Neighbors'' Views of Industrial Accident Management

The prospect of industrial accidents motivated the U.S. Congress to require in the Clean Air Act of 1990 that manufacturing facilities develop Risk Management Plans (RMP) to submit to the U.S. Environmental Protection Agency (USEPA) by July 1999. Industry worried that the requirement to communicate to the public a "worst-case scenario" would arouse unnecessary and counterproductive fears among industry neighbors. We report here the results of focus groups and surveys with such neighbors, focusing particularly upon their reactions to messages about a hypothetical worst-case scenario and management of these risks by industry, government, and other parties. Our findings confirmed our hypotheses that citizens would be skeptical of the competence and trustworthiness of these managers and that this stance would color their views of industrial-facility accident risks. People with job ties to industry or who saw industrial benefits to the community as exceeding its risks had more positive views of industrial risks, but still expressed great concern about the risk and doubt about accident management. Notwithstanding these reactions, overall respondents welcomed this and other related information, which they wanted their local industries to supply. Respondents were not more reassured by additional text describing management of accidents by government and industry. However, respondents did react very positively to the concept of community oversight to review plant safety. Claims about the firm's moral obligation or financial self-interest in preventing accidents were also received positively. Further research on innovative communication and management of accident risks is warranted by these results, even before recent terrorist attacks made this topic more salient.     

Do Reports on Drinking Water Quality Affect Customers' Concerns? Experiments in Report Content

The Safe Drinking Water Act Amendments of 1996 required U.S. utilities to report on drinking water quality to their customers annually, beginning in fall 1999, on the assumption that such reports would alert them to quality problems and perhaps mobilize pressure for improvement. A random sample of New Jersey customers read alternative versions of a water quality report, in an experiment on reactions to water quality information under U.S. Environmental Protection Agency (USEPA) rules. Experiment design was 2 x 3 + 1: two versions each--one with, one without, a violation of a health standard--of a report that was (1) Qualitative (without water quality numbers, thus not meeting USEPA rules); (2) Basic, with minimal information meeting the rules; or (3) Extended, adding reading aids and utility performance information; plus a control instrument without any hypothetical report. Results of ANOVA suggest the reports will have less effect than hoped or feared. These manipulations were successful: people reading the Qualitative versions were less likely to say that the report gave the amounts of substances found in the water, and those reading Violation versions were more likely to report a violation of a health standard. The main differences in responses to the report involved the judged adequacy of the information, and to a lesser extent responses on a Concern scale (constructed from measures of concern, judged risk, clean-up intentions, distrust of utility information, and doubt that the utility was doing all it could to improve water quality). Overall judgments of water quality and utility performance did not change, either relative to the controls or in before versus after responses. Qualitative reports performed worse than others, confirming the decision to have utilities report actual contaminant levels. Extended reports did only slightly better than the Basic versions on these measures. Many respondents had trouble identifying the presence or absence of substance amounts or violations, despite their seeming obviousness (e.g., in a "bottom line" summary on the front page of each report), suggesting many were not processing this information carefully. However, the pattern of responses for those who accurately identified the presence or absence of substance amounts or violations did not differ substantially from that for the group as a whole. Generic risk beliefs (serious local environmental problems; lack of control over risks to one's health) dominated demographic variables, attitudes toward utility water quality or trustworthiness, and the content and format of water quality reports in influencing concern about drinking water quality. Previous empirical and theoretical evidence for lack of change in public risk attitudes due to one-time or infrequent communications--e.g., role of personal experience, perseverance of prior trust or distrust--seems to be confirmed for annual water quality reports.     

On the Measurements of Board Composition: Poor Consistency and a Serious Mismatch of Theory and Operationalization

Over two dozen operationalizations of board composition can be identified from the empirical literature. A structural equations confirmatory factor analysis (LISREL 8.03) suggests that these operationalizations do not constitute a single construct of board independence. Instead, analyses strongly indicate three separate constructs. Common operationalizations of board composition, then, are neither tenable surrogates for one another nor are they interchangeable. Implications for empirical aggregation of studies, theory/measurement convergence, and the current corporate governance public policy debate are discussed.     

Building cultural competence: A systematic evaluation of the effectiveness of culturally sensitive interventions with ethnic minority youth

Growing diversity within the US population accentuates the need for the development and implementation of culturally competent services in the field of child welfare. Despite this, few studies exist that examine the effectiveness of culturally sensitive interventions (CSIs) with ethnic minority populations, who are considerably more at risk of developing psychosocial problems including substance and alcohol abuse, delinquency, low academic achievement, and poor self-esteem. With the demand for evidence-based practice it is imperative that we examine the effectiveness of CSIs designed to prevent and treat youth problems. The main goal of this review was to systematically analyze the methodology of recent CSI research with populations of ethnic minority youth in order to evaluate each intervention's ability to adhere to existing standards of evidence-based practice. The fifteen CSI studies examined in this review illustrate progress in their attempt to adhere to the methodological rigor required of evidence-based practice. This review encourages the field of child welfare to position itself analogously with and embrace the movement toward building cultural competence by further investigating the effectiveness of culturally-sensitive interventions with ethnic minority youth.     

Embedded cultural values and international transaction cost advantages

Culturally embedded differences in self-interested behavior and perceptions of trust violate the universal assumption of moral hazard held by transaction cost economic theorists. A model of transaction cost advantage is derived and propositions developed using the cultural value diversity literature to test the universal assumptions of transaction cost economics.