Measurement Issues in Human Behaviour Theory

Abstract

This paper provides a brief introduction to the use of standardized measurement scales in behavioural theory, discusses some of the limitations of the domain sampling model of measurement for behavioural scientists and practitioners, and identifies central measurement issues in behavioural theory that are germane to all measurement tools. The ever popular and useful behavioural checklist is used to introduce the domain sampling model of measurement and to describe how it can be treated as a standardized measurement tool.     

BIG DATA AND BIG CITIES: THE PROMISES AND LIMITATIONS OF IMPROVED MEASURES OF URBAN LIFE

New, “big data” sources allow measurement of city characteristics and outcome variables at higher collection frequencies and more granular geographic scales than ever before. However, big data will not solve large urban social science questions on its own. Big urban data has the most value for the study of cities when it allows measurement of the previously opaque, or when it can be coupled with exogenous shocks to people or place. We describe a number of new urban data sources and illustrate how they can be used to improve the study and function of cities. We first show how Google Street View images can be used to predict income in New York City, suggesting that similar imagery data can be used to map wealth and poverty in previously unmeasured areas of the developing world. We then discuss how survey techniques can be improved to better measure willingness to pay for urban amenities. Finally, we explain how Internet data is being used to improve the quality of city services. (JEL R1, C8, C18)     

Framing in Mass Communication Research – An Overview and Assessment

This article provides an overview of the use of the framing concept in mass communication research. It focuses on the questions what a frame is and how it is measured, how variation in framing can be explained and what the effects of media framing are. Specific attention will be paid to the sociological origins of framing. The article concludes with recommendations for future research. It argues that a more systematic and conceptually precise measurement of framing is warranted and suggests how the scope of frame‐building and framing‐effects studies can be extended.     

The use of performance measurement in South Carolina nonprofits

Performance measurement is touted by nonprofits as the newest method of ensuring accountability. Knowing the value of the process while having an appreciation for the downside with regard to volume of work can help organizations make more educated decisions about how deeply they want to get involved with performance or outcome measurement. This article provides background information on performance measurement and examples of how it is being used in nonprofit organizations. Next, it presents methods of an exploratory study conducted in South Carolina. Finally, a discussion of results focuses on the effect of outcome measurement on management and funding as well as the need for more in‐depth research.     

Performance measurement in child protection

Abstract

The increasing use of performance measurement by government means that child protection services are under pressure to demonstrate effectiveness in protecting children from harm and efficiency in the use of public funds to help children and families. From a policy perspective, the way performance measurement is conceptualised and implemented can have major consequences for service delivery. This paper examines key issues raised in the literature about performance measurement, the context for its introduction in child protection, how the concepts of effectiveness and efficiency are dealt with, how client outcomes are defined, and assumptions about ‘good performance’. An overview of performance measurement in child protection in Australia is provided. The paper argues that a critical approach to performance measurement in child protection can contribute to improved service delivery to clients.     

Functional measurement, individual preference and discrete-choice modelling: Theory and application

This paper outlines how theory developed in the study of individual judgement and choice behaviour can be implemented and tested using principles from the design and analysis of statistical experiments. The particular method, termed functional measurement is outlined briefly and then applied to a study of the potential for electric vehicles. The natural link with multi-attribute utility theory and the precise requirements for measurement assumptions make the approach appealing as a method for modelling individual's decision processes.     

The contribution of a cultural perspective in understanding and evaluating client satisfaction

Increasing attention is being given to including measures of client satisfaction as a component in the evaluation of health care services. An understanding of cultural factors that may influence attitudes and behaviors in the health care situation can contribute to the theory and measurement of client satisfaction. Through an examination of some of the cross-cultural literature, this paper delineates dimensions of the health care situation in which clients have been shown to have distinctive preferences, behavioral patterns, attitudes, and treatment expectations. A knowledge of the beliefs and attitudes concerning health and illness that are espoused by a particular cultural group can be a valuable asset in understanding how members of that group will evaluate the delivery of health care services. The examination of how cultural and ethnic differences may affect client satisfaction outcomes points to the necessity of avoiding a global measure of client satisfaction without first examining those particular aspects of health care delivery and of the measurement context that may have a predictable influence on client satisfaction ratings.     

The use of concept mapping in measurement development and evaluation: Application and future directions

The past decade has seen an increase of measurement development research in social and health sciences that featured the use of concept mapping as a core technique. The purpose, application, and utility of concept mapping have varied across this emerging literature. Despite the variety of uses and range of outputs, little has been done to critically review how researchers have approached the application of concept mapping in the measurement development and evaluation process. This article focuses on a review of the current state of practice regarding the use of concept mapping as methodological tool in this process. We systematically reviewed 23 scale or measure development and evaluation studies, and detail the application of concept mapping in the context of traditional measurement development and psychometric testing processes. Although several limitations surfaced, we found several strengths in the contemporary application of the method. We determined concept mapping provides (a) a solid method for establishing content validity, (b) facilitates researcher decision-making, (c) insight into target population perspectives that are integrated a priori, and (d) a foundation for analytical and interpretative choices. Based on these results, we outline how concept mapping can be situated in the measurement development and evaluation processes for new instrumentation.     

Measuring quality for human service improvement: How nonprofits meet the quality requirements of public authorities

The growing pressure for service quality has led to an increase in the dissemination of quality models in nonprofit human service organizations. In spite of this, little is known about their implementation. The present study therefore examines how quality management directives imposed by public authorities affect the adoption and use of quality measurement systems under different sets of conditions. Key findings, based on survey data from 536 human service nonprofits in Switzerland, suggest that external quality requirements foster the adoption of measurement systems to the greatest degree, but simultaneously reduce their actual utilization for service improvement. The strength of these effects is contingent on the organizations' resources and the quality of indicators. Managers' commitment to quality measurement shows the strongest effect on the use of quality measurement systems. These findings and the implications for future research and practice will be discussed.     

Measuring facial movement

A procedure has been developed for measuring visibly different facial movements. The Facial Action Code was derived from an analysis of the anatomical basis of facial movement. The method can be used to describe any facial movement (observed in photographs, motion picture film or videotape) in terms of anatomically based action units. The development of the method is explained, contrasting it to other methods of measuring facial behavior. An example of how facial behavior is measured is provided, and ideas about research applications are discussed.The research reported here was supported by a grant from NIMH, MH 167845. The authors are grateful to Wade Seaford, Dickinson College, for encouraging us to build our measurement system on the basis of specific muscular action. He convinced us that it would allow more precision, and that learning the anatomy would not be an overwhelming obstacle. Neither he nor we realized, however, how detailed and elaborate this undertaking would be. Seaford (1976) recently advanced some of the arguments we have made here about the value of an anatomically based measurement system. We are grateful also to those who first learned FAC and gave us many helpful suggestions as to how to improve the manual. We thank Linda Camras, Joe Hager, Harriet Oster, and Maureen O'Sullivan also for their comments on this report.     

Give the Emperor a Mirror: Toward Developing a Stakeholder Measurement of Organizational Transparency

The concept of organizational transparency has vaulted to prominence in recent years. While the virtues of transparency have been tied to trust and credibility, there have been no efforts to measure this abstract concept. Guidelines exist to help organizations be more transparent, but the real test is how stakeholders perceive an organization's transparency. This paper focuses on developing a stakeholder measurement of organizational transparency. Factor analyses, structural equation models, and reliability alphas on the measurement items indicate the instrument measures three transparency reputation traits (integrity, respect for others, openness) and four transparency efforts (participation, substantial information, accountability, and secrecy).     

Understanding Symbolic Boundaries and Improving Quantitative Analysis of Social Exclusion by Improving the Operationalization of Boundary Work

In survey research, how can we measure actors' judgments concerning who is a member of a particular social group and who is not? How can we measure changes in this process over time? Within the sociology of culture, the operationalization of boundary work, or decisions of group membership, has been relatively overlooked. Similarly, measurement of changes in boundaries has been relatively overlooked as well. An important first step is to better match the operationalization of boundary work with theory. To do so, I argue that survey items concerning decisions of social exclusion measured in the Likert scale can be used. In addition to allowing scholars to compare those who exclude with those who include (often measured as Strongly Agree/Agree vs. Disagree/Strongly Disagree), the Likert scale offers a measurement of boundary strength: how strongly respondents are committed to their exclusionary judgments (Agree vs. Strongly Agree, Disagree vs. Strongly Disagree). A novel element of boundary change can then be measured as variation over time in commitment to exclusionary judgment in time series data. A boundary that has increased in strength would be one for which respondents increase in their emotional commitment to their exclusionary judgment over time.     

Measuring Constructs in Family Science: How Can Item Response Theory Improve Precision and Validity?

This article provides family scientists with an understanding of contemporary measurement perspectives and the ways in which item response theory (IRT) can be used to develop measures with desired evidence of precision and validity for research uses. The article offers a nontechnical introduction to some key features of IRT, including its orientation toward locating items along an underlying dimension and toward estimating precision of measurement for persons with different levels of that same construct. It also offers a didactic example of how the approach can be used to refine conceptualization and operationalization of constructs in the family sciences, using data from the National Longitudinal Survey of Youth 1979 (n = 2,732). Three basic models are considered: (a) the Rasch and (b) two‐parameter logistic models for dichotomous items and (c) the Rating Scale Model for multicategory items. Throughout, the author highlights the potential for researchers to elevate measurement to a level on par with theorizing and testing about relationships among constructs.     

Official statisticians as conceptual innovators

Statistics are impossible without concepts. As the sociologist of invention Colum Gilfillan put it in 1952, ‘counting begins with definition of the thing to be counted’. This article is concerned with statistics on science, technology and innovation (STI). It documents how official statisticians have, over time, defined the concepts used for measurement. Debates on definitions of the concepts measured started at the very beginning of STI measurement in the first half of the twentieth century. Then, from the early 1960s onward, methodological manuals were developed to conventionalize the definitions. This article claims that the manuals did not have the expected result. They did not stabilize the definitions and the statistics based thereon.     

Extending monetary values to broader performance and impact measures: Transportation applications and lessons for other fields

This article examines recent progress at assigning monetary values to what are normally considered “hard to quantify” benefits of transportation projects. It focuses on three types of impacts – environmental quality, health and wider economic impacts – to examine how transportation project evaluation methods have evolved in recent years and how they compare to methods used for evaluation of non-transportation programs. Examples of recent practice are provided to show how transport agencies are continuing to refine performance measures to include broader impacts in project evaluation. A classification is provided to distinguish direct traveler effects from indirect effects on non-travelers, a step important to maximize coverage and minimize double-counting of impacts. For each type of impact, the paper discusses the range of variation in monetized values and shows that the variation is due less to imprecision in measurement than to fundamental issues about whether to use damage compensation, impact avoidance costs, stated preferences or behavioral valuation perspectives to define those values. Case studies as diverse as Australian roads, Wisconsin energy programs and Appalachian economic development programs are used to show how common methods are evolving among transport and non-transport agencies to improve impact measurement and its use in project evaluation.     

Measuring Religious Identities in Surveys

The purpose of this article is to acquaint the broader publicopinion research audience with what has been a salient issuewithin the community of scholars of religion. We address thequestion of how best to conceptualize and measure religiousidentities in research on contemporary American society. Weconsider the main approaches to the measurement of religiousidentification with regard to their backgrounds, their assumptionsabout the importance of understanding religious identities inhistorically relevant terms, and the practical considerationsof survey measurement. Using data from the General Social Survey,particularly recent innovative efforts to obtain informationon subjective association with particular religious traditionsand/or movements (e.g., Pentecostal, fundamentalist, evangelical,mainline, or liberal Protestant), we compare the two main approaches:the traditional "denominational" approach, where religious identitiesare assumed to be associated with religious denominations, andthe subjective approach, where religious identities are assumedto be captured by a set of "nondenominational" reference categorieslinked to particular historical religious traditions or socialmovements. We conclude that both approaches have substantialpredictive validity, and the most effective strategy for futureresearch may be one that uses a combination of approaches, ratherthan one that relies entirely on a single method of measurement.     

A Collaborative Approach to Infant Research: Promoting Reproducibility,Best Practices,and Theory‐Building

The ideal of scientific progress is that we accumulate measurements and integrate these into theory, but recent discussion of replicability issues has cast doubt on whether psychological research conforms to this model. Developmental research—especially with infant participants—also has discipline‐specific replicability challenges, including small samples and limited measurement methods. Inspired by collaborative replication efforts in cognitive and social psychology, we describe a proposal for assessing and promoting replicability in infancy research: large‐scale, multi‐laboratory replication efforts aiming for a more precise understanding of key developmental phenomena. The ManyBabies project, our instantiation of this proposal, will not only help us estimate how robust and replicable these phenomena are, but also gain new theoretical insights into how they vary across ages, linguistic communities, and measurement methods. This project has the potential for a variety of positive outcomes, including less‐biased estimates of theoretically important effects, estimates of variability that can be used for later study planning, and a series of best‐practices blueprints for future infancy research.     

Moving toward recovery within clients' personal narratives: directions for a recovery-focused therapy

Recent literature emphasizes that recovery from schizophrenia involves recovery within one's own narrative of an integral sense of identity, agency, social connection, and worth. While this is intuitively appealing and consistent with a wide range of literature, it raises the issue of how to best help people do this in individual psychotherapy. In this article, we explore how psychotherapy might help people construct new narratives or storied understandings of their lives and thereby promote recovery from schizophrenia. Exemplified with two individual examples, we first discuss the barriers that challenge and the techniques that help psychotherapists seeking to enter into dialogue with people with severe mental illness. We also offer a theoretical model of how the revitalization of dialogues within therapy can be conceptualized as a process that promotes recovery and discuss the objective measurement of such outcomes.     

Inference for the measurement of poverty in the presence of a stochastic weighting variable

Empirical applications of poverty measurement often have to deal with a stochastic weighting variable such as household size. Within the framework of a bivariate distribution function defined over income and weight, I derive the limiting distributions of the decomposable poverty measures and of the ordinates of stochastic dominance curves. The poverty line is allowed to depend on the income distribution. It is shown how the results can be used to test hypotheses concerning changes in poverty. The inference procedures are briefly illustrated using Belgian data. An erratum to this article can be found at