Gender equality regimes and evaluation regimes in Europe and their implications for policy design and evaluation

PurposeThis article starts from the assumption that it is important for the evaluation of gender equality interventions in RTDI (Research, Technology Development and Innovation) to consider its context regarding gender equality regime and evaluation culture because this context does influence effects and long term impacts of such activities. It aims to provide key characteristics for the differentiation of gender equality regimes and evaluation regimes to be considered when designing policy interventions or evaluating specific gender equality interventions in RTDI.Design/methodology/approachAfter a literature review of relevant typologies for welfare state and gender equality regimes, it was analysed to which extent the seven EFFORTI countries correspond to certain typologies based on the data basis of the context research performed in the EFFORTI project. For this context research, international/national and qualitative/quantitative data regarding the relevant framework conditions were firstly collected for each of the EFFORTI countries and secondly compared in a cross-country analysis.FindingsThe research showed that when it comes to gender equality policies, most EFFORTI countries can either be assigned to the Social-Democratic category or Conservative Equal Employment Regime category in the typology of von Wahl (2005), with the latter type tending to provide less favourable conditions for women in the general labour market as well as women in RTDI (e.g. overtime culture). In how far these different types of context can have an impact on the evaluation of interventions can be exemplarily illustrated using case studies carried out as part of EFFORTI.Connecting the typologies for gender equality policy with evaluation regimes has not proved fruitful, as the two discourses have only begun to converge in recent years. The evaluation regimes and cultures of the respective countries have therefore been described independently. However, it was shown that countries with more expertise in certain areas (e.g. gender, evaluations) developed more routinized and institutionalised procedures in the respective field.Originality/valueConsidering the different framework conditions is relevant when it comes to evaluation as different national contexts might require different policy and designs of activities, but might also shape the interventions’ effects. This article therefore aims to provide support in this regard for future evaluations of gender policies.     

Women Underrepresentation in R&I: A Sector Program Assessment of the Contribution of Gender Equality Policies in Research and Innovation

Despite decades of efforts to achieve gender equality in research and innovation (R&I), all EU member states still face remarkable difficulties in driving forward the development of their innovation system while at the same time improving gender equality by using all the available research potential. In this paper we focus on the development of the share of women researchers in four national innovation systems, i.e. in Austria, Denmark, Hungary and Spain in the time period 2005-2015. The four selected cases represent countries with significant differences in their innovation capacity, gender regimes and progress of gender equality in R&I. A qualitative comparative analysis (QCA) is carried out to conduct a sector program evaluation based on statistical data and qualitative studies to understand the dynamic development of the proportion of women researchers. The study aims to provide insights into the aggregated gender equality interventions and policies implemented in the four countries studied and their contributions to the development of the proportion of women scientists at the structural level. The analysis reveals that the development of the share of women researchers during the studied period has been particularly influenced by contextual factors, namely the relative size of the business enterprise sector and the share of women among holders of tertiary education. While this is the case, it is found that gender equality interventions need to be more widespread and more effectively designed to be a strong contributing factor to an increasing representation of women in R&I.     

Analysing facilitating and hindering factors for implementing gender equality interventions in R&I: Structures and processes

This article analyses the facilitating and hindering factors that have affected the implementation of gender equality interventions in research and innovation in Europe. It applies the evaluation framework developed in the EFFORTI project that recognizes the complexity of evaluating gender equality interventions in R&I, the importance of factoring in context to any sound evaluation as well as the need to distinguish between the design and implementation of interventions in evaluations. It is based on the analysis of 19 empirical case studies carried out throughout Europe and focuses on those structural and procedural factors that have either facilitated or hindered the implementation process of these interventions. Findings include how the governance framework; top-management commitment; bottom-up participation; framing synergies with other initiatives, strategies for tackling resistance; resources; sustainability of actions; gender competence, experience and knowledge and transparency, targets, standards and monitoring; and accessible data and information all contributed to the successful implementation of the interventions.     

Pragmatic ex-ante evaluation using an innovative conceptual framework: The case of a high-tech entrepreneurship program for women

Program designers at local project level usually design interventions under conditions of scarce resources and bounded rationality lacking sufficient information about the process and impact mechanisms of the complex social programs they design and implement. This paper proposes a model for pragmatic ex ante evaluation using an innovative conceptual framework for practitioners working in the field to improve gender equality in research and innovation.The presented and applied model is pertinent to the theory-oriented evaluation tradition using a logic frame and a theory of change approach adapted to the special requirements of ex-ante evaluation. The model is illustrated by the case study of a program for the promotion of women entrepreneurship in the STEM fields.It is concluded that in a setting with limited resources and information, ex-ante evaluations should aim at defining clear conditions of proportionality and need to take into account the extent to which contextual factors hinder or facilitate the implementation of a program, and the specific institutional and system mechanisms at play, in order to enable proactive risk management from the beginning.     

Integrating the gender dimension in teaching,research content & knowledge and technology transfer: Validating the EFFORTI evaluation framework through three case studies in Europe

Gender equality and gender mainstreaming in research is one of the six European Research Area (ERA) priorities. Integrating the gender dimension in research content and teaching is one of its three objectives. It is arguably the objective where least progress has been made. In this article we contribute to the evidence base by applying the EFFORTI evaluation framework to three empirical case study interventions that aim to integrate the gender dimension in tertiary education and research content. Comparison is based on an evaluation of the design of the intervention, those factors that have enabled/ hindered its implementation as well as an assessment of outcomes and impacts. The findings of the case studies highlight the importance of design, specifically regarding resources, legal status and the definition and operationalisation of the gender concept. Implementation hinges on top-level institutional commitment and mainstreaming gender studies with support of a central unit and crucially gender competence. A lack of recognition and status of gender studies and subsequent innovations was seen to hamper implementation. Outcomes and impacts included an increased awareness and interest in gender, increased gender competence, a push towards gender equality regarding representation and organisational change as well as an improved accreditation process and more and better research.     

Exploring linkages between organisational culture and gender equality work—An ethnography of a multinational engineering company

This article explores linkages between organisation-specific cultural narratives and gender-equality programme planning through the lens of the ‘historicity’ concept. The article argues that to fully understand problem definitions, programme design and organisational change processes related to gender equality, scholars and practitioners cannot focus one-sidedly on expected outcomes and effects; we must also factor in cultural narratives, because gender equality actors never arrive at their work as ‘tabulae rasae’. A community of actors always draws on shared dispositions that give sense, direction and shape to their anticipations of the future hereby guiding their actions in the present. Based on an ethnography of a multinational engineering company, the article shows how cultural narratives may serve in different ways as support factors for gender equality programme planning and implementation, if they are actively but mindfully engaged. This mindfulness is important as positive cultural narratives may entail problematic gender dimensions. On the other hand, negative cultural narratives may entail important learning outcomes that may benefit future gender equality initiatives. The analysis further points to the centrality of strategic communication, leadership commitment and comprehensive evaluation in order to mobilise the potential of cultural narratives as support factors to gender equality work. Finally, this article offers a rich example to scholars and practitioners of how to employ cultural analysis in relation to gender equality activities, and demonstrates the value of the insights produced by this analysis for the case company and its gender equality programme.     

Network assessment: Design of a framework and indicators for monitoring and self-assessment of a customized gender equality plan in the Mediterranean Engineering Education context

With more than 28 active members, the Mediterranean Engineering Schools Network (RMEI) is the most active network on gender equality in the Mediterranean area. Supported by the HORIZON2020 TARGET project ‘Taking a Reflexive Approach to Gender Equality for Institutional Transformation’, in conceptualizing gender equality learning and system change, the network took a context-specific approach based on a theory of change and other STEM organizational frameworks, to design a self-assessment framework and indicators, considering the systemic view of SDG5 horizontally connected to all SDGs, national context complexity, and transdisciplinary requirements. 95 simple, practical, user-friendly indicators were designed, fitting in the specificities of the Mediterranean-Engineering context. The analysis showed that the network realized its vision, made the ‘passage’ from theory to praxis on gender equality change, effectively developed meaningful processes/structures, formulated a policy statement, built a community of practice and inspired members. It also achieved trustful relationships and inspired outputs, effective communication, sharing of information and resources, and top management commitment. Critical aspects are a) the analysis in depth of issues linked to the existence of gender-based stereotypes and bias in engineering schools of the Mediterranean that entails tackling gender ideologies considering the whole national social system and existing structures; b) sustainability of gender equality structures created at the member institutions with the support of TARGET project which is depending on the willingness of institution’s leaders to continue/advance with gender-sensitive strategies in their institution.     

Developing a public health policy-research nexus: An evaluation of Nurse Practitioner models in aged care

A frustration often expressed by researchers and policy-makers in public health is an apparent mismatch between respective priorities and expectations for research. Academics bemoan an oversimplification of their work, a reticence for independent critique and the constant pressure to pursue evaluation funding. Meanwhile, policy-makers look for research reports written in plain language with clear application, which are attuned to current policy settings and produced quickly. In a context where there are calls in western nations for evidence based policy with stronger links to academic research, such a mismatch can present significant challenges to policy program evaluation. The purpose of this paper is to present one attempt to overcome these challenges. Specifically, the paper describes the development of a conceptual framework for a large-scale, multifaceted evaluation of an Australian Government health initiative to expand Nurse Practitioner models of practice in aged care service delivery. In doing so, the paper provides a brief review of key points for the facilitation of a strong research-policy nexus in public health evaluations, as well as describes how this particular evaluation embodies these key points. As such, the paper presents an evaluation approach which may be adopted and adapted by others undertaking public health policy program evaluations.     

Evaluation considerations for community-based gender-informed health interventions

Evaluations of gender-based interventions have been consistently criticized for their lack of methodological rigor. This is largely due to the complex design of many of the interventions, coupled with difficulties in measuring the outcome and impact of these interventions. This article proposes a number of ways to improve these evaluations both at the community and individual level. We recommend use of organizational theory and narrative inquiry methods, such as the appreciative inquiry technique, to examine how communities design gender-based interventions. In addition, we suggest a variety of methods to measure the effects of these interventions on gender norms in the community for example, policy analysis, multilevel modeling, and social conversations. With respect to measuring outcomes at the individual level, we argue for more rigorous evaluation designs in order to improve internal and external validity claims. Additionally, we suggest that evaluations should incorporate different methodologies, for example autobiographical narratives, which allows one to give saliency to the subjective voices of participants. Finally, we emphasize that evaluation designs need to document the long term effects of intervention programs and define the expected outcomes with greater specificity.     

“Not everything that counts can be counted”: Assessing ‘success’ of EU external migration policy

In the context of the ongoing “migration crisis” the externalization of EU migration policy has continued. EU policy documents argue that cooperation with non‐EU countries is essential in order to manage migration flows. But how successful is this policy? The public policy literature teaches us that “policy success” can be defined in varying ways: as goal achievement; as political success; in terms of norms; in terms of the costs associated with the policy; temporally; and in the light of external factors. An application of this analytical framework to the EU Mobility Partnerships uncovers conceptual and methodological challenges, and above all highlights the need for evaluation of EU external migration policy to be taken more seriously.     

Gender mainstreaming and evaluation in the EU: comparative perspectives from feminist institutionalism

This article presents a focused comparative analysis of the institutionalization of two governance practices in the European Commission that levy distinct challenges to the gender status quo: gender mainstreaming (which overtly challenges gender bias) and evaluation (which does not have explicit feminist aspirations). With reference to five dimensions, we identify evaluation as relatively strongly institutionalized, and gender mainstreaming as relatively weakly institutionalized. We draw on the explanatory power of feminist institutionalism to unpack these findings, arguing that a feminist institutional perspective can shed light on this variation, as it provides greater insight into the formal and informal institutions that constrain, enable and shape the implementation of evaluation and gender mainstreaming. We assert that the notion of path dependency, the logic of appropriateness, and the concept of layering serve as useful tools to understand the gendered nature of the European Commission. This research provides insights into the institutional factors that impact the implementation of gender equality strategies (such as gender mainstreaming). In turn, this contributes to the development of more effective strategies to promote institutional change toward greater equality.     

TOOLS AND TOYS : Communicating gendered positions towards technology

With the rising importance of technology in the information and knowledge society, the gender-technology relationship is ever more important when thinking about gender equality. Gender researchers have shown not only that the use and design of technologies is gendered, but that people also position themselves in relation to technology, based on certain gendered assumptions about technology in societies. This article looks at how people working in quintessential information and knowledge society professions, namely information communication technology (ICT) work, position themselves in relation to technology. Using a social constructivist framework and a discourse analysis, it shows how gender differences are achieved in communication: men tend to describe technology as a toy, while women tend to describe technology as a tool. In some instances this pattern is broken, which opens up the opportunity to rethink the gender binary. This article argues that the way in which people position themselves in relation to technology continues to be gendered, which may threaten gender equality in the information and knowledge society, and it also indicates that there is the possibility of change.     

How organizational research can avoid the pitfalls of a co-optation perspective: analyzing gender equality work in Austrian universities with organizational institutionalism

The concept of co-optation offers vocabulary to discuss how concerns and demands of feminist movements are transformed on their way to, and within, mainstream organizations and policymaking. However, applications of this concept can have problematic implications, failing to grasp the complexity of social change efforts and contributing to divisions, rather than alliances, between different groups that work and fight for gender equality. This article argues that conceptual tools from organizational institutionalism can help to avoid these pitfalls by capturing the ambivalence of organizational change initiatives, and allowing us to identify not only counterintentional effects, but also subtle and unexpected opportunities of organizational gender equality work. I illustrate my arguments with empirical examples from research on gender equality work in Austrian universities.     

Seeing the system: Systemic gender knowledge to support transformational change towards gender equality in science

Scholars agree that gender inequality is systemic and that participants in gender equality interventions need knowledge on gender inequality processes. However, a detailed view on the specific characteristics of this knowledge is as yet missing. This article aims to contribute to gender equality interventions by conceptualizing and visualizing systemic gender knowledge as an important condition for transformational change. Combining gender and participatory system dynamics literature, this article first introduces the concept of systemic gender knowledge. This concept captures two main characteristics that make gender knowledge systemic: knowledge on the interaction of gender inequality processes and endogenous thinking, here implying a focus on the organization as the relevant level of analysis. In addition to this conceptual contribution, the research contributes methodologically to the gender inequality intervention literature by designing a visualization process, translating written texts into system dynamics models which enable exploration of systemic gender knowledge. Finally, the research contributes empirically by exploring the systemic gender knowledge of participants in two science research institutes of a Dutch university, finding shifts in both characteristics of systemic gender knowledge. This enables researchers to discern whether gender equality interventions lead to increases in systemic gender knowledge, thus supporting transformational change.     

Informing family policies: The uses of social research

Like the topic of family policy itself, research informing family policies is difficult to characterize. This article discusses how ideology and values influence research agendas and then describes three types of studies informing family policies: research defining social issues, evaluation research, and research about the policy-making process. Two case studies illustrate how social research informs family policy: in promoting gender equality in Scandinavia and in reforming child support in the United States. Values of individualism and the sanctity of the family have traditionally focused policy makers' and, hence, researchers' attention on individuals, not families, as the units of analysis. But dramatic shifts in family structure and functioning along with renewed public concern about family disintegration are placing families high on the policy agenda. Both “basic” and “applied” family scholars can contribute to a research agenda examining the factors promoting strong, effective families. She conducts research on gender and the life course, as well as on aging, families, and social policy. She received her Ph.D. in Sociology from the University of Minnesota. Her research interests in stratification, social policy, gender, and the life course include sex segregation in occupations, fertility, and work decisions and family policy.     

The practice and perceptions of RRI—A gender perspective

Little is known to date about the practice and perceptions of RRI among researchers in Europe as well as the integration of the gender dimension into everyday RRI practices. This lack was addressed by two large-scale surveys that were launched in the course of the EU-funded MoRRI project (Monitoring the evolution and benefits of RRI, Contract number RTD-B6-PP-00964-2013, Duration 09/2013–03/2018). The analysis shows that the institutional environment positively influences the degree of RRI activities and the general attitudes towards more responsible research and innovation: researchers working in an institutional environment that systematically supports the practice of RRI are more active in RRI practices than researchers who do not rely on such structures. For the gender equality dimension, this means that institutions with a gender equality plan (GEP) in place are more inclined to support female researchers than institutions without such institutional incentives. Furthermore, researchers with experiences in EU-funded projects are more likely to be engaged in RRI activities. Even if female researchers have a stronger inclination to engage with society than their male counterparts, gender competence proves to be the relevant distinguishing criterion. Gender competent researchers are more often involved in other RRI activities.     

CARTOGRAPHY OF GENDER EQUALITY PROJECTS IN ICT: Liberal equality from the perspective of situated equality

This paper examines gender equality activities in the context of information and communication technology (ICT), traces the social and cultural relations that intertwine with them and discusses the understandings of gender, equality and ICT maintained in them. The aim of the paper is to analyse how liberal equal treatment actions prevail in ICT, although it is well known that liberal politics alone do not succeed in promoting gender equality, and not even in fulfilling its own goal of raising the proportions of women in technology. The study is based on oral history interviews with 30 women who have committed important parts of their lives to gender equality activities through several decades, as well as follow-up studies of women's ICT groups that aimed to promote equality in ICT expertise, both of these studies being conducted in Finland. The interviewed gender equality workers are competent promoters of gender and equality. However, they need to negotiate their aims, for example, in order to get funding, within national and transnational institutional practices, with actors who have little knowledge regarding the social construction of gender, equality or ICT. The managerial terms 'efficiency' and 'good practice' then take over the understandings of the gender equality activities in ICT, mainly organized as projects, and further emphasize the measurable goals often linked to liberal gender equality actions. These terms have material consequences and while gender equality projects continue to provide possibilities for unexpected changes, they are locked within liberal politics.     

Generalised and Particularistic Thinking in Policy Analysis and Practice: The Case of Governance Reform in South Africa

This article is concerned with the relationship between generalised and particularistic knowledge in the context of policy‐making and policy analysis. It argues that it is problematic to assume that a reform model will generate similar outcomes across a wide variety of contexts. It presents a conceptual framework, including the concepts of transaction domain and domain consensus, that enables context‐sensitive analyses. The argument is exemplified by South Africa's introduction in the 1990s of an Integrated Development Planning model, based on British reform experience and various international public‐management models. With a case study of such planning in Lukhanji Municipality in the Eastern Cape Province, it illustrates how the conceptual framework may be used in policy research and analysis.     

Bringing about social justice through feminist research for monitoring,evaluation, and learning? A conversation from Oxfam GB

ABSTRACT

At Oxfam, we have spent much of the last two years talking about feminist principles. In the context of the #Metoo and #Aidtoo movements, the presence (or absence) of these feminist values and principles is increasingly – and rightly – under the spotlight. In this article, we reflect on what it takes to embed these values in our research practice within the monitoring, evaluation, and learning (MEL) function of an international non-government organisation such as Oxfam. These values shape our understanding of the purpose of carrying out MEL activities as a tool for bringing about social justice, through examining gender and power relations, why they exist and how they change. We reflect on the successes and challenges we have faced while utilising evaluation for greater gender equality. By doing so, we aim to start shedding light on the structural changes needed to bring about social changes through MEL, and research more broadly.     

Macroeconomic policy tools to finance gender equality

Feminist economists and heterodox macroeconomists have contributed substantively to the body of research that explores the distributional effects of macro policies. This work explicitly addresses the livelihood problems created by neoliberalism and, in addition, it provides a pathway for identifying financing mechanisms. Building on earlier work by Seguino and Grown (2006), this article synthesizes and elaborates the major contributions of this body of gender and macro research and, from this, extrapolates macro‐level policies and tools that support gender equality. Among the tools identified is targeted government spending on physical and social infrastructure, the latter a relatively new conceptual tool that is discussed in detail. A key argument is that financing for gender equality that raises economy‐wide productivity can be self‐sustaining. As a result, both physical and social infrastructure spending have the ability to create fiscal space. This possibility offers a financing framework for gender equality expenditures. A contribution of this article is to critique mainstream monetary policies and identify alternative approaches that expand the toolkit to achieve gender equality goals.