MEASURING THE SUCCESS OF D&I: Effect – Progress – Added value concepts and methods at a glance

In an economic environment it seems obvious to measure the success of initiatives – after all, resources flow into the respective activities. Meanwhile, how success should show itself remains controversial. In the context of the women’s quota, the fulfilment of these goals is already regarded by some as the achievement of goals and thus as a success. Others insist that Diversity & Inclusion – like other initiatives – must deliver measurable added value, i.e. ROI. Programme managers and project managers regard themselves as successful when their measures have had the desired effect. It seems as if incompatible different points of view collide – they merely represent different perspectives and elements of a comprehensive measurement of the success of diversity management.

The special nature of Diversity & Inclusion requires that a success measurement must cover different levels and perspectives if it is to do justice to the holistic nature of the concept and relevant contexts. An essential distinction must be made between

  • Measures explicitly implemented for the implementation of D&I and
  • processes, in which D&I is embedded and which show a certain performance under this aspect.

For specific measures, impact measurement (a way of achieving objectives) and value added measurement (cost-benefit calculation) are the main options. In processes, it makes sense to monitor how diversity is taken into account (proportionality). D&I can also improve the quality of some processes – but this adjustment would then be considered a measure and the process would be monitored in its new form. An added value measurement would also be conceivable for the processes, but is not carried out in practice due to permanent use (they are not available). In combination, the specific D&I measures and the processes mainstreamed with D&I are intended to achieve a target image that is ideally described by quantitative and qualitative criteria. The improvement of corresponding indicators forms the overarching performance measurement of diversity (degree of target achievement).

In the beginning was the deed

Many Diversity & Inclusion initiatives began with individual activities: Events, mentoring, networks, later workshops and interactive forms of communication. Experienced programme managers knew from the outset how important it would be to define clear objectives for each measure and to evaluate their achievement afterwards. This original form of performance measurement is called impact measurement and remains relevant for D&I in all areas, because regardless of complex score cards or cost-benefit calculations, the immediate evaluation of an activity forms essential information that can be used for reporting or communication as well as for management or adjustments.

The evaluation of the Gender Diversity Network (GDN) at Infineon is a particularly sophisticated example for measuring the impact of an activity. The company had an external evaluator evaluate not only the internal success of the network (from the participants’ point of view), but also its impact on the relevant environment and the (work) results. Since all measurements showed positive changes, the GDN is considered an all-round success.

A large financial institution presents the success of its corporate communication on diversity twice a year in a comprehensive internal report. It contains information on the reach and readership of various reports as well as changes from previous periods. This may reflect the general ambition for growth in the economy.

Numerous companies evaluate their mentoring or secondment programs – almost as a matter of course – and a well-founded evaluation is part of the professional implementation of the measures at training courses or workshops. A pharmaceutical company has developed a set of impact measurement instruments that covers several strategic fields of action and is being used successively. Other companies, on the other hand, said they deliberately chose not to take detailed measurements, as they are not seeking justification, but commitment out of conviction. From practical work it is also known that many programs are simply too small to be worth a complex evaluation. 70% of respondents in a Roland Berger survey stated that they did not measure the success of their diversity measures.

Nevertheless, the diversity measures as a whole pursue holistic objectives, most of which are of a quantitative and in some cases qualitative nature. They usually refer to an appreciative corporate culture as well as to inclusive behaviours and processes. As a result, companies expect measurable changes in the composition of their workforce, especially at management levels.

Measuring what really matters

Developing the corporate culture is a slow and potentially lengthy process. This makes it all the more obvious to undertake regular measurements. The integration of diversity-specific questions into the general employee survey has long been regarded as the best way to obtain meaningful data. However, only a few companies succeed in including an appropriate number of valid questions in the canon, which is usually already full to bursting. US conglomerates began to include concrete items on diversity or equality in their extensive, mostly annual surveys as early as the 1980s; this is known, among others, from Ford or Johnson & Johnson. IKEA is currently using ten questions on inclusion in its employee survey. Many are therefore still able to evaluate long-term data, even for smaller organizational units. They are sometimes used to agree on objectives for managers – usually concrete improvements of certain survey values. German companies also cover diversity in their employee surveys and sometimes also have the opportunity to compare their results with other companies that use the same tools. These benchmarking and targeting approaches show that measuring corporate culture through employee perception is a quantitative approach. A qualitative measurement of cultural change is carried out at Daimler: For example, seminars for newly appointed team leaders observe how the discussion on diversity management is changing.

The following article provides an overview of other forms of employee surveys as an instrument for measuring diversity success.

The most common method of measuring the success of diversity is KPIs, most of which originate from personnel controlling. Most companies differentiate between process-related KPIs along the HR value chain and overarching objectives for diversity in the workforce or at specific management levels.

KPIs: Continuous improvement of process results

A large financial institution developed a range of ten key figures, seven of which relate to the topic of gender, as part of a project entitled “Women in management positions”. They are regularly communicated internally for the entire company and are also used at divisional level for setting objectives. Infineon uses five KPIs for gender and also uses a holistic cockpit that prevents the sole focus on a few indicators. Various large corporations use KPIs relating to human resources processes to pursue overarching objectives on gender and internationality in senior management. The gender mix is often recorded in talent pools, AC nominations, shortlists, interview panels or career planning. This approach is valid and relevant in several respects: It shows to what extent established processes in the company take existing diversity into account (or, in an unfavourable case, wear out unintentionally). The KPIs also show how effectively diversity management is anchored in the respective processes and whether D&I has the desired effect in the respective context and thus leads to positive effects (a form of success). These forms of results in turn provide occasions for communication. Daimler publishes an internal fact sheet once a year covering all key areas of activity, including parental leave, sabbaticals, stays abroad and generation management.

The selection and use of such KPIs for diversity management shows how strong the focus currently is on personnel processes on the one hand and on the topic of gender (sometimes also internationality) on the other. Long-term observations make it clear that this is neither a matter of course nor has it always been the case. Just a few years ago, international companies used broad indicators to make their diversity progress visible – including innovation indices, customer feedback and other figures not derived from human resources. IKEA combines these aspects with a formula that is as simple as it is consistent: The furniture stores want their staff to reflect the respective local and national markets, and the diversity in management positions should in turn reflect the workforce mix.

The high level of attention (and increasing regulation) that the topic of “women in management positions” has received in recent years is likely to be the main reason for this development. On the other hand, the differentiated range of KPIs covering the entire HR value chain and several hierarchy levels show that the discussion about women on boards of directors and supervisory boards – contrary to frequent reproaches – can also have a positive (indirect) effect on young women.

Overarching objectives are not quotas – but they function that way

Even before the debate about legal quotas or target setting obligations, CEOs who came into their positions repeatedly wanted to make a positive statement for diversity by formulating expectations: “I want 20% women in senior management” or “We are too male and too German”. Once again, the well-intentioned may be the opposite of well-made. For often these statements did not fall into a prepared context and – much more decisively – the formulations conveyed an unwanted message: The proclaimed representation figure acts as a goal in the sense of an end in itself. The stronger the expectation is formulated, the clearer the aftertaste of the threat of discrimination against unnamed persons (mostly German men) becomes.

The fact that diversity, on the other hand, is a means of achieving more far-reaching goals such as market success and innovation is overshadowed by these messages. This also explains the extensive focus on downstream KPIs exclusively from the HR context and there with the focus on “career development”. There is also a real problem with the interpretation of indicators: If there is the impression that certain proportions of women are final targets, harsh criticism of individual figures and of every decline is inevitable. A drastic example of such interpretations that neglect the context and the overall picture is described in this article (German).

In cases where, on the other hand, it was possible to position higher-level measures as indicators of diversity success – and NOT as goals or an end in itself – managers and employees showed great sympathy and support for the concepts. They find it easier to recognize that certain processes or behaviors are not optimal unless the sword of Damocles hovers above them. This and other negative side effects came with the legal quota debate, which is also being continued in the media and is sometimes astonishing. The television flap at the German talk show “Hart aber fair” was only the tip of the iceberg, as this little press review shows (German).

Nevertheless, almost all the squabblers and hens agree on one point. A mixed workforce, especially at management level, brings a number of economic advantages for the company. Quantifying these is another way of measuring the success of diversity.

Measurable added values as a reward – or as a burden of proof

What improvements diversity management brings with it and how these are to be quantified – these questions on the business case or specifically on the return on investment have been the focus of diversity experts and practitioners for more than 15 years. Not surprisingly, many answers were found during this period. There are now more than 190 reliable studies (no surveys!) showing which (positive) effects Diversity & Inclusion can have – some of them also describe specific conditions under which this is the case. The advantages that companies can achieve in the areas of productivity, cooperation, market development, image and finally performance appear to be largely transparent.

However, there are open and sometimes harsh-critical questions about the added value of concrete measures, especially if these are linked to financial and other investments. This question is the continuation of the previously described impact measurement of D&I activities. While the media presence (from the above example) of a company on the subject of diversity can be easily quantified, it seems somewhat more complex to state the positive effect of networks or mentoring programmes in euro benefits. This is often done through staff turnover costs, which are saved if employees – especially identified talents – are retained by the company. Quantifications can also be carried out via increased satisfaction and thus increasing productivity, which are then compared with the costs involved.

One of the most comprehensive examples of ROI calculations was carried out as part of a series of studies on childcare (Prognos AG: 2003, 2004, 2005) which showed that the business benefits are 25% higher than the costs. Similar calculations have been made for companies that use diversity in the marketing environment – here success considerations are as common as they are simple. It is therefore all the more surprising that in a survey conducted by Roland Berger (2012) 53% of respondents said they had no interest in measuring the financial success of their measures or had no opinion on them. The following impression may not fit this figure at all: Above all, employee networks but also diversity officers are often invited by management to submit their proposals together with a business case. This requirement seems to be skewed, especially if the respective positions do not receive adequate basic equipment to be able to set up a business case professionally. This aspect leads to the question that should be clarified at the outset: What are meaningful, relevant and feasible components of a company-specific performance measurement of diversity?

Clarity for measuring the success of diversity

It became clear from the contexts and examples: The sole focus on the proportion of women in management positions does not represent an appropriate approach to measuring the success of diversity management. These and other representation figures nevertheless show whether many D&I – but even more so HR – activities were carried out effectively and thus successfully. Monitoring a clear range of process outcomes – from employer branding data to candidate and hiring statistics to retention programs and promotions – is an indispensable tool for the comprehensive management of diversity management. Nevertheless, these points are generic personnel management tasks and fall within the HR area of responsibility. Other indicators that are influenced by diversity management measures, such as innovation, customer satisfaction, social responsibility or above all employee commitment, can be used as part of a specific performance measurement of D&I. Finally, the professional implementation of D&I suggests evaluating all activities both in terms of their impact and added value.