ISBN 92-2-111061-3
ISSN 1020-5322
First published 1998

Download the document in Adobe Acrobat's Portable Document Format (PDF)

Table of Contents

Foreword

Gek-Boo Ng
Chief
Employment and Labour Market Policies Branch

1. INTRODUCTION

1.1 Objectives of the study

• which different policy approaches and strategies exist in a range of countries to tackle the problem of long-term unemployment;



• which methodologies and techniques have been used to evaluate the different policy measures;



• what the evaluation literature tells us about the overall effects of measures of different types, and in different combinations on the labour market chances of the target group (the long-term unemployed); and



• what conclusions can be drawn about the strengths and weaknesses of different evaluation approaches, and what proposals can be put forward for developing effective target-oriented approaches to answer the central question, 'how does active labour market policy affect the labour market chances of the long-term unemployed?'.

1.2 Target-oriented versus programme-oriented evaluation

'Traditionally, labour market policy evaluation has been characterized by a strong focus on individual policy instruments and programmes. The impacts of these single purpose programmes have been assessed in a top-down approach by contrasting explicit programme goals with measurable programme outcomes. In these conventional approaches, policy formation and implementation processes have been treated largely as a black box.
... In this way programme evaluations have tended to neglect the incentives created by the interaction of different policy interventions and the cumulative impact of policy regimes on the disposition and observable behaviour of the relevant actors in the target area....
...To overcome the shortcomings of the programme-oriented evaluation approach, we propose a new approach: 'target-oriented policy evaluation' ....Instead of merely attempting to measure the impacts of a certain policy programme along the dimensions of its immediate pre-specified programme objectives, the target-oriented approach takes as its point of departure broadly defined policy goals or targets. Examples of such policy targets are increasing individual options for labour market participation, facilitating the school-to-work transition and increasing employment opportunities for the hard-to-place. With target-oriented evaluation research, then, one analyses which policies and policy combinations may be best suited for achieving these targets under different socioeconomic conditions as well as within different policy regimes.
... In contrast to conventional, programme-oriented approaches, target-oriented evaluation research adopts a bottom-up perspective, which entails viewing policy impacts from the angle of the relevant agents. These agents include the people to whom the policy is addressed and their receptiveness to or rejection of a particular policy (policy adoption); the agents involved in, and responsible for, policy implementation; and the decision-makers in the political arena, who are core actors in the process of policy formation.'

• research in Ireland on the effects of active labour market schemes (summarised in O'Connell and McGinnity, 1997);
  
  
• a Swedish study, also focusing on the impact of labour market policies on a youth target group (Ackum Agell, 1995);



• a recent British study (White et al., 1997) which looks at the impact of a range of measures on the job-finding chances of the long-term unemployed;
  
  
• an evaluation of a range of active measures for the unemployed in the eastern Länder of Germany (Hübler, 1997).

In addition, we attempt to draw, in so far as the existing research permits, on some of the other key insights of target-oriented evaluation. In particular we adopt the principle that wherever possible, evaluation should attempt to open the 'black box', and to understand not just what impact a measure or programme has, but the factors contributing to that impact. In so far as a scheme has an impact, policy-makers need to understand not only how big the impact is, and how much it cost, but which elements of the programme were critical to contributing to it (e.g. where a scheme incorporates a range of interventions, such as counselling, work experience and training elements, it is important to understand the relative contribution of each element to the scheme's overall impact). Equally it is important to understand how much of the impact can be attributed to the institutional context, and the operation of the programme delivery agents etc. (it is common to find, for example, when comparing ostensibly similar schemes in different national or local contexts, that the local management and institutional context are critical in determining programme effectiveness).

Thus we hope, by bringing together the findings from a range of evaluations, in different contexts and institutional environments, to extend the limited conclusions which emerge from individual programme evaluations taken in isolation. Keeping the target-oriented perspective in mind as an 'ideal' however, we also consider, towards the end of the paper, some of the practical and methodological issues which need to be tackled in developing future evaluations of measures for the long-term unemployed in the target-oriented direction.

The study draws on existing comparative work, including that of the author (Meager and Morris, 1996), as well as other recent exercises conducted for the OECD (Fay, 1996), and the European Commission (European Commission, 1995).

1.3 Long-term unemployment; the nature and extent of the problem

Long-term unemployment remains one of the most persistent, and in quantitative terms, serious, social issues facing many industrialised economies. To take the Member States of the European Union as an example, the extent of the problem has been well documented.⁽²⁾ Almost exactly half of the unemployed in the Union, some nine million people, have been unemployed for a year or more (Figure 1 shows recent trends in unemployment and long-term unemployment among the 15 EU Member States). Within this group around 60 per cent have been unemployed for at least two years.

Figure 1 - Unemployment and long-term unemployment (E15)

Whilst there has been some fluctuation in the share of unemployment which is long-term, for the Union as a whole, this share has not fallen below 41 per cent (which was achieved in 1992) for many years. Although the incidence of long-term unemployment is somewhat lower than the EU average in the new Member States (Finland, Austria and, especially Sweden), long-term unemployment has risen persistently across the Union since 1992.

Certain groups in the workforce are disproportionately prone to long-term unemployment. Continuing with the example of the European Union, across the Union, and in most Member States, female long-term unemployment rates have been higher than male.⁽³⁾ Further, in most countries, there is clear evidence that older workers are over-represented in long-term unemployment compared with their share of total unemployment (this over-representation would be even greater, but for high rates of withdrawal from the labour force due to early retirement). Older workers becoming unemployed are more likely to remain unemployed than younger workers, and in many countries the data suggest that older workers losing their jobs in traditional industrial sectors are particularly at risk of long-term unemployment. In European countries at least, although youth unemployment rates are typically higher than the average, unemployed young people (under 25) are generally less likely than those in other age groups to become long-term unemployed (although there are exceptions, especially in southern Europe).

In most countries, long-term unemployment has fluctuated over the economic cycle, and tends to move with overall unemployment levels. The rate of long-term unemployment varies considerably between countries, however, (as the data for EU Member States in Figure 2 illustrate), and although there is a positive relationship between unemployment and long-term unemployment rates, this relationship is by no means a perfect one; as Figure 2 shows, countries with similar rates of overall unemployment may have very different rates of long-term unemployment and vice versa.

Figure 2 - Unemployment rates and long-term unemployment rates (1995)

Or, to express the issue slightly differently, countries vary considerably in terms of the share of unemployment which is long-term unemployment. Figure 3 shows, for example, that in 1995, four EU Member States (Sweden, the UK, Greece and Belgium), with similar rates of overall unemployment, slightly below the EU average, had vastly different shares of long-term unemployment (varying from Sweden with 20 per cent to Belgium with 62 per cent).⁽⁴⁾

Figure 3 - Unemployment rates and shares of long-term unemployment (1995)

This suggests that even where prevailing macro-economic circumstances constrain national governments from achieving full employment goals, some countries may have much to learn from others about active labour market policies to minimise long-term unemployment and the associated social exclusion.⁽⁵⁾ Caution needs to be exercised, however, in concluding that Member States with high levels of active labour market expenditure (e.g. Sweden and Denmark) have thereby 'solved' the problem of long-term unemployment.⁽⁶⁾ Whilst large scale active measures are likely, by definition, to reduce recorded long-term unemployment, the level of 'hidden' long-term unemployment may remain high, due to the 'carousel' effect of individuals moving repeatedly between unemployment and active measures. Participants in measures are often not counted as long-term unemployed during their participation, and if they return to unemployment after participation, they are reclassified as short-term unemployed.

It is, however, clear that national performances with regard to long-term unemployment vary considerably. Recent research for the European Commission (DGV),⁽⁷⁾ using Labour Force Survey and administrative data to estimate the likelihood of a 'representative' individual leaving unemployment, after given durations of unemployment, suggests that the main source of variations in rates of long-term unemployment between Member States is variations in outflow rates rather than inflow rates; it is the speed with which people leave unemployment, rather than the rate of entry to unemployment which is important. The key question, therefore, is not which policies are most effective in stopping people becoming unemployed, but rather which are most effective in maintaining the 'employability' of the unemployed so that they are less likely to flow into long-term unemployment. The same study also shows a considerable decline, in most Member States, with increasing duration of unemployment, of an individual's probability of leaving unemployment. In designing appropriate policy interventions, and deciding on the timing of measures (i.e. at what point in an unemployment spell should measures be applied), and the relative importance of preventative measures and re-integrative measures, we need to understand the process by which people become long-term unemployed, i.e. what explains the declining probability of leaving unemployment observable in aggregate data? We discuss, in section 2.3 below, the alternative explanations for this process, the evidence which exists for them, and some of the policy implications.

2. POLICY APPROACHES

2.1 The policy framework

• demand-side-oriented approaches, which attempt to increase the volume of labour demand or to bias that demand towards the recruitment of the long-term unemployed. This category includes measures which do this directly (e.g. through the creation of jobs for the long-term unemployed), as well as those which do it indirectly (e.g. through offering subsidies or incentives to employers to recruit the long-term unemployed;
• supply-side-oriented approaches, which attempt to improve the chances of the long-term unemployed on the labour market, by increasing their skills and human capital (e.g. through training), their ability to access available jobs, or their motivation for job-search (through advice, counselling and assistance with job search), or indeed their willingness to take up certain job opportunities (through benefit conditionality, reduction in the 'benefit trap' etc.)

2.2 A categorization of measures against long-term unemployment

In developing an appropriate categorization, there is a considerable previous literature on which to draw.⁽⁹⁾ The categories often overlap to some extent and many measures incorporate elements from several of the categories. In addition, they are often part of an overall strategy for tackling unemployment in general, rather than having a specific focus on the long-term unemployed. Nevertheless, they provide a framework for interpreting the strategic policy approaches developed to tackle long-term unemployment in advanced economies over the last few decades.⁽¹⁰⁾

2.2.1 Incentives or subsidies for employers

This category includes demand-side measures, which tackle job creation indirectly, through subsidies or other incentives to employers to encourage them to recruit the long-term unemployed.

2.2.2 Direct employment/job-creation schemes

This category includes initiatives which create employment directly for the long-term unemployed, as a bridge between unemployment and the regular labour market. Such measures offer work experience, with participants normally undertaking socially-useful activities, which do not displace market activities (although the extent to which non-displacement is actually achieved is a key evaluation issue).

The wide spectrum of measures under this heading includes:



• traditional job-creation schemes, which create (usually short-term) jobs in the public or quasi-public sectors for the unemployed, in non-market activities;
• small scale, often local or community-based programmes aimed at generating employment in the 'intermediate labour market', often via special 're-integration enterprises', or 'community businesses' set up for this purpose.



The distinction between the two types is not clear cut, and in practice the difference is largely one of scale, and of the agencies involved in policy design and delivery. Generally, intermediate labour market schemes are typically smaller in scale than the traditional job-creation schemes, and although they frequently involve some form of state or municipal subsidy, they are often characterised, in addition, by partnership involvement and support from a variety of non-governmental bodies, in the voluntary, charitable or co-operative sector, and in some cases, the private sector as well. Also, a key element of such initiatives is that they are embedded in their local communities, with the aim (in addition to that of employing disadvantaged people from those communities), of providing socially and economically beneficial goods and services for those communities. Such initiatives often involve the creation of a distinct trading enterprise, run as a community business or co-operative and selling its goods and services on the open market. As with traditional job-creation schemes, the usual intention is that such initiatives do not displace existing private sector activities -- as with the traditional schemes, however, this can be hard to ensure, particularly as it is often the case that the initiatives are set up with a subsidy which declines over time, the intention being that the enterprises should become economically viable and ultimately self-financing. A further distinct feature of some such initiatives is that they form part of wider strategies for local and regional adjustment to structural change, and have, in this sense, a preventative as well as a re-integrative function in the fight against long-term unemployment -- perhaps the clearest example of this has been the role of 'employment companies' in the transition process in East Germany, a model subsequently transferred, with some modifications, to the West.

2.2.3 Work-sharing/reducing labour supply

These measures tackle unemployment from the supply-side, and tend to be focused on unemployment in general, rather than long-term unemployment in particular. At least two variants can be identified:


• Subsidies to those in employment to reduce their labour supply, to share work with the (long-term) unemployed; and
• subsidies to the (long-term) unemployed themselves to reduce their labour supply, and in the extreme case, to become economically inactive (e.g. by early retirement).

2.2.4 Vocational training-based schemes

These measures account for an extremely high proportion of active labour market policy expenditure for the long-term unemployed. Such measures are clearly supply-side in orientation, and may be off-the-job and classroom-based, and/or may involve on-the-job training in some kind of work placement. Training schemes of the latter type overlap with demand-side job-creation measures in the sense that taking unemployed people off the unemployment register for a period of training, may also involve their being on employers' premises and producing marketable goods or services. The key principle underlying such training schemes, however, has been the notion that training the long-term unemployed will increase their subsequent employability.

2.2.5 Job-search training, counselling etc.

Many countries have in recent years introduced counselling, advice and similar support measures focused on areas other than vocational skills, which may improve participants' employability. They may, for example, include training in techniques of effective job-search, or in how to present oneself to an employer. They may also incorporate elements aiming to improve the 'motivation' and 'work attitudes' of the long-term unemployed.

2.2.6 Tackling information deficits and labour market "friction"

These measures, typically delivered through the public employment services (PES), aim to address market failure and information deficits in the job-hiring process. These may occur through lack of information on the part of employers or job-seekers, or the use of inappropriate or inefficient methods of vacancy notification or job-search. Alternatively they may arise because of inaccurate information, e.g. because job-seekers have incomplete understanding about employers' requirements, or because employers have inaccurate or prejudiced perceptions of the abilities, attitudes and likely productivity of long-term unemployed job applicants. Examples of measures in this category include:


• targeted job-matching or broking services, whereby the PES attempt, through information provision and other broking activities, to effect a better or faster match between employer requirements and job-seeker characteristics, in order to reduce the frictional element of (long-term) unemployment;



• subsidised short-term placements with employers. These measures bring the unemployed and the employer together for a short period, at little cost or commitment, so that the unemployed can obtain a better understanding of employer requirements and acquire or refresh their experience of working life, whilst the employer has an opportunity to try out an unemployed applicant. The intention in such schemes, which are often linked to training schemes, is that employers who might otherwise discriminate against the long-term unemployed may be influenced to change their negative perceptions.




2.2.7 Incentives or subsidies to individuals

These include supply-side measures to reduce the costs to the job-seeker in finding, accepting or keeping a job. They are not usually targeted purely at the long-term unemployed. Important categories include:


• provision of 'in work benefits' (or other forms of payment), either to all low paid workers, or to newly employed entrants from unemployment. The object here is mitigate the 'unemployment trap', whereby the unemployed face a disincentive to enter low paid employment, because their loss of benefit on taking work is not sufficiently compensated by the extra income received from employment ;⁽¹¹⁾



• subsidies to the individual unemployed person to support the costs of job-related training/re-training/education.



• subsidies to offset the individual's costs of finding and entering a job (e.g. expenses incurred in job search, such as travel-to-interview expenses; or costs associated with relocation to take up new employment);



• subsidies to support unemployed people becoming self-employed or starting their own enterprises.⁽¹²⁾




2.2.8 Measures aiming to 'activate' the unemployed

Elements of this approach may be associated with many of the other measures identified above, but there is a clear trend in many countries' towards a greater emphasis on 'activation' of the (long-term) unemployed. Measures which reflect this trend often contain an element of pressure or sanctions on the job-seeker. Important examples in this category⁽¹³⁾ are:


• supply side approaches aiming to reduce the 'reservation wage' of the unemployed, by providing a disincentive for them to refuse low-paying jobs. Such approaches often operate by making unemployment benefit conditional on strict job-search and acceptance criteria, or by time-limiting the receipt of unemployment benefit. This can be seen as an alternative or complementary approach to tackling the 'unemployment trap' problem discussed above, i.e. it functions by reducing the attractiveness of remaining on benefit, rather than by increasing the attractiveness of low-paid jobs per se.



• 'workfare' approaches (rare outside the USA and Australasia),⁽¹⁴⁾ in which the unemployed are required to work as a condition for the receipt of unemployment or social security benefits. A concomitant is often that the state also acts on the demand-side, acting as 'employer of last resort' to guarantee public sector employment to the unemployed who fail to find regular jobs after a certain period.

2.3 Preventative strategies: early identification and action

The main distinction between 'preventative' and 're-integrative' approaches may often, in practice, lie in the timing of intervention. Thus, for example, counselling and advice offered by the PES in the early weeks of unemployment may be seen as a preventative strategy, whilst similar measures offered to the long-term unemployed, are likely to be seen as part of a reintegration strategy.

To pursue the analysis further, we can distinguish between two types of so-called preventative strategies:

• those based on 'early identification' of those 'at risk' of long-term unemployment, which is essentially an issue of targeting; and
• 'early action' for the unemployed, with across-the-board interventions at an early stage of unemployment spell. This is primarily a question of the optimal timing of policy intervention.

Why not, therefore, target unemployed people early in their unemployment spell, and address them with the kinds of measures discussed above? A negative answer to this question is typically related to deadweight and the costs of intervention. Flows data in many countries show that most people becoming unemployed leave the register again within a fairly short period. Any approach, therefore, which offers the various job-creation, training and other measures described above, to people as, or shortly after they enter unemployment, would risk a high deadweight cost. It is even possible that such early interventions could have negative effects, if they resulted in people who would have found a job quickly being held out of the labour market longer than would otherwise have been the case.

An important evaluation question, therefore, relates to the optimal timing of policy intervention: at what point in an unemployment spell do the benefits of intervening outweigh the deadweight costs? As noted in the OECD Jobs Study (OECD 1994, p.103), there is a surprising lack of empirical evaluation evidence on the question of optimal timing, although in practice many countries operate a policy regime which makes implicit assumptions about the relevant trade-offs, with low-cost counselling and job-search information offered to the short-term unemployed, and with the intensity and cost of the measures adopted increasing with duration of unemployment (once the 'easy to place' have been filtered out of the system).

All such approaches, however, assume that there is no reliable method of identifying at an early stage of their unemployment spell (or earlier) individuals with a high risk of becoming long-term unemployed. An alternative view stresses the heterogeneity of the unemployed, and their different and individual characteristics which influence their chances of remaining unemployed; this may be due, for example to discrimination (e.g. among ethnic minorities), or because of objective disadvantage (e.g. people with physical or mental disabilities), or because of institutional constraints (e.g. single parents unable to find suitable work because available wages would not offset their loss of benefits and the additional costs of childcare). On this latter view, policies would do better to focus on the sources of the disadvantage, and use these to trigger appropriate 'tailor-made' interventions, rather than relying on the duration of unemployment itself to generate a standardised intervention, at different points in a person's 'unemployment career'.

In practice, although it is possible readily to identify groups with characteristics over-represented among the long-term unemployed (certain age groups, ethnic minorities, disabled people, people with few work-related skills etc.) the relationship is not perfect, and it is also the case that many unemployed with these characteristics re-enter employment quickly. Early identification and targeting therefore requires a more effective mechanism to identify 'at risk groups' than is offered by simple indicators based on easily observed personal characteristics. As de Koning (1995) points out, describing the experience of Dutch wage-subsidy and job-creation schemes, a policy drift towards 'early action' in the absence of an effective method of early identification carries considerable cost and risks rendering existing measures for the long-term unemployed even less effective than they currently are.

Our review of the literature suggests considerable scepticism, on the basis of experience and evidence to date, that an effective early identification process can be found. The discussion in OECD (1992) embodies this scepticism, but also makes a plea for further refinement of existing processes in this direction. Fay (1996) reports some progress in this area, with 'profiling' initiatives in Australia, Canada and the United States.⁽¹⁵⁾ As Fay points out, however, even where profiling models have significant predictive power in identifying individuals 'at risk' of long-term unemployment, they do not in themselves help to identify what kinds of services are required for such people. There are, moreover, significant potential ethical and legal issues associated with the use of personal characteristics such as age, sex or ethnic origin, to determine the allocation of resources to unemployed clients of the PES (such variables were, for legal reasons, excluded from the US experiments reported in Department of Labor, 1994).

An important policy question, therefore, is how much effort should be invested in developing improved techniques for early identification of individuals and groups with a high risk of becoming long-term unemployed. The answer depends, at least in part, on our understanding of the process by which people become long-term unemployed. There is, in the academic literature, an unresolved debate regarding the relative importance of 'heterogeneity' and 'state-dependence' in generating long-term unemployment.⁽¹⁶⁾ In a model based on heterogeneity, becoming (long-term) unemployed is a filtering process, whereby an individual's characteristics, such as skill and education levels and other personal attributes are seen by employers as 'favourable' attributes. As a result, people lacking such attributes (at any duration of unemployment) are less likely to be hired than those possessing them. On this model we would observe, as duration increases, an increasing concentration among the unemployed, of people lacking the 'favourable' characteristics.

In a model based on state-dependence, unemployment itself causes further unemployment. That is, unemployment duration becomes a 'characteristic' reducing an individual's chances of leaving unemployment, or avoiding future unemployment spells. There are a several reasons why this might occur, the most obvious being statistical discrimination, with employers using a job applicant's previous unemployment record as a 'screen', on the assumption that it is likely to be an indicator of the applicant's skills, productivity, motivation etc. An alternative explanation would be that the experience of unemployment itself causes deterioration in skills, motivation and productivity among the unemployed, as their attachment to the world of work progressively diminishes.

Despite more than a decade's research on the question, economists have been unable to show convincingly that either of these two effects dominates in generating long-term unemployment. Early US research with panel data, particularly on young people, suggested that an explanation rooted in heterogeneity had more explanatory power.⁽¹⁷⁾ European research yielded somewhat different results initially, however. Thus youth unemployment studies in the UK⁽¹⁸⁾ indicated a role for both factors, i.e. heterogeneity was important, but so was state-dependence (at least in the short-term). Further UK studies,⁽¹⁹⁾ however, indicated evidence of state dependence, with the probability of a spell of unemployment ending being negatively related to the duration of that spell (after allowing for heterogeneity, both observed and unobserved). Recently, however, the balance of evidence has shifted again. Research in the UK quoted in Elias (1996), as well as that of Portugal and Addison (1995) and van den Berg and van Ours (1996) for the US, and van den Berg and van Ours (1993) for France, the Netherlands and the UK, suggest a limited role for state-dependence, and that most variation in observed durations of unemployment can be explained by heterogeneity (i.e. individual characteristics).

The findings on this issue are, therefore, somewhat mixed. Elias (1996), argues that the evidence increasingly points towards heterogeneity as the key influence. Hasluck et al. (1996) come to similar conclusions, arguing against any significant role for state-dependence. We would argue, however, that this conclusion may be over-stated, and is difficult to square with evidence from employer surveys indicating that a significant proportion of employers do take account, when recruiting, of previous unemployment and its duration, even when this is not the only, or the most important factor in the decision. For discussion of the evidence from employers on this question, see in particular, Atkinson et al. 1996, Colbjørnsen et al. (1992), ESRI (1991), Gazier and Silvera (1993), Meager and Metcalf (1987), Ronayne and Creedon (1993).

2.3.1 The role of employers in policy design and evaluation

This raises a further issue, regarding the role of employers in the whole process. A key factor, influencing the appropriate choice of policy is the decision-making processes of enterprises in recruiting and selecting new employees. It is notable from our review of the literature, how rarely these processes are taken into account in policy design, and how rarely evidence from employers is incorporated into policy evaluations. As noted above, however, from the employer-based research which does exist, there is persuasive evidence that employers discriminate against the long-term unemployed in the recruitment process; and that they often regard them as having less 'favourable' characteristics than otherwise similar job applicants.

In their discussion of an apparently successful Dutch approach to 'tailor-made' placement services, which tackles both the demand-side (employers' attitudes and practices) as well as the usual supply-side (job-seekers' skills, attitudes and behaviour), van den Berg and van der Veer (1991) argue as follows regarding so-called 'unemployable' groups:

'Policy-makers are often inclined to regard the qualifications of the people concerned -- the nature and level of their education and work experience -- as the most important factor influencing their chances on the labour market. However obvious this conclusion may seem, it has nonetheless been demonstrated by various surveys ..... that the relatively high levels of unemployment among certain groups (such as immigrants and refugees) can only partly be explained by their educational qualifications.

For this reason any policy aimed at the effective integration of 'unemployable' people into the labour market must take other factors into account. In our view those other factors are to be sought not so much on the supply-side as on the demand-side of the labour market. Employers try to minimise their risks in the recruitment and selection of new staff...................... Since it is often difficult to obtain reliable information on these matters, employers rely upon general impressions of the people concerned and upon general images and stereotypes of the groups to which they belong.....

Long-term unemployment is thus not synonymous with the characteristics that a person is 'difficult-to-place' but is one of the consequences of it and plays an important role in establishing the vicious circle. The process described above explains why any efforts aimed exclusively at increasing the educational qualifications of the people concerned usually do little more than produce a better educated class of unemployed people with little more prospect of paid employment. Employment services should be aimed not only at the supply-side but also at the processes of recruitment and selection on the demand-side of the labour market. Only when the relationship between supply and demand becomes the object of a systematic policy can the vicious circle be broken.' van den Berg and van der Veer (1991) pp 178-179.

Of critical importance here is an understanding of why employers believe that the long-term unemployed and other disadvantaged groups are 'less desirable', and how such opinions might be influenced. Looking first at the nature of employers' beliefs and attitudes, building on the discussion of state dependence and heterogeneity,⁽²⁰⁾ we may ask whether employers' perceptions of the long-term unemployed as less desirable than other applicants arise:



• because they believe that being long-term unemployed itself detrimentally affects the job applicants' possession of certain desired attributes (e.g. through deterioration of skills, motivation and work disciplines); i.e. do employers implicitly believe in a state-dependence model; or, alternatively



• because they believe that the long-term unemployed are drawn from a group in the workforce lacking the desired attributes, and who have been successively rejected by other employers as a result; i.e. do they implicitly believe in a 'filtering' type model (heterogeneity).



In either case, we observe the same outcome: employers disproportionately rejecting the long-term unemployed, but depending on the perceptions underlying this behaviour, the policy implications are rather different. Also important are the ways in which employers form such beliefs. Again we can posit two hypotheses:


• the first is where the belief is in some sense 'well-founded' e.g. because the employer has previous direct experience of having recruited from the long-term unemployed, and has found them less satisfactory than other recruits;



• the second is where the belief is based largely on a priori prejudices about the long-term unemployed, which have not been tested against the evidence.



These distinctions are important for at least two reasons. First, as we have seen, there is a growing trend for policy measures to focus on 'educating' the long-term unemployed themselves about job-search techniques, and how best to present themselves to potential employers. Clearly, a better understanding of employers' perceptions of the unemployed can be important in the design of such initiatives. Second, the above hypotheses have very different implications for the type of policy measure which is most appropriate to get the long-term unemployed into jobs.

Thus, for example, a policy of training or providing work experience is more likely to influence employers' attitudes, if the latter are 'well-founded' and assume 'state-dependence', than if they are based on prejudice and assume 'heterogeneity'. Unfortunately, there is little research or empirical evidence on employers' beliefs, practices and behaviour in this area, how they are formed, and how they are influenced (or not) by policy measures. This is a significant lack, given the role played by employers, and their critical influence on whether or not policy measures will be successful. It is also in notable contrast to the sustained effort in many countries in research on the unemployed themselves; their characteristics, behaviour, response to incentives and experience in policy measures. There is, in our view, a strong case for further examination of employer behaviour in this area, as part of a wider strategy of evaluation and design of active measures for the long-term unemployed.

Turning to some of the evidence on employer practice and attitudes which does exist, however, Meager and Metcalf (1987 and 1988), found from employer survey evidence in the UK: first that long-term unemployed job applicants risked rejection simply on the grounds of their previous unemployment duration in at least half of the job vacancies covered by the survey; and second that they suffered, in addition, a disproportionate risk of rejection on other grounds, in so far as they had a higher than average probability of lacking some of the other attributes seen as desirable by employers (health/fitness, stable employment record, qualifications, the 'right attitude'). Thus, the survey confirmed that the long-term unemployed were doubly at risk of rejection in the recruitment process; both in so far as they objectively lacked some desirable job-relevant characteristics; but also simply because many employers used unemployment duration as a selection criterion. Further investigation with employers suggested that employers' views on the attributes lacking in the long-term unemployed were dominated by factors such as the presumed lack (or loss) of work habits and disciplines, deterioration of skills, or inadequate initial basic education; issues such as whether their attitudes to work were 'flexible' or 'suitable' were also often raised. Particularly noteworthy was the fact that few of the 'deficiencies' identified by employers in this research were of a kind suggesting a need for extensive labour market training in vocational skills. When it came to the nature of the underlying beliefs, employers more commonly argued that the deficiencies of the long-term unemployed were because their motivation, skills etc. had deteriorated whilst unemployed (i.e. a state-dependence type view), than they did that the 'undesirable' attributes had been present in the long-term unemployed before they became unemployed. Unsurprisingly, employers in the former group were more likely to believe that with appropriate support and assistance the long-term unemployed could be re-integrated into work. A key finding was that managers' personal contact with the long-term unemployed, through social networks, or through having recruited a long-term unemployed person, was strongly associated with positive attitudes towards the long-term unemployed, and a willingness to consider them as recruits.

These results suggest both that employers' attitudes and practices can make an important difference to policy effectiveness, and that they may be susceptible to influence by policy measures. Thus if experience of, and contact with the long-term unemployed is a positive influence on employer attitudes, this favours measures which provide short-term work experience for the long-term unemployed in 'real' work environments, i.e. such measures may influence not only the long-term unemployed participant, but also the enterprise itself.

More recent survey evidence in the UK (Atkinson et al. 1996) reinforces these findings. In particular it suggests that employers demonstrate relatively little evidence of heterogeneity-based beliefs (i.e. a belief that unemployment reflects inherently less 'attractive' characteristics), and a more widespread attachment to state-dependent type beliefs (i.e. beliefs that experience of unemployment itself renders individuals less attractive through demoralization and deterioration). The research also confirmed earlier findings that being long-term unemployed was a particularly negative signal to employers in tighter labour markets; and showed that participation in government schemes (e.g. through taking an unemployed participant on a work-placement etc.) increased a firm's likelihood of subsequently recruiting from the unemployed, although care must be taken in attributing causality here.⁽²¹⁾

2.3.2 Implications for policy

What conclusions can we draw for policy development from the discussion of heterogeneity, state-dependence and the role of employers?

First, the question of the relative importance of heterogeneity and state-dependence in generating long-term unemployment has yet to be resolved. It is increasingly clear from studies of individuals, however, that heterogeneity may be more important than previously believed, which provides a case for investigating the feasibility of early identification. Unfortunately, initial research on this is not promising; although models have been developed (in the UK, for example -- Payne et al., 1994) which identify measurable characteristics associated with an increased likelihood of becoming long-term unemployed, the overall predictive power of such models is poor, and our conclusion from the UK development work (see, in particular, Gibbons 1996, Hasluck et al. 1996) is that such models do not, so far, provide a robust early identification tool. Further research has been undertaken in Australia, however,⁽²²⁾ and a practical pilot of an early identification instrument is taking place within the Australian PES, the longer-term results from which will clearly be of wider relevance to policy development elsewhere.

Second, if practical models for early identification do not yet exist (although there may be less ambitious ways in which to build on good practice within the PES in targeting different resources to different types of client), we are brought back naturally to the question of early action, and the optimal timing of intervention. There is a clear need for more research on this question; what is the cost-effectiveness of trying to bring forward in time some of the more costly measures (training, subsidised work placements etc.), so that more people benefit from them, at an earlier stage in their unemployment 'careers'?

Third, and perhaps most importantly, in our view, the discussion above suggests a need for a better understanding of the role of employers. We have seen that there exists evidence from some countries that many employers see unemployment duration as an important negative signal in recruitment. This is clearly relevant to the question of the optimal timing of policy measures: how 'long' does someone have to be unemployed before it makes a difference to a potential recruiter? If such cut-off points exist, do they vary by sector, occupation, locality or other structural variables, and how are they susceptible to influence by active measures? These are crucial questions.

More generally, we have seen that the evidence from UK employer surveys carries important messages for the development of policy measures, and there is a clear case for extending this kind of investigation elsewhere. So far, for example, the UK evidence suggests that training the (long-term) unemployed in vocational skills may have relatively little impact on employers' likelihood of recruiting them, compared with measures which bring them into contact with employers, and give them what can be demonstrated as recent, 'real' work experience. We consider below how far such evidence from employers is consistent with the evidence on scheme impact on placement rates.

2.4 Trends in policy development for long-term unemployment

• a growing importance of supply-side measures: there has been a marked shift in the balance between demand-side and supply-side measures over time (and particularly since the early- to mid-1980s), with a growing disillusionment with traditional job-creation and subsidy schemes, and an increasing emphasis on supply side measures. Even in countries where the demand-side emphasis remains important, it is possible to observe a parallel proliferation of supply-side measures (although in some countries where the supply-side emphasis went the furthest, such as the UK, there has also been a reversal of the trend in the most recent period, and growing experimentation with some of the traditional demand-side measures, albeit on a smaller scale and in a more targeted fashion);



• within the demand-side approach, a shift towards targeted and indirect job-creation measures: in so far as the demand side approach has remained important in some countries, there has, nevertheless been some evolution within the overall portfolio. In particular, several countries have moved away from mass direct job-creation schemes, by adopting a greater element of targeting (on particular client groups), and have also returned to more indirect approaches of job-creation (through incentive- and subsidy-based schemes);



• within the supply-side approach, a changing balance of measures: at the same time, within the overall range of supply-side measures typically found in the labour market strategies of advanced industrialised countries, we can also detect some important shifts. During the 1970s, and particularly during the 1980s, there was a significant growth in measures for the (long-term) unemployed based on training in vocational skills, and this remains the predominant element in many countries' approaches. There has, nevertheless, in recent years been a growing scepticism about the effectiveness of an approach based on training measures. In part this reflects budgetary concerns (such measures are often high cost), and in part it reflects also evaluation results (which we discuss further below) showing poor performance of training measures on the re-integration chances of the long-term unemployed. The balance has, therefore, begun to shift in many countries towards a greater emphasis on initiatives (which are often much lower in per capita cost terms than training measures) delivered through the PES and focusing on advice, counselling, job-search support and the like;



• associated with the previous two developments, there has been a move towards an 'individualization' of measures, i.e. a trend away from standardised, 'one-size-fits all' schemes, and an development of tailor-made approaches which aim to address the specific needs and characteristics of individual job-seekers. These latter initiatives may be delivered through the PES or other intermediaries, but a common feature is the growing role being played by individual case-workers devising 'individual action plans', and which may involve a mix of specific interventions (job-search support, vocational training, work-experience, help with social problems etc.) at different times and in different combinations, according to the needs and circumstances of the job-seeker in question;



• in recent years (often with the support of international initiatives such as the EU's ERGO programme, and the European Social Fund, for example) there has also been a proliferation of local, community-based, and 'intermediate labour market' initiatives, again seen as alternatives to the traditional centralised mass approaches found in national schemes for the long-term unemployed;



• in some countries, particularly but not solely in the Anglo-Saxon world, there has been a trend towards what is sometimes called 'activation' of the (long-term) unemployed. This can be seen in two broad forms (sometimes they are combined in a single approach): the first is incentive-based (i.e. approaches, based on in-work benefits and the like, aiming to reduce the disincentives faced by many unemployed as a result of the interaction between benefit levels and low paid jobs); the second is based more heavily on sanctions or compulsion, and may involve stricter benefit eligibility criteria, based on job search activity, and availability for work; or it may involve a 'workfare' type approach with compulsory participation in work-experience or training schemes becoming a requirement of benefit receipt;



• Finally it is worth noting, in many countries, the growing interest in initiatives based on early action for and early identification of the (potential) long-term unemployed. This is also in line with the greater emphasis on targeting, counselling and individualization, although it needs to be stressed that this interest has yet to be translated into many concrete practical initiatives incorporating the principles of early action/early identification. This, we would argue, reflects the significant technical difficulties (discussed above) facing the construction of effective early identification tools, and robust assessments of the optimal timing of policy interventions.

3. EVALUATION

In this chapter, we examine:

• some conceptual issues which need to be tackled in evaluation of policies for the long-term unemployed;
• issues concerned with the choice of empirical evaluation methodology, with some emphasis on the 'appropriate' methodologies for target-oriented evaluation research;
• examples of evaluation findings from the literature, and pertaining to the examples of policy types discussed in Chapter 2.

3.1 Evaluation issues

There are some important general points which can be made about the problems which need to be addressed in evaluating such measures for the long-term unemployed, and the kinds of methodological approaches which are appropriate for evaluation purposes.⁽²⁴⁾

3.1.1 Identification of objectives

Policy measures for the long-term unemployed may have a variety of specific objectives, and it is crucial, therefore, that any evaluation be conducted against the stated objectives of the specific measure(s) in question. A key question is 'what effects are policy-makers trying to have on the target group (long-term unemployed), through the measure or measures in question?'

Whilst this point may be an obvious one, it is one that is not always observed in practice. It is common, for example, to see evaluations which conclude that a high substitution rate (i.e. long-term unemployed people entering jobs at the expense of other potential employees) indicates lack of policy effectiveness, whereas it may be a legitimate objective of the measure in question to encourage such substitution. Similarly in evaluations of self-employment schemes for the unemployed, evidence of low enterprise survival rates is commonly seen as evidence that the measures are ineffective. One objective of such measures, however, may be to remove people from unemployment for a short period (at a relatively low cost, typically related to their unemployment benefit entitlement) and, at the same time, through the experience of starting and running a business, to impart some valuable skills and human capital and improve the participant's subsequent employability in the eyes of employers. From this perspective, whilst a high survival rate may be desirable, its lack does not necessarily imply that the programme is ineffective.

3.1.2 Controlling for other relevant variables

Any evaluation needs to take account of the influence of non-policy factors which affect the policy outcome. This may apply at the individual level, so when looking at post-scheme experiences of participants, for example, it is desirable to identify how far those experiences can be attributed to the scheme itself, and how far to other factors. Ideally, therefore, an experimental or control group evaluation (see below) should be conducted.

Similarly when identifying policy impact at the aggregate level, over time or across countries, it is desirable to allow for other factors which affect the outcome. It is not sufficient to observe, say, that the rate of long-term unemployment falls following the introduction of specific measures; rather it is necessary to allow for what would have happened to the rate of long-term unemployment in the absence of the measure(s), ideally by building estimates of the timing and intensity of the measure into aggregate time-series econometric models of employment (see the discussion of aggregate impact analysis below).

3.1.3 Offsetting impacts

As is well-known, it is normally necessary when evaluating a labour market policy measure to allow for effects which may offset the immediate gross impact of the measure. Nevertheless, it is still possible to find a large number of so-called 'evaluation' studies, which concentrate largely or solely on gross effects, e.g. by identifying from a survey or administrative data, the proportion of participants entering employment. Whilst this is useful for monitoring scheme performance, and examining its cost-efficiency, it is no substitute for full evaluation.⁽²⁵⁾

There at least three major potential offsetting impacts evaluations need to test for:



• deadweight; where the apparent 'impact' cannot be attributed to the measure itself. Examples are where a subsidy is paid for jobs which would have been created in the absence of the subsidy, or where an individual unemployed person entering employment after training or counselling would have found the same job without the policy intervention;
• substitution; where the measure has an effect on its target group, at the expense of a non-target group. Examples include cases where a subsidy or training scheme enables a long-term unemployed person to fill a vacancy which would otherwise have been filled by a short-term unemployed person or a new labour market entrant. It should again be stressed, however, that such substitution may be a desirable outcome;
• displacement; where the measure's net effect is lower than its gross effect, because the intervention has market-distorting effects leading to job loss elsewhere. Examples are where a wage subsidy enables a firm to take business from competitor firms; or where a subsidised unemployed person entering self-employment takes business from existing self-employed persons or competing small enterprises.

In principle, aggregate impact analysis (see below) can identify a measure's full net impacts, but it is not usually possible to separate out the relative importance of the above three types of effect from aggregate analysis. For this, micro-level analysis, e.g. through surveys, is often required, but there are severe limits to such evaluations (firms and individuals may find it hard or impossible accurately to identify deadweight; and displacement can usually be assessed at the micro level only through in-depth surveys, not only of the firms directly affected by the measure, but also of their competitors and suppliers⁽²⁶⁾).

3.1.4 Scheme design and implementation

It is increasingly clear that the design features of a particular measure, the method of its implementation, and the skills of and relationships between the various delivery agents, may have important implications for its outcomes. This recognition is not often fully reflected in evaluation studies, although in recent years, particularly in countries where policy implementation is decentralised, allowing local actors a certain amount of discretion in design and implementation, the role of these factors is now being examined. Such studies suggest that the following elements may be of importance:


• eligibility criteria: these may be the formal criteria, e.g. the extent of targeting on particular disadvantaged groups among the unemployed. Informal eligibility criteria may also be important, e.g. where institutional incentives bias participation towards one group or another. Thus, for example, 'creaming effects' in training schemes are often observed, with delivery agents tending to select unemployed individuals with 'positive' characteristics who are more likely to meet scheme performance targets.⁽²⁷⁾

  There are several important trade-offs here, whose net effects cannot be determined a priori. Thus, for example, compared with a scheme with broad eligibility, a scheme targeted on particularly disadvantaged groups may have low deadweight effects, but its effective job placement rates may also be lower, and the cost per placement correspondingly higher;




• institutional context. It is clear that the actors and institutions involved in programme delivery may have an important influence on scheme impact, especially if they have discretion in the design of the scheme, the degree and level of support provided to participants etc. Where possible, therefore, evaluations need to take account of, and isolate the impact of such institutional variety in scheme implementation;.
• scale, duration and coverage of payments. For any given programme design and amount of public expenditure, the scale of payments, their duration and coverage, may have an important impact. Thus, for example, there may be diseconomies of scale, and the observed positive effects of a small local subsidy or training scheme may be dissipated (e.g. through growing deadweight and displacement) if the scheme is expanded to a wider area and with greater coverage. A key question for policy makers is whether it is more effective to spend a given sum on a large number of participants each receiving a small subsidy and/or a subsidy for a short duration, or to reduce the number of participants with a corresponding increase in the amount and duration of subsidy. Once again this is a question which cannot be addressed a priori, but requires empirical evaluations.

3.2 Evaluation methodologies

3.2.1 Experimental/control group evaluations

The most rigorous approaches to policy evaluation at the microeconomic level attempt to identify a measure's impact on participants, in comparison with a control group not affected by the measure. Such approaches are common in the USA and Canada, but are rarely used in European countries, where they are often ruled out in on grounds of cost, ethical objections to experimental evaluations, or simply because political expediency requires a rapid implementation of a measure.

We can identify at least three variants of this approach:



• 'pure' random experiments;
• matched comparisons;
• local pilots (with design variations);

'Pure' random experiments, or experimental approaches with random assignation are often seen as the 'gold standard' for evaluation. A full description of this approach, and its application to labour market policy evaluation can be found in Björklund and Regnér (1996). The essence of the approach is that two groups (a 'treatment' group and a 'control' group) are randomly chosen from the eligible population (e.g. the unemployed). The first group then receives the treatment (participates in the labour market policy measure), whilst the second does not, and the approach avoids the issues of 'selection bias' typical in other control group methods (see below).

This does not mean that there are no disadvantages, however. Apart from ethical and cost issues, biases may arise because the existence of the experiment itself affects behaviour, e.g. if unemployed people in the locality of the experiment postpone or reduce active job search in the hope of being selected for the experiment. Equally, their knowledge that they are participating in an experiment may also influence the behaviour both of participants, and of those administering the scheme.

Nevertheless, random experiments have considerable advantages over the more common alternative, the 'quasi-experimental' or 'matched comparisons' approach. This also involves a treatment group and a control group, the difference being that the evaluation takes place ex post, with the control group chosen from among those not participating in the scheme. Whilst an attempt is made to control for observed personal characteristics (age, duration of unemployment, sex, skill and qualification levels), by definition the approach is likely to suffer from selection bias or 'unobserved heterogeneity', since the decision to participate or not in a scheme may be influenced by unobservable characteristics (motivation, for example). Whilst statisticians have developed sophisticated methods to control for this bias, the complexity of the subsequent analysis, and the judgement required in the selection of the appropriate technique, reduces the transparency of the evaluation, and increases the risk of erroneous conclusions being drawn.⁽²⁸⁾

Finally, it is worth also mentioning an variant of these control group approaches, namely the local pilot approach, in which the randomization is done on a geographical basis rather than an individual one. In this approach, certain localities (e.g. local labour markets, or PES administrative areas) are chosen (ideally at random) in which to pilot the measure, and its impact is assessed through comparison with areas not participating in the pilot (ideally controlling for relevant exogenous variables, such as local industrial structure and labour market tightness; or by choosing 'matched' control areas which are as similar as possible to the pilot areas). This method also allows the possibility of testing the effects of design variation, and variation in administrative systems and procedures.⁽²⁹⁾

3.2.2 Aggregate impact analysis

Aggregate impact analyses use econometric estimates of the statistical relationship between a policy measure's introduction and intensity on the one hand, and observed developments in aggregate employment and unemployment on the other, in a modelich allows for the impact of other relevant variables. Such approaches, the theory and methodology of which have been usefully summarised by Bellman (1996), are also commonly used to assess the macro-economic impact of measures, e.g. their effect on the inflation-unemployment trade-off, and on the 'Beveridge curve' (the aggregate relationship between unemployment and unfilled vacancies, as a measure of the efficiency of the job matching process in an economy). There is a large academic literature on these approaches,⁽³⁰⁾which remains nevertheless somewhat inconclusive on the macro-economic benefits of active labour market policies.

For present purposes, however, our interest is mainly in the more limited use of such approaches, which attempt to pick up econometrically, the impact of measures on aggregate (long-term) unemployment and employment and the flows between the two. The enormous advantage of such approaches is that, in so far as appropriate aggregate data exist, and the measure in question is of sufficient scale and duration for the econometric estimations to be meaningful, they allow us to arrive at estimates of the full net effects of programmes. Thus they can provide an important complement to the micro level evaluations of measures (e.g. those based on surveys of participants -- see below), which often exaggerate their impact, because they are unable adequately to assess factors such as deadweight, substitution and displacement. Typically, however, aggregate impact models leave the process by which the measure affects employment and unemployment as a 'black box', and are unable to distinguish the separate impacts of deadweight, substitution and displacement on the final net outcome. In an ideal world, therefore (see Bellman 1996 op. cit.), the aggregate impact approach should be seen as complementary to micro-level evaluations, with the former providing robust indications of net impact, and the latter providing insights to aid the interpretation of the aggregate results, and helping to steer policy through showing why net effects are lower than gross, and the relative importance of deadweight, substitution, displacement, and the scheme's institutional context. Only micro-level evaluations can provide this complementary insight.

3.2.3 Participant surveys

These are the most commonly methods used to evaluate measures for tackling (long-term) unemployment. They involve administrative data to identify the characteristics of scheme participants, together with surveys of those participants during and/or after scheme participation to identify their perceptions, experiences, destinations etc. The main drawback of this approach, apart from cost and time, is that without external comparisons of non-scheme participants and post-scheme destinations etc. (which many studies do not include), it is difficult to draw conclusions about scheme effect per se. Equally, reliance on participants' or employers' perceptions to identify deadweight, substitution and especially displacement, is fraught with difficulties (although the latter can be eased through seeking collateral information from the competitors and customers of employers involved in, e.g. a subsidy scheme; see Elias and Whitfield, 1987 and Hasluck, 1990).

3.2.4 Monitoring with administrative data

Almost without exception, states and agencies which implement measures to tackle long-term unemployment have systems to monitor administrative data on scheme participants, outcomes and financial performance. As pointed out in Auer and Kruppe (1996), such exercises are typically concerned with issues such as the extent to which the scheme is reaching the target groups, as well as 'outcome' measures such as placement rates, and financial, material and staff costs of the scheme. Whilst such data can provide an important input into scheme evaluations, particularly where they can be integrated, at an individual level, with survey data collected from participants (and ideally from a control group), they rarely provide sufficient material for an evaluation in their own right (although it is common to find descriptive accounts of such data labelled as 'evaluations').

3.2.5 What methodologies are required for target-oriented evaluation?

As Schmid et al. (1996) make clear, target-oriented policy evaluation does not require different methodological tools from those traditionally used in programme-oriented approaches. The strengths and weakness of the different methodologies outlined above, apply in a targeted-oriented evaluation in much the same way as in a programme-oriented evaluation. The difference resides rather in the overall design of the evaluation, which under the target-oriented approach, is ideally comparative, on the one hand, in the sense of facilitating a systematic comparison of effectiveness of the alternative policy approaches which can be applied to the target group (rather than focusing on the evaluation of a single measure). And, on the other hand, such an evaluation will also be comprehensive and multidimensional, with an emphasis not just on 'what works', but also on 'why'; taking account in particular, for example, of financing, implementation and wider institutional factors.

In methodological terms this points towards:



• where possible, greater methodological rigour, e.g. through an experimental or control group approach, but ideally using a sample and control group drawn from the target population (e.g. long-term unemployed), to examine the impact of the full range of measures to which this population has been subject (the recent studies of O'Connell and McGinnity (1997), and White et al. (1997), go a considerable way in this direction);
• where such rigour is not possible (e.g. because of the large samples required to yield sufficient cases to analyse the full range of measures); aggregate impact analysis (again incorporating independent variables to reflect the full range of policy measures to which the target population has been subject) may offer considerable potential;
• finally, at a more general level, reviews such as the present one, or that of Fay (1996), may help illuminate the impact of different measures for the target group in question, simply by bringing together the range of results from a large number of programme-oriented evaluations, and attempting to draw out common conclusions, or to explain differences. In practice, of course, such approaches leave unanswered as many questions as they answer, since the different methodologies and institutional contexts of the various scheme evaluations make reliable comparisons difficult to draw out. The work of Gardiner (1997) in the UK is, however, interesting in this context, since by focusing on a large number of measures, in a single national context, the institutional variety is reduced. Furthermore she adopts a standardised framework for comparing the different measures (supplementing the programme evaluations with further monitoring data from administrative sources on financing and costs), enabling her to draw some general conclusions about the comparative impact of measures per unit cost of public expenditure.

3.3 Evaluation findings and conclusions

3.3.1 The evaluation evidence

The studies examined are grouped according to the type of measure in question, in Tables 1 to 8 below, using the categories of measures in the typology developed in the previous chapter, whilst table 9 reports a small number of studies which use a target-oriented methodology to compare systematically the impacts of a range of measures addressed at the (long-term unemployed) target group. The tables include a brief summary of their main findings and conclusions relating to the measure's effectiveness in contributing to the re-integration of the (long-term) unemployed.

We have drawn on published and unpublished literature uncovered during the course of the research, as well as some of the already-existing internationally-comparative reviews ⁽³¹⁾, but inevitably, certain countries are over-represented in the discussion, particularly those in Northern Europe, North America, and Australia, with a tradition both of active labour market policies, and of policy evaluation. In this respect the research incorporates and builds on the work of Fay (1996), who argues that the evaluation tradition, and the use of rigorous experimental methods, is more highly developed in North America than in Europe:

'A first observation is that two countries -- Canada and the United States -- evaluate their programmes more than others. There are very few evaluation studies available from most European countries and none from Japan' Fay (1996) p.12.

Arguably, however, Fay's conclusion on the relative lack of European evaluation research is exaggerated, as suggested by the large number of European studies included in the tables below.

In addition to the variation between countries in availability of evaluation material, however, it must also be noted that certain types of measure are less well-represented in the evaluation literature than others. This particularly applies to measures which have attained prominence more recently (including, for example, guidance, counselling and job-search initiatives; and more innovative individualised approaches), as well as for those which operate on a small scale, or at a local level (such as the many of the 'intermediate labour market initiatives⁽³²⁾). By contrast, most of the major traditional large-scale measures (including employer subsidies, mass job creation schemes, training schemes, and self-employment subsidies) have been subject to numerous evaluations over the years, and are disproportionately represented in the discussion below.

Table 1: Subsidies for employers

Table 2: Direct employment: Traditional job-creation and intermediate labour market schemes

Table 3: Work-sharing/reducing labour supply

Table 4: Training (Vocational skills)