The effects of more intensive counseling for disadvantaged unemployed youth

The Austrian PES acts as a one-stop-shop for the unemployed: it administers unemployment benefits and unemployment assistance, offers counseling and placement services, and is responsible for the implementation of training, along with several other ALMP measures.

The PES comprises a federal office and nine provincial offices – one for each of Austria's nine federal states – as well as 98 regional employment offices (REOs), twelve of which are located in the Austrian capital Vienna. Central coordination is carried out by the federal office, which is responsible for management, controlling, evaluation, analysis and strategic planning. The REOs are coordinated by the provincial offices; these provide information, advice, support and labor market assistance tailored to the regional environment. Unemployed people are assigned to the REOs according to the postal code of their place of residence, because this is generally the nearest REO. Until recently (during the period under review), the PES Vienna bundled its special offers and competencies for young people under the age of 21 in a separate Youth Employment Service (YES).

Each of the REOs has three zones for the purpose of customer segmentation: an information zone, a service zone, and a counseling zone. The information zone provides general – anonymous – labor market information to the public, including many self-service options. The service zone is primarily for newly registered unemployed clients and those considered “ready for work” with little need for assistance. Here, applications for unemployment benefits are processed, and the unemployed receive counseling and job offers. The counseling zone is designed to assist unemployed persons who have been unemployed for at least 6 months or who are difficult to place for other reasons. They receive more intensive counseling and support here than in the service zone.

The PES Vienna developed and tested a new model of intensified support (“case management”) in the counseling zone of its YES from November 2018 to March 2020. The target group consisted of low-skilled, 15-to-21-year-old unemployed young people receiving social assistance. Ten additional positions were established to implement case management, eight of which were established directly in the counseling zone. For the treated, this increased the ratio of caseworkers to unemployed from originally 450–460 to 120–130 clients per caseworker.

There is no information on the resulting caseworkers to clients ratio specifically for the control group, i.e. the target group persons served in Department 3, and on the duration of the meetings. In our empirical analysis, we find that the average number of meetings increased by a good 50% and the meeting interval decreased from about once a month to about once every three weeks (see section 4.1.).

Initially the pilot project was set-up as a randomized controlled trial (RCT). To allow a comparison between intensified counseling and the status quo, case management was only introduced in two of the three departments in the counseling zone of the YES, while the third department continued with the previously existing method of counseling. The three departments had identical tasks. Already before the start of the pilot project, clients were assigned to departments on the basis of their date of birth: Department 1 was responsible for clients born between January 1 and May 10, Department 2 for those born between May 11 and September 14, and Department 3 for those born between September 15 and December 31. Persons entitled to asylum or subsidiary protection

Asylum seekers whose application for asylum is rejected must be granted subsidiary protection if there is a threat of violation of the ban against torture or the prohibition of inhuman or degrading punishment or treatment or the right to life or a serious danger to body and life in conflict situations. In contrast to asylum, a right of residence is limited to one year, which upon request can be prolonged by two years.

The case management was set up in Departments 1 and 2, starting from November 2018 for all target group persons, i.e., clients who met the clearly defined target group criteria. Three case management desks were created in each of the two departments. All customers assigned to Department 3 continued to be served by the eleven existing counters in this department, where the service remained unchanged. The target group clients in this department served as the reference group in the pilot project. Since their counseling did not change, their histories could be used as a reference and as a counterfactual result in the hypothetical case of unchanged counseling.

The case management consisted primarily of intensified counseling. In Departments 1 and 2, the aim was to reduce the meeting intervals from around six to eight weeks to around two weeks. In addition, the initial meeting with the clients was to be extended, lasting 50 minutes instead of the usual 15 to 20 minutes. The counseling additionally included detailed anamnesis based on an anamnesis form, a support folder and an agreement form. All documents were collected in a support folder and the topics discussed were recorded on the agreement sheet; all of this was aimed at supporting the commitment, structure and organization of the young people. Further elements of the case management included additional feedback from clients after scheduled appointments and workshops via tablet, regular supervision and moderated case discussions for the caseworkers, as well as same-day support for spontaneous client contact.

The target group consisted of young people between 15 and 21 years of age drawing on social assistance (exclusively or in addition to unemployment benefits from the PES), who either did not have a compulsory school-leaving certificate or had a compulsory school-leaving certificate and had already been registered as unemployed or had been looking for an apprenticeship for more than three months. A few groups were considered exceptions to the pilot's target group and continued to be served at their own counters, regardless of their date of birth: (1) health-impaired persons who had been assigned to a specific rehabilitation desk, (2) pregnant women, and (3) men with a draft order.

If, as of November 2018, YES clients met the target group criteria and were assigned to Department 1 or 2 based on their date of birth, they were assigned to a case management desk. For those already in care before the beginning of November, the case management criteria had been reviewed in the two weeks prior. If a person served in Department 1 or 2 did not meet the target group criteria immediately upon his or her first appearance at the YES, but only later, he or she was assigned to case management accordingly. Once clients were assigned to case management, they remained in their departments, even if they temporarily left unemployment or no longer met the target group criteria (e.g., by obtaining an apprenticeship diploma or ending their social assistance receipt).

The increase in staff was intended to create more time for mentoring and supporting the young people, i.e., for motivation, vocational orientation and the selection of a suitable support program. Part of the strategy was to provide more continuous support for low-skilled young people in the PES program and, where sensible and possible, to offer them more training instead of quickly placing them in an unskilled job – in the hope that this would improve their longer-term employment prospects.

The case managers knew they were participating in the pilot project. They received extra coaching and training in case management. The treated clients should also have been aware that they were participating in a pilot project. However, the clients in the reference group, for whom nothing changed, did not know about it.

The pilot project was implemented by AMS Vienna, jointly financed with AMS Austria and accompanied by a project steering group, which consisted of representatives of the provincial management (AMS Vienna), the management of the regional office (YES), the federal office (AMS Austria), as well as experts and works council of the REO. The group networked with the department of the City of Vienna responsible for social affairs, social and health law (“Magistratsabteilung 40”) and drew on the expertise of other in-house experts.

The AMS itself commissioned the evaluation of the pilot project by an independent research institution in order to examine the effects of the case management and to learn from the experience for the large-scale project “U25” to be created in 2021 (after the study period): a joint service center of AMS Vienna and the City of Vienna, which offers all young people under 25 all services related to work, education and social issues as a “one-stop shop”. We undertook this evaluation and in the course of it we found that the intended randomization was not achieved and that we therefore need to adjust for selection bias with non-experimental methods.

To evaluate the impact of intensified counseling on the placement process and labor market integration of the youths served, we primarily use two sources of linked individual administrative data: the Austrian unemployment register (AUR) and the Austrian Social Security database (ASSD). The AUR contains detailed information on individual characteristics of the unemployed, such as age, gender, formal education, health constraints or care responsibilities, which may affect individual labor supply. We use daily information on unemployment episodes, receipt of unemployment benefits, PES caseworkers’ interventions such as client-meetings, job offers and benefit sanctions for non-compliance with job search requirements, as well as information on participation in active measures. In addition, all persons registered with the PES are provided with monthly information on their receipt of social assistance. We compare this information on social assistance recipients with data from the social welfare office in Vienna. The ASSD is a matched employer-employee-dataset, which provides a full record of labor market histories and earnings of all private-sector workers in Austria on a daily basis from 1972 onwards.

Our evaluation sample includes individuals who benefited from case management or were part of the reference group in the period from November 2018 to March 2019.

In the subsequent period from April to October 2019, no new persons were admitted to case management, only once again starting in November 2019.

The treated are monitored in the three years after the month of entry into case management, and the controls are monitored in the three years after the month in which they first entered Department 3 and met the target group criteria. For the effects on the placement process, we focus on the first year, since many of the clients were no longer registered as unemployed and thus no longer exposed to treatment later on. Labor market effects are examined over a period of three years. Average wages are an exception. We measure this indicator of post-unemployment job quality in two time periods: (1) in the calendar years 2019 and 2020 and (2) in the two years following the month of pilot project entry. Both the duration of the follow-up period and the definition of outcomes are exactly the same for all jobseekers in the treatment and the control group.

Each person is included in the evaluation only once, from the time of its first appearance in the pilot project. For reasons of data availability, all persons considered must have been registered with the PES and must still have met the target group criteria at the end of the month in which they joined the pilot. We exclude the extremely few participants who were over the age limit of 21 years at the end of the month, as well as one individual who died during the first year. Our final sample consists of a total of 1,811 individuals, 977 (54.0%) of whom were in the treatment group and 834 (46.1%) in the control group. About three quarters of them (73.7%) entered the case management or reference group as early as November 2018, with relatively few joining in subsequent months.

The reason for this high proportion lies in the definition of the target group: In the following months, only persons joined who (1) were either newly unemployed and did not have a compulsory school-leaving certificate or (2) had a compulsory school-leaving certificate and only then exceeded the threshold of 3 months of unemployment or apprenticeship search. A large proportion of the pilot participants were already registered with the PES in November 2018, and if young people had been looking for a job or apprenticeship for longer than three months, this was generally already the case at the start of the pilot project and not in the months thereafter.

Our objective is to estimate the average treatment effect of intensified counseling by comparing outcomes between the treatment and the control group. If randomization had been successful in the pilot project, the two groups would have been directly comparable in their relevant outcome characteristics, and average causal effects could have been determined in an unbiased way by a simple mean comparison of outcomes (cf. Imbens & Wooldridge, 2009; Angrist & Pischke, 2009; Harrison & List, 2004; Levitt & List, 2009; List, 2011; List & Rasul, 2011). However, we observe systematic pre-treatment differences between the two groups in relevant characteristics at the time of entry into the pilot project. This would lead to a “selection bias” in a simple mean comparison of outcomes.

To be clear, randomization based on the month of birth is a promising approach. In a field experiment in Austria with an increase in counseling staff for adults, this randomization mechanism achieved complete random assignment (see Böheim et al., 2017). One reason for the incomplete randomization in the present case is the systematic assignment of all persons granted asylum and subsidiary protection without a date of birth upon entry into the social security system to Department 3 (control group). These individuals were unable to provide information on their actual date of birth. Therefore, their birth month was set to 13, and they were consequently cared for in Department 3. However, this is obviously not the only cause. As can be seen from the selected variables in Table 1 and the full summary statistics in Table 8 in the Appendix, the observed differences are more diverse. They relate to personal characteristics such as gender, nationality, education level and residence status, as well as to employment histories and past welfare receipt. Treatment and control group differ significantly in most of the observed characteristics. Therefore, the problem cannot be solved by simply excluding certain subgroups.

Remaining imbalances could be due to the fact that we do not measure the fulfillment of the target group criteria at exactly the same time as the PES. The PES immediately assigned clients to case management if they were served in Department 1 or 2 based on their date of birth and met the target group criteria at the time they entered the pilot (as of November 2018). If a person served in Department 1 or 2 did not meet the criteria immediately, but later, it was assigned to case management accordingly. For individuals who were already registered with the PES prior to the beginning of November (i.e., the start of the pilot), the case management criteria were already reviewed in the previous two weeks. We know the allocation of clients to the three departments and have all the information to define the target group persons exactly the same as the PES. However, we do not have all relevant information on a daily basis, but we can only check at the end of the month whether clients fulfilled the target group criteria. Our sample therefore necessarily consists of those individuals who were registered with the PES and met criteria at the end of the month in which they entered the pilot project.

The differences in pre-treatment characteristics must be controlled for using appropriate empirical methods in order to establish the ex-post comparability of the groups and ensure causal interpretability of the results. For this purpose, we combine 4-to-1 propensity score matching (Rosenbaum & Rubin, 1983) within a caliper of 0.1 with exact covariate matching on residence status, social assistance receipt and last economic activity. Thus, we estimate a propensity score by way of a logit model with a very large number of individual characteristics (see Table 9 for the estimates

According to the logit estimation, personal characteristics such as gender, age and health have a significant influence, as do previous unemployment duration, numerous aspects of previous employment history, and employment status and benefit receipt at the time of entry into the pilot project. Previous participation in labor market policy measures and contact with the PES in the last two years are also relevant.

By means of exact matching, we only compare persons with the same status of residence: asylum, subsidiary protection, or no asylum status. In addition, we condition on whether individuals had partial receipt of social assistance as a non-single person at the month-end cutoff date and were last employed in the public sector. With this matching approach, we achieve an extremely good balance of covariates, at an acceptable loss of 49 persons to common support. After matching, treatment and control group do not differ significantly on average in any of the 78 control variables used (see Table 8 for balancing indicators

The median absolute standardized bias, determined according to Rosenbaum & Rubin (1983), is 2.1% in the main estimate after matching. According to T-tests, there are no significant differences in the mean values between the treatment and the control group in any of the 78 control variables after matching. The pseudo-R² of the logit estimate of the propensity score for the matched populations is 0.022. The p-value of the likelihood-ratio test of the joint significance of all regressors in the logit model after matching is 0.977.

Separate estimates are needed for outcome measures that cannot be considered for the entire population. For example, average monthly earnings can only be examined for those with employment in the relevant period. In these other cases, the chosen matching procedure also balances the contribution of covariates very well.

The similarity of the matched groups ensured by the matching procedure refers to a wealth of characteristics that potentially have an influence on the probability of case management, the placement process and labor market integration. First, these include numerous personal characteristics: gender, age, education, presence of health impairment, nationality, naturalization, asylum status, and German language skills. Second, we match on previous unemployment duration (distinguishing between registered unemployment, training periods, and the search for apprenticeship), time elapsed since last employment, industry, occupation, and earnings in last employment, the employment status at the cut-off dates one year, half a year, and three months ago, as well as detailed indicators of the five-year labor market history (sum of days in different employment statuses, namely unsubsidized employment, different forms of subsidized employment, apprenticeships, temporary absence for reasons such as child care, elderly care or education, registered unemployment, periods of PES training, apprenticeship search or other unemployment statuses). In addition, we take into account sickness benefits received during unemployment. Third, we control for past participation in various ALMP measures, for PES contacts and job offers received in the last two years. Fourth, we adjust for differences in the specific employment status and in the receipt of unemployment benefits and social assistance immediately on the day before the month under consideration.

Most characteristics (labor market history and current employment, subsidy, and benefit receipt status) are measured at the end of the previous month, i.e., immediately before the month of pilot access. Only for personal characteristics and previous unemployment duration does the status at the end of the pilot access month have to be used for consistency, as our sample by definition is to be unemployed and meet the target group criteria on this cut-off date.

We include all these variables in the analysis because they are all potentially related to outcomes on the placement process or labor market integration. There is sufficient overlap in the distribution of covariates between the treatment and comparison groups. Moreover, because we can observe such a large number of factors by combining multiple data sources, we are confident that the conditional independence assumption (CIA) on which our matching approach relies is satisfied: namely, that, conditional on the propensity score, treatment assignment and potential outcomes are independent.

We estimate the average treatment effect on the treated (ATT). This corresponds to the difference between the actual outcomes of the treated and the hypothetical outcomes they would have achieved if they had not benefited from intensified counseling. Regarding the variance of the matching estimator, we provide heteroskedasticity-consistent analytical standard errors proposed by Abadie & Imbens (2006).

We use the following conditional variance formula Abadie and Imbens (2006) propose for the case of matching with replacement with a fixed number of matches: σ^2Xi,Wi=JJ+1Yi−1J∑m=1JYlji2 {{\hat{\sigma }}^{2}}\left( {{X}_{i}},{{W}_{i}} \right)=\frac{J}{J+1}{{\left( {{Y}_{i}}-\frac{1}{J}\sum\limits_{m=1}^{J}{{{Y}_{{{l}_{j}}\left( i \right)}}} \right)}^{2}} .

We compare a variety of outcomes on the counseling and placement process on the one hand and the integration into training, education and employment on the other across the up to three-year observation period after the (individual-specific) pilot entry month.

First, we examine effects on the counseling and job placement process using the share of unemployed clients having at least one meeting with a PES caseworker in the one-year observation period, the number of meetings (including individuals without a single meeting), and the interval of meetings (number of days in registered unemployment or looking for an apprenticeship between meetings). Only one meeting is counted per day. In addition to face-to-face visits, online meetings are included. Second, we compare the share receiving at least one job offer through the PES and the overall number of job offers in the one-year period. Third, we look at the share receiving at least one sanction. We also distinguish between a suspension of unemployment benefits due to a failure to accept a job or participate in training and a suspension of benefits due to a failure to meet a PES appointment.

Furthermore, we look at the share of unemployed young people with at least one entry into an active labor market policy measure during the year of interest. With regard to employment measures, we separately identify three types that are most common among the target group: two forms of direct job creation in the public or non-profit sector, namely socio-economic enterprises (SEEs) and non-profit employment projects (NEPs), and non-profit personnel leasing. Other employment measures such as integration subsidies, wage top-up systems and short-time work are included in the total but are not presented separately because they hardly ever occur. Similarly, for training measures, we only present specific effects for selected instruments, namely vocational orientation, basic training, training and further education provided by external educational institutions, PES subsidies for company-based apprenticeships, supra-company apprenticeship training, as well as extended apprenticeships or partial qualifications (EAPQ) for young people with disadvantages or personal placement obstacles who cannot complete a conventional apprenticeship. For other measures, such as course cost subsidies for participation in the free education market and labor foundations, we refrain from presenting these separately due to their lack of significance. In addition to employment and training measures, labor-market-related counseling and support provided by external institutions (BBEs) round out the list.

We examine labor market effects by looking at (1) the sum of days spent by youth in different labor market positions during the three-year period after entry into case management and (2) the shares in each labor market position at the cut-off date after three years. Since part of the objective was to increase the number of young people in apprenticeships or other training, we make a basic distinction between apprenticeships and other forms of employment. In addition, (3) we compare the duration of unemployment and the receipt of benefits from unemployment insurance and social assistance remaining after the pilot project entry month in the three-year period. Finally, as an indicator of post-unemployment job quality, we choose average monthly earnings during employment, in a first variant in the two years following the pilot project entry month and in a second variant in the calendar years 2019 and 2020. In each case, we only consider months with an average income above the low-income threshold (€ 446.81 in 2019).

The increase in staff in Departments 1 and 2, and the subsequent implementation of the case management, resulted in a significant intensification of support for target group persons, which can be seen in more frequent meetings between PES caseworkers and unemployed clients. This was accompanied by a significant increase in placement activities. It led to more job proposals from the PES, more assignments to active labor market programs, in particular to qualification measures, and to more frequent sanctions in the form of suspension of benefit payments due to a failure to meet PES appointments.

As shown in Table 2, the proportion of persons having at least one meeting increased by 4.3 percentage points or 5.3%, and the average number of meetings increased by a good 50% (+2.1 contacts, +53.6%) in the year under review. If all days of registered unemployment or apprenticeship search are taken into account, the meeting interval decreased, on average, by a good third (−12.6 days or −39.7%). On average, persons from the treatment group had a meeting approximately every three weeks instead of once a month. The frequency of contact has thus increased significantly, although obviously less than planned.

The greater intensity of support was accompanied by an increase in job proposals: the share of clients who received at least one job proposal from PES in the year under review increased by 10.6 percentage points or 24.2% as a result of the increase in PES counseling staff. The average number of job proposals, taking into account persons without a single proposal, increased by almost two thirds (+1.6 job proposals or +63.9%).

With more counseling staff, more time becomes available, not just for counseling, but also for monitoring clients’ compliance with benefit obligations, particularly their willingness to accept available work or participate in active measures. Additionally, when more appointments are arranged, there is a higher chance that appointments will not be kept (without a good reason), which can result in benefit suspensions. Both of these aspects are possible explanations for why sanctions were imposed more frequently as a result of the introduction of the case management. In the year under review, the proportion of individuals with a sanction due to a failure to meet an appointment increased by 2.1 percentage points or 50.3%. The case management had no effect on sanctions due to refusal to accept a job or participate in active measures.

Intensifying counseling led to more frequent participation in employment and qualification measures (see Table 3). More specifically, the percentage of persons who entered basic training increased by 30%, from an already high starting level (+10.7 percentage points). Basic training plays the most prominent role in the target group. Here, general basic qualifications are taught in order to create the basic conditions for entry into the labor market or participation in further education or training. Typical examples are German courses, literacy courses, courses for catching up on compulsory school-leaving competencies, and basic IT courses. As a further result of intensified counseling, the share of those with in-company apprenticeship support (PES subsidies to support companies and training establishments, which offer training slots to certain target groups) increased by 2.7 percentage points or 63.5%. In addition, there was a significant increase in the number of persons entering a socio-economic enterprise (+1.8 percentage points or +52.8%).

One year after entering the pilot project, 55.2% of the young people in case management were (still) taking part in an ALMP measure, compared to 46.9% in the control group. Significant differences exist in the placement in basic training, training and further education, company-based apprenticeship support, and direct job creation in SEEs and NEPs.

These results are available upon request from the authors.

Turning to effects on direct labor market integration, we find that the partial goal pursued within the pilot project was achieved, i.e., to encourage more of the low-skilled young people to take up training and further education instead of being quickly placed in some form of “unskilled job”. As a consequence of intensified counseling, in the three years following access to the pilot, the young people were more frequently taking part in a (subsidized) apprenticeship and training measures provided by the PES. At the same time, they were less likely to be engaged in other employment (except apprenticeship) as (low-skilled) workers, i.e., in “unskilled jobs”.

As can be seen from Table 4, in the three years following access to case management, the treated spent on average 16 days more in subsidized company-based apprenticeships. This represents a significant increase of 60.3%. They spent 39 days more in PES training (+19.1%). Conversely, they spent on average 21 days (−11.7%) less in other dependent employment (except apprenticeship). The decisive factor here is that they spent fewer days as (low-skilled) workers in the primary labor market (−29 days or −24.6%).

After three years, the treatment group had a significantly higher proportion of apprentices than the control group (+4.3 percentage points or +25.2%). Also, the share in PES training was higher (+2.5 percentage points or +21.7%). Conversely, the treated were significantly less often in other dependent employment (no apprenticeship) (−5.1 percentage points or −18.0%). More specifically, they were less likely to be employed as workers in the first labor market (−3.2 percentage points or −18.8%) (see Table 5).

The main reason for the higher share in an apprenticeship after three years was an increase in apprenticeship participation subsidized by the PES: treated persons were more often in subsidized company-based apprenticeships (+1.4 percentage points or +61,3%) and in extended apprenticeship or partial qualification (EAPQ) for young people with disadvantages or personal placement obstacles who cannot complete a conventional apprenticeship (+1.7 percentage points or +56.8%). Unsubsidized apprenticeships were relatively rare in both groups, reflecting the special problems of the young people studied.

If all forms of employment, including apprenticeships, are combined, no effect of intensified counseling on integration into employment can be observed in the three years after entry into case management. Individuals from the treatment group spent an average of 422 days in employment, which was not significantly different from the control group (426 days). Three years after pilot project access, 49.1% of the treated and 48.5% of the controls were employed. Intensified counseling also had no effect on labor force participation.

As seen in Table 6, intensified counseling had no effect on the average remaining duration of unemployment in the three years under review (treated 530 days, controls 515 days). Nor did it change the average number of days during which the treated received unemployment benefit or unemployment assistance. The target group under consideration generally rarely received these benefits, presumably because they did not meet the eligibility requirements. However, case management resulted in a more frequent receipt of subsistence allowance (+25 days or +18.5%). This is due to more frequent participation in subsidized apprenticeships and PES training. The subsistence allowance is paid during participation in such ALMP measures. At the time after three years, the treated and controls did not differ in their receipt of unemployment insurance benefits or social assistance.

Finally, with regard to post-unemployment job quality, we find that the young people who received intensified counseling after participating in case management had a similarly high monthly income as comparable young people who did not receive case management, once they were in employment (covered by social insurance). In both groups, the average was slightly above 1,000 €.