The Marginal Benefit of an Active Labor Market Program Relative to a Public Works Program: Evidence from Papua New Guinea

UYEP is conducted in Port Moresby, the capital and largest city in PNG. Across the country, the number of jobs in formal sector employment reduced by 10% from 2013 to 2017 and more than half the population is under the age of 25 years (BPNG 2018; UNFPA 2014). Formal sector employment peaked at 16% of the labor force in 2013 following 6 years of strong economic growth driven from some mining projects; however, since this time the non-mining economy has been undergoing a significant downturn (Jones and McGavin, 2015; BPNG, 2018). Two-thirds of formal sector jobs are in the private sector (heavily concentrated in the wholesale and retail trade sector and construction), with the remainder in the public sector (NSO, 2013; Jones and McGavin, 2015). There are large skill shortages in a variety of trades, such as carpentry, hospitality, retail, and office administration (Imbun, 2015). In addition, the cost of labor is quite high as the minimum wage was around 1.22 United States Dollars (USD) per hour in 2018, which is the same as countries with Gross Domestic Product (GDP) per capita four times that of PNG (such as Malaysia) (Jones and McGavin, 2015).

Despite the large skill shortages, there are few opportunities for youth to enter the formal sector labor market in Port Moresby. Since the end of the mining boom in 2013, private sector employment has fallen by 20% in Port Moresby, led by large declines in the construction industry followed by wholesale and retail trade (BPNG, 2018). In addition, each year more than 80,000 youth leave secondary school across the country but most are not trained well enough to transition into formal sector roles (Voigt-Graf, 2017). There are only limited places available at technical colleges and universities (just over 5,000 across the country), which restricts the options available for youth to be trained to fill some of the skill shortages (Voigt-Graf, 2017). This issue is particularly acute in Port Moresby as a large number of young people migrate there in search of better job opportunities (Voigt-Graf, 2017). It is estimated that there are at least 40,000 unemployed youth currently in Port Moresby (World Bank, 2018). This is set to grow as the number of people between the ages of 15 and 35 years in PNG is expected to increase by more than 50% over the next 25 years (UNFPA, 2018).

The government and aid donors have invested in a range of strategies to address the lack of opportunities in the formal sector for youth and the largest program to date is UYEP. There are two components of UYEP, all participants complete the first component while around 25% of participants complete the second. The first component of UYEP is a 40-day LIPW called Youth Job Corps (YJC), which primarily focuses on the provision of temporary employment through activities like collecting trash off the streets. The second component is a comprehensive ALMP called On-the-Job Training (OJT), which begins with a vocational training course for 20 days followed by a fully subsidized placement with a formal sector employer for 5 months. These placements are in occupations where there are large skill shortages, primarily hospitality, retail, and office administration. UYEP begins with a 1-week Basic Life Skills Training (BLST) course, which includes a basic numeracy and literacy test. Only individuals that perform well on the test and satisfactorily complete the BLST course are eligible to progress to the ALMP component (OJT).

UYEP is an established program, having existed since 2012 and with over 20,000 participants, of which over 2,000 graduated from the comprehensive ALMP component. To be eligible for UYEP, individuals need to have been out of the formal sector labor market for over 6 months, be between the ages of 16 and 35 years, have been based in Port Moresby for at least 1 year, and attend a screening interview held at various locations throughout Port Moresby. Each of these aspects of the eligibility criteria were verified through consultation with community leaders. New intakes of youth are enrolled every 3–4 months with over 20 intakes occurring since the program began. Youth first undertake 5 days of BLST and 40 days of a LIWP (YJC) shortly after the baseline survey and those who are eligible undertake the ALMP component (OJT) once job placements are arranged. The project management unit has relationships with over 100 employers who regularly provide placements for youth.

To determine the impact of the ALMP component of this program (beyond just the LIWP component), we conduct a difference-in-difference analysis and exploit a one-off, unexpected change in the implementation of the program. Specifically, during intake 13 of UYEP, there was an unanticipated intervention in the program and the standard ALMP component was removed part way through the LIWP component. Instead, participants were offered the opportunity to be paid to be a volunteer at the 2016 Under 23 Women’s FIFA World Cup. Youth who undertook this placement were only engaged for around one-third of the period of time as youth who undertook the standard ALMP (2 months compared to 6 months). It seems unlikely that if the duration of the volunteer program had been longer, employability of the youth in this intake would have been improved. The volunteer placement only involved basic manual labor, limited training opportunities, and there was no potential for ongoing employment with the employer as this was a one-off event. As such, program participants in intake 13 who were eligible to complete the ALMP component because of their academic performance in the screening test and satisfactory completion of BLST did not complete an ALMP. Following intake 13, the usual program resumed whereby people who were eligible completed the ALMP component as was the case in earlier intakes. Importantly, the change in the program for intake 13 was not made public until after youth for intake 14 had expressed interest in UYEP and all youth in the treatment who were eligible to participate in the ALMP component of the program took up this opportunity.

The youth who were eligible for the ALMP component in intake 13, but missed out due to the temporary change to the program (“placebo” control group), are comparable to the youth in the intakes just before and just after (intakes 12 and 14) who completed the ALMP component (treatment group) for five reasons (Figure 1). First, all youth in each of the three intakes had not worked in the formal sector in the last 6 months and on average between 75% and 80% of the youth had never worked in the formal sector throughout their life. Second, intakes 12, 13, and 14 occurred at a relatively similar time (the baseline surveys occurred in April, July, and November 2015, respectively). Third, the youth are sourced from the same areas in Port Moresby and expressed interest in the program through the same channels. Fourth, the removal of the ALMP component of the program during intake 13 was not known by participants at the time the intake began and they were only informed of this change part way through the LIWP component. Finally, statistical balance calculations show that there were almost no statistically significant differences between the demographic characteristics of treatment and control groups (section 4.2).

Figure 1

The main outcomes of interest are whether the treatment group outperformed the control group in terms of employment in the formal sector, earning a cash income, and seeking employment. The exact wording of each of the questions is shown in Table 1 and they are based on what has been used in other impact evaluations on this topic (McKenzie, 2017).

The baseline survey was conducted from April to November 2015 when youth were registering to participate in the program. The follow-up survey was conducted in July to August 2017, which was around 9 months after all youth from these three intakes had completed the program.

There were 355 program participants who met the criteria outlined above and completed the baseline survey. Among these respondents, 215 were re-interviewed in the follow-up survey (113 in the control group and 102 in the treatment group). Respondents in the treatment group were only three percentage points more likely to complete the follow-up survey than respondents in the control group. We show in Appendix A1 that on average there was no difference in the demographic characteristics between respondents who were re-interviewed and those that were not, except in regards to education. Respondents in the treatment group who were re-interviewed were more likely to have lower levels of education at baseline (there were no statistically significant differences between respondents in the control group who were re-interviewed and those that were not). The relatively small sample size (compared to other impact evaluations of ALMPs) means we are only adequately powered to detect statistically significant differences between treatment and control groups of at least 8 percentage points.

Statistical balance calculations show that there were no statistically significant differences (at a p value below 0.05) between the demographic characteristics of respondents who completed both surveys in the treatment and control groups. Table 2 shows that the only statistically significant difference between these groups is that those in the treatment group were slightly older.

Sections 4.1 and 4.2 outline how the treatment and placebo control groups were clearly comparable at the time of the baseline survey; however, the key assumption of a difference-in-difference analysis is that in the absence of the program the comparability of the groups would have lasted over time (i.e., there would be “equal trends”). We take two steps to illustrate that this equal trends assumption is reasonable. First, we show that youth in the treatment and control groups experienced very similar employment and job-seeking histories prior to the program. While we only have two rounds of survey data (before and after the program), the baseline survey includes a number of questions that provide insight into respondents’ lives at various points in time (Table 3). There are no statistically significant differences between the treatment and control groups across a range of characteristics that capture their employment and job-seeking histories. This provides good reason to believe they would have had very similar trajectories for employment and job-seeking outcomes in the absence of the program.

In addition, there were few opportunities for these unemployed youth to enter the formal sector. As discussed above, the levels of formal sector employment in Port Moresby declined by 20% from 2013 to 2017 and most employers were not hiring new staff (BPNG, 2017). As such, it is highly unlikely that opportunities in the formal sector for the youth in these intakes that are 3 months apart would have varied dramatically in the absence of the program.

The second step we take to illustrate the credibility of the equal trends assumption is to follow Duflo (2001) and conduct a placebo test. This involves selecting groups of individuals in the same intakes as the treatment (intakes 12 and 14) and control groups (intake 13) who were not affected by the program, and conducting difference-in-difference analysis. If there are no statistically significant differences in outcomes, this provides further evidence that in the absence of the program the trajectory of outcomes of the actual treatment and control groups would have been comparable. The design of this study naturally lends itself to conducting a placebo test using UYEP participants who were not eligible to complete the ALMP component due to their performance in the academic test in intakes 12, 13, and 14. We show in Table 4 that there are no statistically significant differences between these groups across demographic characteristics.

To estimate the effect of the ALMP component on the main outcomes in Table 1, we conduct a difference-in-difference, ordinary least squares (OLS) regression in the form of a linear probability model that can be written as follows:

where Y_it is a dummy variable for each of the outcomes of interest discussed in Table 1, Treatment_i is a dummy variable that takes the value of 1 if the respondent belongs to the treatment group and the value of 0 if the respondent belongs to the control group, After_t is a dummy variable that takes the value of 1 for the follow-up survey and 0 for the baseline survey and ε_it is the model error term. β₃ captures the average difference in differences in regards to the outcomes of interest (Y_it) between respondents in treatment group and the control group (i.e., the treatment effect).

The estimates of the OLS regression above are compared to those obtained from a binary logit model and they are qualitatively similar (see Appendix A2). In addition, the OLS regression above is conducted controlling for baseline characteristics and the results are very similar (see Appendix A2).

The ALMP component has a large effect on formal employment, with a 41.2 percentage point increase in the share of participants in the treatment group who are currently working or worked in the last 6 months compared to a 23.0 percentage point increase in the control group (see Table 5). This is from a starting point of no formal employment in the last 6 months at the time of the baseline survey, which means the 18.2 percentage point difference can be attributed to the effect of the program. This difference in the overall rate of formal sector employment

resulted in youth in the treatment group having on average higher weekly wages and working more hours in the formal sector. However, conditional on being employed there were no statistically significant differences in wages and hours worked between the treatment and control groups. On average, participants who worked in the formal sector currently or in the last 6 months in the treatment group worked 47.3 hours a week (compared to 43.3 in the control group) and earned 170 Kina a week (approximately USD50) (compared to 179 Kina [approximately USD53] in the control group). This suggests the benefits from the ALMP component of the program are entirely on the extensive margin as opposed to the intensive margin (i.e., more people were employed as opposed to employed people working more hours and/or earning more).

The large effect on employment from the program is due to youth continuing to work with the employer they were placed with after the end of the program (Figure 2). In the treatment group, 20% of participants were employed with their ALMP employer at the time of the follow-up survey, compared to no one in the control group as this option was not available for them. In comparison, there was no statistically significant effect from the program on formal sector employment with a different employer, as 21% of youth in both treatment and control groups were employed somewhere other than their ALMP employer. This suggests the effect of the ALMP on formal sector employment is a level shift in outcomes (i.e., increase in the likelihood of having employment immediately following the program) as opposed to an increase in the rate at which an individual might be employed in the future.

Figure 2

Similar to the results of other impact evaluations of ALMPs (McKenzie, 2017), the program did not lead youth in the treatment group to be more likely to have earned cash or searched for a job compared to youth in the control group (Table 5). However, the program does have a positive effect on job-seeking behavior if we restrict our analysis to youth who did not have a job in the formal sector in the last 6 months. Among these youth, there was a 24.2 percentage point difference between changes in job-seeking behavior between treatment and control groups.

We illustrate the robustness of the results in Section 5.1 by conducting a placebo test of the equal trends assumption. As discussed in Section 4.3, we compare the change in outcomes for individuals in the same intakes as the treatment (intakes 12 and 14) and control groups (intake 13) who were not eligible to complete the ALMP component due to their performance in the academic test. These individuals only completed the LIWP component of the program. The results of the placebo test are shown in Table 6. There are no statistically significant differences in outcomes, which provides further evidence that the equal trends assumption is reasonable.

The outcomes shown in Table 6 are not directly comparable to those shown in Table 5 because the time between the follow-up survey and when youth exited the program is substantially different. Youth in the treatment and control groups did not exit the program until a much longer period of time had passed (in some cases, over a year) than youth in the placebo analysis above (see further discussion in Appendix 3). As such, the follow-up survey was around 18–24 months after the youth in the placebo analysis exited the program, whereas the follow-up survey was around 9–12 months after the youth in the treatment and control groups completed the program. This means that the youth included in the placebo analysis (those that were ineligible for the program) had spent on average almost twice as long in the labor market following the conclusion of the program as eligible youth in the same intakes. As a result, they have had substantially more time following the program to find alternative employment opportunities than youth in the treatment and control groups.

Employers involved in UYEP did not need to pay the ALMP participants throughout the 5-month placement (their wage was provided by the program) and this appears to be essential for at least three reasons. First, most employers would not have participated in the program to the same extent in the absence of a subsidy. We captured employers’ willingness to pay by asking how many ALMP placements they intended to offer over the next year with the full subsidy, if only half the subsidy was offered, or if no subsidy was in place. On average, only 30% of employers would keep the same number of placements if they received half the subsidy and less than 15% of employers would keep the same number of placements if there was no subsidy. Second, 45% of employers stated the number one change they would make to the program would be to increase the time period of ALMP beyond 5 months (assuming the full subsidy remained). Finally, the subsidy allowed some employers to reduce the total labor costs of their business as 28% reported that they would have more staff in the absence of UYEP (compared to 10% that said they would have fewer staff). This suggests a displacement effect from the program as fewer jobs may have been available in the labor market for non-participants. However, the extent of the displacement effect is unknown and it may not fully offset the employment benefits created for the program participants.

The importance of the training provided to youth to overcome the skill shortages in the labor market was consistently highlighted by employers. One of the main barriers to employers hiring more staff is that “training new staff is too costly,” with this option being listed as one of the top three reasons provided by 52% of employers. The popularity of this response is even higher than “economic conditions” (listed by 49% of employers), which is noteworthy given the downturn in the non-mining sector in PNG. In light of this, it is not surprising that 51% of employers listed the training provided to youth prior to ALMP placements as the most important aspect of the program. Furthermore, 49% of employers wanted the amount of training provided to youth prior to the placement increased, which was the highest response provided in regard to a question about how to improve the program.

Employers appear to have used the ALMP placements as a low-cost way to screen youth prior to deciding whether they wanted to offer them a job. More employers stated that the main reason they offered jobs to ALMP youth was due to the performance of the youth undertaking the placement as opposed to any need for additional staff (52% compared to 37%). This is reinforced by the finding that one of the main barriers to hiring more staff employers face is that staff with the right skills are difficult to find (52% of employers included this as one of the top three answers). Furthermore, employers indicated that UYEP was the third most common way they recruited new staff, after newspaper advertisements and job agencies.