Directing young dropouts via SMS: evidence from a field experiment

Recent empirical evidence shows that planned and assisted job search strategies are effective for young job seekers trying to find employment (Abel et al., 2019; Belot et al., 2019). Yet, many countries face hard times identifying young school dropouts and directing them to public agencies through which they can receive such assistance.

See, for instance, the Organisation for Economic Co-operation and Development (OECD) collection Investing in Youth.

Nowadays, most individuals communicate through SMS daily.

I use the terms SMS, texts, text messages, or text messaging as synonyms throughout the whole paper.

The style adopted in the SMS seems to matter, especially for young people. Some studies show that texts addressed to young people should be carefully analyzed if they are to provide them with better advice about education-, health-, or life-related choices (Hudson et al., 2012; Graham, 2013; Ehrenreich et al., 2014). For instance, the US firm American Telephone & Telegraph Company (AT&T) saw an increase in positive reactions from young people after broadcasting a series of television commercials in which the protagonists spoke like young people’s text messages (Jones and Schieffelin, 2009). Accordingly, if public assistance agencies were to adopt an informal language for their communication, would they be more effective in directing young dropouts?

This paper is the first to address these two questions in the field by sending SMS randomly, with either formal or informal content, to direct young dropouts who are not in employment, education, or training (NEET) toward public assistance agencies in France.

This paper mostly contributes to the literature on program take-up through the provision of information. In the United States, some studies find positive effects on disability insurance take-up (Armour, 2018), on the demand for tax credits (Bhargava and Manoli, 2015), on social security subscription (Finkelstein and Notowidigdo, 2019), on voting for political elections (Gerber et al., 2008), and on labor force participation from letters correcting misconceptions about social security earnings (Liebman and Luttmer, 2015). Barr and Turner (2018) find a positive effect on higher education enrollment from letters pointing out the benefits of training for displaced workers after the financial crisis. Bettinger et al. (2012) find positive effects on college enrollment of American high school students by assisting them throughout the application process. In Canada, Oreopoulos and Dunn (2013) find that online information and video tutorials increase the willingness of high school students to pursue higher education. In Germany, Berkes et al. (2019) find positive effects in terms of improving graduate students’ beliefs about the benefits of graduation returns by providing online information via an interactive survey. Altmann et al. (2018) find a positive effect on exit from unemployment from an information brochure pointing out the harm of being unemployed and suggesting strategies for a return to job-seeking, but only among long-term high-risk unemployed job seekers. In France, Goldzahl et al. (2018) find no effect from information letters related to breast-cancer screening uptake, which describe the risks of this form of cancer and suggest a free-of-charge service with a voucher. However, it is difficult to disentangle the effect of the information itself from the channel through which it is delivered in those papers, especially for those that involve several communication media or multiple information content over time. In the present study, young dropouts received simple SMS – whose language style is either formal or informal – directing toward local public agencies delivering job search assistance programs. It concludes that sending SMS, whatever style is adopted, does not increase the number of young dropouts enrolled in those agencies.

The remainder of the paper is organized as follows. Section 2 presents the relevant French institutions and some characteristics of young dropouts. Section 3 describes the experimental design. Section 4 shows the results of the experiment. Section 5 concludes the paper.

All French youths are required to remain in the education system until the age of 16 years. While at school, they start a citizenship pathway built on three compulsory stages. The first stage concerns classes related to national defense, all during the 9th and 11th grades. The second involves registering at the town hall at most 3 months after the 16th birthday. The third stage is the army day, called “La Journée Défense et Citoyenneté” (JDC).

Young people attend the army day once after receiving an official invitation from the Ministry of the Army. Attendance is required by law when taking any diploma or competitive exam under the control of the public authority at <25 years of age. Army days have taken place every year at different military centers since 1998. About 40–50 young people attend a given army day at a specific military center. The army day agenda is set between 8:30 am and 5:00 pm. At the beginning of the day, all participants must fill in a form pertaining to their situation with respect to schooling or the labor market. They take a 30-min test before lunch to assess their proficiency in French. During the remainder of the day, military instructors aim to raise the participants’ awareness of national security and other social issues, such as drug abuse, road safety, racism, and so on. They also inform participants about public institutions that supply active labor market programs.

Every year, about 800,000 young people participate in army days. According to a report from the French general accounting office, 96%–98% of all French-born individuals do their army day before they turn 25 years old (Cour des Comptes, 2016). Information filled by youth at the beginning of the army day is recorded by military men in an information system called the “Système d’aide à la gestion des administrés” (SAGA). This database is primarily used as an up-to-date census of French people who could be called up in wartime.

Table 1 shows the aggregated values of some characteristics averaged over the period January 2013–July 2019. Information on all youths who attended the army day is shown in Column (1), while Column (2) restricts the sample to school dropouts. It appears that dropouts are more often males, do their army day more often when older, more often have an educational level equivalent to middle school, are less proficient in French, and are more directed toward a partner institution that supplies mostly active labor market programs.

Among them, the mission locale (ML) agencies comprise a network of French institutions dedicated to dealing with 16- to 25-year-old youths who potentially face problems in relation to employment, health, housing, transport, psychology, and so on. There are about 440 agencies spread over the whole territory and 13,000 caseworkers performing individual or collective meetings. At the local level, each agency is free to publicize its service through an appropriate medium. Agencies may variously put up posters on walls, communicate through social media, participate in school or business meetings, and so on. However, there is no record or follow-up about the effects of such attempts. At the national level, the main call for young people to join is made by military instructors during the army days. Table 1 shows that about one third of the dropouts are directed to an agency.

It is only possible to verify whether directing the youths was successful by merging SAGA together with the information system of the agencies called Informations Missions Locales (IMILO). Table A1 in Online Appendix A shows that the effect of military guidance is positive when controlling for individual characteristics and month fixed effects. Military men increase the enrollment rate of young dropouts in public agencies by about 8 percentage points (pp) on average, which corresponds to an increase of approximately 17%. This result is thus driven by selection effects and cannot be interpreted as causal. The experiment presented in the section aims to analyze the probability of going to such public agencies following the receipt of an SMS.

The experiment involved sending two SMS to youths identified as dropouts during army days to direct them to the nearest ML public agency.

Dropouts were randomly allocated to one of three groups. One fifth of the dropouts did not receive SMS and thus constituted the control group. Another fifth made up the first treated group and received SMS, with formal language and content giving the name and the postal address of the nearest agency. The remaining three fifths were allocated to a second treatment group, with more informal language and subdivided according to an additional specific piece of information. All the second subtreatment groups received the same basic information as the first treated group and additionally received information about the distance in kilometers, the past enrollment rate, or both. Full texts related to each group are shown in Table 2.

Table B1 in Online Appendix B shows the text content in the original version, and Figure B1 shows how they are displayed on a smartphone screen.

SMS sent in the first treatment are similar to typical texts sent by some public agencies, with whom the design of the messages was discussed. It contains basic information (name and location of the agency) and no particular design for the text content. In contrast, SMS in the second treatment group added content on the basis of literature in psychology and brain science with the hope of better matching the way young people communicate. Jones and Schieffelin (2009) note that young people communicate through texting in a way that differs from the standard forms of speaking and writing, by playing with words, grammar, and so on. Though structurally equivalent to the SMS in Treatment 1 for statistical comparisons, the SMS in Treatment 2 include specific elements, such as an informal tone or a smiley associated with positive emotions when referring to oneself or undertaking actions after reading the texts.

The dissimilarity between the SMS in Treatment 1 and Treatment 2 and the informal tone are far more perceptible in the original version in French than in English because of the difference implied by the tutoiement (“tu”) and vouvoiement (“vous”), which does not exist in English (only “you”). See Table B1 in Online Appendix B and Table C1 in Appendix C for more details.

The experiment includes youths who did their army day between January 1, 2019 and May 31, 2019. Two conditions were needed to be satisfied in making the selection:

The youth was a dropout and had never attended an ML agency;

I used SAGA and IMILO databases to carry out the experiment. Both databases are updated monthly with a 1-monthlag, i.e., the SAGA database of February 2019 included all youths who did their army day up to January 31. The same applies for IMILO. After obtaining a copy of the two databases, I cleaned the information related to personal records (last name + first name + gender + date of birth + place of birth). Once the two databases were cleaned, I extracted the sample by merging them on names, using the Jaro-Wrinkler distance algorithm and exact matching on gender, date of birth, and place of birth. The output file listed dropouts who had never registered with an agency.

The next task was to assign a particular agency to each youth. Agencies accept youths who live in the same geographical area, generally at the commuting zone level. Otherwise, they redirect them to the appropriate agency. Since the postal addresses of both youths and the agencies were available in the database, I assigned the agency located nearest to each individual, based on the geodesic distance algorithm, provided it was in the same geographical unit.

In total, 4,457 youths participated in the experiment, and 3,540 youths were contacted from March 6, 2019 to July 17, 2019.

The SMS sender’s name was restricted to eight characters by the digital platform (twilio.org) and set to “INFOS ML” for “Informations Missions Locales”.

Table E1 in Online Appendix E shows that the randomization process was successful. Therefore, I estimate the following linear probability model with ordinary least squares (OLS) estimators to analyze the effect of the SMS: (1) yij=α+βkSMSi=k+X′γ+εij# {y_{ij}} = \alpha + {\beta _k}SM{S_{i = k}} + X'\gamma + {\varepsilon _{ij}}\,\# where y_ij is a dummy variable equal to one if youth i went to the ML agency j, zero otherwise. SMS_i=k is a dummy variable equal to one if youth i received SMS k ∈ {Formal; Informal}, zero otherwise. X is a vector of control variables including information displayed in the text, individual characteristics, agency characteristics, location characteristics, and month fixed effects. ∊_ij is a residual term, orthogonal to treatment variables because of randomization. Turning to parameters, β_k is of interest and measures the intention-to-treat (ITT) effects, i.e., the differential in probabilities of going to an agency in comparison to the control group (which receive no SMS at all) with a group receiving SMS of type k. The OLS estimates of β are reported in Table 3.

Directing young dropouts through low-cost SMS has little chance of triggering the expected behavior since the field experiment presented in this paper shows no effect of SMS on enrollment in public agencies delivering mainly job search assistance. As noted by Babcock et al. (2012), information transmitted to job seekers “should be simple and personalized” and “the presentation and context of this information can potentially invoke frames of loss or gain”. While the former point can be considered satisfied in this experiment as the SMS were quite concise, written in a simple language, and individualized to some youth and agency characteristics, there is room for improvement given the latter point; however, there are inherent difficulties considering the nature of SMS.

Insights from behavioral economics indicate that the psychological and external barriers encountered by young job seekers may be too deep, and so their expectations are misaligned with reality. Some may overestimate their propensity to exit from a nonemployment situation and find a sustainable alternative by themselves.

Spinnewijn (2015) shows that 80% of US job seekers underestimated their unemployment duration. Mueller et al. (2018) show that about 10% of the incidence of long-term unemployment can be attributable to optimistic bias in the job-finding rate.

Future information campaigns designed by public institutions should try to incorporate elements that clearly state the gains and losses that young dropouts might face when choosing options over a default. The language style adopted in the information campaign and its potential connotation associated with the type of institution might therefore be correlated to the specificity of information. Information related to gains may be framed or displayed in a way that differs from information related to losses. More research in this direction is needed to provide information to young dropouts in a salient way if one wants to increase participation in labor market policies. This point implicitly assumes that information on agencies’ performances is available. Small-scale tests on a preselected panel of young dropouts and cooperation among policymakers, economists, psychologists, and behavioral scientists should be at play to design and evaluate future information campaigns.