School district investments in general skills: The case of principal residency programs

Due to concerns about the quality of school principals combined with turnover rates around 20 percent annually (Goldring and Taie 2018), CPS launched a principal residency program called the Chicago Leadership Collaborative (CLC) in November 2011 (Chicago Public School 2020). In the spirit of well-known teacher preparation programs like Teach for America, the CLC sought to attract candidates with proven track records of success into leadership roles in Chicago schools. Specifically, the goal of the CLC was to strengthen the pipeline of aspiring school leaders, starting with classroom-based principal training focused on the skills needed to work in CPS specifically and urban schools more broadly, followed by a school-based residency under the guidance of a mentor principal and a transition into a CPS principal position. According to CPS promotional materials, the hope was that the CLC would make Chicago a “destination of choice for urban school leaders.” The programs advertised the rigor of their admissions processes, suggesting that at least some of the value of these programs may be in the screening and selection of principal candidates. Requirements for admission often included prior classroom experience, as well as prior leadership positions such as department or grade-level chairs, teacher mentors, or leadership positions outside the field of education. We note that current principals are not eligible to participate in the CLC.

The CLC includes both classroom and residency components. Students enroll in a participating graduate program at either a local institution or in partnerships with nationally known principal preparation programs (e.g., New Leaders for New Schools), in contrast to the New York City Aspiring Principals program run by the district which compresses classroom instruction into a six-week summer course offered prior to the yearlong residency (Corcoran et al., 2012). The content of the CLC training focuses on issues faced by leaders in urban schools and is aligned with the State of Illinois Principal Standards. A key component of the CLC is the year-long residency that candidates complete in CPS schools under the tutelage of a “high-performing” principal. The CLC training programs also continue to support candidates once they attain principal positions, usually for 1–2 years following program completion. Upon completion of the program and depending on the program requirements, candidates receive either a masters in school administration (MSA) or a doctoral degree in education (EdD).

The district commits substantial funding to the CLC program. Between 2012 and 2017, the district has awarded subcontracts to the principal preparation programs totaling $16,190,004. Per district contracts, that funding goes toward development of curriculum, recruitment of candidates to the programs, and compensation to mentors for candidates who are placed as principals. Candidates participating in the residency receive compensation equivalent to $80,000 per year or their annual salary if they were already in a CPS position, whichever is greater. They also receive the same benefits offered to CPS employees. Candidates must cover the costs of tuition and fees for the preparation programs. Recognizing that the training may make program participants more attractive to other districts, participants are contractually bound to repay the district up to $7,500 if they decline an offer as a CPS administrator or they do not work for CPS for four years upon completion.

The Chicago School Reform Act of 1988 established four-year principal contracts along with a decentralized governance structure in which democratically elected “local school councils” (LSCs) comprised of teachers, parents, and community members maintain authority over principal hiring and evaluation unless the district intervenes for cause (Ryan et al. 1997; Schiman et al. 2018) the 1988 Chicago School Reform Act decentralized school governance by forming elected local school councils (LSCs. Consequently, while the district may influence principal placement, they do not control who is offered a principal contract in most schools.

The few exceptions are 1) schools that are persistently low performing on the district's accountability system, and 2) schools that do not have full membership of their local school councils (e.g., there were not enough candidates who ran to sit on the LSC).

Upon entering a CPS principalship, the principal has multiple roles and expectations. Across school districts, and including CPS, the role of the principal is quite extensive. Principals are expected to lead the school operations, including setting the budget, managing personnel, and maintaining facilities. In CPS, school funding occurs through a student-based budgeting process, which means that dollars are allocated to schools based on enrollment. CPS principals then have discretion around allocating those funds to hire teachers and other school personnel when there are vacancies, as well as evaluating teachers and providing teachers with actionable feedback to improve their instructional practice and, thus, improve learning. CPS principals are also the school's designated instructional leader, meaning that they are supposed to make key decisions as they relate to learning and teacher development. Thus, there are many mechanisms through which principals can ultimately affect student outcomes.

Another important consideration is how principals are compensated in CPS. Similar to other districts and to teacher pay, school administrators in CPS are awarded compensation packages of salaries and benefits as established by the school district and approved by the school board.

Information on the CPS pay policy is available at the following link: https://www.cps.edu/sites/cps-policy-rules/policies/300/302/302-8/ (last accessed 10 January 2023).

Consider the investment decision for an employer who operates in perfectly competitive and frictionless output and labor markets, where there are no hiring costs, and no geography so that workers bear zero cost to change jobs. In the presence of training costs, firms consider both current and future costs and revenues in making hiring decisions. To simplify the presentation, assume that all training costs occur in period zero, the increase in marginal revenue product (MRP) remains constant following training and firms hire until the point that the expected present value of the marginal revenue products over (y + 1) years equals the expected present value of wage and training costs for (y + 1) years (1) Expected present value of MRP for y+1 years=(1−γ)⋆MRP0+∑y=1Y(MRP0+Δ)y(1+r)y {\rm{Expected}}\;{\rm{present}}\;{\rm{value}}\;{\rm{of}}\;{\rm{MRP\, for}}\left( {y + 1} \right)\;{\rm{years}} = (1 - \gamma ) {}^\star {MRP_0} + \sum\limits_{y = 1}^Y {{{{{({MRP_0} + \Delta )}_y}} \over {{{(1 + r)}^y}}}} (2) Expected present value of wage and training cost for y+1 years=w0+k+∑y=1Ywy(1+r)y {\rm{Expected}}\;{\rm{present}}\;{\rm{value}}\;{\rm{of}}\;{\rm{wage}}\;{\rm{and}}\;{\rm{training}}\;{\rm{cost}}\;{\rm{for}}\left( {y + 1} \right)\;{\rm{years}} = {w_0} + k + \sum\limits_{y = 1}^Y {{{{w_y}} \over {{{(1 + r)}^y}}}} where w is wage, k is the financial cost of training, Δ is the training-induced increase in MRP, MRP₀ is marginal revenue product prior to training, and γ is the share of productive time lost to training. Assume the firm incurs all training costs in year 0.

In equilibrium, firms hire until (1) and (2) are equal. Given the assumptions of a perfectly competitive and frictionless labor market with zero hiring costs, outside firms will be willing to pay a wage equal to MRP₀ + Δ. Therefore, to retain a trained worker the firm must raise the wage equal to MRP₀ + Δ in all subsequent years. This produces the familiar result that MRP₀ + Δ = w_y in all periods, the summation terms in equations (1) and (2) exactly cancel out, and firms receive no benefit from the increase in productivity. Consequently, profit maximizing firms would not be willing to bear any training costs. As equation (3) shows, financial (k) and time (γMRP₀) costs of training are borne by the worker in the form of lower wages in period zero. (3) w0=MRP0−γMRP0+k {w_0} = {MRP_0} - \left( {\gamma {MRP_0} + k} \right)

Of course, CPS and other large urban districts do not function as profit maximizing firms, and the labor market for principals diverges from perfect competition in myriad ways including rigidities in district salary structures, the treatment of experience gained in other districts, the geographic specificity of schools, licensing requirements, and imperfect information about productivity. This highlights the importance of incorporating relevant institutional features into the analysis. Three particularly important factors for CPS would appear to be 1) delayed hiring as a principal resulting from local school autonomy over personnel decisions rather than centralized district assignment of candidates to schools; 2) salary compression within CPS which is typical for large urban districts; and 3) the fixed costs of switching districts.

Dinerstein, Megalokonomou, and Yannelis (2019) find that a longer delay between teacher training and hiring leads to skill depreciation, and Autor et al. (2015) finds that employment delay leads to human capital deterioration.

As mentioned previously, principal contracts in CPS are managed in a decentralized way, relying on school-based governance, which means that central office does not determine who is hired to be a school principal. Therefore, residents must compete for positions school by school, and that likely decreases the probability a resident is hired simply based on participation in the program. Additionally, an LSC may prefer a candidate over a resident for reasons other than leadership skills. These circumstances would result in increased time between program completion and entrance into the principalship with most candidates entering assistant principal or other positions immediately following completion of the CLC program rather than going straight to the principalship. Regardless of the reason, to the extent that the CLC training develops skills that are more applicable to a principal's role and responsibilities than an assistant principal's job, the delays in procuring a principalship likely lower the training benefit to the district.

Spending additional years as an assistant principal likely also degrades the value of the training. In the standard career path, a teacher transitions to an assistant principal position and subsequently to a principalship. A primary rationale for residencies is that assistant principal responsibilities hinder the acquisition of leadership skills and that residents will learn much more from the mentor principal who is selected based on performance. However, if the resident remains an assistant principal following program completion, some of the knowledge acquired during the residency will be lost or potentially irrelevant as the district changes. A longer delay likely amplifies this skill depreciation. Even if the training enhances productivity as an assistant principal, this is unlikely to compensate for the negative effects of the delay. Note that such skill degradation becomes far less relevant if selection on pre-existing skills rather than skill acquisition during the residency determines the higher productivity of residents.

Regardless of whether program participants serve as principals or assistant principals, the largely fixed CPS salary structure based on years of experience and size of school introduces a divergence between productivity and salary. Higher pay does not accompany program completion nor is pay typically determined by performance, so if residencies elevate productivity the salary in CPS will not reflect the increase in effectiveness. As Acemoglu and Pischke (1999a; 1999b) emphasize, such salary compression with respect to productivity potentially redistributes some of the returns to the acquisition of general skills from the worker to the firm, or in this case from the program participant to the district.

Equations (4) and (5) incorporate the delayed ascension to the principal position and CPS salary compression into equations (1) and (2) (4) Expected present value of MRP for y+1 years working in CPS=(1−γ)⋆MRP0+∑y=1A(MRPa+Δa)y(1+r)y+∑y=A+1Y(MRPp+λAΔp)y(1+r)y \matrix{ {{\rm{Expected}}\;{\rm{present}}\;{\rm{value}}\;{\rm{of}}\;{\rm{MRP}}\;{\rm{for }}\,y + 1\;{\rm{years}}\;{\rm{working}}\;{\rm{in}}\;{\rm{CPS}} = } \hfill \cr {(1 - \gamma ) {}^\star {MRP_0} + \sum\limits_{y = 1}^A {{{{{({MRP^a} + {\Delta ^a})}_y}} \over {{{(1 + r)}^y}}}} + \sum\limits_{y = A + 1}^Y {{{{{({MRP^p} + {\lambda ^A}{\Delta ^p})}_y}} \over {{{(1 + r)}^y}}}} } \hfill \cr } (5) Expected present value wage and training cost for y+1 years=w0+k+∑y=1Aw¯ya(1+r)y+∑y=A+1Yw¯yp(1+r)y \matrix{ {{\rm{Expected}}\;{\rm{present}}\;{\rm{value}}\;{\rm{wage}}\;{\rm{and}}\;{\rm{training}}\;{\rm{cost}}\;{\rm{for }}\,y + 1\;{\rm{years}} = } \hfill \cr {{w_0} + k + \sum\limits_{y = 1}^A {{{\bar w_y^a} \over {{{(1 + r)}^y}}}} + \sum\limits_{y = A + 1}^Y {{{\bar w_y^p} \over {{{(1 + r)}^y}}}} } \hfill \cr } where the benefits of training (Δ) and the wage depend upon whether the participant is an assistant principal (α) or principal (p), the wage is determined by salary schedule and not productivity (w̅) and the value of the training to productivity as a principal depreciates geometrically with the length of delay (A) at the rate 1 − λ. We assume Δp > Δa, i.e. that the benefit of the training is larger when working as a principal than as an assistant principal.

Equations (4) and (5) illustrate that the fixed salary structure and delayed employment as a principal have offsetting effects on the benefits of the program to CPS. On the one hand, the longer is the delay and the larger is the annual delay-related knowledge depreciation the lower is the training benefit. On the other hand, the wage compression for both assistant principals and principals redistributes at least a portion of the benefit to the district.

Of course, the duration of employment in CPS also affects the net return of the residency investment to the district. Differences in total compensation between a position in CPS and in an alternative district depend upon both pecuniary and non-pecuniary factors, and any weight placed on the opportunity to lead an urban school raises the probability of turning down a competing offer outside of CPS. In addition, commuting costs for those who have a strong preference for living in Chicago may be lower for a position within as opposed to outside of CPS. Finally, residency participants agree to pay CPS a nominal amount if they leave the district prior to an agreed upon number of years of employment, though the value of the promise likely exceeds the small monetary penalty.

Taken together, job and location preferences and the agreement to work in the district following program completion may introduce a large wedge between the CPS salary and the reservation offer that would induce a transition to a position outside of CPS even if the residency program substantially raises productivity. Therefore, the district may realize a substantial net benefit from investments in principal residency training programs that exceeds the net benefit of the next best use of education funds even if the program primarily augments general leadership skills. A more effective principal elevates the skill acquisition of the entire school rather than only a single classroom as is the case with a more effective teacher. Therefore, the influence on leadership quality would not have to be large to generate a positive return on the district investment.

A growing body of research highlights the difficulties of identifying principal effectiveness as measured by value added to achievement or other outcomes Branch et al (2020), (Chiang, Lipscomb, and Gill (2016), Grissom, Kalogrides, and Loeb (2015), and Miller (2013)). The separation of principal effects from time-invariant and time-varying school effects including student composition presents a primary challenge, and we adopt an event-study framework with schools that hire non-resident principals during the same time period as controls. Restricting the controls to schools with principal transitions accounts for the dynamics identified by Miller (2013), which finds substantial average decreases in achievement just prior to principal transitions. Information on pretrends and changes in demographic composition illuminate divergent trends in unobserved school factors and student characteristics that could introduce bias.

Because of the endogeneity of transitions, we do not restrict the sample to the years in which the new principal remains in a school. Rather, the parameters capture the intent to treat and reflect differences in both productivity and transition costs resulting from turnover. Alternative estimates from samples that exclude principals who either leave within three years or are hired too recently to have three years of data are quite similar. The hiring date is determined by the first principal transition observed during the 2014 to 2018 sample frame, so a subsequent transition does not alter the initial classification based on residency status.

Equation 7 presents the basic event-study specification: (7) Aist=α+∑j=−5−2ρjlag jst+∑k=12τklead kst+θs+δt+∈ist {A_{ist}} = \alpha + \sum\nolimits_{j = - 5}^{ - 2} {{\rho _j}{{\left( {lag\;j} \right)}_{st}}} + \sum\nolimits_{k = 1}^2 {{\tau _k}{{\left( {lead\;k} \right)}_{st}}} + {\theta _s} + {\delta _t} + { \in _{ist}} where achievement for student i in school s in year t (A_ist) is a function of school and time fixed effects, a random error and indicators for the periods leading up to and following the hiring of a resident principal. Period j = −1, the year prior to the new hire, is the omitted period, and k=0, the first year of a new principal spell, is dropped because of concerns about a recovery from a dip in the final year of the previous principal and the more limited influence over staffing, climate and other school operations in the first year. This two-way fixed effects (TWFE) framework controls for fixed differences between schools and time periods, and the parameters on the lags and leads capture the time-pattern of achievement around the hiring of a resident principal.

The key estimates of interest, ρ_j and τ_k, represent the periods j and k differences in achievement between schools that hired a new resident and the controls. The omitted period in the regression is the year before the new resident was hired (period t = −1), meaning that ρ_j and τ_k are benchmarked against that period. The pre-period estimates of ρ_j provide evidence on the common-trends assumption, and the estimates of τ_k capture resident-nonresident differences in achievement after the hiring of the new principal. If the hiring of a resident does not improve leadership quality and raise achievement, the estimates of γ_k would not be expected to be significantly different from zero. However, if residents are associated with improvements to test scores, we would expect to see a positive divergence from the pre-trend and positive and significant estimates of γ_k.

Because the effectiveness of resident principals may vary by program cohort, year hired, and school, concerns that TWFE models do not produce easily interpretable estimates of treatment effects certainly arise in this context.

A growing number of papers address this issue including Borusyak et al. (2021), Callaway and Sant’Anna (2021), De Chaisemartin and d’Haultfoeuille (2020), Goodman-Bacon (2021) and Sun and Abraham (2021).

Todd and Wolpin (2003) model achievement as a cumulative function of family, school and neighborhood factors, leading to a focus on achievement gain rather than level in the estimation of school, teacher or principal effects. We therefore replace the level of achievement with achievement gain in our preferred specifications. Differencing out the prior-year score accounts for unobserved differences in academic preparation owing to the cumulative effects of student, family, school and community factors. In this specification, differences in principal effectiveness appear as differences in learning and annual achievement growth. Importantly, the gain measure equals (A_it −0.7^* A_i,_t−1), thereby allowing for knowledge depreciation of 30 percent. Estimates are similar under the simple gain model equal to the difference in test scores between the current and previous year, though the simple gain model imposes the assumption of no knowledge depreciation which is not consistent with the evidence.

Meghir and Rivkin (2011) discuss the assumptions imposed by the achievement gain specification.

Even in a gains specification, endogenous changes in student composition in response to the hiring of a new resident could introduce bias if they are related to achievement growth. Importantly, pretrends would be unaffected by such responses, and we therefore utilize an event-study framework to investigate changes in observed student characteristics following the hiring of a resident.

Similar to Grissom, Mitani, and Woo (2019), our results reflect changes in performance that occur in the first 2 to 3 years after hiring a new principal. It is possible that the benefits of a residency do not appear in the first three years in the leadership position. However, Branch et al. (2020) show evidence of substantial variation in principal value added to achievement within the first three years that is significantly related to teacher survey responses on leadership effectiveness. Moreover, roughly half of principal spells are four years or shorter, suggesting that the impact of a successful intervention would almost certainly appear within the first three years. Because we have only two years of data for principals hired in 2018, the set of schools that contribute to the third-year effect differs from the set that contributes to the second-year effect. Therefore, some estimates come from samples restricted to schools that contribute three treatment years. Note also that the main estimates ignore any subsequent transitions, meaning that the estimates capture the effect of hiring a resident on outcomes two or three years later regardless of whether the resident remains or transitions out of a school. Alternative estimates come from samples limited to new principals who remain in their positions for at least three years.

This section reports the residency effects on principal productivity as measured by achievement level and gain. Table 4 reports treatment effects for math and reading achievement level and gain based on the Equation 7 event-study specifications and the Gardner (2022) two-stage method. Subsequent tables report estimates based on restricted samples that either exclude schools with fewer than three years of outcomes following the principal hire (Table 5) or principal spells lasting fewer than three years (Table 6). Standard errors are clustered by school. Appendix Tables a1 to a3 present TWFE estimates that do not account for heterogeneous treatment effects for the respective samples.

The pre-period coefficients in Table 4 (period=−5 to period=−2) for schools that subsequently hire a resident participant reveal modest fluctuations in the level of achievement in periods −5 to −3, none of which are significant at conventional levels. However, the combination of modest negative effects in period −5 and positive effects in period −4 produces significant and somewhat larger jumps in achievement gain for both math and reading in period −4. The magnitude of the coefficient in period −4 is roughly 40 percent as large as the treatment effects in math and even larger in reading. In contrast, the pre-period coefficients in periods −3 and −2 are roughly one tenth as large in math and one fourth as large in reading and insignificant. Thus, math and reading achievement and achievement gains appear to be moving in parallel in treatment and control schools in the three years leading up to the hiring of a resident principal following a modest jump in period −4. In fact, none of the math coefficients are larger than 0.007 in the levels specification or larger than 0.002 in magnitude in the gain specification in the years following period −4; the magnitudes in the reading specifications are only slightly larger.

The resident interaction-term coefficients for periods 1 and 2 showing the treatment effects in period 1 (second year) and period 2 (third year) reveal positive effects of hiring a resident principal that are larger and more significant in math. The coefficients of 0.0516 (period 1) and 0.0503 (period 2) in the preferred gains specification are highly significant and large: they exceed in magnitude estimates of a one standard deviation difference in principal effectiveness for the Chicago Public Schools (Branch et al., 2022). The reading coefficient in Column 4 is roughly 40 percent smaller though highly significant in period 2 and half as large and not significant in period 3.

The unbalanced panels used to produce the Table 4 estimates include principals hired in 2018 who do not contribute to the estimates of period 2 effects, meaning that the differences between period 1 and 2 effects come from variation in composition and tenure. Table 5 reports estimates for the same specifications estimated over balanced panels of schools that all have three years of post-period observations. The estimates are quite similar to those for the unbalanced panel in Table 4: only slightly smaller but still significant effects on math achievement gain in both periods, a highly significant effect on reading achievement gain in period 1, and a smaller and not significant effect on reading achievement gain in period 2. The strength of the resident effects on math achievement gain supports the belief that residents significantly improve math instruction and achievement growth.

The estimates in Tables 4 and 5 come from samples that include observations following the departure of a resident or nonresident principal, meaning that they capture the effect of hiring a resident, not of the effectiveness differences between residents and nonresidents. It is informative to produce estimates for the restricted sample of schools where the newly hired principal remains at least three years, recognizing that these principals are not randomly drawn from all new hires. The slightly larger estimates, particularly in period 2, are consistent with the notion that turnover imposes transaction costs that adversely affect learning and achievement, though the differences may also reflect the positive selection of resident principals who remain in their positions for at least three years.

A comparison between the estimates in Table 4 and Appendix Table a1 reveals that the failure to account for heterogeneous treatment effects leads to much larger coefficients in the gains specifications in both the pre and treatment periods; the same pattern holds for the more restrictive samples used to produce the estimates shown in Tables 5 and 6. These differences highlight the importance of accounting for heterogeneous treatment effects in this context.

A final sensitivity test examines the importance of accounting for any achievement dip in the year immediately prior to the hiring of a nonresident principal. Appendix Table a4 shows that estimates are slightly smaller but quite similar in magnitude and significance to those in Table 4 that do not include any controls.

Changes in student composition following the hiring of a resident could potentially introduce bias, and Table 7 reproduces the event-study analysis substituting student characteristics in place of achievement as outcome variables. The estimates suggest a decline in share Black and roughly corresponding increase in share Hispanic of approximately 1.5 percentage points following the hiring of a resident principal. Estimates of the relationship between achievement and achievement gain on the one hand and student characteristics on the other from TWFE models that include covariates reported in Appendix Table a5 indicate that such a compositional change is likely to increase achievement. However, the coefficients for Black and Hispanic students in the math achievement gain specifications of −0.18 and −0.11, respectively, suggest the compositional change effect is quite small: multiplying the difference in coefficients by 0.015 indicates that the change in race-ethnic composition increases achievement gain by 0.001 standard deviations, which is less than 2 percent of the resident effect. Even in the presence of a positive change in orthogonal unobserved student characteristics that has an effect that is twice as large as that introduced by the change in racial composition, compositional changes would account for less than 10 percent of the resident effect for math and an even smaller amount for reading given the smaller difference between coefficients for Black and Hispanic students reported in Appendix Table a5.