Scaling-up Education Reforms in Kenya: An Evaluation of the Nationwide 'Teacher Internship Programme'

This project will evaluate the nationwide roll-out of a controversial education programme in Kenya that has already been shown to raise student performance in small-scale randomized trials. In the process the project will provide insights on the external validity of micro evaluations, and on the political economy of scaling-up effective interventions. In August 2009 the Kenyan Minister of Education, Prof. Sam Ongeri, announced that the Ministry would bypass the Teacher Service Commission (TSC) – responsible for employing all government teachers – and transfer funds directly to local School Management Committees (SMCs) to hire 12,000 teachers on an informal basis. These plans are currently on hold, pending political negotiations. In the meantime, the current funding proposal describes a collaboration between the Kenyan National Examination Council (KNEC) and World Vision Kenya together with the University of Nairobi, Oxford University and Stockholm’s University to rigorously evaluate a pilot version of this new “Community Support Teacher Program” (CST) before it goes to scale. This project aims not only to provide lessons about the possibility of extrapolating from micro evaluations, but also to work hand-in-hand with government to build rigorous policy evaluation into the policymaking process in Kenya’s education sector. From the research side, the project takes as its starting point recent results by Duflo, Dupas and Kremer (DDK 2009a), who evaluate a similar intervention implemented by a faith-based NGO in Kenya’s Western province. Duflo et al. find that locally-hired contract teachers – particularly when combined with a programme to increase parental monitoring of teacher performance – significantly raise student exam scores relative to students in classes taught by civil-service teachers. This project will examine the ability of the Kenyan government, through the CST initiative, to replicate these results on a national scale. The impetus for the CST initiative has complex political origins outside of academic research, but the Technical Committee overseeing the details of its design and implementation – specifically, the researchers listed here – have made frequent reference to the results of randomized evaluations conducted in Western Kenya in deliberations about the programme’s design (Glewwe, et al. 2004, Duflo et al. 2009a, 2009b). Thus the results of DDK 2009a provide a relevant benchmark for gauging the success of the CST initiative. With the recent proliferation of randomized evaluations within development economics, questions of external validity and the capacity to scale up successful projects have come to the fore. The opportunity to rigorously evaluate a nationwide, government initiative such as the CST provides a unique vantage point from which to address a number of these issues: 1. Political economy. Political constraints have inevitably influenced the Ministry’s chosen programme design (and, not coincidentally, prevented it from perfectly replicating the design piloted in Western Kenya). Will these political compromises undermine the effectiveness of the project? By cross-cutting various sub-components of the programme, we will investigate whether de jure differences in salary levels and contract structure for teachers diminish the impact of the project. Conceptually, we focus on three features of teacher contracts that distinguish community support teachers from TSC and PTA teachers: salary levels, local control, and the link between a teacher’s performance and their job tenure (i.e. the risk, if any, of dismissal for poor performance). 2. Institutional capacity. Holding constant the observable features of teachers’ contracts and the project design, does implementation by government rather than a faith-based NGO significantly affect its success? We propose to investigate this question with the collaboration of World Vision Kenya, who will replicate several variants of the CST programme in a sub-sample of the schools in the research sample. This will provide a rare opportunity to examine the effectiveness of public and NGO provision of an ostensibly identical public service. 3. External validity due to heterogeneity of treatment effects. In our context, this issue boils down to the question of whether Western Kenya is special. The evaluation will test whether previous results from a single province give an accurate indication of impacts outside the original sample. The project investigators are: Dr. Tessa Bold, Institute for International Economic Studies Prof. Mwangi Kimenyi, Brookings Institution Prof. Germano Mwabu, Department of Economics, University of Nairobi Mr. Mukhtar Abdi Ogle, National Coordinator, the Assessment Centre, Kenya National Examinations Council Dr. Justin Sandefur, Centre for the Study of African Economies, Oxford University Dr. Paul M. Wasanga, Council Secretary and Chief Executive, Kenya National Examinations Council