American Institute for Research
Results from a Three-Year i3 Impact Evaluation of the Children’s Literacy Initiative
Overview
Learning to read by the end of third grade is a strong predictor of a student’s future academic achievement. Teachers’ literacy instruction plays a vital role in students’ reading development, but many teachers fail to receive adequate training in this area. A newly completed study from the American Institutes for Research found Children’s Literacy Initiative’s professional development program for teachers has a measurable impact on teacher and student literacy outcomes.
In 2013, the Nation’s Report Card determined that only 35% of U.S. fourth grade students were proficient readers. This means more than 10.6 million five-to-eight year-olds in kindergarten through third grade are not reading on grade level.1
Students living in poverty are at the greatest risk of not reading on grade level; 83% of low-income students test below proficient in reading at the start of fourth grade.2 This situation is particularly dire considering that reading by the end of third grade is an important predictor of later-in-life outcomes, such as graduating from high school on time. Students who are not proficient readers by the end of third grade are four times more likely to leave high school without a diploma.3 Furthermore, adults who did not complete high school are more likely to be chronically unemployed and to live at or below the poverty line,4 reinforcing the cycle of economic disadvantage.
Teachers can play a pivotal role in turning these statistics around by ensuring students reach appropriate reading milestones. In fact, teacher effectiveness can account for as much as 90% of the variation in literacy achievement among students.5 Although teachers cannot change their students’ economic situation, they can make a difference for students from low-income families. When a student has an effective teacher for three years in a row, the negative impact of socioeconomic disadvantages is eliminated.6
In order to be effective, teachers require ongoing professional development. Unfortunately, a majority of these programs are unfocused, disconnected from in-class lessons and infrequent enough to actually improve teacher practice.7 In addition, there are few interventions that target teacher professional development to improve students’ reading achievement.
Children’s Literacy Initiative (CLI) is a non-profit organization focused on strengthening education in the U.S. by ensuring students can read by the end of third grade. CLI has honed its professional development program around early childhood literacy practices to improve both teacher and student outcomes. CLI professional development differs from other available programs in that it is evidence-based, connected to teacher daily practice and intensive. Teachers participate in training seminars where they learn practices that are reinforced by in-class coaching sessions with CLI Professional Developers. This provides teachers with the hands-on training they need to succeed.
For 30 years, CLI has been working with educators to transform instruction so that children can become powerful readers, writers, and thinkers. During the 2013/14 school year, CLI served 248 schools and more than 34,000 students, coached 1,841 educators and distributed more than 81,000 books to schools around the country.
- NAEP, 2013
- ibid
- AECF, 2011
- Sum, Khatiwada, & McLaughlin, 2009
- National Commission on Teaching and America’s Future, 1998
- Education Trust
- Gates Foundation, 2014
i3 Evaluation
In 2010, CLI received a federal Investing in Innovation (i3) grant to implement its early-childhood literacy program with kindergarten through second-grade teachers in four U.S. school districts with high-need student populations: Philadelphia, PA, Chicago, IL, Newark, NJ, and Camden, NJ. This program provided teachers more than 460 hours of seminars and in-class coaching sessions. In addition, CLI mentored school leadership on how to best promote an environment of learning in classrooms and provided schools with book collections and other literacy resources.
CLI enlisted the help of the American Institutes for Research (AIR), to conduct the independent evaluation of CLI’s impact. AIR randomly assigned participating schools to either the treatment group (received CLI services immediately) or control group (received CLI services following data collection). Therefore, any differences observed between the two groups at the end of the study are attributable to the CLI program and not to preexisting differences between the two groups and teachers and students receiving CLI services can be compared to similar teachers and students who did not receive services.
Results
Teachers’ Literacy Instruction Practices & Classroom Environment
Teacher practice and classroom environment were measured by the Early Language and Literacy Classroom Observation (ELLCO), an observational measure of the presence of literacy best practices. This included how well and how often a teacher encouraged extended classroom discussions, facilitated oral language development and whether the teacher regularly used and updated a word wall in the classroom. Each element was rated on a scale of 1 (poor) to 5 (excellent).
The study found that:
- Teachers who received CLI training had a significantly more positive classroom environment than teachers who did not receive CLI services
- Teachers who received CLI services exhibited significantly more language and literacy practices in the classroom
Reading Achievement – Kindergarten
Student reading achievement for kindergarten students was measured using the Predictive Assessment of Reading (PAR). The PAR is an individually administered test of pre-reading skills, including students’ letter-word reading, picture vocabulary, phonemic awareness, and rapid naming fluency. First and second-grade students’ reading achievement was measured using the Group Reading Assessment and Diagnostic Evaluation (GRADE). The GRADE is a group administered a multiple-choice assessment that is completed in two sessions and assesses students’ word reading as well as meaning and comprehension. Overall, each assessment measures students’ literacy skills. We refer to overall scores on each measure below as “reading achievement.”
Kindergarten CLI students exhibited significantly greater reading achievement than non-CLI students.
- Especially students in classrooms with teachers who had received CLI training for two years
Average non-CLI kindergarten student would perform better than 50% of peers, but with CLI-trained teacher would perform better than 57% of peers.
- Translates to an improvement index of 7 for kindergarten students (calculated using WWC methods: http://ies.ed.gov/ncee/wwc)
Reading Achievement – First Grade
Overall, first grade CLI students did not exhibit greater reading achievement relative to first-grade non-CLI students
- In year 1, effect size of the difference between CLI and non-CLI first-grade students was nearly zero
- In year 2, effect size was larger (.06), but was still not significant
However, it is important to consider that students enter first grade with a wider range of pre-existing achievement levels than in later grades. Therefore, AIR examined whether CLI services helped reduce gaps in reading achievement for those entering first grade as low performers versus high performers. They found that students entering CLI classrooms as low performers showed significantly greater reading achievement compared to low-performing control students, reducing gaps between low performers and high performers in CLI classrooms.
Reading Achievement – Second Grade
Second grade CLI students also exhibited significantly higher reading achievement than non-CLI students
- On average, second grade CLI students scored at the 38th percentile for their age, while non-CLI second graders scored at the 35th percentile
Based on reading achievement scores, second grade CLI students received the equivalent of two additional months of instruction as compared to non-CLI students.
- That is, without the expense and logistical difficulties of keeping school open for summer, CLI teachers provided students with the instructional equivalent of attending school nearly year-round
Conclusion
- Rivkin, Hanushek, Kain, 2005
Download the Summary
Fill out the form below to download the American Institute for Research's Three-Year i3 Impact Evaluation summary.