American Institute for Research: Results from a Three-Year i3 Impact Evaluation of the Children’s Literacy Initiative

Overview

Learning to read by the end of third grade is a strong predictor of a student’s future academic achievement. Teachers’ literacy instruction plays a vital role in students’ reading development, but many teachers fail to receive adequate training in this area. A newly completed study from the American Institutes for Research found Children’s Literacy Initiative’s professional development program for teachers has a measurable impact on teacher and student literacy outcomes.

In 2013, the Nation’s Report Card determined that only 35% of U.S. fourth grade students were proficient readers. This means more than 10.6 million five-to-eight year-olds in kindergarten through third grade are not reading on grade level.1

Students living in poverty are at the greatest risk of not reading on grade level; 83% of low-income students test below proficient in reading at the start of fourth grade.2 This situation is particularly dire considering that reading by the end of third grade is an important predictor of later-in-life outcomes, such as graduating from high school on time. Students who are not proficient readers by the end of third grade are four times more likely to leave high school without a diploma.3 Furthermore, adults who did not complete high school are more likely to be chronically unemployed and to live at or below the poverty line,4 reinforcing the cycle of economic disadvantage.

Teachers can play a pivotal role in turning these statistics around by ensuring students reach appropriate reading milestones. In fact, teacher effectiveness can account for as much as 90% of the variation in literacy achievement among students.5 Although teachers cannot change their students’ economic situation, they can make a difference for students from low-income families. When a student has an effective teacher for three years in a row, the negative impact of socioeconomic disadvantages is eliminated.6

In order to be effective, teachers require ongoing professional development. Unfortunately, a majority of these programs are unfocused, disconnected from in-class lessons and infrequent enough to actually improve teacher practice.7 In addition, there are few interventions that target teacher professional development to improve students’ reading achievement.

Children’s Literacy Initiative (CLI) is a non-profit organization focused on strengthening education in the U.S. by ensuring students can read by the end of third grade. CLI has honed its professional development program around early childhood literacy practices to improve both teacher and student outcomes. CLI professional development differs from other available programs in that it is evidence-based, connected to teacher daily practice and intensive. Teachers participate in training seminars where they learn practices that are reinforced by in-class coaching sessions with CLI Professional Developers. This provides teachers with the hands-on training they need to succeed.

For 30 years, CLI has been working with educators to transform instruction so that children can become powerful readers, writers, and thinkers. During the 2013/14 school year, CLI served 248 schools and more than 34,000 students, coached 1,841 educators and distributed more than 81,000 books to schools around the country.

Methods Included in the What Works Clearinghouse
  1. NAEP, 2013
  2. ibid
  3. AECF, 2011
  4. Sum, Khatiwada, & McLaughlin, 2009
  5. National Commission on Teaching and America’s Future, 1998
  6. Education Trust
  7. Gates Foundation, 2014

i3 Evaluation

In 2010, CLI received a federal Investing in Innovation (i3) grant to implement its early-childhood literacy program with kindergarten through second-grade teachers in four U.S. school districts with high-need student populations: Philadelphia, PA, Chicago, IL, Newark, NJ, and Camden, NJ. This program provided teachers more than 460 hours of seminars and in-class coaching sessions. In addition, CLI mentored school leadership on how to best promote an environment of learning in classrooms and provided schools with book collections and other literacy resources.

CLI enlisted the help of the American Institutes for Research (AIR), to conduct the independent evaluation of CLI’s impact. AIR randomly assigned participating schools to either the treatment group (received CLI services immediately) or control group (received CLI services following data collection). Therefore, any differences observed between the two groups at the end of the study are attributable to the CLI program and not to preexisting differences between the two groups and teachers and students receiving CLI services can be compared to similar teachers and students who did not receive services.

 

Results

Teachers’ Literacy Instruction Practices & Classroom Environment

Teacher practice and classroom environment were measured by the Early Language and Literacy Classroom Observation (ELLCO), an observational measure of the presence of literacy best practices. This included how well and how often a teacher encouraged extended classroom discussions, facilitated oral language development and whether the teacher regularly used and updated a word wall in the classroom. Each element was rated on a scale of 1 (poor) to 5 (excellent).

The study found that:

  • Teachers who received CLI training had a significantly more positive classroom environment than teachers who did not receive CLI services
  • Teachers who received CLI services exhibited significantly more language and literacy practices in the classroom

Reading Achievement

Student reading achievement for kindergarten students was measured using the Predictive Assessment of Reading (PAR). The PAR is an individually administered test of pre-reading skills, including students’ letter-word reading, picture vocabulary, phonemic awareness, and rapid naming fluency. First and second-grade students’ reading achievement was measured using the Group Reading Assessment and Diagnostic Evaluation (GRADE). The GRADE is a group administered a multiple-choice assessment that is completed in two sessions and assesses students’ word reading as well as meaning and comprehension. Overall, each assessment measures students’ literacy skills. We refer to overall scores on each measure below as “reading achievement.”

Kindergarten CLI students exhibited significantly greater reading achievement than non-CLI students.

  • Especially students in classrooms with teachers who had received CLI training for two years

Average non-CLI kindergarten student would perform better than 50% of peers, but with CLI-trained teacher would perform better than 57% of peers.

  • Translates to an improvement index of 7 for kindergarten students (calculated using WWC methods: //ies.ed.gov/ncee/wwc)

Overall, first grade CLI students did not exhibit greater reading achievement relative to first-grade non-CLI students

  • In year 1, effect size of the difference between CLI and non-CLI first-grade students was nearly zero
  • In year 2, effect size was larger (.06), but was still not significant

However, it is important to consider that students enter first grade with a wider range of pre-existing achievement levels than in later grades. Therefore, AIR examined whether CLI services helped reduce gaps in reading achievement for those entering first grade as low performers versus high performers. They found that students entering CLI classrooms as low performers showed significantly greater reading achievement compared to low-performing control students, reducing gaps between low performers and high performers in CLI classrooms.

Second grade CLI students also exhibited significantly higher reading achievement than non-CLI students

  • On average, second grade CLI students scored at the 38th percentile for their age, while non-CLI second graders scored at the 35th percentile

Based on reading achievement scores, second grade CLI students received the equivalent of two additional months of instruction as compared to non-CLI students.

  • That is, without the expense and logistical difficulties of keeping school open for summer, CLI teachers provided students with the instructional equivalent of attending school nearly year-round
Image
Image
Image
Image

Conclusion

As the above results demonstrate, CLI professional development positively impacts teachers, which in turn translates to improved student learning. Teachers’ practices and classroom environment significantly improved as a result of CLI services. Further, student reading achievement improved as a result of CLI services and teachers’ improvements, particularly for kindergarten and second grade students.

Many of the nation’s students are struggling to read at grade level. Economic disadvantage and inadequate professional development for teachers have prevented this situation from improving. Fortunately, high quality instruction throughout primary school can substantially offset disadvantages associated with low socioeconomic background.8 CLI’s validated program is an effective way to ensure that both teachers and students have the tools and support they need, not only to meet, but to exceed expectations.

  1. Rivkin, Hanushek, Kain, 2005
Download the Executive Summary