Effective Implementation

Effective Implementation of Reading Recovery

1. Assessing Effectiveness of the Implementation of Reading Recovery Within an Educational System
   An evaluation design for Reading Recovery must address three key questions and it must collect information on all factors that affect outcomes, including extent and quality of implementation.
    
2. Using Data to Assess Effectiveness With First-Grade Students
   Data collected and reported by the National Data Evaluation Center can be used to assess both program effectiveness and program efficiency.
    
3. Assessing the Effect of the Intervention on the Educational System
   The goal of Reading Recovery is to reduce reading failure within a system. Evidence that this is happening is the best response to challenges to Reading Recovery as too expensive or not serving enough children.
    
4. Evaluating Two Positive Outcomes
   The Reading Recovery children who accelerate their learning and whose lessons are successfully discontinued are one positive program outcome. The children who do not accelerate are positive outcomes also: some have been efficiently identified as needing further evaluation; others have made learning progress at various rates and many can continue to the next grade with or without some further supplemental support.
    
5. Assessing the Quality of Reading Recovery Implementation
   The quality of implementation has significant effect not only on outcomes, but also on the degree to which Reading Recovery can affect the system. Factors in effectiveness include full implementation, administrative support, and adherence to the Standards and Guidelines for Reading Recovery in the United States.
    
6. Assessing Perceptions of Reading Recovery as Essential to the System
   It is important to assess perceptions of Reading Recovery regularly to know what must be done to develop understanding, appreciation, and feelings of ownership at all levels.

The following discussion gives more information on the methods and benefits of fully evaluating the effectiveness of any Reading Recovery implementation.

1. Assessing Effectiveness of the Implementation of Reading Recovery Within an Educational System
Reading Recovery may be judged as highly effective if

1. data show that the intervention is significantly reducing the numbers of children with severe reading problems,
2. there is clear evidence of strong support for Reading Recovery, and
3. Reading Recovery is perceived as an integral part of the educational system.

Assessing the effectiveness of Reading Recovery is a complex issue for several reasons:

1. Reading Recovery does not work in isolation. It must work in conjunction with classroom instruction and other educational support and management systems.
2. Reading Recovery implementation is gradual, by design. Assessment must recognize the multiyear process of implementation.
3. Implementing any high-quality educational change such as Reading Recovery is a slow process that involves changes in people's beliefs and allegiances, organizational patterns, allocations of resources, roles, and communication patterns.

Therefore, any assessment of Reading Recovery effectiveness

1. must recognize the time required for change to occur. Teachers and schools will not easily realize the full potential of the intervention in the early years of implementation.
2. must be evidence-based, but the evidence may need to be understood in relation to incomplete or partial implementation and in relation to classroom practice.
3. must seek to understand how the system is supporting the implementation as well as how Reading Recovery teaching is effective.

Return to Top

2. Using Data to Assess Effectiveness With First-Grade Students
Comparison of local data to national data (from the National Data Evaluation Center) yields initial indications of effectiveness with first-grade children. Site and school data should also be examined annually, comparing results to previous years, to determine year-to-year gains and to set goals for further improvement. The teacher leader has expertise in analyzing and communicating the annual results.

Look for evidence from the following groups:

1. children whose series of lessons were successfully discontinued (children who are now able to continue their learning in classroom situations),
2. complete intervention children (children who have had ample opportunity to learn)
3. incomplete intervention children (children for whom time ran out at the end of the year),
4. children recommended for action (children who have had a complete intervention but whose lessons were not discontinued),
5. all Reading Recovery children (every child who received any service at all).

Look for evidence on the following points:

1. relative size of each group (percentage of all children served)
2. entrance- and exit-level performance on all tests, but especially on text reading
3. comparison of each group to a random sample control group identified by the National Data Evaluation Center
4. change in reading group placement for children in each group

For evidence of program efficiency, check

1. the number of children served per teacher,
2. the number of weeks and progress of children with an incomplete intervention,
3. the number of lessons per week for teachers (evidence of daily consistent lessons), and
4. the number of weeks in the intervention for children whose lessons were discontinued.

Return to Top

3. Assessing the Effect of the Intervention on the Educational System

1. Assess the percent of coverage — the degree to which this intervention is available to all the children in each first-grade cohort that need this intervention. Entry-level data from Observation Survey testing will help determine the extent of need for this intervention.
2. Examine how Reading Recovery has changed the paths of progress for these children within the system. Look for evidence of reduction in the numbers of children retained as well as reductions in the numbers referred for testing and classified as qualifying for exceptional services.
3. Examine longitudinal data on Reading Recovery children beyond the first-grade year using tests given to all children in Grades 3 and 4. Examine this in three groups: (1) Reading Recovery children whose lessons were discontinued, (2) children who had a complete intervention, and (3) all Reading Recovery children. It is best to collect data from a number of sources (standardized tests, informal reading inventories, teacher perception, and letter grades), since no one assessment provides a full depiction of the child's competencies.
4. Compare the progress of various majority and minority groups to determine the degree that Reading Recovery closes the achievement gap.
5. Watch for signs of improvement over time in the entering scores of first-grade children (evidence that kindergarten programs build early literacy skills).
6. Look for change over time in average reading levels of grade-level cohorts.
7. Collect data on classroom teachers' perceptions of the performance of Reading Recovery children.

Return to Top

4. Evaluating Two Positive Outcomes
The first positive outcome of Reading Recovery is that most children accelerate their learning so that they are able to learn in the classroom and meet grade-level expectations. These are the children whose series of lessons are successfully discontinued.

The second positive outcome of Reading Recovery is that the children who need long-term learning support are identified and referred for further evaluation. These children are referred to as recommended for further action. Some of these children have made good learning progress and with some supplementary small-group assistance will be able to make adequate school progress in the future. Others will need the long-term intensive support of special education services. The Reading Recovery intervention reduces the number of students who need longer services, especially those who will receive special education services, saving the system time and expense.

Even though the children whose lessons are not successfully discontinued may be viewed positively, the continuing goals of the system and of Reading Recovery teaching should include

1. reducing the percentage of children recommended for further action,
2. providing efficient and timely evaluation of those who need referral, and
3. providing additional support for the children whose lessons were not discontinued who can learn in small-group contexts or with other additional tutoring.

Return to Top

5. Assessing the Quality of Reading Recovery Implementation
Implementation is of high quality if

1. the program is implemented according to Standards and Guidelines for Reading Recovery in the United States, published by the Reading Recovery Council of North America,
2. resources are committed to achieve full implementation (Reading Recovery teaching time is available to all children who need it),
3. administrators lead, encourage, and develop strong classroom teaching and close collaboration and communication among Reading Recovery and classroom teachers, and
4. coordination and collaboration is established among all basic and support educational services.

Return to Top

6. Assessing Perceptions of Reading Recovery as Essential to the System
Barriers to the acceptance of Reading Recovery are almost inevitable. These include

1. established expectations about what is possible for low-achieving children,
2. expectations for children with variant social and behavioral characteristics,
3. traditional curriculum expectations, and
4. conflicting ideas about how children learn to read and write.

Beliefs and values can change as people perceive a need to change and as they confront strong, contradictory evidence. To facilitate this change administrators and the teacher leader will

1. share entrance data and progress data with classroom teachers,
2. establish mutual interest (with classroom teachers) in children's performance,
3. arrange to have classroom teachers observe Reading Recovery lessons,
4. invite teachers to bring children to the training class and to come as observers,
5. arrange for children to read books to their teachers and to their class,
6. work with the teacher leader and principals to establish school teams, and
7. share data and arrange for direct contact to help upper-grade teachers to see how an early intervention can make a significant contribution to their work.

Positive social and intellectual interactions also contribute to belief and value changes. These can be fostered in the following ways:

1. Reading Recovery teachers must take responsibility for establishing relationships.
2. Principals can arrange positive contexts and opportunities for sharing expertise.
3. Reading Recovery teachers can exchange helpful suggestions with classroom teachers.
4. Reading Recovery teachers can ask specialist educators for information or advice.
5. Reading Recovery teachers can enlist parent participation and visitation from families that have not previously engaged with school staff.

Return to Top