Discussion
Identifying Nonresponders
The first purpose of this study was to explore the usefulness of a dual-discrepancy approach to identifying nonresponders (see L. Fuchs & Fuchs, 1998). Findings show that our CBM measures were reliable and valid indicators of students' reading skill, and that the dual-discrepancy approach identified students performing statistically significantly below their peers. This result is consistent with recent evidence that dual-discrepancy can successfully distinguish nonresponders from responsive at-risk and average-achieving children on reading-related measures (e.g., Speece & Case, 2001; Speece et al., 2003).
Findings also suggest that dual-discrepancy holds promise as a better method of identification than performance-level-only and growth-rate-only approaches. In this study, the performance-level-only approach would identify a student as a nonresponder who scored below the 30th percentile on the WRMT-R Word Identification or Word Attack subtests. The dual-discrepancy approach would not identify such a student as a nonresponder if she were making growth similar to average-performing readers. Likewise, using growth-rate only criteria, a student making limited growth may be identified as a nonresponder. However, if this student were reading at a level commensurate with his average-performing peers, he probably would not need additional instruction. Using the dual-discrepancy approach, both his growth rate and performance level would need to be below those of average performers for him to be considered unresponsive. Our dual-discrepancy approach provided performance-level criteria (based on average students' performance) needed to make such a decision.
Nevertheless, we offer two important caveats about our favored approach. First, just as performance-level and growth-rate criteria used by previous researchers are arbitrary, our discrepancy criterion of .5 SD below average is arbitrary, too. Currently, there is no consensus regarding how far below average a student must perform to warrant a change in intervention. Second, nonresponders were identified using word-level CBM measures rather than other indicators of reading skill. These measures were selected because they were likely to be sensitive to weekly growth in low-performing first graders. Yet, others have used measures of phonological awareness, spelling, fluency, or comprehension to identify younger and older nonresponders. Continued research is needed to determine which measures are most appropriate to identify nonresponders at different points of reading development.
Responding to Nonresponders
Our second purpose was to compare the effects of PALS, Modified PALS, and Tutoring to determine which was most effective in improving the reading performance of nonresponders to PALS and in reducing rates of unresponsiveness. There were no statistically significant differences across the treatment approaches on the monitoring measures and on any of the other reading-related measures.
There are several possible explanations for this. First, the treatments' relatively low intensity and short duration may simply have been insufficient to yield dramatic gains in students with severe reading difficulties. A number of researchers (e.g., Berninger et al., 2002; Blachman et al., 1999; Foorman et al., 1998; Torgesen et al., 2001; Vellutino et al., 1996) have conducted treatments for similar students that have totaled 65 hours to more than 300 hours, some spanning 2 years or more. Moreover, these interventions have often been implemented by highly trained teachers or reading specialists. Even such intensive interventions have not succeeded in improving the reading achievement of a subset of at-risk readers, with proportions of nonresponders similar to those reported in this study. Another explanation is that PALS, Modified PALS, and Tutoring all incorporated comparable activities. Whereas the treatment activities and formats were deliberately designed to be similar so we might explore the importance of varying levels of individualization, it is possible that they were not sufficiently different from each other to produce different results.
A third explanation, already discussed, may be the low statistical power of the study. In recognition of this fact, we calculated effect sizes and proportions of nonresponders at the end of the study as additional ways to compare the three treatments. Effect sizes favored Tutoring over PALS and Modified PALS on a number of reading-related measures, and favored Modified PALS over PALS on Rapid Letter Sound naming, Word Attack, and comprehension. A similar pattern is seen in the proportions of students who continued to be unresponsive at the end of treatment. Tutoring reduced unresponsiveness by 50%, whereas the Modified PALS and PALS treatments lessened unresponsiveness by 25% and 19%, respectively.
The Tutoring program deserves some additional comment. The addition of one-to-one instruction provided by a trained research assistant may explain why half of the tutored students were responsive. The research assistant was trained to ensure that immediate, correct feedback was provided, and that students mastered the content of the tutoring lessons before moving to new content. Although peers have been demonstrated to be effective tutors for many students, perhaps the most difficult-to-teach students require the more individualized support that trained adult tutors are able to provide. Also, tutored students had more opportunities to respond than PALS and Modified PALS students, since tutored students were always the "Readers" and there were fewer transitions.
A final feature that may have enhanced the Tutoring treatment is difficult to quantify, but was mentioned by many of the tutored students' teachers. The teachers often spoke of the special relationship shared by the students and their tutors, reporting that many of the students looked forward to the tutoring. Also, many of the tutored students enjoyed sharing their progress with their teachers, a behavior not observed among students in the other two treatments. Perhaps there is an important motivational component associated with one-to-one instruction from an adult that leads to a stronger desire to engage in reading activities--leading, perhaps, to increased learning. Future researchers may wish to investigate this aspect of individualized tutoring through more systematic observation and interviewing.
We hoped that Modified PALS would also benefit nonresponders. Modified PALS effects would indicate that the needs of struggling readers might be met using less costly resources more readily available to teachers than individualized tutoring. However, Modified PALS was only half as successful as Tutoring in reducing rates of unresponsiveness. Specific implementation features may have contributed to this outcome. First, teachers were asked to supervise the students' implementation of the Modified PALS activities, but, given that they had an entire class to oversee at the same time, it is not clear that they were able to ensure that Modified PALS was conducted correctly. The wide range of Modified PALS fidelity (49% to 100%) indicates that it was conducted incorrectly at least some of the time. In addition, making modifications to a PALS program that was not benefiting the nonresponders was likely an inadequate response to their reading problems. Unfortunately, and importantly, this reflects what can happen when the general curriculum is modified for struggling students: classroom teachers are not always able to monitor them closely, and instruction is basically "watered down" rather than individualized (D. Fuchs & Fuchs, 1994; Vaughn & Schumm, 1995).
Findings indicate that classroom instruction--even generally effective classroom instruction--can be inappropriate for struggling readers, with or without modifications. For those students for whom modifications are ineffective, it is important that options such as one-to-one or small group tutorials are available. Whereas some may see such intervention as "restrictive," especially if located outside the general education classroom, it is important to remember why federal law compels educators to provide such an option--to ensure that education is not only provided in the least restrictive environment, but that it is also most appropriate for meeting students' unique learning needs.
Study Limitations
There are at least several study limitations, the first of which is that the sample size was small, reducing statistical power. It should be noted, however, that the participants were drawn from a pool of 323 students, which represents a large field-based study. A much larger pool of students would be needed to generate sufficiently large numbers of nonresponders to power the necessary inferential statistical analyses. Second, 15 of the 56 nonresponders (27%) were ELL students. Although these students spoke English well enough to interact with their peer or adult tutors, language differences may have complicated treatment effects. For example, several ELL students were observed to make very rapid gains, suggesting that, as they learned more English, they began to overcome their reading difficulties. Conversely, some ELL students made little growth; it is difficult to determine whether their unresponsiveness was due primarily to reading deficits or to severe language problems. Third, this study did not include a no-treatment control group, which would have been useful in determining whether the three treatments were more beneficial for unresponsive readers than more traditional classroom instruction. Finally, our pretreatment measures were administered before PALS began in October. Modified PALS and Tutoring did not begin until January. Whereas we used the students' December Dolch level as a covariate in comparing posttest scores, additional information about group equivalence on other reading related measures immediately before the additional treatments were implemented would have strengthened our analyses.
Implications for Research and Practice
Further research is needed to explore features of early intervention that should be in place to maximize the learning of struggling readers. For example, standardized instructional programs should be compared with instruction tailored to individual needs. Characteristics of children unresponsive to treatment should continue to be studied closely. It may be important to include components that address important child characteristics such as attention, motivation, and behavior. The ideal size of instructional group (e.g., whole class vs. small group vs. one-on-one) should also be further explored, and interventions that are supplemental to regular instruction should be compared with interventions that take the place of regular instruction. In addition, the length of intervention needed to produce strong and stable growth must be investigated. And follow-up studies are needed to better understand long-term benefits of early reading intervention.
Finally, researchers should examine ways of implementing instruction that is effective but also feasible, given finite school resources. Interventions that can be implemented by classroom teachers, paraprofessionals, parents, and school volunteers are likely to be more accessible, and thus more widely beneficial, than those that require special training and many hours outside the general classroom. Someday, perhaps, we will know the conditions necessary for all children to learn to read. However, if and when that day arrives, the work of early intervention researchers will still not be complete. A critical goal of future research should be not only to develop the most effective interventions, but also to determine the most appropriate and efficient means of delivering them.
We offer several implications for practice based on findings from this study and previous research. First, whereas we suggest all teachers implement evidence-based instruction that meets a wide range of needs, we also caution that such instruction should not be viewed as a "cure-all." Ongoing progress-monitoring of students, especially those at risk for, or identified as having, reading disabilities, is critical for determining whether an instructional strategy is beneficial. We suggest that teachers use frequent curriculum-based measures to ensure that their students are making progress; to identify students who are discrepant from their peers in performance level and growth rate; and to make changes to group or individual instruction when students are not progressing as expected. Second, when modifications are made to general education instruction, they should be implemented with fidelity, and concurrently with ongoing progress monitoring. Finally, when evidence-based instruction and modifications in general education fail to meet students' unique learning needs, we encourage practitioners to consider more intensive, individualized instruction. Recent research suggests that such "special-education-like" instruction is the best response to children most at-risk for reading failure.
Previous Page | Next Page
(Results: After Treatment) | (References)
