Douglas Fuchs and Lynn S. Fuchs, Peabody College of Vanderbilt University; Patricia G. Mathes, University of Texas--Houston Health Science Center; Mark W. Lipsey and P. Holley Roberts, Peabody College of Vanderbilt University
Learning Disabilities Summit: Building a Foundation for the Future White Papers
This paper is available in alternative formats: | Download Word | Download pdf |
Pages: | 1 | 2 | 3 | 4 | 5 | 6 |
Our first major analysis indicated considerable disagreement among the studies with respect to the magnitude of the differences between LD and LA groups in reading performance: The Q statistic indicated substantially greater variability among ESs than would be expected from sampling error alone, Q(111) = 535.75, p < 0.001. This finding led us to explore which study characteristics might be associated with variation in ES.
Before examining the relation between study features and ESs, we consolidated some study features. First, based on analyses we conducted, we consolidated our definitions of LD and LA samples to five levels of LD/LA definitional pairings; this resulted in 109 ESs.
Second, we conducted several focused factor analyses on sets of variables that seemed to be related conceptually and were better represented as multivariate composites. A varimax-rotated solution seemed to fit these variables nicely. We thereby reduced the LD-LA student comparability data to three factors: achievement, which incorporated variables related to reading comparability; demographic characteristics, including age, race, and socioeconomic status (SES); and gender comparability, IQ, and SES comparability. We refer to these three factors as (a) achievement comparability, (b) demographic comparability, and (c) gender comparability, respectively.
Finally, we conducted another factor analysis to examine relations among variables describing the research method used for constructing the LD-LA samples. This analysis produced a sensible two-factor solution. The first factor showed a co-occurrence of the following: (a) lower IQ scores, (b) higher grade levels for the LD sample, and (c) referral of LA samples for special education testing. We called this factor "other sample features." The second factor cleanly combined the two variables describing whether the samples were district or researcher identified. We called this factor "identification source."
How Do the Clustered Study Features Relate to ESs? We used weighted least-squares regression, weighting each ES by the inverse of its variance. Our pool of predictor variables included the LD/LA definitional pairings, the five factor scores (reading comparability, demographic comparability, gender comparability, other sample features, and identification source), the three locale variables, technical adequacy, test format, study quality, and date of study. Predictors were entered simultaneously; then, the weakest was dropped and the model was refit. We repeated this process until all remaining variables were significant.
The regression model accounted for a statistically significant 41% of the variance among the ESs. The following variables made significant, independent contributions to the prediction of ES.
First, measurement format contributed to the prediction of ES, with a beta of 0.34. ESs were greater for the timed than the untimed measurement formats. For example, on tests requiring students to work in a fixed time (such as the Stanford Achievement Test or curriculum-based measurement), the difference between students with and without LD was larger than when tests permitted students as much time as they needed (e.g., Woodcock Reading Mastery Tests). This was true across reading domains.
Second, other sample features contributed to the prediction of ES, with a beta of 0.16. ESs were greater for LD samples with lower IQ and with higher LD grade; ESs were greater when LAs had been referred but had never qualified as appropriate for special education.
Third, LD/LA definitional pairings contributed to the prediction of ES. ESs were greater when LD samples were defined by discrepancies and when LA samples were defined by teacher judgment; the associated beta was 0.51. ESs were smaller when LD samples were identified by multidisciplinary team judgment and when LA samples were defined by data-driven methods; the associated beta was -0.27.
Fourth, the three comparability factors contributed to the prediction of ES. ESs were greater when achievement and demographics were not comparable for LD and LA samples; the associated beta was 0.13. ESs were greater when gender and, to a lesser extent, IQ and SES were comparable; the corresponding beta was 0.08.
Finally, methodological study quality contributed to the prediction of ES, with somewhat greater ESs for lower quality studies. The associated beta was 0.12.
Across the many substantive and methodological variables associated with studies in this meta-analysis, ESs demonstrated considerable heterogeneity. Analyses were, however, successful in identifying a large proportion of the variance among ESs. Ten variables operated independently to explain the variation. In particular, three variables maximized the degree of reading impairment associated with the LD label and, therefore, provide insight into the theoretical nature of the disability. They also may help practitioners and researchers develop more effective assessment and intervention procedures for students with LD, as well as more precise measures of treatment success.
On the basis of these meta-analytic findings, we offer three conclusions, which may guide future research and practice. First, across the many different ways in which students become identified as LD, results leave no doubt that these students' reading achievement differs dramatically from other LA, nondisabled students. Averaged across all the methodological and substantive variations in the studies, the mean effect size was 0.61 standard deviations units. This effect is sizable; it means than 72% of the LA population performs better in reading than the mean of the LD population. Moreover, regarding ESs for timed measurements, whereby students were required to perform (i.e., read aloud, read silently, answer questions, match words to meanings) within a fixed time, the ESs increased to well beyond one full standard deviation unit. And, in a similar way, when LD and LA samples had been identified using data-based methods, the overall ES of 0.61 rose to beyond a full standard deviation unit. Findings, therefore, suggest that researchers and school personnel in fact do identify as LD those children who have appreciably more severe reading problems compared to other low-performing students who go unidentified. As with any comparison of two populations, some overlap between these populations occurs; that overlap, however, is not sufficient to call the LD label into question. Consequently, in light of the more severe magnitude of LD students' reading problems, it seems reasonable and desirable that more intensive forms of reading instruction be directed at this group of students.
Second, the ESs associated with timed tests were larger than those associated with untimed tests. The beta associated with this effect was an impressive 0.34. This strong effect associated with timed measurement format suggests theoretical and practical implications. Failure at achieving automaticity may represent an important characteristic of students with LD, which may be associated with the low performance on rapid-naming tasks (Wolf, 1991) of many of these children. The possibility that difficulties in achieving automaticity may represent a key feature of students with LD warrants additional study. Methods of identifying LD children might incorporate timed reading assessments to focus on students' failure to achieve automatic word-reading performance. In addition, with respect to treatment, researchers should develop methods for helping students with LD transition from accurate to automatic word reading. Finally, results suggest that the effectiveness of interventions for students with LD should be evaluated at least in part by how they influence students' performance on timed reading measurements.
Finally, results underscore the importance of objective measurement of reading performance in the identification process. Larger differences between LD and LA students emerged when definitional and selection criteria for inclusion to studies relied on objective forms of reading measurement--that is, the administration of tests. By contrast, when individual or team judgment was involved, differences between LD and LA samples on reading measures grew smaller. On one hand, this finding provides a basis for questioning human judgment in the identification process. On the other hand, it suggests that other considerations, such as a focus on social behavior, may play a viable role in the identification of children whose overall performance profiles warrant special treatment. Practitioners should be mindful of the advantages and disadvantages associated with reliance on nonobjective forms of input to the multidisciplinary team process. Future research should continue to identify which types of nonobjective data may be important in the identification process and should continue to examine the role of social behavior deficits and the possibility of comorbidity in children with LD.
Funding for NRCLD is provided by the U.S. Department of Education, Office of Special
Education Programs. Award No. H324U010004. Opinions expressed herein are those of the National
Research Center on Learning Disabilities and do not necessarily represent the position of the U.S.
Department of Education.
© 2007 National Research Center on Learning Disabilities
Last Updated: November 19, 2007
Contact: nrcld@ku.edu