Frank M. Gresham, University of California-Riverside
Learning Disabilities Summit: Building a Foundation for the Future White Papers
This paper is available in alternative formats: | Download Word | Download pdf |
Pages: | 1 | 2 | 3 | 4 | 5 | 6 |
The process employed by public schools can be conceptualized as consisting of three steps: (a) the decision to refer by a child's general education teacher, (b) the psychological evaluation of the child, which yields a combination of psychometric scores corresponding to criteria specified in the state as a prerequisite for eligibility, and (c) the team placement decision arrived at after review and discussion of all evidence by a school placement team. It is significant that these three steps occur in a set sequence as presented above. As a result, a student who is not referred by his or her general education teacher is not at risk for being identified as LD. Only those students passing through this first gate--referral--are even considered for psychological evaluation. In addition to the steps in the above sequence, consideration must be given to the fact that at each gate there are differences in the weighting given to various factors that result in three competing paradigms for the identification process. These factors are (a) the nature and role of professional judgment permitted at a specific gate, (b) the concept or question addressed by those involved in the decision making at a particular gate, (c) the use of local versus national norms employed at various gates, and (d) the extent to which sociocultural and contextual factors are considered (Bocian, Beebe, MacMillan, & Gresham, 1999).
Viewing the identification process through the lens of competing paradigms may serve to clarify the process by which schools identify children as LD and why there is often a gap between who is identified by schools and research criteria. The following sections expand on how each of the four factors just noted operate in concert or in competition with each of the three paradigms or gates in the identification process.
Being referred by a general education teacher is a necessary but insufficient requirement for being school-identified as LD. Although teachers refer students to prereferral teams for academic and/or behavioral difficulties, the referral issue with LD is almost always academic deficiencies. The child's academic performance relative to the modal performance of the class or the gap between the target child's reading level and that of members of the lowest reading group is more salient in reaching the referral decision than are standardized test scores. This perspective reflects what has been referred to in the literature as "teachers as imperfect tests" (Bahr, Fuchs, Stecker, & Fuchs, 1991; Gerber & Semmel, 1984; Gresham, MacMillan, & Bocian, 1997; Gresham, Reschly, & Carey, 1987). The principle guiding the teacher at this step is one of relativity--that is, what is the likelihood that this teacher will be able to close the gap in achievement relative to the target child's peers in both the classroom and grade level, given class size, past responsiveness of the child to intervention, and the resources available in the classroom? When the teacher concludes that this relative gap cannot be substantially narrowed without assistance, the decision to refer is highly probable.
Although the referral decision is almost never influenced by information from nationally normed scales, the decision can be and sometimes is tempered by sociocultural and contextual factors. Even in cases where the teacher judges a child's academic performance to be deficient, he or she might refrain from referring because of circumstances involving the home, the facility of both regular and special education teachers with the child's native language, or health concerns. The point here is that although local norms are employed to determine academic performance at the referral step, sociocultural and contextual factors are considerations that sometimes influence the referral decision.
It is likely that children who were referred and fail to respond to prereferral efforts will ultimately be subjected to the second gate in the referral process--psychoeducational evaluation. MacMillan and Speece (1999) characterized this gate as representing a cognitive paradigm intended to detect or document the existence of a within-child problem. It is through psychoeducational assessment that the referred child's eligibility for special education as LD is established as 98% of the states include a discrepancy in either their definition of or criteria for identifying students with LD (Mercer, Jordan, Allsopp, & Mercer, 1996).
The concept guiding the decision to pass the child through this gate and on to the school-placement team is one of acceptability. Through the assessment with standardized tests, one can determine whether the referred child's low level of academic performance is acceptable. If it is severely discrepant from the aptitude score, a low performance in reading is unacceptable (i.e., the child should be doing better). This situation reflects the concept of LD as unexpected underachievement. Conversely, if a child with a very low reading score performs equally low on an individually administered measure of intelligence, he or she is doing about as well as can be expected. This situation reflects the notion of expected underachievement. Finally, although teachers weigh sociocultural and contextual factors in deciding whether to refer the child, the testing step is devoid of such factors.
Multidisciplinary teams (MDTs) are responsible for determining eligibility and recommending placement. These teams are permitted to exercise judgment, but unlike the teacher in the referral step, it is a "team judgment," not an individual one. It brings together the two major players involved in referral and testing--that is, the general education teacher who referred the child and the school psychologist or educational diagnostician who performed the psychoeducational assessment.
The decision reached by the MDT reflects a considerable amount of team judgment, as opposed to individual judgment which is reflected in the referral process. The general education teacher assesses the child's academic performance relative to local norms and the school psychologist assesses the child's academic performance discrepancy relative to aptitude and national norms. However, although local norms predominate at the referral step and national norms predominate at the testing step, all three perspectives are considered by the MDT in arriving at a placement decision: local norms, national norms, and sociocultural and contextual factors.
The concept guiding the team decision regarding placement is profitability, which reflects the collective perception that the specific special education services provided at that school site will or will not benefit the child. As such, the anticipated profitability gauges the interaction between child characteristics (derived from the comparisons of this child's level of performance to both local and national norms) and the quality of special education services on site. Parental wishes and concerns also will factor into the ultimate decision regarding placement.
The dynamics of specific MDTs will result in assignment of differential weighting to local norms, national norms, and sociocultural and contextual factors in arriving at placement decisions. Thus, team decisions are likely to vary, even in the face of hypothetically identical information, because of the relative forcefulness of particular players serving on the team. Any effort to understand school-identified LD students must consider the importance of these three steps (referral, testing, and team recommendation) and the relative weighting given to available data at each step.
Presently, research on LD students often examines a group of students who are screened according to criteria for only one of the gates. For example, a "sample" will sometimes be selected from children with a certain psychological profile reflective of the testing gate even though a referring teacher did not initially screen the sample. Such sampling results in a group that overlaps with but is not identical with children who will be school-identified as LD. Findings over the past 15 years have pointed out the lack of consistent definition in policy or practice in the identification of LD students, a circumstance that has been a major stumbling block to effective research and practice (Lyon, 1996). Response to this challenge has ranged from impugning the concept of LD as neither valid nor instructionally relevant, to criticizing teachers and schools for failing to implement criteria correctly. Some researchers have suggested that schools seek flexibility and the opportunity to exercise professional judgment rather than being held to a rigid code of precise formulas (Keogh & Speece, 1996; MacMillan et al., 1997; McLeskey & Waldron, 1991).
A second implication of the competing paradigm model is the accuracy with which teachers identify within-child variables relevant to the classroom that are later validated by psychoeducational assessments. Teachers' accurate evaluations of students' abilities should be sought after rather than continually challenged. Teachers may be "imperfect tests," but in terms of classroom relevance, their perceptions often outrank students' performances on psychoeducational assessments on isolated tasks conducted under ideal, pristine conditions.
A third implication is recognizing the severe limitations and the ability of the discrepancy concept of LD to both plan instruction and identify students for early intervention. The recent national downward trend in reading achievement and the public pressure for student outcomes and accountability have led to an enhanced focus in the field on reading disabilities (Lyon, 1996). This approach surely holds more promise for students and teachers alike, particularly given the ability of teachers to identify reading disabilities based on curriculum-achievement discrepancy or an achievement discrepancy relative to peers. Perhaps of greater import is the need to train and encourage teachers to exercise their judgment at even earlier points in a student's career. The research field should work to validate that judgment with operationalized criteria, particularly with reading problems.
Although the competing paradigm multiple-gate system now in place does work to identify students in need of services, the competition between expensive, time-consuming assessments at three different steps could be streamlined and articulated in a fashion more respectful of both teacher and school professional judgment to meet students' need for immediate intervention services. The most serious flaw in the current process is the absence of a direct link between assessment procedures used for identification and subsequent interventions that might be prescribed based on these assessment procedures (i.e., treatment validity). In fact, it is clear that most reading difficulties exhibited by students now classified as LD are caused by inadequate literacy experiences, inadequate instruction, or some combination of both (Clay, 1987; Vellutino, Scanlon, & Tanzman, 1998). This being the case, an alternative approach to the identification of students with LD is justified. Therefore, the focus of the current paper is to describe how such an assessment process can be developed and used in identifying and instructing students with LD.
The purpose of this section is to provide a very brief overview of the history of and difficulties in defining LD and some of the issues inherent in using a discrepancy approach to operationalize the LD construct. Other chapters in this book provide a much more detailed analysis of the issues involved in the definition of LD. This overview is intended to provide a context for discussing a different approach to LD definition: responsiveness to validated intervention procedures.
Kirk (1962) first used "learning disabled" to describe a group of children who have retardation, disorder, or delayed development in one of more of the processes of speech, language, reading, writing, arithmetic, or other school subjects. This definition was the first to introduce the concept of psychological process disorders and how these processing deficits adversely affect academic achievement (Kavale & Forness, 2000). Shortly thereafter, Bateman (1965) proposed the notion of underachievement as a fundamental aspect of LD. In Bateman's definition, the idea of an "educationally significant discrepancy" between intellectual potential and actual level of academic performance was emphasized. This definition did not specify what constituted an "educationally significant discrepancy" and did not provide information on how to measure intellectual potential and actual level of performance (Kavale & Forness, 2000). More than three decades later, the field of LD still has not arrived at a consensus in terms of resolving these definitional and measurement issues.
Rutter and Yule (1975) defined two types of reading underachievement difficulties: general reading backwardness (GRB) and specific reading retardation (SRR). GRB is defined as reading below the level expected of a child's chronological age, whereas SRR is defined as reading below the level predicted from a child's intelligence. Rutter and Yule estimated the prevalence of GRB in the school-age population to be 7% and 20% (rural and inner-city settings, respectively), whereas the prevalence rate of SRR was 4% and 10%, respectively. It should be noted that, according to Hinshaw (1992), almost all children with SRR could be classified as GRB, but only half of children with GRB are classifiable as SRR.
Children such as those described by Rutter and Yule (1975) as having SRR may be considered as having LD in most states using a discrepancy-based definition of LD (Mercer et al., 1996). In fact, the prevalence rate of SRR of 4-10% in Great Britain is consistent with the 5% prevalence rate of children served as LD in the United States. Moreover, children who might be described as low achievers might meet the definition of GRB. SRR and GRB capture the concepts of unexpected and expected reading underachievement, respectively.
Differentiation among groups of children having mild disabilities such as LD and MMR as well as low achievement (LA) has always been problematic. Children functioning around the margin of what might be considered a disability group create special problems in assessment, measurement, and eligibility determination for special education programs. At what point, for instance, is low academic achievement considered to be due to MMR and not to LD? How is MMR different from LA? Is LD different from LA, and if so, how is it different? Are LD and LA primarily reflective of differences in degree or kind of academic underachievement? Although these questions remain fundamental to the identification of students having difficulties in school, definitive answers to these questions have not been forthcoming.
Researchers have debated the similarities and differences between students classified as LD (discrepant low achievers) and those classified as LA. The heart of these debates centers on the degree to which LD can be differentiated from LA and the extent to which distributions of these groups' intellectual, academic achievement, and social behavior functioning overlap (Epps, Ysseldyke, & McGue, 1984; Fuchs, Mathes, Fuchs, & Lipsey, 2001; Kavale, Fuchs, & Scruggs, 1994; Ysseldyke, Algozzine, Shinn, & McGue, 1982). Perhaps the most widely cited study in this debate was reported by Ysseldyke et al. (1982) in which school-identified children with LD were compared to a group of LA children on a variety of psychoeducational measures. This study suggested that LD could not be differentiated from LA, with 96% of the scores on psychoeducational measures being in a common range. Ysseldyke et al. argued that LD and LA are essentially identical constructs, and they questioned the diagnostic validity of the term "learning disabilities."
Kavale et al. (1994) criticized the interpretation and analyses of Ysseldyke et al. (1982), indicating that the data had been misused to support policies from the Regular Education Initiative. Kavale et al. reanalyzed Ysseldyke et al.'s original data, using a meta-analytic statistic (Cohen's d) that compares the means of each group relative to the groups' variability (pooled standard deviation [SD]). On the basis of 44 comparisons, Kavale et al. showed that 63% of the LD group could be differentiated from the LA group (effect size = 0.338), with 37% overlap between the groups. This 37% overlap figure is substantially less than the 96% overlap reported by Ysseldyke et al. With respect to academic achievement, almost 80% of the LD group could be differentiated from the LA group, with LD children scoring lower than the LA group.
The results of the Connecticut Longitudinal Study added further fuel to the debate concerning the differentiation of LD and LA (Shaywitz, Fletcher, Holahan, & Shaywitz, 1992). This investigation compared children with LD (defined as a 22-point discrepancy between aptitude and reading achievement) with low achievers (defined as children scoring below the 25th percentile in reading, but who did not show a severe discrepancy). Using a variety of child-, teacher-, and parent-based measures, these authors found more similarities than differences between LD and LA groups, suggesting that both groups could be considered eligible for special education services.
The separate analyses and interpretations of the same data set by Ysseldyke et al. (1982) and Kavale et al. (1994), coupled with the longitudinal study by Shaywitz et al. (1992), leave a fundamental question unresolved: Are LD and LA quantitatively or qualitatively different? The studies and analyses by Ysseldyke et al. and Shaywitz et al. suggest that LD and LA groups are more alike than different. The analyses by Kavale et al. suggest these groups are more different than alike, particularly in the area of academic achievement. Researchers and practitioners are left with the decision of deciding which group's analyses and conclusions to believe. This distinction is important, given that important educational decisions are made for children with these characteristics and that these decisions have rather substantial economic and legal consequences for school districts.
Recently, a meta-analysis of 79 studies on this topic was completed by Fuchs et al. (2001), the purpose of which was to determine whether LD and LA reflect differences in degree of underachievement or differences in kind of underachievement. That is, is LD quantitatively different or qualitatively different from LA in terms of reading achievement? On the basis of 112 effect sizes, the mean weighted effect size was 0.61 (95% CI: 0.56 to 0.65); however, there was considerable heterogeneity among studies concerning the magnitude of differences in reading between LD and LA groups.
Fuchs et al. (2001) interpreted the 0.61 effect size as being large, thereby suggesting that LD could be differentiated from LA (LD < LA in reading achievement). However, this 0.61 effect size translates into only a 9-point (M = 100, SD = 15) standard score difference between LD and LA groups. In fact, Cohen and Cohen (1983) would define a 0.61 effect size as moderate and a large effect size as being 0.80 or greater. Assuming a median reliability coefficient of 0.90 for reading domain measures used in calculating effect sizes and a standard error of measurement of 4.74 (SD = 15), a 95% confidence interval calculates to +9.48 standard score points. Clearly, this difference is not large, particularly when taking into account measurement error of the dependent measures.
It is difficult to make the case that a standard score difference which is within the range of measurement error represents a substantial difference in kind rather than degree and therefore somehow validates the LD construct. Certainly, these data do not support a two-groups approach to LA like that found in the field of mental retardation (Zigler, Balla, & Hodapp, 1984). For the sake of argument, the average IQ scores of students with MMR is around 70 and the average IQ score of students with profound mental retardation is about 25. Few would argue that these two groups do not differ in kind on a number of variables such as identification prior to school entry, severe deficits in independent functioning, and frequent comorbid biomedical conditions (MacMillan, Gresham, & Siperstein, 1993).
The Fuchs et al. (2001) meta-analysis suggested that a standard-score point difference of 9 (0.61 SD) was sufficient to conclude that LD students differ in kind from LA students, particularly on timed reading tasks reflecting deficits primarily in automaticity of reading skills. By comparison, in the area of sensory disabilities, there are clear distinctions between hearing impaired and deaf as well as visually impaired and blind based on rather substantial differences in the magnitude of hearing and visual loss, respectively. Clearly, the field of LD must be able to present more convincing evidence to conclude that LD students differ in kind from LA students and thus legitimately deserve special education and related services based on this minimal difference.
There have been a variety of ways of operationalizing the LD construct using some variation of a discrepancy-based notion. Berninger and Abbott (1994) suggested that four major methods have been used to compute discrepancy; all of which have measurement difficulties: (a) deviation from grade level, (b) expectancy formulas, (c) simple standard-score difference, and (d) standard regression analysis. A deviation from grade-level approach makes the fallacious assumption that all students should be functioning on grade level. Of course, this assumption ignores the most fundamental principle of standardized achievement tests: In a normal distribution of test scores, half the students will be above level and half will be below grade level. How far below grade level one must be to qualify for LD using this approach is influenced by a variety of factors such as level of intelligence, socioeconomic status of the school, and measurement problems with grade-equivalent scores.
Another approach is to compare a child's expected and observed grade level in an academic area controlling for IQ (expectancy approach). To determine this discrepancy, this approach uses grade-equivalent scores which vary greatly across grade levels in terms of the raw scores underlying these scores and are not comparable across test instruments (Berninger & Abbott, 1994). A third approach uses standard-score differences between IQ and achievement measures (sometimes called the simple difference method) to quantify LD. This approach, however, does not account for measurement error in IQ and achievement measures, the unreliability of difference scores, and the attendant effects of regression toward the mean. In a final method, the regression discrepancy approach, the measurement errors using the simple difference method are accounted for by calculating aptitude-achievement discrepancies using the parameters of reliability of aptitude, reliability of achievement, and reliability of aptitude-achievement difference scores (Reynolds, 1984). This approach, like the standard-score difference approach, assumes that IQ is the exclusive and self-limiting determinant of achievement.
The aforementioned approaches to quantifying LD have been used to qualify students for special education and related services. Each method, as briefly reviewed, has a number of conceptual and statistical drawbacks. A major controversy in discrepancy-based notions of defining LD is the central importance assigned to IQ tests in this process (Gresham & Witt, 1997; Kavale & Forness, 2000; Siegel, 1989). Perhaps the most important criticism of IQ tests is that they contribute little reliable information to the planning, implementation, and evaluation of instructional interventions for children and youth. Moreover, according to the research contrasting LD and LA populations, IQ tests are not particularly useful in diagnostic and classification purposes for students with mild learning problems. What appears to be needed is an approach to defining LD based on how students respond to instructional interventions rather than some arbitrarily defined discrepancy between ability and achievement.
Funding for NRCLD is provided by the U.S. Department of Education, Office of Special
Education Programs. Award No. H324U010004. Opinions expressed herein are those of the National
Research Center on Learning Disabilities and do not necessarily represent the position of the U.S.
Department of Education.
© 2007 National Research Center on Learning Disabilities
Last Updated: November 19, 2007
Contact: nrcld@ku.edu