Donald L. MacMillan, University of California, Riverside, & Gary N. Siperstein, University of Massachusetts, Boston
Learning Disabilities Summit: Building a Foundation for the Future White Papers
This paper is available in alternative formats: | Download Word | Download pdf |
Pages: | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
Ysseldyke and Algozzine (1983) noted years ago that the most important decision in the assignment of children to LD programs is the decision by the regular classroom teacher to refer. Zigmond (1993) echoed this sentiment when she wrote: "The referral is a signal that the teacher has reached the limits of his or her tolerance of individual differences, is no longer optimistic about his or her capacity to deal effectively with a particular student in the context of the larger group, and no longer perceives that the student is teachable by him- or herself" (pp. 262-263). Any understanding of the population of SI LD students begins with a consideration of those students that general education teachers consider "difficult to teach." Given the process prescribed in IDEA, different perspectives are dominant at different stages of the referral-assessment-placement process that contribute to false positive and false negative identifications. Decisions to refer made by a general education teacher are influenced by factors beyond child characteristics. That is, two hypothetical children with identical reading deficits enrolled in different districts are not equally at risk for being referred by their classroom teacher. The extent to which respective teachers are optimistic about their ability to successfully teach the child (i.e., the teacher's self-efficacy) enters into the decision. Zigmond (1993) reported on one of her projects that explored the extreme differences in rates of special education services. Fiscal and demographic variables failed to explain why some districts served large proportions of their students (11-15%) while others served small percentages (2-4%); neither did the availability or use of prereferral options at the school building level explain the disproportion. Teachers in schools serving small proportions of their students in special education did, however, express greater optimism about the likelihood of success of non-special-education strategies and interventions than did teachers in the schools serving high proportions of students in special education.
A teacher's decision to refer is also influenced by a comparison of a given child's academic performance to that of classmates or some absolute standard held by the teacher regarding "how well a second-grade student should be reading." The modal level of achievement in a given classroom is the baseline against which teachers judge the adequacy of specific children's performance. The two hypothetical students with the same level of reading performance are differentially at risk for referral if enrolled in two different classes where the modal level of achievement in one is 2 years below grade level and the other is 2 years above grade level. Presently, judgment by the general education classroom teacher that a child's performance is inadequate and unresponsive to materials and methods available in that teacher's classroom prompts referral, a necessary but insufficient step in becoming SI as LD. A parent less frequently initiates referral; however, even in such cases it is prompted by a perception that the child's progress in general education is inadequate and treatments provided are ineffective. These judgments are, by their very nature, subjective because they are made in the context of a specific teacher's classroom or a parent's experience with other children, neighborhood peers, and family relatives and friends.
General education teachers do not refer all students who have, if tested, a psychometric profile that meets state education code criteria. As noted above, referral is necessary for SI as LD. Cases meeting criteria that are not referred will not be available for sampling as SI LD, but would be included in a RI population of LD. However, the only way one could "catch" unreferred students meeting RI LD criteria would be by doing massive screenings of all schoolchildren with nationally standardized scales and then rigidly applying criteria. In subsequent discussion we will refer to this group of students (psychometrically eligible but not SI as LD) as "false negative" cases of LD. Students who are SI as LD but do not meet eligibility criteria will be referred to as "false positive" cases of LD.
The false negatives first emerge at the referral stage. If students are not referred, they will not be SI as LD. We can't begin to estimate the magnitude of this group, although we will subsequently describe situations in which students are referred by their teacher and found to be psychometrically eligible as LD, yet they elude being identified by the schools as LD.
The general education teacher serves as an "imperfect test" (Gerber & Semmel, 1984; Gresham, MacMillan, & Bocian, 1997), thereby determining which children get referred and which do not. The imperfections include the optimism discussed above, but could also result from other factors independent of the specific academic deficiencies noted as a concern. One line of research has examined whether general education teachers are racially and/or gender biased in their referrals, with somewhat mixed findings. Zucker and his colleagues (Prieto & Zucker, 1981; Zucker & Prieto, 1977; Zucker, Prieto, & Rutherford, 1979) used vignettes and manipulated the ethnicity and gender of the child when presenting the vignettes to teachers. Their findings suggested that teachers were more inclined to judge a child as appropriate for special education placement if he or she were described as Black or Hispanic; however, no effects for gender were found. Shinn, Tindal, and Spira (1987), however, found that both racial and gender biases were plausible for the referral behavior of elementary school students with severe reading deficits.
Two other investigations (Bahr, Fuchs, Stecker, & Fuchs, 1991; Tobias, Zibrin, & Menell, 1983) extended the design to consider the race of both the referring teacher and the child being considered for referral. In the Tobias et al. study, neither the race of the teacher nor that of the student exerted a significant effect on the referral recommendation. However, Bahr et al. found a significant effect for the race of the student (i.e., with Black students being judged as more appropriate for placement) but not for the effect of the race of the teacher or the interaction.
Similarly, MacMillan, Gresham, Lopez, and Bocian (1996) examined a sample, stratified on the basis of ethnic group, of primary-grade students who had actually been referred by their teachers for prereferral interventions. They concluded that these teachers defined "difficult to teach" (i.e., the reasons for their referral) in terms of absolute low achievement and problem behaviors, primarily externalizing behaviors. In essence, those students whose academic performance deviates significantly from that of classmates and those whose behavior is disruptive and threatening to the smooth running of the classroom are at heightened risk for referral. Several findings from that study bear directly on the issue of potential bias. Comparisons on the basis of ethnic group (White, Black, Hispanic) and gender resulted in the following significant differences: (a) referred White students had significantly higher verbal IQ scores and reading achievement scores than did referred students from the other two ethnic groups, (b) referred Black students were more likely to have a higher incidence of behavior problems than were Hispanic students, and (c) gender differences were evident in the problem behaviors exhibited (males having more) but did not emerge on cognitive or achievement measures. Contrary to some of the previously summarized studies, these data suggest that teachers were more reluctant to refer Black students. That is, Black students who were referred exhibited achievement deficits more severe than those prompting the referral of White students and behavior problems more serious than those prompting referral of Hispanic students.
We call attention to the fact that when teachers make decisions about a child's academic progress, it is teacher judgment that is employed, using local norms as the child's performance is compared to that of classmates and grade peers (Bocian et al., 1999). When teachers refer a child they do not know if that child is LD or mildly mentally retarded (MMR). Instead, they know that the child's progress is unacceptable in comparison to local norms. We know of precious little evidence that addresses the magnitude of the population of children who would psychometrically qualify as LD but who are never referred by their general education teacher. These would be the "false negative" cases (if one assumes the RI LD are the "real LD"). Ironically, the limited evidence forthcoming from MacMillan, Gresham, and Bocian (1998) reveal that among referred students, the traditional LD student with above-average intelligence and discrepant achievement was among those not placed in special education.
Referral by general education teachers then initiates a process that can ultimately lead to a child being SI as LD. We have noted that the teacher employs local norms in deciding that a child's academic performance is deficient. The achievement of the child's classmates or the teacher's subjective standards for acceptable achievement provide the basis for comparison. While the evidence is somewhat conflicting, it would appear that teachers might employ slightly different standards when evaluating the achievement of children of different racial or ethnic groups. Moreover, the referral decision is grounded in the child's absolute level of achievement rather than comparing it to an expected level of achievement based on the individual child's aptitude or achievement in other subjects. As a result, subjectivity is introduced into the process in this initial stage that precludes any possibility of yielding the population of LD children defined in education codes and authoritative definitions. The false negative cases, where teachers fail to refer, are of unknown magnitude but will not be SI as LD. Already some students who meet RI LD criteria have been eliminated from possible identification. The cases moving to the next stage (assessment) also include students who do not meet criteria for LD, but whose exclusion will depend on being detected and eliminated from possible classification during one of the two following stages in the process. As we will see, however, subjectivity is present at these stages as well, making the ultimate decision contaminated by additive subjectivity.
For those students who are resistant to interventions provided in general education, something has to be done, be it retention in grade or consideration for special education services. In order to qualify for special education services, the child must qualify for one of the disability categories, determination of which requires psychological evaluation. MacMillan and Speece (1999) characterized this gate, the psychological assessment, as representing a cognitive paradigm intended to detect or document the existence of a within-child problem. In the case of LD, a common definitional criterion for eligibility is a severe discrepancy between aptitude and achievement; 98% of the states include a discrepancy of some magnitude in either the definition or criteria for LD (Mercer, Jordan, Allsopp, & Mercer, 1996). Bocian et al. (1999) expanded on this characterization of the assessment gate by suggesting that the psychometric data on aptitude and achievement permit determining whether the child's level of achievement is acceptable. If achievement is far below predicted levels (based on aptitude), then the placement team would probably conclude that it is unacceptable--the student should be doing much better. On the other hand, if the measures of intelligence and achievement are consistent (i.e., both very low), then one would reluctantly conclude that achievement is "acceptable" (if not desirable) and presumably consider "exclusionary" criteria that might prevent eligibility as LD.
In comparison to the referral stage, the assessment stage employs national norms. The use of "objective" evidence is a cornerstone of psychological assessment, and information from standardized tests is used--administration protocols scripted, the scoring carefully prescribed, and comparisons of a child's performance made to norms established on nationally representative samples. While teacher judgment was employed in decisions concerning referral, the assessment stage is devoid of such factors.
When one considers the process prescribed under IDEA, these two "competing paradigms"--teacher referral employing local norms and the assessment employing national norms--the tension between "those whom teachers perceive as needing help" and "those whom psychometric profiles indicate are entitled to receive help"--result in overlapping populations. Were the psychometric template applied to all public school students, a segment of children who, for whatever reason, are not referred by their teachers (referred to previously as "false negatives") would emerge. Moreover, of those referred by their teachers, some number are found ineligible when the psychometric template is applied. The psychometric data suggest that either the level of achievement is not sufficiently discrepant from aptitude to warrant eligibility as LD or the IQ score may be below the criterion for mental retardation. In other words, the assessment stage serves to screen the referrals made by the teacher and has historically been used to make a differential diagnosis, differentiating between cases of LD, mental retardation, emotional disturbance, and speech and language.
In previous writings (MacMillan, Gresham, Siperstein, & Bocian, 1996; MacMillan et al., 1997, 1998) we have described the difference between what is supposed to be and what really is in the use of assessment in qualifying children for special education services. The compendium of results from studies examining the degree of congruence between criteria specified in authoritative definitions or state education codes and the characteristics of students actually served under a given disability rubric is not very high. For example, when we examined the group of children that the public schools ultimately qualified as LD, less than half (29 of a total 61) evidenced the required discrepancy using the Wechsler Intelligence Scale for Children III (WISC-III) and Wide Range Achievement Test-Revised (WRAT-R) scores. MacMillan et al. (1998) wrote, "...public school practices for diagnosing children with LD bear little resemblance to what is prescribed in federal and state regulations...defining LD..."(p. 323). The models suggested in the federal and state regulations, particularly concerning criteria, are "measurement bound," specifying cutoff scores, requisite discrepancies, and various other psychometric profiles on tests and rating scales that are to be applied "objectively" in establishing eligibility. Below we argue that despite the appearance of "objectivity" at the assessment stage, considerable subjectivity is introduced which serves to further distance SI cases of LD from RI cases of LD.
What do the public schools do? On the basis of our findings and rather extensive discussions with school personnel in several states, we conclude that the concept of LD used in the schools is not defined by psychometric profiles prescribed in legislation or employed by researchers. First, school personnel knowingly classify children with very low cognitive skills (mentally retarded?) as LD, despite exclusionary criteria and a lack of required discrepancies. Moreover, they express particular disrespect for tests of intelligence, which they perceive to be unfair and totally lacking in instructional validity. In addition, placement committee members and special education directors believe that the label "mentally retarded" is extremely pessimistic in its prognosis and are reluctant to use it. As one administrator put it, there is no upside to classifying a child as mentally retarded--the child is stigmatized, the parents resent the label, and we can develop an appropriate program for the child through the IEP process regardless of what we call the child.
These conclusions are consistent with those of Gottlieb, Alter, Gottlieb, and Wishner (1994) who also noted the fact that the discrepancy component is ignored by school professionals. They concluded that one reason is that urban practitioners knowingly ignore the absence of the required IQ-achievement discrepancy in "an effort to marshal scant resources for low-achieving students" (p. 459). The same sentiment is reflected in Shepard's (1983) comments: "Specialists would be more willing to make tough decisions about whether a child was really LD if rejecting the label was not tantamount to denying help" (p. 8).
A second observation about the assessment process and how school personnel use the results is in order. Placement committee members are painfully aware that certain assessments are mandated by state regulations. Moreover, school personnel dutifully, if unenthusiastically, comply with these regulations, although seeing them as "necessary evils" because that is the mandated process required in order to get services to children. As MacMillan, Gresham, Siperstein, & Bocian (1996) expressed it, school personnel are more concerned with "what to do" than with "what kind of kid this is." In this same spirit, one also encounters "creative testing" employed in order to record a combination of numbers that justifies the classification as LD. For instance, if the aptitude estimate using the WISC-III is "too low," then a nonverbal test of intelligence or an older version of the WISC (e.g., WISC-R) with outdated norms might be employed in order to secure a higher aptitude score and thereby the requisite discrepancy. Once convinced that this child needs and will benefit from services available in special education placement, school personnel seek ways to justify the action. School personnel repeatedly note the lack of instructional validity of intelligence tests for teachers' instructional planning and object to the cost of performing these assessments merely for qualifying the eligibility of children. We reiterate a point made in the early portion of this chapter: The schools' reason for classifying students LD is primarily for planning for services. In order to plan for services, the process prescribed under IDEA first requires that a child be qualified as eligible and in most states that requires the right combination of numbers (Mercer et al., 1996).
The subjectivity noted at the referral stage is exacerbated by additional subjectivity being introduced during the assessment stage. While objectivity is the hallmark of psychometric assessment, the selection of specific tests to be used (and the combination thereof) in order to justify the decision to serve this child results in most cases in false positive identifications. That is, a child who would not display the requisite discrepancy if one combination of tests is used is assessed with another combination of tests. Increasingly, the discrepancy between SI LD and RI LD is increased even further. As will be discussed in the next section, there is also the potential for increasing the number of false negatives, but it occurs in the deliberations of the committee assigned responsibility for placement. That committee may attend to the perceived need of a child and choose to disregard an IQ-achievement discrepancy when the absolute level of achievement is considerably higher for a nondiscrepant profile with very low reading achievement, for example. Let us now turn our attention to the third stage in the referral-assessment-placement process: the committee decision making that serves as the final arbitrator in the SI process prescribed in IDEA.
The recommendation of the placement committee (we use this term here recognizing that various other terms are used to describe it) ultimately determines if a given child will be classified as LD after considering all of the evidence brought to its attention. IDEA specifically prescribes that a multidisciplinary team decision must be made, and specifies the role of the parent in this decision. These specifications make it clear that the psychometric profile alone cannot be used to determine eligibility--to do so would be out of compliance. This interdisciplinary team is responsible for determining eligibility and, when appropriate, crafting the IEP and determining least restrictive environment (LRE) for a given case. Like the teacher at the referral stage, the team is permitted to exercise professional judgement, but it is "collective judgement" rather than individual judgment, as was the case at the referral stage.
Bocian et al. (1999) reasoned that the team decision regarding eligibility and "placement" is guided by the concept of profitability, which reflects the collective judgment on whether the specific special education services provided by the special education staff at that school site will or will not be beneficial to the child. At this stage, the information and perceptions that prompted the general education teacher to refer the child, the results of the formal psychological assessment comparing that child to norms based on national samples, and sociocultural and contextual factors that inform the decision are all considered by the placement team. The team must weigh evidence coming from the competing paradigms described above (e.g., local vs. national norms). Bocian et al. noted that a number of contextual factors are considered prior to making its decision:
Ideally, this team decision will weigh evidence provided by the general education teacher, the school psychologist, the parents, and all members regarding the perceived efficacy of the services that accompany alternative decisions. In addition, very practical considerations enter into the decision: openings in a special day class, caseload of resource teachers, second language issues, and the stridency of the parents when they oppose a course of action. (p. 3)
It was also noted that the relative forcefulness and competence of participants in specific team meetings play a role in the course of action ultimately taken. For example, a forceful general education teacher pressing for placement and a school psychologist with borderline discrepancy evidence may arrive at a different decision than a team with an ambivalent teacher and an articulate and forceful school psychologist.
When one examines the decisions made by a multidisciplinary team in light of the evidence available from the teacher's perspective and the school psychologist's perspective, the process prescribed by IDEA apparently does yield rational decisions. Bocian et al. (1999) examined the team recommendations to certify students as LD. In certain cases the teacher rated the child as having very severe achievement deficits, while in other cases the teacher ratings indicated only modest deficits--these cases reflected "local norms" as we've used the term. National norms were reflected in these cases by the presence or absence of the required IQ-achievement discrepancy. When there was congruence between these conflicting paradigms (i.e., either [a] the teacher rated the achievement deficit as very severe and the discrepancy was present, or [b] the teacher rated the achievement deficit as only modest and the discrepancy was not present), the decisions reached were consistent: 78% of [a] were classified as LD; 100% of the cases in situation [b] were not classified as LD. However, when the teacher rating of the severity of the achievement deficit and the presence or absence of a discrepancy were misaligned (i.e., achievement rated relatively high-discrepancy or achievement rated low-nondiscrepant), the rate at which the LD label was appended was much lower--45% and 39%, respectively.
Payette and Clarizio (1994) also examined team decisions on LD eligibility and found that three fourths of the children classified as LD did meet the severe discrepancy. In this investigation, the authors went on to examine factors related to "two kinds of misclassification observed: ineligibility with a severe discrepancy, and eligibility without a severe discrepancy" (p. 43). We have referred to these two situations elsewhere in the paper as "false negatives" and "false positives." In false negative cases, which constituted 16.57% of the referred sample, these investigators found that this group had significantly higher IQ scores than did students with a severe discrepancy found eligible by the placement team. In addition, they were significantly more likely to be White and older and have higher achievement scores. In the false positive cases, which constituted 9.59% of the sample, when compared to those found not eligible, findings indicated that they had lower Full-Scale IQ (FSIQ), were more likely to be girls, and were significantly lower in achievement. It is worth noting that the rate of false positive cases reported by Payette and Clarizio is considerably smaller than noted earlier for other investigations. However, the mean FSIQ for their ineligible cases was 102.30 and for the eligible children it was 95.03. In MacMillan, Gresham, Lopez, & Bocian (1996) study on students referred to prereferral intervention, the mean Verbal IQ scores were as follows: 87.42 (White), 79.93 (Black), and 78.17 (Hispanic). Such variability in characteristics across school districts attests to the "relativity" of LD and, in the MacMillan et al. study, the difficulty in demonstrating a discrepancy is evident.
Funding for NRCLD is provided by the U.S. Department of Education, Office of Special
Education Programs. Award No. H324U010004. Opinions expressed herein are those of the National
Research Center on Learning Disabilities and do not necessarily represent the position of the U.S.
Department of Education.
© 2007 National Research Center on Learning Disabilities
Last Updated: November 19, 2007
Contact: nrcld@ku.edu