Joseph R. Jenkins, University of Washington & Rollanda E. O'Connor, University of Pittsburgh
Learning Disabilities Summit: Building a Foundation for the Future White Papers
This paper is available in alternative formats: | Download Word | Download pdf |
Identifying early those children most likely to encounter reading problems may constitute the first step in reducing the incidence or severity of RD. Because schools tend not to identify these children until the middle elementary grades, these children's reading difficulties grow stronger roots, and possibly become more intractable. For the most effective intervention, schools must find ways to identify these children much earlier than they usually do.
Research on early identification originates from studies of potential causes of reading difficulties, in which a range of children's preliteracy skills are measured in kindergarten or first grade, and then the strength of the correlations between these skills and reading ability is calculated either concurrently, or 1 or 2 years later. As one might expect, virtually all studies in which letter knowledge was measured in preschool, kindergarten, or early first grade documented its significant contribution to reading. Other contributing factors have been explored, such as vocabulary (Bowey & Patel, 1988; Mantzicopoulos & Morrison, 1994; Scarborough, 1990, 1995), short-term memory for language-related information (Baddeley, 1986; Leather & Henry, 1994; Mann & Ditunno, 1990; Rapala & Brady, 1990), and efficient retrieval of verbal labels (Badian, 1993; Bowers & Swanson, 1991; Doi & Manis, 1996; Seidenberg & McClelland, 1989; Wagner et al., 1987; Wolf, 1991), however, findings on the unique additional variance in reading that each factor contributes have been inconsistent. Some of these differences appear to depend on whether or which control variables were used (e.g., indices of IQ, socioeconomic level, age, or phonological awareness), and whether these skills and reading achievement were measured concurrently or predictively.
Since the mid-1980s, most studies that focus on predictive correlations have also included measures of phonological awareness (e.g., Berninger, 1986; MacLean, Bryant, & Bradley, 1988; Majsterek & Ellenwood, 1995; Mann & Ditunno, 1990; Share et al., 1984; Stanovich, Cunningham, & Cramer, 1984; Uhry, 1993; Wagner, Torgesen, Laughon, Simmons, & Rashotte, 1993). These measures include matching tasks, in which children match spoken words with similar beginning or ending sounds or rhymes; representational tasks, in which children tap the syllables or phonemes they hear within words spoken by an examiner; production tasks, in which children blend speech sounds together to generate words, or articulate the first, last, or all phonemes within spoken words; or deletion tasks, in which children listen to a word (e.g., baseball, street) and say the word without a particular syllable (e.g., /base/) or phoneme (e.g., /s/). A convergence of findings across these studies builds a strong case that the combination of phonological awareness and letter knowledge accounts for a large portion of the attributable variance in reading--generally 40-60% of the variance concurrently and 1-2 years later. In addition to reports of the relative stability of phonological processing over the elementary years (Elbro, Borstrom, & Petersen, 1998; Wagner et al., 1993), MacDonald and Cornwall (1995) found that phonological awareness measured in kindergarten was still more predictive of word identification and spelling 11 years later than was socioeconomic status or vocabulary.
Because sensitivity to the phonemic elements of spoken words is necessary for reading acquisition (see Figure 3), researchers have examined various ways to assess children's phonemic sensitivity as a means of early identification of RD. Yopp (1988) compared the strength of correlations and factor loadings of a range of measures of phonological awareness and a reading analogue task with kindergartners, and found that rhyme production was too easy, deletion too difficult, and segmenting the most highly correlated with reading analogue scores. Because her participants were kindergartners, she was unable to measure "real" reading. Segmenting tends to develop among typical readers during kindergarten and early first grade (Kaminski & Good, 1996; Vandervelden & Siegel, 1997), and so measures used with preschoolers, such as rhyme (MacLean et al., 1988) or alliteration (Bradley & Bryant, 1983), are often predictors of later predictors (i.e., segmenting). Murray, Smith, and Murray (2000) tested the validity of a measure of phoneme identity ("Do you hear /s/ in moon or soon?") to predict prereading children's ability to read with phonetic cues (choosing between the printed words mad and sad when sad was spoken). Their measure correlated more strongly with trials to criterion on phonetic cue reading than the Comprehensive Test of Phonological Processing (Wagner, Torgesen, & Rashotte, 1999) or the Yopp-Singer segmenting test (Yopp, 1988); however, again, the dependent variable was not "real" reading achievement, and the measures were used concurrently, rather than predicting reading later in time.
Despite a strong correlational knowledge base connecting children's phonological language skills to later reading acquisition, predicting exactly which children will develop RD has proved problematic. The relative accuracy of prediction varies with the specific measures used as predictors and as outcomes, the timing of their administration, and the degree and direction of classification error the researchers consider acceptable, such that differences among selection processes have resulted in confusion over how reliably and early children with RD can be identified (Felton, 1992; Hurford et al., 1993; Torgesen, Burgess, Wagner, & Rashotte, 1996). Moreover, as Tymms (1999) suggested, "assessment has its work cut out simply getting a good general measure," given the tendency of many kindergartners to become easily distracted or bored.
Two types of errors reduce the prediction of RD. Errors of underprediction occur when the predictive measures miss children who subsequently develop RD. Coleman and Dover (1993) developed the RISK screening battery, which required teachers to estimate the ability of each of their kindergarten students along several different dimensions, including school competence, task orientation, social competence, behavior, and motor ability. The overall accuracy of the scale was high (94% of children correctly identified); however, 21% of the children who later needed special education services were overlooked by this screening tool. Over half of the missed children were girls, suggesting the possibility of teacher bias related to gender. Mantzicopoulos and Morrison (1994) investigated the accuracy of the SEARCH procedure developed by Silver and Hagin (1981) for identifying children at the end of kindergarten who were likely to develop RD. SEARCH, which uses 10 measures of reading readiness (visual and auditory discrimination, immediate visual recall, visual-motor copying, rote sequencing, articulation, sound-symbol associations, directionality, finger schema, and pencil grip) underidentified relatively advantaged youngsters (missing 50%), while overidentifying children from low socioeconomic backgrounds.
Measures of vocabulary or concepts about print, although moderately related to later reading achievement, can lead to underprediction of RD because some children who will develop RD, especially those who are older than their peers at the time of testing or those who come from homes rich in literacy experiences, perform better on these measures than non-RD children who are younger or who come from more impoverished literacy circumstances. Measures that underpredict RD are of concern for those interested in early intervention because they directly undermine the intent of early intervention efforts (i.e., identifying those students who require early, intense, and targeted instruction).
The second type of error, overprediction, occurs when predictive measures mistakenly identify non-RD children as at risk for becoming RD. Indeed, most efforts to identify reading problems before children receive reading instruction overpredict RD (Badian, 1994; Catts, 1991; Felton, 1992; O'Connor & Jenkins, 1999; Torgesen et al., 1996; Uhry, 1993). Sixty-nine percent of the children predicted to be at risk in Felton's (1992) study, for example, were good readers by third grade; only 58% of Badian's (1994) at-risk preschoolers had confirmed reading problems 2 years later.
Over time, attempts at early identification of RD have been linked to the theoretical models of the causes of learning disability as understood at that time. Uhry (1993) used measures of sound categorization, segmenting, fingerpoint reading, and writing in kindergarten to predict poor readers in first grade. She established cutoff scores for the measures, which increased the potential usefulness of the battery; however, establishing a low cutoff score missed 28% of the future poor readers, and raising the cutoff scores to correctly identify most at-risk students resulted in a prediction that 42% of her private school sample would develop reading difficulties. Torgesen et al. (1996) used measures of phonological awareness, rapid naming, and letter knowledge in kindergarten and first grade to predict beginning second-grade reading. Although measures administered in first grade were more predictive than in kindergarten, they still missed 35% of the poor readers 1 year later.
Nicolson and Fawcett (1996) developed the Dyslexia Early Screening Test (DEST), a set of screening measures and cutoff scores widely used in the United Kingdom at school entry. Rather than identify a small set of predictive measures with cutoff scores, their test yields a profile of current ability across rapid naming, phonological and letter tasks, copying, and balance, which is summed to a risk index. With cross-validation, however, they needed to adjust the cutoff scores for this index to avoid problems of underprediction, particularly for children who began kindergarten at an age greater than 6.5 years.
Some of the language measures that have the highest correlation with subsequent reading achievement (e.g., phonological segmentation) are difficult for many typically developing children when tested early in kindergarten, leading to substantial overprediction errors. Other language measures, such as receptive or expressive vocabulary, have strong relations with reading comprehension by second or third grade, but may weaken classification accuracy for first-grade reading because they exert a protective factor in a discriminant function, making children with RD with strong vocabularies more difficult to detect--even though, on average, children with RD earn lower verbal scores (O'Connor & Jenkins, 1999). When early intervention services are in short supply, overprediction may squander limited educational resources. Part of the challenge facing researchers is to identify early-developing reading-related skills, and design age-based measures that are at an appropriate level of difficulty. As Badian (1998) noted, "as the nature of reading changes, so change the predictors" (p. 478).
Researchers have tried to solve the problem of overprediction by fine-tuning measures to make them more sensitive to small differences among children, or to growth in the same children over time. For example, even though kindergartners' ability to segment spoken words is tied (r = 0.62) to their reading achievement in grade 1 (Share et al., 1984), segmentation ability nevertheless overpredicts RD because many normally developing readers are unskilled segmenters in kindergarten, leading to floor effects for the measure. To better distinguish children with RD from late-developing segmenters without RD, researchers have attempted easier levels of segmenting, such as syllable tapping (Badian, 1998), alliteration matching (Bradley & Bryant, 1983), or first sound production (Good, Simmons, & Kame'enui, 2001). Although these tasks correlate with reading, they lose substantial predictive power when administered late in kindergarten (O'Connor & Jenkins, 1999). Others have used discrimination indexes or item-response theory to order the items within a segmenting task (Wagner et al., 1999) from easier to more difficult. This approach allows children with low skill levels to engage in some of the testing items; however, the number of low-level items is limited.
Another approach to controlling the difficulty of segmenting is to adjust the tasks to offer more opportunity to learn, or to assess growth in a skill, rather than merely static achievement. Spector (1992) used a dynamic segmentation measure that provided children with varying levels of prompts to help them perform the task. Dynamic segmentation proved more predictive of later reading achievement than did static segmentation. Kaminski and Good (1996) provided variable scoring on the items of their segmenting test, so that children received credit for partially segmenting a word (e.g., identifying the /f/ or the /sh/ in fish), with more points for completely correct attempts. Scoring adjustments that reflect partial knowledge of a complex task (e.g., isolating the first phoneme within a word) and progress toward a goal (i.e., to completely segment a three- or four-phoneme word) may also provide teachers with insight into children's instructional needs.
O'Connor and Jenkins (1999) tested over 400 children in kindergarten and followed their reading development through first grade, layering the investigation by testing various cohorts from diverse geographic, community, and economic conditions. They began with measures that have been identified in studies that sought component skills with high concurrent (Badian, 1993; Perfetti, Beck, Bell, & Hughes, 1987; Tunmer, Herriman, & Nesdale, 1988) or predictive (Felton, 1992; Hurford et al., 1993; Juel, 1988; Share et al., 1984) correlations with reading, including timed letter recognition, first sound identification, syllable and phoneme blending and segmenting, deletion, short-term memory for sounds, and rhyme production. Next, they set criteria by calibrating indicators of RD on one cohort of children; testing the parameters on a new cohort; and exploring the relative accuracy of predictors gathered over time. Across the three cohorts, rapid letter naming and segment phonemes were included among the subset of strong predictors of RD at all three screening points (beginning and end of kindergarten and beginning of first grade). The stability of these two tasks across three test periods in this study may be tied to their capacity to detect fine-grain individual differences. Their letter naming task represented not only children's accuracy of letter knowledge, but also their speed in accessing that knowledge. Likewise, their segmenting measure tapped various levels of insight into the phonemic structure of words, because items were not scored simply as right or wrong; rather, credit was awarded for gradations of phonemic awareness (isolating the initial sound in a word, separating onset from rime, complete phonemic segmentation), much like that of Kaminski and Good (1996). In addition, borrowing from Spector's work (1992), they provided corrective feedback to children during administration of the measure, which offered learning opportunities within the task itself. They suggested that the combination of graduated scoring and corrective feedback increased the predictive validity of the segmentation task by reducing floor effects that otherwise would have been pronounced, particularly for the November kindergarten test period, had scores been based solely on complete phonemic segmentation.
This sensitivity to children's partial and developing knowledge of segmentation probably contributed to reduced overselection rates (4-17% across cohorts) relative to earlier prediction studies. It also reduced ceiling effects associated with tasks like identifying the first sound in words, a task that was difficult in November for all three cohorts, but mastered by April for many students, including a few children who later developed RD. Depending on the timing of the screenings and the cohort, overprediction ranged from 4 to 17% and underprediction from 0 to 9%, but like Nicholson and Fawcett (1996), O'Connor and Jenkins warned that the patterns that predict poor reading among children of typical kindergarten age may not apply as well to older kindergarten children (> 6.1 years in September) who are repeating the grade. Some children who repeated kindergarten had learned enough about letter names and segmenting first sounds to score above established cutoff criteria, even though their performance still fell below the average of first-time kindergartners.
It appears, then, that for prereaders in kindergarten, tests that incorporate some form of learning, such as providing feedback on test items (O'Connor & Jenkins, 1999), variable scoring to indicate partial knowledge (Kaminski & Good, 1996; O'Connor & Jenkins, 1999), or trials to criterion (Murray et al., 2000; Spector, 1992) may be more sensitive indicators of future reading achievement.
Prediction studies attempt to select (a) all children (i.e., no underprediction) whose reading scores at the end of first or second grade reveal a pattern of RD and (b) few children (i.e., small overprediction) whose later reading scores do not reveal an RD pattern. None of the studies we have reviewed have met these stringent expectations. Discriminant analysis provides information about the extent to which tasks in kindergarten distinguish children who eventually develop an RD profile. To take the next step in developing a screening instrument requires establishing criterion, or cutoff, scores for each of the primary predictors. Few research studies provide specific criteria for interpreting scores on predictive measures, and specific measures are rarely cross-validated with other samples.
Selecting specific tasks that are most useful in distinguishing children who will exhibit RD is dependent on the timing of the screening effort. Second, cutoff scores on various screening measures that accurately distinguished RD in one cohort tend to have reduced predictive validity for other same-age cohorts. In studies that included cross-validations (Badian, 1998; Fawcett, Singleton, & Peer, 1998; O'Connor & Jenkins, 1999), the researchers liberalized the preceding criterion scores with each successive cohort in order to capture every child who subsequently developed an RD profile. As expected, raising criterion scores increased overprediction rates, sometimes substantially.
As we noted before, which error is most egregious depends on the consequences. Some researchers recommend screening later than kindergarten to reduce the overidentification (Torgesen, Burgess, Wagner, & Rashotte, 1996). Accuracy rates of the predictive tasks for correctly classifying RD and non-RD groups tend to be higher with later screening. Moreover, the accuracy of prediction in kindergarten is somewhat dependent on the instruction children receive in first grade. This phenomenon was documenting in a year-long study conducted by Perfetti, Beck, Bell, and Hughes (1987) in which ability to blend and segment at the beginning of first grade was predictive of reading at the end of the year for children who received instruction organized around whole language, however, early phonemic awareness lost predictive power in classes that included frequent instruction in phonics as part of the reading approach, perhaps because the instruction in sound-symbol relations and word analysis quickly established the alphabetic principle for most children who had not already acquired it.
Badian (1998) suggests that many children predicted to fail by her kindergarten measures in fact succeeded because of the instructional approach in first grade. This approach was based on Bradley and Bryant's (1985) instructional procedures for children at risk for reading problems, which included integrating letter sounds with phonological blending, segmenting, and spelling. She believed that her rate of overprediction would have decreased if children had received a less structured reading program.
Tradeoffs between increased accuracy of identification and provision of early intervention affect the choice of a screening window. Another alternative is to incorporate some of the features of early intervention (e.g., stronger emphasis on letter knowledge, phonological blending and segmenting, and activities to promote the alphabetic principle) in general kindergarten routines, so that children are less likely to score poorly on kindergarten screenings because of lack of exposure, and are more likely to succeed in first grade.
Reasonably accurate prediction of RD is essential for evaluating the outcomes of early intervention. It is obvious why the predictive net must capture all or most of the children with RD--they are whom treatment is meant to help. Unless we set liberal cutoff scores (resulting in sizable overidentification), no set of predictors appears to be 100% accurate in identifying all children who eventually develop RD. Moreover, if RD samples in early intervention studies include many non-RD children, researchers and practitioners may be misled by the cure rate for children who did not really have RD. Prediction batteries that can be administered more than once over time may decrease overprediction by allowing the evaluators to determine growth in response to good instruction, as well as absolute levels of skills.
Some researchers (e.g., Fawcett et al., 1998; O'Connor, 2000; Simmons, Kuykendall, King, Cornachione, & Kame'enui, 2000) advocate layered approaches to screening and intervention, such that prediction of reading problems and increasingly intense interventions are interfaced over time. The interplay between small-group instruction on early literacy skills and ongoing measurement may ease the problem of overidentification, while offering low-cost early intervention to the children captured in the predictive net. Some sensible actions to identify the children most likely to need intensive support in reading are shown below.
Early Identification of Reading/Learning Disabilities: Sensible Actions
Funding for NRCLD is provided by the U.S. Department of Education, Office of Special
Education Programs. Award No. H324U010004. Opinions expressed herein are those of the National
Research Center on Learning Disabilities and do not necessarily represent the position of the U.S.
Department of Education.
© 2007 National Research Center on Learning Disabilities
Last Updated: November 19, 2007
Contact: nrcld@ku.edu