Description of Study

Sample

We screened all first grade children in two consecutive cohorts (n= 679) with two LSF probes in late September/early October to identify samples of at-risk (AR) and not-at-risk children (NAR). AR children were those who earned mean scores in the bottom 25% of their classrooms. NAR children scored at the 30^th, 50^th (two children), 75^th, and 90^th percentiles for their classrooms and represented the range of reading ability in the classroom. There were 140 AR children and 136 NAR children in the sample. In order to interpret the data from a normative perspective, the data for the NAR children were weighted by a factor of four as each NAR child represented approximately four children in the population. This procedure yielded a weighted N of 684, which is close to the population n of 679. From this point forward the AR and NAR designations refer not to the above subject selection procedures but to classifications formed by the various screening methods used (described below).

The characteristics of the weighted sample generally reflected a normal distribution: WJ-R Basic Reading Skills Cluster Score M = 107.2 (SD = 15.5); Full Scale IQ M = 103.1 (SD = 16.9). The May Oral Reading Fluency descriptive statistics were Level M = 46.0 (SD = 37.0); Slope M = 1.42 (SD = .97). The slope estimate approximates other early elementary school samples reported in the literature (e.g., 1.5 in Fuchs, 2003; Deno et al., 2001), while the level data are lower than some reports (e.g., M = 66 [SD = 41.9], Marston et al., 2003).

Screening Groups

Screening groups were formed using cutoff scores - 1 SD below the mean for the following variables: LSF Fall Screen, LSF January Level, LSF January Slope, LSF January Level and Slope (DD), and LSF Growth. The cutoff score for ORF January Level was defined as a score below the 25th percentile. The LSF Growth variable was calculated by subtracting LSF Fall Screen from LSF January Level. The - 1 SD criterion was selected because a more reasonable percentage of children was identified as AR (Mdn. = 16.96, range 8.04 to 23.8) compared to a -.5 SD criterion (Mdn. = 33.3; range 18.71 to 42.7).

Criterion Variables

Two approaches were used to assess the validity of the screening groups. First, the AR and NAR children from each screen were compared on the available end-of-year reading variables: WJ-R Basic Reading Skills Cluster standard score (BRS SS; composite of Letter Word Identification and Word Attack), WJ-R Passage Comprehension standard score (PC SS), ORF level and ORF slope. ORF level was the average of the last two data points administered in May and reflected the number of words read correctly in one minute from connected text; ORF slope was determined through OLS regression and reflected growth from January to mid-May in terms of weekly increase in words read correctly. Children classified as AR should have lower end-of-year reading scores than children not classified.

Second, several "reading disability" classifications were created so that classification analyses could be conducted with the screen groups. These classifications were based on procedures either recommended or used in the literature to define reading problems. Four classifications were created: BRS SS 90 (BRS standard score less than 90 or the 25th percentile); BRS SS 78 (BRS standard score less than 78 or approximately 1.4 SD below the normative mean); ORF 40 (ORF score below the benchmark criterion of 40 words per minute); and ORF DD (ORF slope and level -1SD below the weighted sample's mean).

All of the screening groups were crossed with all the reading disability groups to produce 2 x 2 tables for calculation of diagnostic utility indexes. Four indexes are reported: Sensitivity (TP/TP + FN), specificity (TN/TN + FP); Positive Predictive Power (TP/TP + FP); and Negative Predictive Power (TN/TN + FN). The extent to which each index approaches 100% is an indication of classification accuracy.

Results

Group differences. Table 1 presents the percentage of children classified as AR for each screening group and the effect size (d adjusted for sample sizes) for each end-of-year reading variable. The largest AR group was produced by the ORF January cut point of the 25th percentile and the fewest AR children were identified by the LSF January DD criteria. Large effect sizes (ES) (all differences were statistically significant; p < .0001) were associated with the following screening groups: LSF Fall, LSF January Level, and ORF January Level. Moderate ES (all p values < .0005) were found for LSF Growth and LSF January DD with small ES for LSF January Slope ( p > .01 BRS and ORF Level; p > .05 PC and ORF Slope). The groups that used level to define the cut points produced larger AR/NAR differences than did the groups that had some aspect of growth in the definition. Although it is possible that growth does not make for a good screen, it is just as likely that LSF is not a good measure for first grade progress monitoring. That is, it worked well as an initial measure to define research groups but by late Fall/early January word measures may be more appropriate.

---

Table 1. Effect Sizes for Reading Variables Comparing At-Risk and Not-At-Risk Children Defined by Screening Criteria

Screen Group	% At Risk	End of Year Reading Variable	ES/d
LSF FALL	17.4	BRS SS PC SS ORF LEVEL ORF SLOPE	1.25 1.40 1.05 0.89
LSF GROWTH	16.96	BRS SS PC SS ORF LEVEL ORF SLOPE	0.54 0.45 0.37 0.50
LSF JAN LEVEL	19.3	BRS SS PC SS ORF LEVEL ORF SLOPE	1.30 1.21 1.04 0.86
LSF JAN SLOPE	14.33	BRS SS PC SS ORF LEVEL ORF SLOPE	0.30 0.20 0.29 0.05
LSF JAN DD	8.04	BRS SS PC SS ORF LEVEL ORF SLOPE	0.83 0.73 0.73 0.52
ORF JAN LEVEL*	23.83	BRS SS PC SS ORF LEVEL ORF SLOPE	1.52 1.81 1.52 1.35

---

Classification Analyses

Table 2 presents the percentage of children identified as reading disabled (RD) for each criterion group and the classification indexes. As expected, the BRS SS < 78 group produced the fewest RD children (6.9%) and the ORF < 40 group produced the most (55.9%). Overall, the specificity indexes were quite high indicating that children who were not RD in May were identified as such by the screen. For the most part, the negative predictive power (1 - false negative rate) was also high. Children who were NAR on the screen were likely not RD at the end of the year. The exception to this is the RD group defined by ORF < 40 which is predictable since over half the sample was identified as RD.

---

Table 2. Diagnostic Utility Indexes for Screening and Diagnostic Groups and Percentage of Reading Disabled Children in Each Diagnostic Group

Reading Disability Criteria
Screening Group	BRS SS 90 (18.3%)				BRS SS 78 (6.9%)				ORF 40 (55.9%)				ORF DD (11.6%)
	Sens	PPP	Spec	NPP	Sens	PPP	Spec	NPP	Sens	PPP	Spec	NPP	Sens	PPP	Spec	NPP
LSF FALL	48.8	51.3	89.6	88.7	40.3	16	84.3	95	27.8	89.1	95.7	51.2	55.7	37	87.6	93.8
LSF GROWTH	32	34.5	86.4	85	32	12.9	84.1	94.4	20.4	67.2	87.4	46.5	33.6	49.4	87.3	93
LSF JAN LEVEL	51.2	48.5	87.8	89	42.6	15.2	82.4	95.1	30.1	87.1	94.4	51.6	69.6	41.7	87.3	95.7
LSF JAN SLOPE	19.2	24.5	86.7	82.8	23.4	11.2	86.3	93.9	17	66.3	89.1	45.9	27.8	22.5	87.4	90.3
LSF JAN DD	15.2	34.6	93.6	83.2	14.9	12.7	92.5	93.6	12.0	83.6	97	46.6	27.9	40	94.6	90.9
ORF JAN LEVEL	63.2	48.5	85	91	57.5	16.6	78.7	96.2	41.6	97.6	98.7	57.2	94.9	46	85.5	99.2

---

Sensitivity, the percentage of RD children picked up by the screen, and positive predictive power, percentage of AR children who were classified as RD, were more variable. For all four reading disability criteria, ORF January Level produced the highest sensitivity index and in two of four cases it produced the highest positive predictive power. In most cases, however, the level of accuracy would be considered unacceptable. The ORF January screen and the ORF DD reading disability criterion produced the most accurate classification: sensitivity = 94.9%, positive predictive power = 46% specificity = 85.8%, and negative predictive power = 99.2%.

Previous Page | Next Page
(Introduction) | (Discussion)