Test Scores, Achievement Gaps, and District FOIA Policy: Concerning New Information

This article addresses three related areas of concern:  the drop in test scores across the district, the wide achievement gaps across demographic groups, and the district’s policy that keeps more information inaccessible.

Test scores

In early December, the state released the 2020 version of the Illinois Report Card, the website which collects and makes available to the public key data on schools across Illinois, including test scores (for high schools, the SAT) broken down by demographic groups, the demographics themselves, AP test participation and pass rates, school funding, and other information.  This made it possible for the first time to see how the school closures affected student performance in a manner untainted by grade inflation/changes in grading policies.

Here is a summary table for all schools, for all demographic groups.  It is not possible to compare the district to the state because the state has not published statewide averages.  My understanding is that this is not a mere lack of time to process the data but because enough schools received waivers that statewide averages would not be comparable to prior or future years.  Also note that only four years are shown because I had previously compiled 2019 and 2017; in any case, no comparable data is available prior to 2016 because of the change from the ACT.

Note that this is based on the SAT scores, in which a cut-off has been pre-defined as marking grade-level expectations.  One could argue about whether that cut-off is defined appropriate or not, and I suspect that District officials would do so.  One could make the claim that scores are lower than they otherwise would be because of a refusal to “teach to the test,” and I suspect the District does that as well.  But these drops belie any claim that the district’s remote education has been “robust.”

All schools had significant decreases, even the “golden twins” of Prospect and Hersey.  What is surprising is that Wheeling had only a small decrease in Math, and none in ELA (English-language arts) from 2019, but had a dramatic drop from 2017 to 2019, so here is a closer look at Wheeling.  Note that “low income” is a category based on eligibility for free or reduced price lunch, which means poverty-level or a little higher.

It is important to notice that in some of these groups, scores actually improved from 2019 to 2021 — as if the school took a very aggressive anti-SAT tact in 2019, then dialed it back in 2021, but was only successful with some groups.  And certainly, the drop in test scores for low-income and Hispanic kids should be raising serious alarm bells.  In fact, for Math, over half of kids from these two demographic groups fall into the lowest of the four categories; though there is no ready translation into a “grade level equivalent,” this is a serious problem.

Achievement Gaps

“Achievement gaps” refer to differences between more disadvantaged and less-disadvantaged students (Hispanic vs. white, low-income vs. non-low income) in test scores and other measures of performance.  Some of this can be viewed with the test scores above.  For example, here is an analysis of scores in 2019, comparing District 214 schools with statewide results and using a percentile metric (50th percentile is the median; 99th percentile = tops in the state) to show where the schools stand relative to all district schools, comparing students of a given demographic group only.  I have highlighted in red those where the school is below median and yellow those only somewhat above median.

Again, this is 2019 only because of the issues with 2021 data.  And this data cannot be explained away by the disadvantages that immigrant or low-income students face — this is comparing Hispanic students at District 214 schools to Hispanic students at all other Illinois schools, and likewise for low income students, and I am aware of no reason why “our” disadvantaged students are more disadvantaged than average.

Here is another form of gap, the differences in AP enrollment, in this case at Prospect High School:

Again, the district has a high AP participation rate, but for Hispanic students, it’s only a smidge above the state average for the district as a whole, and not much higher for this school in particular.  In addition, what is striking about this school is that here a clear gap between the test-taking rates of male and female students is visible as well, most pronounced at Prospect but present at other schools as well.

FOIA Policy

Earlier in the fall, I had begun to see articles in the mainstream press about the growing gap between men and women in terms of college attendance and success, with the proportion of men and women becoming 40%/60%.  These reports indicated that problems started much sooner, so I wanted to know how things looked at District 214, then expanded my request to include any available data and data analysis regarding achievement gaps between boys and girls or by demographic group.

I was denied — not because they claimed that this information was exempt but because they claimed they had no such information of any kind.  After multiple follow-up questions, I received this statement from Dr. Schuler:

As our team has stated to you previously, our data warehouse system is housed in Tableau, which we call D214 Insight. It is a dynamic, web-based system that is continually updated from our student information system (SIS), which is Infinite Campus. District staff is able to access, explore, research and disaggregate data as needed to review student performance information. Staff and teams regularly meet and review the data but we do not have access to reports or other artifacts that would be responsive to your request. For example, teams may project the data from their computer onto a screen for their Professional Learning Community to review the data or individuals may login to the system on their computer to run their own analysis.

To clarify the matter, I replied with the question,

Would you please tell me whether there is an explicit policy against saving/downloading Tableau data, or deficiencies in the software which prevent this?  Also, are board members given access to Tableau, or, if not, have they ever been provided any information on achievement discrepancies?

To which I received no response.

Separately, I have heard from individuals with more expertise in the matter than I, that this is a deliberate policy some government bodies adopt, to go about their business in a manner that deliberately avoids creating FOIA-able material, and, in fact, Schuler states categorically that there are no reports created using Tableau data, nor are Tableau charts/tables ever saved, not merely that a search across systems turned up nothing.

Finally, I requested an export of student data, without personally identifying information, thinking that it would be possible for me to perform some analysis myself to answer questions such as, “controlling for recency of immigration, is the achievement gap for Hispanic or low-income students reduced or moderated?” even if I would not be able to access any reports or documents indicating what District staff knew and how they were dealing with the issue.  This, too, was denied, with the claim that student data was protected, in any form, that there was no obligation to take the action of stripping out identifiers, and that running a data export excluding personally-identifying fields constituted “stripping out identifiers” and was therefore among the FOIA exclusions.

In other words, District 214 wants to claim the title of “destination district.”  Its board members are happy to claim that they are one of the top districts in the country, or are on their way there.  But they demand that the community simply take it on faith that the district is doing all it reasonably can to educate the disadvantaged students among us.  Based on what I’m seeing, they don’t deserve that level of trust.

 

— Authored by Elizabeth Bauer