Project description

Palm Beach County public school leaders touted a survey to claim that their teachers and staff were increasingly optimistic about their schools’ direction.
But it turned out that it wasn’t true. An analysis by The Palm Beach Post’s Mahima Singh and Andrew Marra proved it.

In analyzing the survey for a digital presentation on the paper’s website, the reporters discovered major errors in the district’s results.

Stunned by the obvious discrepancies, Singh and Marra ran the numbers again. They wrote separate code in Python to check their data. They re-did the analysis in Excel. They calculated the data by hand.
Then they confronted the school district.
At first, the district ducked.
District officials canceled two interviews. They conducted their own review and confirmed The Post’s findings, identifying two main problems with their own calculations.

District administrators acknowledged their mistakes, disclosing to Singh and Marra that their research staff made a significant, previously undisclosed change to how they calculated teacher-satisfaction rate that made the latest data look rosier that previous years. At the same time, administrators said, a computer program built to compile survey results tallied the results incorrectly, falsely inflating the teacher satisfaction rates even further.

In the end, the district conceded that the reported spike in school morale was largely an illusion and issued an apology to the school district’s teachers and other employees via email and a recorded video.

What makes this project innovative?

This was the first time the results of school effectiveness survey was visualized and presented in a user-friendly way. The school district does put up the results on their website but that is hard to get to and not very user-friendly. The Post wanted to make it more accessible to the people who had the most stakes in the results, the parents, and the teachers. The team set out to help our target audience understand their school better, make informed choices about where parents send their children to and ultimately showcase all the schools in the county.
This is also the first time the Palm Beach Post has undertaken a project like this.
The Post’s Mahima Singh and Andrew Marra took what seemed to be a happy story about school district morale and found the truth, forced the school district to own up to its mistakes and correct false impressions and gave readers an in-depth and close look at how teachers really feel about their schools.

What was the impact of your project? How did you measure it?

While we were analyzing the data and creating our own percentiles for the presentation, we discovered discrepancies in the survey results. Our averages didn’t match up to what the district had on their websites. In order to be sure of our findings we ran the numbers again. We wrote separate code in python to check our data, we even re-did the analysis in Excel and to add another layer of certainty we calculated the data by hand as well.
The results remained the same. The data that the district had on their website was inflated. This data was also used in presentations across school boards to paint a rosy picture of happy teachers.
In the beginning, when the Palm Beach Post reached out to the administrators and their data team, they canceled two interviews scheduled to discuss the discrepancies while they conducted their own review and discovered two main problems with their calculations.
They discovered that district researchers made a significant change to how they calculated teacher-satisfaction rate and at the same time, administrators said a computer program built to compile survey results tallied the results incorrectly, falsely inflating the teacher satisfaction rates.
In the end, the district conceded that the reported spike in school morale was largely an illusion and also issued an apology to the school district’s teachers and other employees via email and a recorded video.


Source and methodology

The information for this project was taken from the Palm Beach County School District’s 2017 school effectiveness questionnaire. For the survey, teachers and staff at each school were asked to respond to more than 60 statements. For most of the questions, educators could respond with “strongly agree,” “moderately agree,” “slightly agree,” “slightly disagree,” “moderately disagree,” “strongly disagree.” Agreement with a statement indicated a positive view. To measure satisfaction rates at each school, the school district measured the percentage of questions that were answered “positively.” The school district organized the prompts into five categories: “instructional leadership,” “high expectations,” “school climate,” “decision-making” and “student conduct.” Based on the responses, separate scores were assigned for each category at each school. Administrators also compiled the percentage of positive responses to create an “overall percent positive” which we calculated as the “Overall Satisfaction” rate.
Based on the “Overall Satisfaction” rates, The Post created a relative ranking of the schools and divided them into four quartiles: “very happy,” “pretty happy,” “less happy,” “among the least happy.”
The landing page of the web app lists all the schools sorted by satisfaction rating. The user can click on the sort icon to change how the schools are listed. They can even type in a schools name to directly see it in the listing. Once the user clicks on a particular school they are taken to that school's page. Here they can see how the school compared to its previous year’s results. We also present the user with the detailed results of the survey along with the specific questions.

Technologies Used

For the project, we used Python for the data analysis. We used HTML, CSS, and JS to build the web app and Flask to bake the pages. We used the templating language, Jinja to make the individual school pages. All the data was stored in a Python dictionary and was used in to populate the individual pages. We built the pages inside a bootstrap framework to make them more responsive on all devices.

Project members

Mahima Singh, Data Reporter, The Palm Beach Post. Andrew Marra, Education Reporter, The Plam Beach Post, Mike Stucka, Data Reporter, The Plam Beach Post.

Link

Additional links

Project owner administration

Contributor username

Followers

Click Follow to keep up with the evolution of this project:
you will receive a notification anytime the project leader updates the project page.