Expanded story: Data dashboards

Written by Mica Pollock, Jedd Cohen, Josh Wairi, and Seth Woodworth for the dashboard project.

Click here for the Summary on this project; click here for the Overview and key findings on this project.

The Details of the Work

The expanded story behind our efforts, our communication and implementation ¡Ahas!, and our turning points!

Click here for the Summary on this project; click here for the Overview and key findings on this project.

Starting off, and developing our goals:

As we began the OneVille Project back in summer 2009, we proposed to create a “dashboard” displaying basic data linking many youth- and child-related databases in the city. That was because many policymakers and researchers have a sense these days that the more data seen by more people, the better (see our Research Base page). But we quickly became unsure that “seeing everything” on young people was necessarily good – and especially, not clearly necessary at the level of the individual, family, teacher, and school administrator (does a teacher actually need to know a student’s police record in order to serve him better? Who exactly should see health data on children?). And anyway, the district was first interested in getting all of its own basic data viewable, quickly -- and that’s what we ended up working on in the end.

We also moved away from multi “sector”-database linking because SomerPromise, the Mayor's new site-based initiative to provide comprehensive youth services, was also interested in tackling the issue of linking databases across agencies, and we felt they were better positioned to pursue that goal even as we worked to lay groundwork for the effort by creating administrative and family-level views of school data alone. So, the goal became to create a simple data display that could help educators and families of youth in a single school see some basic data from district’s data warehousing software, Aspen X2, in a single view. This included creating a translated display easily understandable by an immigrant parent.

Throughout, we also became convinced that a dashboard shouldn’t just show data – it should help people communicate about it. Our rationale: Data displays in schools have traditionally been a) on paper and b) one-way. Think a report card or a quarterly report on one’s “scores”: schools or districts just “display” student scores to students and parents or show parents their child’s absences. Since OneVille’s goal is to support diverse partners in running communication about pursuing the success of young people, we wanted to make sure that parents could communicate back ABOUT data, to teachers -- and that tutors, teachers, and parents could over time communicate with one another.

¡Aha! In addition to having the ability to quickly see and sort such basic data, diverse partners in young people’s lives need supports to communicate ABOUT basic data.

In talking to families, teachers, and other service providers, we realized that just "getting data" on a student is never enough: people need to then converse (online, in person, or otherwise) about how the young person is doing and how they might be assisted. Just knowing how many days a child is absent is the first step, but then you need to have a conversation about why and what to do about it!

The community’s need for the work

Our work was shaped by a number of insights about necessary communications at the levels of student and family, teacher and classroom, school, and community.

¡Aha! One-Stop Shopping: It is crucial to be able to see different kinds of student data at the same time, in a single display.

Our dashboard sits “above” X2, the district’s current database, and displays its data in a more easily readable format for more people.

Parents, teachers, and afterschool providers all had a range of concerns about the accessibility of the data as stored in the district’s current database, Aspen X2:

At the Healey School, older students and parents can log into X2, the student information system, to see updates on their grades, absences, etc. But many don't know that they do have passwords, and students told us they often forgot them. Any dashboard needs a password too, though, so it was really other issues that made a dashboard seem necessary: more importantly, once some of these users would get to X2, they found the format of the data there hard to understand. In particular, information isn't translated for non-English speakers.

When a teacher logged into X2, for another example, it was hard to see any student’s growth over time (i.e., test score growth) As Josh pointed out, test scores are kept in pure chronological order and since students take many tests, it was hard for anyone looking at X2 to see growth on a single test from year to year. Further, the “fields,” or “boxes,” keeping data in X2 didn’t have calculations like test score growth (they just showed one score, then the next). (Our technologist, Seth Woodworth, spent lots of time building “tubes” to make the dashboard automatically calculate this growth.) While the new report card importantly had sections for comments by teachers, those comments -- on particular skills listed on the report card -- only could be chosen from a drop-down list. Teachers could add longer summary comments on student progress only once per quarter, in the days when report cards “open” for updating and before they “close.” X2 also cuts off teacher comments at a certain length.

From an administrator’s perspective, different data sets were not automatically linked. For example, a request to the central office was required to link a table of attendance by student name with a table of MCAS score by student name. Principals also had to click through numerous choices within X2 before seeing any data at all. While queries to a busy central office could get a data report from staff, getting new data on demand -- during a staff meeting, for immediate discussion -- had not been feasible. 2010-11 principal Jason DeFalco explained that often, he was in meetings where people had to pull folders out to compare different spreadsheets on the same young people.

Finally, while some afterschool providers have direct access to X2, many don’t, meaning they had trouble knowing basic information like whether students who were coming to afterschool were going to school. For that reason, some important data fields were not kept in X2 at all yet, like the afterschool program in which children were enrolled or their attendance in that program. These providers typically kept their information in separate computer databases or even on paper. Again and again, people voiced the need to see all the data in one place.

And perhaps most interesting, the existing X2 system was set up only to house student data -- not to help people talk about it. In talking about our dashboard prototypes with parents, the major feature everyone emphasized was the ability to immediately comment ON data, rather than simply “look at it.”

Our dashboard conversations with DeFalco and Vadhera showed us that all our data did not come from the same place. Test score growth and years at Healey had to be calculated based on other data in X2, and crucial data fields were not kept in X2 at all, i.e., MEPA scores, ELL and IEP status, home language, and afterschool program. Because data in X2 was also not “linked,” or organized in the same way we wanted for our dashboard, our technologist, Seth Woodworth, spend lots of time building “tubes” from the different parts of the district’s X2 database to our dashboards. Incorporating this data into our dashboards impressed upon us how much work, by many different people at the school, was necessary to track comprehensive data on any student.

We ultimately had to leave out some data we considered using because it was not yet kept in X2 (e.g., afterschool attendance, and specific in-school tutoring services students received). If our dashboards prove useful and become adopted by the school, staff can reconsider whether they want to enter additional fields into X2.