Expanded story: Data dashboards

Written by Mica Pollock, Jedd Cohen, Josh Wairi, and Seth Woodworth for the dashboard project.

Click here for the Summary on this project; click here for the Overview and key findings on this project.

The Details of the Work

The expanded story behind our efforts, our communication and implementation ¡Ahas!, and our turning points!

Click here for the Summary on this project; click here for the Overview and key findings on this project.

Starting off, and developing our goals:

As we began the OneVille Project back in summer 2009, we proposed to create a “dashboard” displaying basic data linking many youth- and child-related databases in the city. That was because many policymakers and researchers have a sense these days that the more data seen by more people, the better (see our Research Base page). But we quickly became unsure that “seeing everything” on young people was necessarily good – and especially, not clearly necessary at the level of the individual, family, teacher, and school administrator (does a teacher actually need to know a student’s police record in order to serve him better? Who exactly should see health data on children?). And anyway, the district was first interested in getting all of its own basic data viewable, quickly -- and that’s what we ended up working on in the end.

We also moved away from multi “sector”-database linking because SomerPromise, the Mayor's new site-based initiative to provide comprehensive youth services, was also interested in tackling the issue of linking databases across agencies, and we felt they were better positioned to pursue that goal even as we worked to lay groundwork for the effort by creating administrative and family-level views of school data alone. So, the goal became to create a simple data display that could help educators and families of youth in a single school see some basic data from district’s data warehousing software, Aspen X2, in a single view. This included creating a translated display easily understandable by an immigrant parent.

Throughout, we also became convinced that a dashboard shouldn’t just show data – it should help people communicate about it. Our rationale: Data displays in schools have traditionally been a) on paper and b) one-way. Think a report card or a quarterly report on one’s “scores”: schools or districts just “display” student scores to students and parents or show parents their child’s absences. Since OneVille’s goal is to support diverse partners in running communication about pursuing the success of young people, we wanted to make sure that parents could communicate back ABOUT data, to teachers -- and that tutors, teachers, and parents could over time communicate with one another.

¡Aha! In addition to having the ability to quickly see and sort such basic data, diverse partners in young people’s lives need supports to communicate ABOUT basic data.

In talking to families, teachers, and other service providers, we realized that just "getting data" on a student is never enough: people need to then converse (online, in person, or otherwise) about how the young person is doing and how they might be assisted. Just knowing how many days a child is absent is the first step, but then you need to have a conversation about why and what to do about it!

The community’s need for the work

Our work was shaped by a number of insights about necessary communications at the levels of student and family, teacher and classroom, school, and community.

¡Aha! One-Stop Shopping: It is crucial to be able to see different kinds of student data at the same time, in a single display.

Our dashboard sits “above” X2, the district’s current database, and displays its data in a more easily readable format for more people.

Parents, teachers, and afterschool providers all had a range of concerns about the accessibility of the data as stored in the district’s current database, Aspen X2:

At the Healey School, older students and parents can log into X2, the student information system, to see updates on their grades, absences, etc. But many don't know that they do have passwords, and students told us they often forgot them. Any dashboard needs a password too, though, so it was really other issues that made a dashboard seem necessary: more importantly, once some of these users would get to X2, they found the format of the data there hard to understand. In particular, information isn't translated for non-English speakers.

When a teacher logged into X2, for another example, it was hard to see any student’s growth over time (i.e., test score growth) As Josh pointed out, test scores are kept in pure chronological order and since students take many tests, it was hard for anyone looking at X2 to see growth on a single test from year to year. Further, the “fields,” or “boxes,” keeping data in X2 didn’t have calculations like test score growth (they just showed one score, then the next). (Our technologist, Seth Woodworth, spent lots of time building “tubes” to make the dashboard automatically calculate this growth.) While the new report card importantly had sections for comments by teachers, those comments -- on particular skills listed on the report card -- only could be chosen from a drop-down list. Teachers could add longer summary comments on student progress only once per quarter, in the days when report cards “open” for updating and before they “close.” X2 also cuts off teacher comments at a certain length.

From an administrator’s perspective, different data sets were not automatically linked. For example, a request to the central office was required to link a table of attendance by student name with a table of MCAS score by student name. Principals also had to click through numerous choices within X2 before seeing any data at all. While queries to a busy central office could get a data report from staff, getting new data on demand -- during a staff meeting, for immediate discussion -- had not been feasible. 2010-11 principal Jason DeFalco explained that often, he was in meetings where people had to pull folders out to compare different spreadsheets on the same young people.

Finally, while some afterschool providers have direct access to X2, many don’t, meaning they had trouble knowing basic information like whether students who were coming to afterschool were going to school. For that reason, some important data fields were not kept in X2 at all yet, like the afterschool program in which children were enrolled or their attendance in that program. These providers typically kept their information in separate computer databases or even on paper. Again and again, people voiced the need to see all the data in one place.

And perhaps most interesting, the existing X2 system was set up only to house student data -- not to help people talk about it. In talking about our dashboard prototypes with parents, the major feature everyone emphasized was the ability to immediately comment ON data, rather than simply “look at it.”

Our dashboard conversations with DeFalco and Vadhera showed us that all our data did not come from the same place. Test score growth and years at Healey had to be calculated based on other data in X2, and crucial data fields were not kept in X2 at all, i.e., MEPA scores, ELL and IEP status, home language, and afterschool program. Because data in X2 was also not “linked,” or organized in the same way we wanted for our dashboard, our technologist, Seth Woodworth, spend lots of time building “tubes” from the different parts of the district’s X2 database to our dashboards. Incorporating this data into our dashboards impressed upon us how much work, by many different people at the school, was necessary to track comprehensive data on any student.

We ultimately had to leave out some data we considered using because it was not yet kept in X2 (e.g., afterschool attendance, and specific in-school tutoring services students received). If our dashboards prove useful and become adopted by the school, staff can reconsider whether they want to enter additional fields into X2.

Development Process

From the very beginning, in the 2009-2010 school year, we collaborated with a range of Healey community members to design the dashboard’s user interface. As it turned out, Greg Nadeau, Somerville resident and Healey dad, had already made a color-coded Excel spreadsheet for the Healey Principal the year before we began work on the dashboard. It was an early Excel version of what later became our Administrator Dashboard View (what we came to call the Admin View). Here’s the earliest version of our Admin View:

Before we made the dashboard version of this view “live” by connecting it to X2, the school’s main need at the time was entering updated data into the Excel spreadsheet -- by hand. We knew that we would eventually develop a dashboard to automatically transfer data from X2, but the school needed to integrate the data sets immediately. So Susan Klimczak, from the South End Technology Center at Tent City, who would later become the lead organizer for the eportfolio project, did some valiant handiwork, assisted by colleague Al Willis. They cleaned up the current year’s spreadsheet and added new data typically not kept in X2 that the principal also wanted to see, like afterschool enrollment. We also had some regular meetings via phone conference with Greg and the principal to consider the patterns the principal wanted to see and the new "fields" for data that the principal felt needed to be created permanently in the district student information system (e.g., attendance in the afterschool programs. Afterschool programs weren't keeping this data in Somerville's core data system, X2, and still don’t.)

Turning point: The Teacher View

Josh Wairi, a 5th grade teacher at the Healey, got interested in the dashboard design when we stopped by his classroom one day after school in the winter of 2011. Looking together at his computer and printouts, we realized he was already creating spreadsheets of student data from X2 by hand. He was interested in quickly displaying and sorting basic data, to supplement his face-to-face and phone conversations with students and parents. His work showed us the need for what would become the “Teacher View,” a version of the Administrator View in which all the students come from the same homeroom.

Between this stage and the final version of the admin and teacher views (below), Mica, Seth, and Jedd met with Principal DeFalco several times during the 2010-2011 year and new Principal Vadhera beginning during the summer of 2011 to figure out the highest priority data fields for the admin view and to brainstorm potential uses. Based on DeFalco’s feedback, and with Seth’s programming skill, we developed a dashboard to automatically transfer data from X2 to a user-friendly view. Seeing Greg’s initial prototype, above, sparked DeFalco to suggest additional data fields: years at Healey, score growth on the MAP, ELL status, MEPA scores (English language learner assessments), IEP status, and afterschool program name (in the end, we did not add program name to the dashboard, because it was never made a real field in X2.) From this feedback, we developed the view below (names are fictional to ensure anonymity):

All data fields are visible on one screen – so there is no need to click through multiple windows to view the desired data. Viewers can sort up to three columns at a time simply by clicking at the top of each while holding down the shift key. With Vadhera, over the summer of 2011, we brainstormed additional uses for the admin staff, which are described below in the second-to-last ¡Ahas!, “Data really can launch a conversation.”

Turning Point: Making the report card online and interactive Stories of parents and afterschool providers trying to communicate with teachers about students’ report cards prompted us to push forward on an “Individual View” that included the report card instead of just absences, grades, and test scores. Mica, Seth, Josh, and Jedd had many meetings in the spring of 2011 to brainstorm the design for this view, considering parent needs for accessibility and common communication challenges between teachers and parents. Josh was a critical resource on these issues.

We had also been inspired by the New Visions for New Schools project, which has had success with a view that gives parents and students a quick understanding of student performance on quantifiable measures. (http://www.hfrp.org/var/hfrp/storage/fckeditor/File/9thGradeTracker.pdf) Compared with New Visions' view, we wanted to include the additional feature of allowing parents - and afterschool providers - to comment on any child's progress, send these comments to the teacher, and begin a conversation about how to support the student at home and in school. New Visions emphasizes a verbal agreement between family and teacher, called an “Improvement Plan,” so we created space on the dashboard page to write down this agreement. In our initial prototype, we did this by adding actual “Improvement Plan” text boxes in the right-hand side of the screen:

As we continued to develop the dashboard and add new types of basic data for viewing, we wanted to keep all the info on a single page for easy access, but new developments convinced us to spread the info out across several pages. At the time we were designing the “Individual View,” the District was ceasing to assign its elementary students numerical grades, or their equivalents on the typical A, B, C scale, and moving to a standards-based K-6 report card that asked teachers to grade students as proficient on a range of grade-level skills. So, we made Somerville's new K-6 report card (handed out on paper to families) online and color-coded. We also decided that we wanted the text boxes to prompt more of an ongoing conversation, rather than a formal quarterly agreement. So, we put these text boxes next to each chunk of data, with a prompt embedded in the “save comment” click button to encourage parents to write comments or questions. With the help of local technologist Evan Burchard, whom Seth recruited, we developed a revised version. The image below includes a sample tutor comment. That’s because in the months to come, we would realize that any approved viewer of the dashboard could comment:

As we added the report card to the dashboard, we realized it was substantial enough that it needed its own page (above). In order to make the individual view as accessible as possible, even to families that are not used to looking at lots of quantitative data in one place, we spread the rest of the individual view’s information across several other pages, the names of which are visible as tabs at the top of each page. The individual view is now organized like a slideshow: Clicking on different tabs allows the viewer to see and comment on different parts of each student’s profile. We combined info about the student’s attendance and MCAS and MAP scores into the Individual View, which also includes the teacher’s quarterly summary comments from the report card: