Dashboard/ahas

Here’s where we’ll talk about how we figured things out, over time. Our main goal is to share our “ahas” -- the moments when we figured something important out.

We’ll share our COMMUNICATION AHAS. In the process of doing the work, what did the working group realize about improving communications in education?

IMPLEMENTATION AHAS. In the process of doing the work, what did the working group realize about implementing these innovations?

TURNING POINTS. Moments when we redirected the project accordingly, after a communication aha or an implementation aha.

We’ll share visual examples and use photos or videos of people whenever we can!

In diverse districts across the country, educators are often unable to share comprehensive student data, due to the high cost of cutting edge student data systems. Families, for their part, are often unsure how to find all the relevant data on their children, and how to communicate with schools about it. Over the past two years in our “dashboard” project, we – local technologists, teachers, researchers -- have been working with families, afterschool providers, principals, and central administration in the Somerville School District to design ways that everyone could go to a single place – on the web – to find comprehensive data for each student, class of students, and the entire school. Our ideal has been to design tools that not only display data, but also launch a useful conversation between necessary partners about how to support each student.

We’ve asked for feedback on the tool throughout, showing it to administrators, families, and afterschool providers, including doing focused interviews with parents and students from Josh’s own class. Mica, Seth, Josh, and Jedd had many meetings in the spring of 2011 about the teacher and individual views, and we met with Principal DeFalco several times during the 2010-2011 year and Principal Vadhera after that for feedback on the admin view.

COMMUNICATION AHA: Viewing data piecemeal is not enough

DeFalco explained that often, he was in meetings where people had to pull folders out to compare notes on young people, such that updates viewable in a common place would move the conversation and decision-making much farther and much faster. Even within X2, different data sets are not automatically linked. For example, DeFalco would have to email a request to the data office in order to link, e.g., a table of attendance by student name with a table of MCAS score by student name. Since X2 has no default data display setting, DeFalco had to click through numerous choices within X2 before seeing any data at all.

X2 also does not show trends over time. For example, X2 doesn't show test score growth, one of the key measures people are thinking about in education. As Josh pointed out, test scores are kept in chronological order and since students take many tests, it is hard for anyone looking at X2 to see growth on a single test from year to year.

Comments on particular skills listed on the report card only can be chosen from a drop-down list. Teachers can add longer summary comments on student progress only once per quarter, in the days when report cards “open” for updating and before they “close.” X2 also cuts off teacher comments at a certain length.

IMPLEMENTATION AHA: When designing tools for communication within a system, no single person has enough info OR PERSPECTIVE to do it alone. Ongoing collaboration is necessary.

Greg, Josh, and principals Jason and Purnima all suggested data columns; Josh grounded us in reality during through numerous conversations about who needs to communicate what to whom; Seth built the tool; Jedd and Mica managed the communication among the OneVille staff, Healey staff, and Healey families, tracked the details and deadlines, and created documentation; and everyone involved brainstormed different uses and design possibilities.

Throughout, we tried to address our use cases with the most technologically simple design solutions - in terms of programming elegance and ease of use for educators and families. We’ll be piloting these solutions and refining our tools this coming fall!

IMPLEMENTATION AHA: It takes time and multiple perspectives to develop and build the communication channels that connect different supporters of young people.

The admin and teacher views appear in the form of a colorful chart that allows sorting by up to four columns at a time. Based largely on feedback from former Healey Principal Jason DeFalco, we have added to the original, below, produced by Somerville parent Greg Nadeau:

[Image]

We’ve added MEPA scores, score growth on the MAP, ELL status, IEP status, years at Healey, and afterschool program name:

[image]

Based on conversations with new Healey Principal Purnima Vadhera, we’ll also add average attendance over the past several weeks, to compare to the current week’s attendance, and 504 status. We may still add MCAS score and growth, MAP writing score and growth, and DIBELS and MELA-O scores. The updated admin view also creates scatter plots and bar graphs to display the relation demographics and other data, i.e., achievement or attendance:

[image]

DeFalco also suggested some data that we have not been able to include in the current version of the dashboard because it is not currently collected and would require too large of a change in staff behavior, i.e., agreeing on the different types of disciplinary infractions, creating a numerical code for each, and recording the appropriate code each time a student is sent to the office. We are comfortable creating an easier way to view data, but not working to implement a rule that staff must enter new kinds of data into the system.

IMPLEMENTATION AHA: The details of caring that actually “care for:” Painstaking attention to the details - spreadsheets, excel, cutting and pasting - necessary for educators to effectively care for their students.

The flipside of “a gap in data = a gap in student service” is that serving student requires us to track many kinds of data from many sources, rely on the data input from all relevant educators, and strategize about which data sets we can incorporate into a dashboard view without too much work for the staff doing data entry. For example, our dashboard conversations with principals DeFalco and Vadhera showed us that all our data did not come from the same place. Data in X2 required our programmer, Seth, to build “tubes” from the district’s X2 database to our our dashboards. Plenty of data was not simply waiting in X2, ready to be exported. Test score growth and years at Healey had to be calculated based on other data in X2, and crucial data fields were not kept in X2 at all, i.e., MEPA scores, ELL and IEP status, home language, and afterschool program. Incorporating this data into our dashboards impressed upon us how much work, by many different people at the school, was necessary to track and assemble comprehensive data on any student.

COMMUNICATION AHA: Our approach is to create tools that enable, rather than require behavioral change.

In contrast to the spreadsheet-like admin and teacher views, which display the same types of data for many students, the individual view is organized like a slideshow: Clicking on different tabs allows the viewer to see and comment on different parts of each student’s profile. The narrative structure, as well as many decisions about exactly what to display in this view, came out of numerous brainstorming meetings last spring with author and Healey teacher Josh Wairi. We’ll pilot the individual and teacher views in his class in the fall. The individual view presents data such as attendance, grades, MCAS and MAP test scores and growth, and teacher comments - each type of data on its own page accessible by tabs at the top. (Most of this data is in Somerville’s “student information system,” just more scattered; we wanted to get it easily all in one place for a teacher and family/providers to see.) We’ve also made Somerville’s K-6 report card data “live”[I thought “live” meant that viewers can change it. Seth?]: Parents usually get it on a piece of paper. We plan to add each student’s yearbook photo and data on allotted support services. Next to each 'chunk' of student data, “comment/question” boxes provide a space for the parent or afterschool provider to comment on the data by entering text that gets sent to the homeroom teacher’s email:

[image]

On the “Comments” page, the parent or afterschool provider can request the teacher reply to their comments or make an appointment with them. They can specify any new contact info and convenient meeting times. After receiving these comments, the homeroom teacher can forward any relevant parts to the appropriate subject area teachers. (Josh feels that teachers would like to take the lead in responding to and informing other teachers about families’ comments.)

The Parent Connectors will help to make the user interface available in English, Spanish, Portuguese, and Haitian Creole (and we'll use the district's own translation for the report card). For more ongoing translation (email messages sent to and from parents, parent-teacher conferences, the summary comments on the dashboard), we’re adding links to Google Translate and to the Parent Connector calendar for setting up meetings with interpreters.

Knowing all the work that families and schools face on a daily basis, we’ve designed these tools to spark specific kinds of interaction around particular chunks of student data. How people use the tools will be up to them – but rather than have the tools just “display” data, we wanted the individual view, in particular, to also prompt and encourage communication about data.

Feedback: Use Value

COMMUNICATION AHA: Many parents welcome an invitation into a conversation with their child’s teachers, and these parents see technology as an opportunity for connection, rather that an obstacle.

In recent interviews, several immigrant parents emphasized the way the individual view dashboard sparks parent involvement: Smiling, one said, "Parents are not just left out of the school. With this, you are bringing them in, sucking them into the school curriculum!" When asked whether the dashboard might feel like extra work, another parent articulated his/her vision of parent involvement: “Not extra – you have children, you spend time to communicate. The more time you spend, the better students do.” One English-speaking parent with three children at the school explained that the dashboard’s comment and scheduling features solved a long-standing problem for her: After being a Healey parent for 11 years, she has only ever had time to meet with each of her children’s core academic teachers during PTA nights, but never the specialty teachers, e.g., music, art, support room teachers. Our dashboard enables and encourages parents like her to submit their questions, requests for meetings, and updated contact info to the student’s homeroom teacher, who will forward it to the specials teachers. Another parent was especially enthusiastic about online access: “I do everything on the computer now.” And another immigrant parent said he does “everything” on his smart phone!

COMMUNICATION AHA: One-stop shopping: It’s crucial to have a display that shows all the relevant data together

In a recent meeting with OneVille staff, Principal Vadhera described the value of the integrated dashboard tools, in contrast to the old system of requesting info from many different people: “Right now, in just five minutes, I have seen a complete picture of the kid. Without even checking in with folks [other staff]. Normally, I would have to wait for them to get back to me, and bring charts and graphs to meetings. What a great way to launch conversation.”

Online access to this data also helps close an even more basic communication gap, as Vadhera noted: “Even having this [individual view] up there [online] for parents to go back to,” helps when “the report card didn’t get in the backpack, or whatever.” She can see it being a powerful tool in conversations with parents who come in to see her.

Purnima explained that on the admin view, she is interested in “anything that shows a gap,” such as “What were their math scores in 5th grade, what level are they at now? Mapping a class of 2010 – here’s where their scores were in 2008, 2009, 2010.” Sorting attendance data by grade, for example, would save hours for the secretary has print the daily reports and then spend “a week” looking or patterns among the different pages.

Students with IEPs and 504 plans sometimes need accommodations on the MCAS, and Purnima often spends “hours” going over the paper lists and checking with the teachers that everyone’s needs have been met. Our tool will allow her to sort by IEP and 504 status, so that all these students appear together, and, as she said, “so we don’t have moments when things fall through the cracks.”

MAIN COMMUNICATION REALIZATION: In addition to having the ability to quickly see and sort such basic data, diverse partners in young people’s lives need supports to communicate ABOUT basic data.

Purnima and Josh both suggested that the dashboards may enhance teamwork among educators at Healey: In staff team meetings, access to each view could allow teachers and administrators to collaboratively assess a student’s needs, design targeted interventions, and, if desired, record their plan by submitting it as subject-specific comments that get archived in the homeroom (lead) teacher’s email. Such team conversations could involve the school’s “student support team,” a standing group of educators that Purnima described as “the central nervous system of the building,” including Purnima, the Vice Principal, nurse, adjustment counselor, redirect person (discipline). Another relevant team is made up of each student’s individualized group of supporters, e.g. their homeroom teacher, Special Ed or ELL specialists, and reading/math resource room staff. Josh explained that another advantage to the individual view is that it allows him to present a single student’s data in one of these team meetings without revealing all the other students’ grades unnecessarily (a breach of confidentiality).

IMPLEMENTATION AHA: When institutional change is concerned, especially with experimental technology, an incremental and iterative approach can be most appealing for everyone involved.

We hope that these dashboards are useful enough to ensure that students’ needs are met, spark collaboration among educators, and catch on. As Principal Vadhera explained about our dashboards, “A lot of ideas start like THIS (gestures big with hands). And then they fail. This is a guinea pig, Josh can always share back, move forward in small increments. Teachers would just want to get on board with this!”

Come the fall, we can’t revise the dashboard each time someone suggests a change. So: IMPLEMENTATION AHA: We will keep a record of their suggestions/desires, prioritize them, and implement those that are most pressing.