All ISC-DC schools had the opportunity to participate in four years of school climate survey data collection using the U.S. Department of Education School Climate Survey tools (ED-SCLS),[2] and to receive annual detailed data reports based on their survey results. An additional two schools that were not part of the evaluation—and that did not implement SSC—also participated in school climate survey data collection and received a detailed data report.

Child Trends developed and provided schools with school climate data reports designed to contextualize and explain the data (see example data report). The reports did this in several ways:

Child Trends evaluated implementation[3] and outcomes associated with SSC.[4] As part of the implementation evaluation, we conducted semi-structured interviews with school points of contact and technical assistance specialists working with a subset of schools to understand their perspectives on implementation. These interviews, combined with informal feedback from schools, shed light on how schools reacted to, interpreted, and used the school climate data provided in the data reports.

---

Strategies to Help Schools Use School Climate Data

Across four years of collecting and sharing school climate data with participating schools, our research team identified five strategies that researchers, school districts, or others providing data to schools can use to ensure that schools are able to use those data for decision making.

Report data in a timely manner and in a format that supports data interpretation.

Data reports should be timely and go beyond summary statistics to present detailed, disaggregated, and contextualized findings that address the information needs of school decision makers.[5],[6],[7],[8] In the ISC-DC project, many school leaders mentioned that they were originally drawn to the project because of its focus on data. In fact, some schools had been actively looking for ways to become more data-driven. Moreover, of the framework’s eight elements, the data element was the most consistently implemented across schools. Even schools that never succeeded in assembling a core leadership team for the project still participated in survey data collection year after year. We found that delays in receiving the data report during the first year of the project led some school teams to dismiss survey results as outdated. We also learned, during the first year of the project, that school teams valued receiving data in a way that allowed them to readily identify their school’s strengths and challenges and track progress over time, but that they did not value the ability to compare their school climate data with those of other schools in the project. We incorporated this feedback beginning in the second year of the project and received positive feedback from school points of contact about the subsequent analysis included in the data reports.

Provide data-focused consultation or technical assistance, along with training on data interpretation and data-informed decision making.

Given competing priorities and limited time and resources, it is important to help school leaders gain the capacity to understand, compare, and make use of a wide variety of data sources.[9],[10],[11],[12] School leaders in the ISC-DC project often had to juggle multiple demands on their time. This made it difficult for them to dedicate the necessary time to the project, including the time to review data reports and use the survey data for decision making. The task of reviewing the data was sometimes delegated to staff with more time available but with limited data interpretation and analysis capacity and/or knowledge of the school context. Technical assistance specialists reported that school climate data were most readily used for decision making by school leaders who were already comfortable with reading and analyzing data. Moreover, when school teams found time to engage with technical assistants in a guided discussion of the data, they were able to hone their data interpretation skills and more strategically use school climate data for decision making.

Build buy-in for data use by ensuring that data are high-quality and reflect the needs of the school community.

Working with school leaders in advance of data collection can help them understand the potential value of school climate data and build the sort of buy-in that can make a difference in whether schools use the data. When school leaders communicate the value of the data to the school community, their demonstrated support can improve response rates. This is critical because low response rates can lead members of the school community to question the validity of the data. In the ISC-DC project, technical assistance specialists noted that some school teams did not trust survey results when response rates were low or when they had concerns about the representativeness of survey respondents.

Engage multiple stakeholders in the review of data.

Encourage schools to engage diverse perspectives; this is critical for making sense of school climate data.[13] Students, teachers, school-based mental health professionals, and parents are often able to contribute pertinent information that can help contextualize results—for instance, by sharing personal insights or bringing to light other data sources that help explain why a certain score is high or low. This dialogue can lead to a more robust understanding of what the data mean for a school and how to best address the issues identified within the data. It also helps guard against bias—such as the tendency to rationalize or dismiss findings that do not align with one’s expectations—and builds data capacity among stakeholders. In the ISC-DC project, schools varied in the extent to which, and the ways in which, they used the survey data for decision making. At some schools, reviews of the school climate data led to meaningful conversations with various stakeholder groups. These conversations, in turn, elicited additional context for the data and helped engage a broad range of perspectives on best addressing the needs identified in the report.

Emphasize the utility of school climate data for improvement, rather than for accountability.

Recognize that, historically, data have been used in punitive ways and be clear about the purpose of school climate data to promote continuous improvements. Help school decision makers reflect about their data-related hopes, fears, and expectations to minimize bias when reviewing data and increase the likelihood that data are used for school improvement. In an era of high-stakes assessments of student and staff performance, school leaders may be wary of school climate data.[14] In the ISC-DC project, technical assistance specialists observed that many school leaders and teams displayed an initial distrust or discomfort with data, stemming from past experiences in which data were used in ways that were punitive toward schools, school leaders, and educators.

For instance, a number of high schools in Washington, DC have been labeled as failing—and administrators have felt that their jobs were at risk—when high percentages of students did not meet grade-level performance indicators on standardized assessments such as the Partnership for Assessment of Readiness for College and Careers (PARCC) test. Stressing the value of school climate data for improvement, rather than for accountability, can help attenuate negative past experiences with data and increase data use, as can investing in the lessons outlined above—presenting data in easy-to-understand formats, building the capacity of school leaders to interpret and use school climate data, building buy-in, and engaging multiple stakeholders.

---

This project was supported by Award No. 2015-CK-BX-0016, awarded by the National Institute of Justice, Office of Justice Programs, U.S. Department of Justice. The opinions, findings, and conclusions or recommendations expressed in this report are those of the authors and do not necessarily reflect those of the Department of Justice.

Photo courtesy of https://deeperlearning4all.org

---

References

[1] Jordan, P. W. & Hamilton, L. S. (2019). Walking a fine line: School climate surveys in state ESSA plans (Report). FutureEd. Retrieved from https://www.future-ed.org/wp-content/uploads/2019/12/FutureEdSchoolClimateReport.pdf

[2] Ryberg, R., Her, S., Temkin, D., Madill, R., Kelley, C., Thompson, J., & Gabriel, A. (2020). Measuring school climate: Validating the Education Department School Climate Survey in a sample of urban middle and high school students. AERA Open, 6(3). http://doi.org/10.1177/2332858420948024

[3] Solomon, B. J., Stratford, B., Steed, H., Sun, S., & Temkin, D. (2020). Implementation of a capacity building framework to improve school climate in an urban school system. [Manuscript submitted for publication].

[4] Ryberg, R., Her, S., Temkin, D., & Rodriguez, Y. (2020). Associations between a school organizational capacity building model and improvements in student perceptions of school climate. [Manuscript submitted for publication].

[5] Lachat, M. A. & Smith, S. (2005). Practices that support data use in urban high schools. Journal of Education for Students Placed at Risk, 10(3), 333-349. http://doi.org/10.1207/s15327671espr1003_7

[6] Marsh, J. A., Pane, J. F., & Hamilton, L. S. (2006). Making sense of data-driven decision making in education: Evidence from recent RAND research (OP-170). RAND Corporation. Retrieved from https://www.rand.org/content/dam/rand/pubs/
occasional_papers/2006/RAND_OP170.pdf

[7] Means, B., Padilla, C., DeBarger, A., & Bakia, M. (2009). Implementing data-informed decision making in schools—teacher access, supports and use. U.S. Department of Education.  Retrieved from https://files.eric.ed.gov/fulltext/ED504191.pdf

[8] Rankin, J. G. (2016). Data systems and reports as active participants in data interpretation. Universal Journal of Educational Research, 4(11), 2493-2501. http://doi.org/10.13189/ujer.2016.041101

[9] Knapp, M. S., Swinnerton, J. A., Copland, M. A., & Monpas-Huber, J. (2006). Data-informed leadership in education. University of Washington, Center for the Study of Teaching and Policy. Retrieved from https://www.education.uw.edu/ctp/sites/default/files/
ctpmail/PDFs/DataInformed-Nov1.pdf

[10] Lachat, M. A. & Smith, S. (2005).

[11] Marsh, J. A., Pane, J. F., & Hamilton, L. S. (2006).

[12] Means, B., Padilla, C., DeBarger, A., & Bakia, M. (2009).

[13] Schildkamp, K. (2019). Data-based decision-making for school improvement: Research insights and gaps. Educational Research, 61(3), 257-273. https://doi.org/10.1080/00131881.2019.1625716

[14] Lachat, M. A. & Smith, S. (2005).