How school systems use data to implement instructional strategies for elementary students

¶ … School Systems use Data to Implement Strategies and Drive Instruction for Elementary Students?

In any setting, there is a best way to do something, and elementary school systems are no exception. In the Digital Age, though, trying to make sense of the enormous amounts of information available is often like trying to take a sip from a fire hose and many educators might be at a loss concerning how to use the available data to its best effect. Therefore, identifying ways that other educational school systems have used data to help them implement strategies and drive instruction represents a timely and valuable enterprise.

In an era of dwindling state budgets for education and increasing calls for accountability on the part of teachers and principals as the result of a long string of federal mandates such as the Individuals with Disabilities Education Act and the No Child Left Behind Act, the need for cost-effective methods of assessing the effectiveness of curricular offerings and teacher performance in achieving established goals is paramount. It is axiomatic that in order to improve something it must first be measured, so there is also an ongoing need for innovative ways to use existing secondary data in order to identify opportunities for improvement that might otherwise be overlooked. Therefore, the overarching purpose of this study was to identify current approaches to using data of different types to implement strategies and drive instruction for elementary students.

Problem question

The problem question that guided this study was, "How do school systems use data to implement strategies and drive instruction for elementary students?"

Research question

The research question used to guide this study had two parts: (a) What type of data is available for performance-related analyses as well as to identify opportunities for improvement; and (b) How can this data be used to implement strategies and drive instruction for elementary students?

Brief Review of the Literature

In the Age of Information, there is typically a wide array of data about any elementary school system available, whether in print or online, that can be used to track changes in performance indicators and identify trends, but the specific types of data needed for any particular educational initiative may not be readily available or may require fine-tuning existing computer-based applications. Therefore, it would be useful to establish some common areas in which data is widely used for these purposes from the outset. Certainly, performance-based metrics such as standardized test scores can be used to assess the effectiveness of current instruction and this data are already used for this purpose across the country and globe. Some educational analysts have taken this data and applied it across several realms of elementary school operations to determine the effectiveness of their current performance as well as what could be done to improve educational instruction. For instance, a study by Bryk (2010) (currently president of the Carnegie Foundation for the Advancement of Teaching) examined innovative ways to use data to implement strategies and drive instruction for elementary schools in the Chicago school system. The five key data-driven realms for improving elementary school instruction developed by Bryk are set forth in Table 1 below.

Table 1

Five Key Data-Driven Areas for Improving Elementary School Instruction

Key Area

Description

Coherent instructional guidance system

This key area is characterized by schools that have demonstrated student learning improvements based on coherent instructional guidance systems that specify how instruction will be delivered and what curricular content will be offered. Some of the ways this dimension can be measured include the learning tasks required of students which are an essential analytical element, as well as the results of any standardized testing regimens that are in place that are used to provide feedback to students and inform instructional development. A part of this assessment concerns the quality of the tools, materials, and instructional routines that are shared by faculty that support instruction. While individual teachers enjoy a significant amount of discretion concerning the allocation of resources, the overall effectiveness of individual teacher will relate to the quality of the supports and the local community of practice that guide their use and refinement.

Professional capacity

Elementary school teachers and principals can work together to improve instruction, but for them to become proficient at the process, they must have access to ongoing training opportunities and collaborative exchanges to improve their delivery of educational services. Some of the ways this dimension can be measured include: 1. The professional development that supports their learning;

2. The faculty's capacity to work together to improve instruction;

3. The school's ability to recruit and retain capable staff;

4. The efficacy of performance feedback and professional development; and,

5. The social resources within a staff to work together to solve local problems.

Strong parent-community-school ties

Recruiting and sustaining the active involvement of parents is an essential ingredient in the overall mix of efforts needed to use data-driven methods to improve instruction and measuring this dimension can provide some useful insights into existing gaps in the school-parent-community continuum. In fact, this is probably the most cost-effective approach to improving academic outcomes which of course is one of the main goals of all elementary schools. More active involvement by parents with schoolwork and curricular content may also have some significant tangential benefits that transcend mere grades, though, making this an especially important realm for assessment.

Student-centered learning climate

The type of climate that is cultivated by the responsible adult educators in an elementary school will have a profound effect on the quality of instruction that is provided. This dimension an be measured by the extent to which a climate that:

1. Encourages young people to consider themselves as being learners is facilitated by educators;

2. Creates a positive learning climate conducive to learning such as a safe and orderly environment;

3. Creates a climate that celebrates ambitious academic work ethics together with the provision of support for each young learner.

Leadership drives change.

Although all of the five support indicators are essential for success, a growing body of knowledge suggests that if any one could be singled out as being "most important," it would be this dimension. According to Bryk, "Principals in improving schools engage in a dynamic interplay of instructional and inclusive-facilitative leadership. On the instructional side, school leaders influence local activity around core instructional programs, supplemental academic and social supports, and the hiring and development of staff" (2010, p. 24). Some of the ways that this dimension can be measured would include the extent to which a principal:

1. Established strategic priorities for using resources and buffer externalities that might distract from coherent reform;

2. Developed relationships across the school community because improving teaching and learning places demands on these relationships.

3. School leaders advance instrumental objectives while also trying to enlist teachers in the change effort in their daily activities.

4. Principals cultivate a growing cadre of leaders (teachers, parents, and community members) who can help expand the reach of this work and share overall responsibility for improvement.

Source: Adapted from Bryk, 2010, p. 24

Based on the results generated by extensive primary survey data collected from teachers, principals, and students in the Chicago area, Bryk and his colleagues developed secondary data indicators for each of the five foregoing essential supports, tracked changes in these indicators over time, and then associated these performance-related outcomes to changes in student attendance and learning gains in reading and mathematics. Some of the main findings that emerged from this data analysis included the following:

1. Schools with strong indicators on most supports were 10 times more likely to improve than schools with weak supports.

2. Half of the schools strong on most supports improved substantially in reading.

3. Not a single school weak on most supports improved in mathematics.

4. A material weakness in any one support, sustained over several years, undermined other change efforts, and improvement rarely resulted (Bryk, 2010, p. 25).

In fact, Bryk frequently equates these five supports with pillars, and the absence (or sustained weakness) in one pillar threatens the performance of the entire school on the others. Like an automobile that will not run properly without a carburetor or a set of tires, simply focusing on one pillar will not provide the complete well-oiled machinery of educational success that can be achieved when all five supports are in place. In this regard, based on the analysis of the secondary data indicators for the elementary schools in this study, Bryk emphasizes the need for timely and relevant data to achieve the broad-based types of support needed to help elementary schools achieve their organizational missions. According to Bryk, "This statistical evidence affords a strong warrant that how we organize schools is critical for student achievement. Improving schools entails coherent, orchestrated action across all five essential supports. Put simply, there is no one silver bullet" (2010, p. 24). Although there is no "silver bullet" available, it is clear that the secondary data-driven approach used by Bryk could be applied to elementary schools in any setting to help implement strategies and drive instruction, but they might require additional primary data to formulate their own analysis.

For example, with respect to the leadership support area developed by Bryk, Korkmaz (2006) reports that in some cases, collecting primary data are required to make informed decisions, particularly with respect to human resource decisions. In his study of 842 teachers working in 42 elementary schools, Korkmaz operationalized the dimensions in this area as described in Table 2 below.

Table 2

Operationalization of Elementary School Climate and Leadership Dimensions

Dimension

Definition

Instruments Used

School vision

This dimension was defined as being a top-down process that begins with the elementary school principal and purposely extends to all school stakeholders. The two important elements of effective leadership identified were:

1. Building up positive interpersonal relations; and,

2. Developing a school vision.

To measure the perceptions of teachers working in these schools about the vision in their schools, the study used the Robustness Semantic Differential (RSD), developed by Licata and Willower (1978) (see proforma copy at Appendix a).

Organizational health

This dimension measures the perception of teachers concerning the health of their schools. Some ways to measure this dimension include the extent to which:

1. Teachers working in schools see their school's success in building positive communication with its environment as a strong characteristic of their school.

2. Schools provide a learning conducive atmosphere.

3. Schools are flexible and can respond to changes in their environments to achieve their organizational goals and promote the common values of the educators.

4. Technical, managerial and institutional levels are in harmony wherein students, teachers and principals respond to the school vision.

5. Teachers are committed to the school vision and see that their colleagues are working towards a better future.

First developed by Hoy and Miskel (1991), the Organizational Health Inventory (OHI) was used in the study. A middle school form of the OHI scale was published by Hoy and Tarter (1997) and Hoy and Sabo (1998); the OHI developed by Licata and Harper (2001) was comprised of 33 items across 6 subscales representing approximately 77% of the cumulative variance. Relatively high alpha reliability coefficients have been established for these subscales. In addition, the instrument used by Licata and Harper (2001) for their research Organizational Health and Robust School Vision was also used in the study.

Source: Modified from Korkmaz, 2006

Based on his analysis of the data collected to measure these aspects of the leadership pillar, Korkmaz identified a significant relationship between elementary school teachers, organizational health and a robust school vision. According to Korkmaz, "As a result of multiple regression analysis, it was found that collegial leadership and academic emphasis related to school health and the resource support subscale were related to the robust school vision" (2006, p. 15). In order to place these statistical results in context, Korkmaz also suggested an explanation for this relationship. In this regard, this researcher noted that, "Statistics used to collect data and analyze the findings seem to support the hypothesis of the study that there is an important positive relationship between teachers' perception of organizational health and their perception of robust school vision" (2006, p. 15). This researcher is also quick to qualify these findings in other ways as well while still drawing some important and relevant conclusions. In this regard, Korkmaz concluded that, "It can comfortably be stated then that where technical, managerial and institutional levels are in harmony in a middle school, there is a healthy professional atmosphere. Probably, a school with such an atmosphere meets its needs and directs its potential energy towards the realization of its mission" (2006, p. 15). This same approach has been used by other academic researchers as well, including Licata and Harper (1999) who advise, "Apparently, when schools are healthy and robust, academic emphasis is a predominant organizational theme" (p. 463). In order to make the maximum use of the data collected for these types of studies, identifying others ways to analyze existing data to inform practice just makes good business sense. In this regard, Korkmaz provides the following suggestions for further research:

1. School health could be compared to the managerial style of the school (e.g., School-Based Management) or to determine whether there is a mutual relationship between managerial style and school health;

2. The manner in which managerial style affects the organizational health of the school could be investigated; and,

3. The effect of a robust school health on students' success can be studied.

The need to use this type of data analysis to develop meaningful findings, though, will likely require the addition of other data sets that can help identify potential cause and effect relationships and eliminate false leads. For this purpose, Korkmaz (2006) recommends using the instruments described in Table 2 above or comparable instruments with established reliability and validity to evaluate the entire range of factors that may contribute to school health. In sum, Korkmaz concludes that, "By studying types of relationships between school health and organizational conflict, the results can be used to develop school health. To develop a school vision, teachers' perceptions of principals' effectiveness can be measured and the results could be used to discover the possible positive relationship between school health and school vision" (2006, p. 15). This analysis, though, is not a static affair but rather requires administration every few years in order to ensure that its findings remain relevant and on-point (Korkmaz, 2006).

Other sources of data that can be used to implement strategies and drive instruction can be databases provided by the National Assessment for Educational Progress (NAEP) that provide a wide range of elementary school-related performance data that can help identify general trends in performance gaps. For instance, using the NAEP database, Philipp (2008) examined problem areas in mathematics instruction at the elementary school level across the country. According to Philipp, "The way most students are learning mathematics in the United States is problematic. In particular, students learn to manipulate mathematical symbols without developing the underlying conceptual meanings for the symbols" (2008, p. 7).

Design and Methodology

This type of study requires a qualitative research design supplemented by quantitative data sets such as standardized test results and other performance measures already being used, as well as the results of any primary research conducted to replace, augment or otherwise inform the analytical process. Based on the qualitative research design selected, a purposive sampling technique would be a suitable methodology for collecting the data needed for the assessment of each of the five pillars described by Bryk. According to Neuman (2003), purposive sampling is "a type of nonrandom sample in which the researcher uses a wide range of methods to locate all possible cases of a highly specific and difficult to reach population" (p. 542).

An exploratory data collection strategy would be used to collect the statistical data needed for this type study, with as many relevant data sources being located and integrated into the study as possible. All statistical data would be analyzed using Excel or SPSS Version 11.0 for Windows (Student Version) and the results of this analysis qualitatively synthesized. By focusing on an individual school, the results that are generated by this research approach may not be generalizable beyond a single school, but the framework that was used for the analytical process in transferable in a wholesale fashion to any academic setting.

Reflection

Rationale in support of a qualitative design.

The literature was consistent in showing that strictly quantifiable measures are most useful for the specific purposes for which they were collected, but this information can be used in other ways to gain insights into strengths and weaknesses of a given elementary school as well. Typically, though, in order to develop truly original and significant contributions to the body of knowledge, researchers have been required to include a qualitative element in their analysis as well. Indeed, within the five indicators developed by Bryk and described above exists the entire range of possible outcomes, making some type of subjective analysis an important component to understand the context in which these outcomes were achieved. Therefore, simply "counting the beans" will not be sufficient to developing a robust and thorough understanding of the results of any data analysis that is used that can be provided through the inclusion of a qualitative approach.