Measuring student engagement with mathematics through dependent variables and test items

¶ … student engagement within mathematics, create a set of dependent measures. Follow the rules for writing test items, and please include the correct answers, as well.

Dependent Measures

Student Engagement

Five Multiple Choice Items

What is the square root of 49?

Which number is not a prime number?

How many degrees is a right angle?

A diagonal line cutting through two parallel lines creates angles which are?

unequal

congruent

immeasurable

What is 3 to the third power?

A composite engagement score would have to be tallied in order to total the average score of all of these variables so that one could assess how all of these variables interacted with one another, giving the researchers a sense of the total and complete interest, enjoyment, capability, and confusion that all students experienced when engaging with these problems, and to determine which variables were most often experienced simultaneously.

Five True-false items

The infinity symbol symbolizes a number

True/False

2. 0 to the Nth power is zero.

True/False

3. An acute angle is smaller than an obtuse angle.

True/False

4. A rhombus has eight sides.

True/False

5. Eight is always divisible by 2.

True/False

Matching Section

Figure

Form

Right Angle

Has four sides

Obtuse Angle

Has 90 degrees

Acute angle

Has less than 90 degrees

Parallelogram

Has between 180-and 91 degrees

2. After reading the article addressing middle school student note taking, discuss the value of each phase of the study. That is, describe the value of the experimental phase, and then do the same for the qualitative phase. Next, be critical of the study. What, in your opinion was lacking? What were possible weaknesses (e.g. you might address the use of the Latin square design)? Finally, comment on the value of using both quantitative and qualitative phases together in a single study. (2 pages)

The Qualitative Phase

The qualitative phase of the study was made up of interviews which found that students had a preference for copying and pasting their notes for logistical reasons and found that typing notes was generally distracting to them (Igo et al., 2006). This portion of the research was followed up with a textual analysis (Igo et al., 2006). In this sense, it demonstrates how so much of the research construct was based on rudimentary and overused forms of gathering and analyzing data. Little was done that was innovative or ground-breaking. Rather the unremarkable methods only succeeded in supporting unremarkable results.

In my opinion, it would have been interesting and far more illuminating to determine how the students could have reacted to note-taking in a more creative fashion. This experiment simply tested students against constructs which have been tested and retested for decades. If the researchers were really looking for progress, they really should've sought to test a more interesting form of note-taking. Creative note-taking, visual and image-based forms of note-taking are all elements which should have been incorporated into the research.

For instance, if the research looked at tools which were able to engage in word prediction, such an event would be ideal in order to make real strides in the realm of note-taking. That sort of research would look more at some of the tools which are available to LD students so that they can better engage in constructive habits of study. Tools with "…word prediction, which augments spelling and syntax to enable users to make choices, find words, and complete sentences. Word prediction programs display words based on frequency of use, grammatically correct usage of words, and most recently used words. Read & Write 5.0 by textHELP (1999) is an invaluable tool for students with reading and writing difficulties" (Quenneville, 2001). The fact that this most recent research had a truly inadequate means of harnessing and using technology is truly revelatory about the paucity of the research in general and the narrowness of the overall focus. When it comes to LD students, assistive technology for note-taking and general education needs has a tremendous amount of potential. The benefits that it offers include the possibility of enhanced academic achievement through clearer and more cohesive note-taking along with improvements in the overall written expression and in spelling (Quenneville, 2001).

The consequences of such moves are tremendous: it can lead to improved organization and bolstered social acceptance and can help students facilitate writing and other skills that they generally find very frustrating. The fact that there is little to no recognition about the possibility of technology in this research experiment along with the fact that there is almost no recognition of the more advanced forms of technology out there to benefit students, is truly demonstrative of the limited potential of the entire research project.

3. Assume you are creating a Likert-type scale addressing teacher interest of teachers engaged in a professional development activity. Include 10 items and two versions of the anchors -- "one set of four anchors and one set of six anchors.

Likert Scale on Teacher Interest/Engagement in Professional Development Activity

Six Anchors

Completely Agree

Mostly Agree

Somewhat agree

Somewhat disagree

Mostly Disagree

Completely Disagree

Four Anchors

Agree

Neutral

Need time to reflect

Disagree

10 Items

1. I am satisfied with my competency within this professional development activity.

2. I am satisfied with my level of knowledge within this professional development activity.

3. I take pride in doing professional development activities to the best of my ability.

4. I have at least one other colleague that I trust completely while doing this professional development activity.

5. My colleagues listen to me attentively throughout the course of this activity.

6. I feel like I have an adequate number of choices when I engage in these professional development activities.

7. If I have a problem or a question during any of these professional development activities, I know that I can get help.

8. I can use these professional development activities to help track and further my own specific goals and needs.

9. These professional development activities help challenge the way that I think.

10. These professional development activities are presented in a manner that is relevant and interesting to me.

4. Using the research website www.socialresearchmethods.net/kb/index.php distinguish the differences among Likert-type, Gutman, and Thurstone scales. Next, describe your opinion of their usefulness in research by creating a rank order of them. Describe why you have ordered them that way. In other words, which do you believe is best, second best, and worst? (1 and half pages)

This website on social sciences research methods really is incredibly useful when it comes to distinguishing the differences between a range of research scaling methods. The website is able to truly lucidly distinguish between these different types of scales and what their differences and benefits are. The following order of usefulness has been created: Likert, Thurstone, and Gutman as a personal order of preference and utility.

Likert was selected as the most useful because it excels at presenting information in the most straight-forward and simple to understand manner, which capitalizes on the natural intuition of the user. Likert is one of the most effective an direct uni-dimensional scaling methods which can help one define very cohesively what one is attempting to measure: "it is assumed that the concept you want to measure is one-dimensional in nature. You might operationalize the definition as an instruction to the people who are going to create or generate the initial set of candidate items for your scale" (Trochim, 2006). The Likert scaling process also involves generating items and then rating the items which are generated. This is followed by selecting the items and then applying the scale. In the end, the final score for the respondent on the scale is generally the sum total of all their ratings for all of the items. Essentially, this method is able to process the data in a highly accessibly and intuitive manner that is practical for both the user and researcher.

Thurston methods were selected next because of their dependence on a scale of relatively few items. All Thurstone methods "begin by focusing on a concept that is assumed to be unidimensional and involve generating a large set of potential scale items. All of them result in a scale consisting of relatively few items which the respondent rates on Agree/Disagree basis" (Trochim, 2006). The Thurstone method uses a truly interesting method of data collection and uses paired comparisons but in a manner so that the paired comparisons aren't too taxing or overwhelming. This method was selected as the second most useful, because it illustrates how a simple unidimensional scale might be created (Trochim, 2006).

Guttman scaling has slightly less accessibility and is thus rated last. In the technique of Guttman scaling, engaging in cumulative scaling is paramount so that a one-dimensional continuum is established (Trochim, 2006). Essentially this method is arrange in a manner so that if a respondent agrees with a specific question in the list, it indicates that he or she will agree with all previous questions (Trochim, 2006). In a more formal fashion, this just means that researchers use the Guttman method of scaling as a means of predicting the respondent choices, knowing only the total score for each user (Trochim, 2006). It's this sort of backwards and backhanded method which explains why this scaling choice was selected last.

5. Consider the differences in the experimental research designs discussed in class (NOTE: refer to the powerpoint slides). Also consider your areas of interest. Generate two examples of each of the following designs, describing the Dependent and Independent variables for A, B, and C, and only the two levels of independent variables in D.

a. One-group

Research design: A project which determines whether a problem-solving, data-driven approach is more effective at creating new interventions focused on disrupting the archaic methods of burnt-out teachers. This would be based on the creation of a new software tools to assist researchers in analyzing the patterns of low-scoring classrooms and to pinpoint the pathways by which students stop coming to class. In this research, the findings would be used to integrate a working method to prevent teacher burnout and to prevent student truancy.

Research Design: A project which seeks to determine whether providing students with clear rewards (in the form of gift-cards or other desired items) can assist in raising scores of low-scoring students and classrooms. In this case, the independent variables would be the gift cards and the dependent variables would be the student test scores.

b. One-group pre-test post-test

Research design: This would consist of an informal classroom-based pro-creative teaching program vs. An anti-textbook-based education program. These programs would be delivered over several weeks in classrooms all over a select city. The independent variables would be the different teaching programs. The dependent variables would be the changes in student knowledge, attitudes, and experience.

Research design: This research design would be made up of a more holistic forms of professional development, versus more research-based forms of professional development. The programs would occur with different groups in conjunction with one another and would be instituted in schools and faculties all over the city. The independent variables would be the two separate professional development programs; the dependent variables would be the skill sets and teaching abilities of the faculty members who participate in these programs.

c. Static-group comparison

Research design: One group of teachers is shown a documentary on gang violence and public schools, and the same film is not shown to another group of teachers. An awareness of violence and the reasons for violence among public school students is then measured giving both groups of teacher's actual surveys to use. In this case, the independent variable would be the documentary and the dependent variable would be the teacher knowledge of violence in public schools.

Research design: One group of students is shown a documentary on drug use, abuse and the spread of drugs. Another group of students is not. Both groups of students are given surveys or questionnaires to test their understanding of drug use and the sale of drugs. In this research, the documentary would be the independent variable and the student understanding of drug use and abuse would be the dependent variable.

d. Two-way factorial (2x3)

Research design: The study looks at the impact of nap-times and holistic therapies on teacher burnout rates.

Research design: The study looks at professional development weekends for teachers vs. professional development night classes for teachers.

6. A colleague of yours is interested in conducting a correlational study. However, he does not seem to understand the difference between correlation and causality. Explain it to him using plain, conversational language and two practical examples: one for correlation and one for causality. Please make up these examples. There is no need to cite actual research. Rather, you should be creative. Do NOT use examples from the PowerPoint slides. (1 page).

People are constantly getting the difference between correlation and causality mixed up and that is a problem as the difference between them is definitive and distinctive. Correlation refers to the measured mathematical relationship between two things. For instance, the relationship between two things is measured and given as a figure between 0 and 1. If there is a correlation of 0 it means that the two things compared have no relation (that means if given the first value there would be no way to anticipate the second. A correlation of 1 means that two things are completely related, as the first thing always predicts the second. A common example of this dynamic would be using height as a predictor of weight, as the first thing generally has a high correlation to a person's weight. In this case, the correlation would be somewhere around .8 (w-uh.com). Likewise, causality is also a relationship between two factor, but it has no mathematical connection; rather there is a physical or philosophical connection between two things. Something can cause something else in a chain of events along a timeframe: this creates an unsymmetrical relationship. For instance, consider the following: there is a high correlation between smoking and getting lung cancer, though smoking doesn't cause lung cancer, even though there is a high correlation between them.