School Program Evaluation This School

School Program Evaluation

This school program evaluates the necessity in the dissemination of currents events in the area of critical thinking skills and the CSCL tool's contribution in this area. History is a subject that students generally consider to be dry and boring because it is typically delivered in the form of a lecture. The objective of this school program is to make the learning of history more insightful, meaningful and engaging for Normal Academic students. The project also aims to inculcate the school values of Graciousness, Determination, Loyalty and Sincerity.

Evidence has been provided in the work of many researchers that the approach known as "drill and kill" where reading voluminous textbooks, will reduce students' motivation to learn. Development of critical thinking skills is a consideration of this study. Wright (2002) wrote about the necessity of challenging students through tools of critical thinking and providing the integration of critical thinking skills into history instruction through the usage of CSCL tool. The effectiveness of the CSCL tool will be measured through an experimental design, making use of qualitative and quantitative measures in the form of questionnaires during test in pre- and post-study. The results found in this study suggest that a Tablet PC -- or a Notebook -- based classroom interaction system can be harnessed to enhance the argumentation process. The teacher is able to collect and review students' work in real time, incorporating selected students' answers into the discussion. The researcher believes that the application of technology as illustrated in this work can bring similar benefits to the instruction in other disciplines.

Research Questions

The research question for this study is whether dialogic approach through CSCL has an impact upon students' learning of history. The impact will be measured not only by the product but also through the process of the intervention itself. Questions that this research sought to answer were: Are there changes in students' critical thinking skills and history knowledge in the process of the intervention? And (2) To what extent will the 20-week treatment in the CSCL environment help improve students' results?

Introduction

This study highlights the necessity in the dissemination of currents events in the area of critical thinking skills and the CSCL tool's contribution in this area. History is a subject that students generally consider to be dry and boring because it is typically delivered in the form of a lecture. Rittel and Webber (1973) held that critical thinking skills cannot develop without debate, negotiation and conflict. Lehtinen et al. (2004) held that education must serve the most basic requirement of the preparation of learners to fully participate in an information society that is highly networked and in which the most critical resource for development in social and economic aspects is knowledge. To this end, schools and institutions of education must find more advanced pedagogical tools and methods in meeting the challenges presented in the networked information society of today's world. According to the work of Lehtinen et al. (2004), cognitive interaction among peers drives the development of critical thinking and argumentative skills. Lehtinen et al. (2004) also related that the CSCL tool is one that seems to engage students in an in-depth inquiry over a substantial period of time and one that provides socially distributed cognitive resources for monitoring comprehension and other metacognitive activities.

Method

The method employed in this study is an experimental static-group comparison design, which will be used in the assessment of the effectiveness of the CSCL tool in the development of students' critical thinking skills in the subject area of history. The length of this study will be one semester (20 weeks). A questionnaire and test will be administered during pre- and post-study. The researcher and teachers will further, through observations, evaluate the study. The data generated will be used for analysis in answering the questions that this research posed.

Sample

This experiment will be conducted through a sampling of secondary one Normal Academic students undertaking history tests and teachers involved in completing the questionnaires, which will be in the form of self-reflection. This study identifies two dependent variables; the first being students' attitude toward the study of history measured through questionnaires designed by the researcher. These questionnaires looked at factors such as students' perception of their achievement in history and students' attitude toward learning history after using CSCL. The second dependent variable is students' achievement in history as measured by their pre and post-test results.

Design

This study will be conducted through an experimental comparison design (Gall, Borg & Gall, 1996) to assess the effectiveness of the CSCL tool in developing critical thinking skills in history students over the course of one semester (20 weeks).The main sources of data include pre- and post- survey for students, pre- and post-tests, and the discussion forum. Criteria set out for measurement in this study is that of the effectiveness of the use of Computer-Support Collaborative Learning Tools in teaching history specifically as to "collaborative argumentation."

The subjects are Secondary One students in the Normal stream undertaking history. The history test and questionnaires for the students will be administered pre- and post-study. The researcher is teaching all the four classes of Secondary One students in the Normal stream in the school. From each class, five students will be selected randomly to undergo the CSCL teaching method. These students will be classified under Group A, while the rest will be classified under Group B. A more detailed discussion about the participants will be elaborated in later paragraphs. Group A will undergo the CSCL teaching method, while Group B. students will experience the existing teaching method used by the school. Group B's outcome will be used to compare with Group A's results based on the research questions.

Independent variables are often regarded as the "treatment" given to one group in a study. In this study, the treatment will be the use of CSCL with one group of students. The control will be the researcher-led face-to-face discussions with the teachers, which is the normal mode of instruction in the classroom.

There are two dependent variables in this study: First, students' attitude toward the study of history measured by questionnaires designed by the researcher. These questionnaires looked at factors such as students' perception of their achievement in history and students' attitude toward learning history after using CSCL. The second dependent variable is students' achievement in history as measured by their pre- and post-test results.

CSCL Tool

The CSCL tool chosen is called EduLearn, a software provided by the company AsknLearn. This software permits any number of users to collaborate online in real time. Its design provides all the tools necessary for successful collaboration, such as general chat capabilities (with the added feature of printing verbatim transcripts of the chat sessions), file sharing capabilities, private chat, document review capabilities, voice communications, a shared calendar, a web links tool, email, and an outline tool. Many of these tools may be used simultaneously by participants with continuous updates of any changes that are made for each participant. Participation in collaborative meetings may be synchronous or asynchronous (Tan, 2000). For the purpose of this study, the main tool used will be the online forum, where questions will be posted via the online forum and students will discuss it via the online forum. In either case, the software provides all participants with instant updates of any changes which have occurred during the collaboration. Furthermore, the software provides full encryption of all data before it leaves a user's computer and travels over the Internet, and full decryption of all data after it enters another user's computer. This will ensure a fully secured peer-to-peer environment. Transcripts of each collaborative session can be printed verbatim for further studies to gather data that might not be expressed by the subjects in their answering of the questionnaires or during the writing of their self-reflection.

Research questions

To reiterate, this study is designed to examine the impact of CSCL in history classrooms for Secondary One students in the Normal stream. Specific research questions include:

1. Are there changes in students' critical thinking skills and history knowledge in the process of the intervention?

2. To what extent will the 20-week treatment in the CSCL environment help improve students' results?

Research Approach/procedure

For this research the researcher has decided to take a sampling from secondary One Normal Academic students from the school that the researcher teaches. The rationales for selecting secondary One Normal Academic students are as follows:

1.

They have not, through their one year of study since primary school, accumulated a certain degree of history knowledge required for secondary school history curriculum and have not grown accustomed to the need for individual thinking.

2.

The researcher has not taught them before and will be in a better position to interpret their achievement and perceive the changes in their learning.

3.

In secondary one, they are not in the critical period of sitting for the GCE "O Level" examination. Thus, this study will not place additional stress on students.

From each of the four classes, the researcher will randomly select 5 students to undertake the CSCL tool teaching method, this will ensure that the students are selected without any conscious or unconscious prejudices. These students (20 from the four classes) will form Group A. The rest of the students (Group B) will be taught using existing teaching methods used by the school for teaching the subject. In the event that more time is required to gather more conclusive data, the researcher will extend the study to the second semester. Before the commencement of the study, both groups will undergo a pre-study test and questionnaire to determine the level of their competency in history, level of critical thinking skills and attitude towards history. The researcher will grade the test using a marking scheme. The marking scheme has incorporated different levels of answers which correlate to the level of critical thinking expressed by students. This will be repeated after the study with a different set of questions but with the same level of difficulty for the two groups and a comparison study between the two sets of data will be conducted.

Before the commencement of the research, a familiarization session with the CSCL tool will be provided for Group A students. The researcher will give the students a user's guide, which detail the training, introduce the most common screens and features of the platform.

Throughout the semester, the researcher will observe students during their classes. The researcher will take notes pertaining to the following:

1.

Students' behaviors during the study of the history

2.

Interest level of the two groups when attending history lessons.

3.

How well students absorb history knowledge over the span of one semester

4.

How students express their critical thinking skills over the span of one semester.

The researcher's observations will supplement the data gathered from the pre and post-study questionnaires and tests.

Group A will complete CSCL online assignments as part of their curriculum. The researcher will design the online assignment in accordance with the school's history curriculum. These CSCL exercises will take place online (synchronously) using Groove software. All collaboration sessions of the students will be electronically recorded and these transcripts will be used to help teachers in the assessment process for all collaborative projects that the students produce. The student-computer ration will be one student to one computer. Students will not depend entirely on the tool. The researcher will guide students in the discussion. With the presence of computers in class, the researcher may opt to vary the method of teaching by letting these students be exposed to various historical places, events and people that can be shown on the computer. Also, instead of quizzes that are traditionally written on paper, quizzes and exams will now be taken using the computer as well as essay type questions that will actually challenge them to think critically about issues. Another means of developing critical thinking among these students is to provide them questions about some current issues in the country or society that needs to be addressed. These issues may not be very familiar to students, so they will be given ample time to research for meaningful and helpful information to assist them in answering the questions. The research of information can be done using computers which are connected to the World Wide Web. This will provide students with information in a faster manner, so that they can compose their answers and present them to the class during that same time when the question was given to them. Each student is to answer a different issue so that there will be no duplication of researched information and that everyone will be better informed about various happenings around them.

In a CSCL environment, students can easily collaborate with one another as they learn. The teacher's role when monitoring the collaboration is essential to keep the collaborative discussion "on track." By guiding students in their online discussions, the researcher can ensure that all students participate constructively and teacher moderated discussions will ensure that all students participate equally, thus eliminating domination by any individual. Good collaboration among students should enhance their knowledge of the subject and positively influence their attitude toward history.

The researcher will develop the additional elements to the history curriculum and assessment practices for the online collaboration project of Group A. Part of this preparation will occur during the semester prior to the implementation of the study while the remainder will occur during the study itself.

Schedule for the research

Assessment of the CSCL learning will be through usage of AskNLearn platform, which permits any number of users to collaborate in real-time, online. This design provides all the necessary tools for successful collaboration such as chat, file-sharing, private chat, document review and voice communications, a shared calendar, a web links tool, email and an outline tool. These tools are of the nature that they can be used simultaneously by students and with continuous updates for each of the study participants to make any changes. The pre -- and post-study surveys will include questions pertaining to the student's enjoyment of studying history and enquire if history tends to make the student think critically.

In the month of February to March 2009, the researcher drew lots for each of the four secondary one Normal class to randomly select five students and group them into Group A (a total of twenty students will be using CSCL tool), the online forum by AsknLearn and the remaining students from the four classes formed Group B. The pre-study test was administered to the four classes to determine their level of historical knowledge and critical thinking skills. Each group was given an access to a designated computer for this study to enable them to access the online forum and to the internet. The CSCL teaching method was used to teach the class for four weeks. Each session lasted for 1 hour and 30 minutes. There were two sessions each week for a total of eight sessions. The researcher facilitated and monitored the group discussion to ensure that the students were on the right track. The history topics identified for discussion for this study were "The First Emperor of China" and "The Caste and Class system." The online discussion was done in the form of a debate. This method was used because the basis of this study is that critical thinking skills can be developed using CSCL tool through augmentation. The discussion thread was tracked and archived.

In April 2009, the researcher administered the post-study test and questionnaire to both Group A and B. students. The purpose was to determine whether there was any significant change in critical thinking, knowledge of the selected history topics and their attitudes towards the study of history, where Group A was compared with Group B, which did not have the benefit of using the CSCL tool.

The evaluation of the collated data was done in May 2009. The researcher analyzed the data and determined the effects of the CSCL tool on history learning, critical thinking skills and attitude towards history. The findings and lesson learnt from this study will help to advance the teaching of history in terms of developing critical thinking skills, using CSCL tools and students' attitude towards history.

Pre- and Post-Study Questionnaire Responses for Group A (n=20)

Reponses to this question in the post-questionnaire study shows findings that approximately 69% of the study participants enjoy the study of history following this study through the use of CSCL tools whereas the study participants stated in the pre-questionnaire study that only 65% agree that they enjoy the study of history. 49% of the participants stated that the study of history tends to make them think critically whereas in the pre-questionnaire study 75% of participants in Group A stated that the study of history did not tend to make them think critically. When asked in the pre-study questionnaire if the study of history is found to be boring 88% stated that the study of history was boring however when asked the same question in the post-study questionnaire only 65% of the participants stated that they were bored by the study of history. When asked about the method students used to study history 95% of Group A participants stated in the pre-study questionnaire that it was through 'rote memory' while in the post-study questionnaire, responses in Group A were equally divided, with 50% stating that 'rote memory' had been used in the study of history while the other 50% stated that 'understanding the perspective' had led to the study of history. When Group A was asked in the pre-study questionnaire if history class revolved around group discussions, only 69% agreed with this statement while in the post-study questionnaire, 75% of Group A agreed that study of history in class revolved around group discussion.

It is very interesting to note the findings in question six as in the pre-study questionnaire, 97% of Group A participants stated that they had never participated in group discussion of history. However, responses in the post-study questionnaire revealed a result of 98% of Group A participants stating that they had been involved in group discussion in learning history. Question number 7 in both the pre -- and post-study questionnaire asked whether history discussions traditionally have been stimulating and informing. In the pre-study questionnaire, 97% of participants state that discussions of history traditionally have not been stimulating and informing. However, in the post-study questionnaire, 50% of participants in Group A stated that history discussions were stimulating and informing, while 50% of participants in Group A stated that history discussion were not stimulating and informing. This shows positive evidence that the usage of CSCL tools in the instruction and learning of history does aid in students' enjoyment of history since 50% stated in the post-study questionnaire that they now find discussion of history to be stimulating and informing. This is in contrast to pre-study questionnaire statistics which showed that 90% of the Group A study participants did not find history discussions to be stimulating or informing. In question 8 of both the pre- and post-study questionnaire, the question "Do you believe that using computer assisted learning tools would make history more interesting?" was asked. In the pre-study questionnaire, 98% of Group A study participants stated that "no" use of computer assisted learning tools would 'not' make history more interesting However, post-study questionnaire responses show that 98% of students agreed that use of computer assisted learning tools made the study of history more interesting. Question nine in the pre -- and post-study questionnaire asked if using computer assisted learning tools will make history easier to learn. In the pre-study questionnaire, 75.5% stated "no," yet in the post-study questionnaire, 95% of the participants in Group A stated "yes," that the use of computer assisted learning tools made the learning of history easier. Question ten of the pre- and post-study questionnaire asked if participants believed that using computer assisted learning tools would make history discussions more interesting. 97% of students stated in the post-study questionnaire that the use of computer assisted learning tools did make history discussions more interesting, while in the pre-study questionnaire, 75.5% of the students stated that computer assisted learning tools would not make history discussions more interesting. When asked in the pre-study questionnaire if they believed that the study of history offered knowledge that is both important and critical for individuals to possess, only 6% of participants agreed with this statement, yet in the post-study questionnaire approximately 80% of students agreed with the statement that the study of history offered knowledge that is both important and critical to individuals to possess.

Pre- and Post-Study Questionnaires Responses for Group B (n=140)

The pre- and post-study questionnaires administered to Group B (n=140) in this study showed little to no change from pre-study to post-study questionnaires responses. This is the group which did not used computer assisted learning tools in this study.

The comparative analysis of the findings from the questionnaires showed that there was a change in Group A students' critical thinking skills and history knowledge in the process of the treatment. The students seemed to have fewer problems related to critical thinking and history learning after the 20-week instruction period. The feedback gathered from the interviews and the examples provided by the observations made by the teachers during lessons also suggested that the critical thinking skills learned by the students through the CSCL tool approach might have swayed the students' perception on their ability to answer exam questions. This might had resulted in positive responses in the post-questionnaire.

For instance, in the first conference, one teacher stated that she knew the students had learnt critical thinking skills when she saw how the students were applying the skills in the assignments. The circling and underlining of the exam questions in the workbooks and assignments provided additional evidence. Questions raised by students also revealed to the teacher that they had benefited from the lessons. For instance, when asked an evaluation question, students would pause longer and were more careful in answering. Students did not plunge into answering the question without further thought.

However, the findings from the students' interviews seemed to indicate that in some cases, the observations made by teachers in the classrooms did not match the answers provided by students in the questionnaire. For example, a student might seem to enjoy the lessons yet gave negative responses in the second questionnaire.

Nonetheless, most of the findings from the questionnaire converged with the observations made by teachers during the 20-week treatment period. Informal interviews with teachers and students reinforced the notion that the registered change in the questionnaire findings could be the result of increased familiarity with critical thinking skills learned, specifically, through the forum.