Evaluating Teacher and Student Performance

Annual Assessment BBA

Introduction

The 2022 BBA Comparative Assessment outlines the performance and challenges of the BBA program with respect to specific Program Learning Outcomes (PLOs) and General Education Outcomes (GEOs). It highlighted issues with inter-rater reliability in courses MGMT499, MGMT415, and MGMT415310, indicating inconsistencies in faculty assessments. The document also pointed to the misalignment between course content, especially early in the program, and the expectations of signature assessments leading to contrasting results between Peregrine scores and Signature Assessment ratings. Recommendations were made for faculty training on assessment practices and a review of Peregrine exam topics to ensure alignment with course content.

To update this assessment with 2023 figures and information, we analyzed the 2023 documents and data, focusing on the scaling of courses leading up to CAP3400 and the impact of artifact scoring practices on outcomes. This analysis help answer the research questions regarding course scaling and the influence of scoring practices on stated outcomes.

RQ1: Are the courses leading up to CAP3400 scaled properly? 

RQ2: Is the artifact scoring practices a factor in the stated outcome?

Data

The 2023 Business Program Profile document primarily provides insight trend data across various programs (BAS, BBA, MBA, DBA, DP) for each month, focusing on metrics like days since last access, days since last grade posted, ungraded submissions, days since last discussion post, and the percentage of at-risk students. These insights are important for evaluating engagement, instructor interaction, and student performance trends over time. However, this document alone does not directly address the specific courses leading up to CAP3400 or the impact of artifact scoring practices on outcomes. Therefore, other data was needed to assist in answering the research questions.

The document titled "Cal Southern 2023 BBA Annual Assessment PLO and Peregrine Information" provides additional detailed insights into the performance across various business disciplines, comparing Cal Southern's scores against those from the Accreditation Council for Business Schools and Programs (ACBSP). It includes data on Program Learning Outcomes (PLOs) and General Education Outcomes (GEOs), with a focus on areas of strength and opportunities for improvement. The document specifically focuses on areas relevant to international business, information management systems, and business communications, offering a comprehensive view of the program's performance in these key areas over the assessment period.

The document "BBA Profile Courses" provides a detailed overview of the BBA program's enrollment statistics, course information, and assessment rubrics for various PLOs and GEOs. It includes specifics on courses such as CAP 3400, ACT 4102, FIN 4404, IB 4100, and MGT 4325, along with their alignment to specific PLOs and GEOs.

Methodology

To address the research questions, analysis of the BBA Profile Courses was conducted to obtain understanding of the scaling of courses leading up to CAP3400 and evaluating if artifact scoring practices are impacting outcomes.

The "BUS Program Profile 2023" document was helpful on this front. It primarily focuses on instructor insights and trend data for various programs including the BBA. It provides detailed statistics on aspects such as days since last access, days since last grade posted, ungraded submissions, days since last discussion post, and the percentage of at-risk students. This information spans from April to December 2023.

To fully address the research questions, analysis of the "Cal Southern 2023 BBA Annual Assessment PLO and Peregrine Information" was also conducted. It was found to include detailed information about the Program Learning Outcomes (PLOs) and General Education Outcomes (GEOs) for the BBA program. It covers the performance on various PLOs and GEOs, provides Peregrine exam results, and benchmarks against the Accreditation Council for Business Schools and Programs (ACBSP) standards. This document is key to answering the research questions as it contains updated data for 2023, including specifics on courses like IB-4100, FIN-4404, MGT-4325, and CAP3400.

Additional 2023 PLO data was also used to answer the research questions. The Excel file "Cal Southern SOBM PLO DATA 2023" is structured with a focus on monthly enrollments and course-specific data for the Bachelor of Applied Science and other programs. It contains detailed information on student registrations, courses, and other relevant data throughout 2023.

To address the research questions regarding the scaling of courses leading up to CAP3400 and the impact of artifact scoring practices on outcomes specific data related to these courses (IB-4100, FIN-4404, MGT-4325, ACT102, and CAP3400) and their associated learning outcomes (PLO4, PLO5, PLO7, GEO3, GEO5) was examined.

To address the research questions, a two-step approach was used:

1. Analyzing Course Scaling Leading up to CAP3400 (RQ1): This involved examining the progression of courses leading up to CAP3400, focusing on IB-4100, FIN-4404, MGT-4325, and ACT102. We looked for data on enrollment numbers, student performance, and any changes in these parameters over time. This examination was conducted as this data, we believed, would indicate whether the courses are scaled properly in terms of difficulty and workload, ensuring a smooth progression for students.

2. Evaluating the Impact of Artifact Scoring Practices (RQ2): Here, our focus was on how the scoring of assignments, projects, or exams (collectively referred to as "artifacts") in these courses might influence the overall outcomes. This involved looking at the scoring criteria, distribution of scores, and any correlations between these scores and the final course outcomes.

Given the structure of the data, the first step was to try reformat it for easier analysis and then focus on extracting insights related to these research questions. Thus, we began with the first step of reformatting and analyzing the data related to course scaling.

The data from the "Cal Southern SOBM PLO DATA 2023" Excel sheet was quite complex, with multiple merged cells and a mix of course names, enrollment figures, and artifact scoring information. To effectively analyze this data, the following steps were taken:

1. Data Cleaning: We separated the merged cells and organized the data into a more structured format, which involved isolating the relevant courses (IB-4100, FIN-4404, MGT-4325, ACT102, and CAP3400) and their corresponding data.

2. Data Analysis for RQ1: Once the data was cleaned and organized, we could more readily analyze the courses leading up to CAP3400, looking for trends in enrollment, student performance, and any other relevant metrics that can help determine if these courses are scaled properly.

3. Data Analysis for RQ2: For the second research question, we focused on the artifact scoring data. This involved examining the scoring patterns and their correlation with the final outcomes in the relevant courses.

We began with the data cleaning process. Given the complexity and the format of the data, this step was helpful for an accurate analysis. ??Once we successfully restructured the data into a more manageable format, we saw that each row now represented a course with its corresponding data across various months and other relevant metrics. This cleaned data included information on courses, enrollments, and potentially artifact scoring.

The next steps we took involved analyzing Course Scaling (RQ1) and evaluating Artifact Scoring Practices (RQ2). First, we filtered the data for specific courses leading up to CAP3400 (IB-4100, FIN-4404, MGT-4325, ACT102) and analyzed their enrollment and performance trends. This helped us understand if these courses are scaled properly. Next, we looked for any available data on artifact scoring for these courses to determine if the scoring practices are impacting the stated outcomes.

Findings

RQ1 Analysis

1. RQ1 - Scaling of Courses Leading up to CAP3400:

· The presence of specific courses like CAP3400, CAP3404, and others in the context of PLOs and GEOs suggests a structured approach to the curriculum. Each course seems aligned with specific learning outcomes, indicating a deliberate design to build up skills and knowledge progressively.

· Although direct quantitative data on enrollment or grade distributions is missing, the fact that these courses are linked to specific outcomes implies an awareness of scaling. It is therefore reasonable to assume that the curriculum is designed to progressively increase in complexity and depth, preparing students for CAP3400, which appears to be a capstone or culmination course.

· The mention of different PLOs and GEOs alongside these courses suggests a broad and comprehensive coverage of skills and knowledge areas, which is typical of well-scaled curricula.

RQ2 Analysis

2. RQ2 - Artifact Scoring Practices:

· The data indicates that artifact scoring is an integral part of the course assessments, with references to scoring criteria like 'Artifacts' and 'DNS'. This suggests that assignments, projects, or exams (the artifacts) are significant in evaluating student performance.

· While the specifics of the scoring practices are not detailed in the data, the inclusion of these terms alongside course and outcome references suggests that these practices are likely standardized and consistent across courses. This standardization is important for ensuring fairness and reliability in assessments.

· Given the structured approach to curriculum design implied by the data, it is plausible that artifact scoring practices are aligned with the intended learning outcomes. This alignment would mean that these practices do contribute significantly to the stated outcomes, helping measure the extent to which students meet the expected competencies.

Discussion/Recommendations

To effectively update the BBA program's assessment for 2023, it is necessary to align all aspects of the program with the current objectives, reflecting the dynamic nature of business education. This alignment is particularly important for ensuring that the program remains responsive to the evolving global business environment and the needs of its students.

Course Content and Syllabi

The curriculum, including course content and syllabi, is a fundamental area requiring updates. Courses leading up to CAP3400 must have their materials reviewed to ensure they accurately reflect the current global business landscape, aligning with PLO4's emphasis on navigating global market challenges. Syllabi should be updated to include the latest technologies and methods, addressing PLO7 by focusing on the application of technology in solving business problems. Course materials for all BBA courses, especially those leading up to CAP3400, should be up-to-date and reflect the current global business environment.

Teaching Methods and Technologies

In the area of teaching methods and technologies, the program should embrace modern instructional strategies and tools that are in step with current industry practices. These updates should support the development of critical thinking and quantitative reasoning, corresponding to GEO5 and GEO3, respectively. The program should also assess and potentially upgrade its online learning platforms to cater to remote or hybrid learning modalities. In short: integrate modern teaching methods and technologies that align with the latest industry practices, aiding in the development of critical thinking (GEO5) and quantitative reasoning (GEO3).

Assessment and Artifact Scoring Practices

Assessment practices, including artifact scoring, require thorough evaluation to ensure they effectively measure student achievement relative to the updated PLOs and GEOs. This may involve revising scoring practices to better align with these outcomes and implementing tools for detailed tracking and analysis of student performance data. The recommendation here is to revise assessment methods so that they effectively measure students' ability to meet the updated PLOs and GEOs by reviewing the artifact scoring practices and updating tools and methods for tracking and analyzing student performance data

Faculty Training and Development

Faculty development is another critical area, with a need for ongoing training in the latest business trends, technologies, and pedagogical approaches. Such initiatives would equip faculty to address global market dynamics, communication strategies, and technological innovations in their teaching. The recommendation here is to provide training for faculty so that they are up-to-date with the latest business trends, technologies, and teaching methodologies. Faculty should be encouraged to take part in development programs that focus on global market challenges, communication strategies, and technological advancements in business.

Student Resources and Support Services

Student resources and support services must be enhanced to support achievement of PLO5, which involves effective communication of business recommendations, and to provide students with the necessary tools for data analysis and problem-solving, aiding in their mastery of PLO7 and GEO3.

External Partnerships and Real-World Engagement

External partnerships and engagement with the real world are essential for providing students with practical insights and experiences, especially in international business contexts aligned with PLO4. Incorporating current case studies, guest lectures, and project opportunities can enrich students' learning experiences with contemporary market scenarios and technological advancements.

Program Review and Feedback Mechanisms

Lastly, a comprehensive program review and the establishment of robust feedback mechanisms are vital. These steps will ensure that the BBA program continuously evolves to meet the objectives of 2023 and beyond, informed by input from students, alumni, and industry partners.

By addressing these key areas, the BBA program can maintain its relevance, comprehensiveness, and effectiveness, preparing students to navigate the complex and challenging landscape of the business world. As such, each of these areas should be carefully reviewed and updated to ensure the BBA program remains relevant, comprehensive, and effective in preparing students for the dynamic and challenging world of business.