8.2.b     Student outcomes: general education

The institution identifies expected outcomes, assesses the extent to which it achieves these outcomes, and provides evidence of seeking improvement based on analysis of the results for student learning outcomes for collegiate-level general education competencies of its undergraduate degree programs.

 

Non-Compliance

 

The general education program has been developed with student outcomes aligned with the Louisiana Board of Regents, with departments aligning their courses to the student learning objectives, and with the identification of key assessments to measure and evaluate the student learning outcome competencies. Most of the departments responsible for the assessing and using data for program improvement provide evidence of compliance with the Louisiana Board of Regents requirements.

 

According to the Gen Ed Assess Matrix, all the programs required the majority of their students to meet expectations, pass assessments/courses, or score satisfactory or higher. For Natural Sciences the assessment criteria is “Outcomes will be considered successful or achieved if at least 50 percent of students meet or exceed Developing performance on rubric.” Additionally, when scores are provided, there is no disaggregation of the results by established indicators: Developing, Developed, and Exemplary. The aggregation of the scores reveals exceptionally high outcomes, and there is no apparent use of the data for improvement.

 

In the Humanities, there are multiple outcome expectations; however, for the Diversity Assessment of syllabi there is no rating scale. The data provided include the expectation of a 2.0 with averages noted for each course syllabus used in the assessment. Although, average scores can be used to achieve the required outcome of 2.0, it is difficult to interpret without the scale and indicators of the assessment. Furthermore, course syllabi analyses, by themselves, are not direct indicators of student learning outcomes.

 

In addition, the assessment criteria and use of data for improvement was not clear in the areas of Natural Sciences and Humanities.

iNSTITUTION’S rESPONSE

For its general education competencies of its undergraduate degree programs, UL Lafayette identifies and assesses outcomes and seeks improvement based on analysis of results. In its original analysis, Natural Science and Humanities did not disaggregate results. The disaggregation of results as well as how data was used for improvement are provided below by discipline.

Natural Sciences

The Natural Sciences portion of the general education curriculum has two student learning outcomes: 1) to draw reasonable conclusions within natural sciences by applying key processes and scientific reasoning, and 2) to evaluate credibility of information with scientific content by using critical and logical thinking, knowledge of accepted scientific methods, and appropriate sources. Nine courses within the departments of Biology, Chemistry, Environmental Sciences, Geology, and Physics assess these competencies. Outcome 1 is assessed in BIOL 121, CHEM 101, ENVS 100, GEOL 105, and PHYS 160 while Outcome 2 is assessed in BIOL 122, CHEM 102, GEOL 106, and PHYS 170. Two cycles of assessment (2017-2018 and 2018-2019) are available in the Natural Sciences Assessment Report. The following table demonstrates the disaggregation of results by established indicators.

Table 8.2.b – 1: 2017-2018 Assessment Results for Natural Sciences

 

Table 8.2.b – 2: 2018-2019 Assessment Results for Natural Sciences

 

Although all targets were met in these two assessment cycles, the departments within Natural Sciences used the data for improvement. A detailed explanation of each improvement is available in the Natural Sciences Assessment Report, but examples are provided here for each of the two competencies.

Between Fall 2017 and Fall 2018, CHEM 101 saw an increase in “exemplary” (from 45% to 65%, respectively). In Fall 2017, the instructor observed challenges in engaging students on questions and topics involving analytical thinking requiring students to draw conclusions using principles of chemistry that also required deeper thinking. In response to these findings, the instructor developed new instructional material using several examples, reinforcing learning through homework, and interacting inside and outside of the classroom. In Fall 2018, these pedagogical changes were implemented. The results improved this student learning outcome, as demonstrated by an increase in the “exemplary” category.

Physics provides another example of how data was used for improvement. In Fall 2017, 5% of PHYS 160 students were rated as “underdeveloped;” by Fall 2018, no student was rated as “underdeveloped.” In reviewing the Fall 2017 results, the instructor prepared several course changes, including adding more challenging questions on homework, improving PowerPoint slides for classroom instruction, and including videos from NASA and PBS on pertinent topics. In reviewing the Fall 2018 results, the instructor observed that the videos from NASA and PBS proved to be particularly beneficial in student learning, and will continue in future courses.

Finally, Physics 170 also demonstrated how data was used to make improvements. In Spring 2018, 80% of students were rated as exemplary (20%), developed (40%), and developing (20%) with the remaining 20% “underdeveloped.” After analyzing results in Spring 2018, the PHYS 170 instructor noted some of the best practices described above from PHYS 160 as well as the implementation of iClicker and Pearson homework. By Spring 2019, 90% were rated as exemplary (30%) and developed (60%), with none “developing” and 10% “underdeveloped.” The instructor attributed the positive changes to the implementation of instructional videos and more engaging homework.

Humanities

The Humanities portion of the general education curriculum has three student learning outcomes in the areas of communication and language, literature and humanities, and historical perspectives.

·         Communication and Language: Communicate effectively in verbal language.

·         Literature and Humanities: Read, interpret, and write cogently and critically about diverse literary and cultural texts.

·         Historical Perspectives: Demonstrate an awareness of diverse historical perspectives and their significance for the present.

Communication and Language

Communication and Language used disaggregated results and data for improvement within the departments of Communication, Theater, and Modern Languages.

The department of Communication assessed 200 student presentations in CMCN 100 in the academic year 2018-2019, as detailed in the Communications Assessment Report. Six competencies were assessed using a four-point scale, where an achievement of a 1 or 2 was deemed as not having met the individual competency and an achievement of a 3 or a 4 was deemed as having met the individual competency. The following table displays the disaggregated results for those presentations reviewed in Fall 2018 and Spring 2019.

Table 8.2.b – 3: 2018-2019 Assessment Results for Communication

 

Upon analyzing the disaggregated results, the department of Communication sought improvements to address all competencies. A detailed explanation of each improvement is available in the Communications Assessment Report, but two examples are provided here for the competencies of organization and subject knowledge.

In Fall 2018, 57% of students scored either a 1 (11%) or 2 (46%) for the competency of “organization,” while 43% scored either a 3 (27%) or 4 (16%). Changes were made to the homework schedule which allowed students to begin working on a rough draft of their speech outline two weeks earlier than previously scheduled. This additional time allowed students to work on the organizational flow of their outline, which ultimately allowed them to deliver a more organized speech. This change proved to be successful; in Spring 2019, only 40% scored a 1 (7%) or 2 (33%) and 60% scored a 3 (35%) or 4 (35%) in this competency indicating that the adjustment to the schedule allowed students to prepare their outline in more detail, aiding them in their ability to maintain organization when delivering their speeches. After analyzing the Spring 2019 results, additional improvements are being developed. For example, CMCN 100 will incorporate a peer review exercise. By having a peer review their outlines and provide feedback, students will benefit from receiving additional feedback and gaining ideas by reviewing other examples. Additionally, the exercise provides students another opportunity to review and edit multiple outline drafts.

Another example of how the department of Communication sought improvements was in relation to the competency of “subject knowledge.” In the Fall 2018 semester, 54% did not meet the criteria, earning either a 1 (21%) or 2 (33%), while 46% did meet the criteria with a 3 (29%) or 4 (17%). Changes were made to the course schedule to allow more time for students to work with their approved, individual topics. By having students turn in their potential topics a week earlier and providing them immediate feedback with an approved final topic, students were allowed an extra week to research their topic, become acquainted with the information they needed to know for their speech, and prepare accordingly. This minor change contributed to the students’ ability to meet the goal for this individual competency: in Spring 2019, only 34% did not meet the criterion (8% scored a 1 and 26% scored a 2). The remaining 66% of students met this goal (46% scored a 3 and 20% scored a 4). These findings suggest that more time with the topic has a favorable impact on the students’ overall ability to grasp subject knowledge. In light of this finding, the course schedule will be adjusted once again to provide students with yet another week to have their approved topics to prepare and work on their research; specifically, students will now have six full weeks with their topics compared to the four weeks initially allotted. 

Theater’s Assessment Report outlines its assessment protocol, including the use of the Communications Rubric to assess five of the six competencies (the competency of “Mechanics” was not evaluated because this is not an appropriate competency in a theater presentation). 20% of students enrolled in THEA 261 were assessed in Spring 2019. The following table displays the disaggregated results for those presentations reviewed in Spring 2019.

Table 8.2.b – 4: 2018-2019 Assessment Results for Theater

 

Upon analyzing the disaggregated results, the Theater Department sought improvements to address all competencies. A detailed explanation of each improvement is available in the Theater Assessment Report Addendum, but two examples are provided here for the competencies of speaker engagement and elocution.

In THEA 261, speaker engagement deals with the students’ ability to capture their audience through an energized and engaged performance that incorporates dynamism in a way that carries the audience through an arc of the scripted scene. In Spring 2019, 0% of assessed students scored a 1, 12.5% scored a 2, 12.5% scored a 3, and 75% scored a 4. While the outcome was met, opportunities for improvement were nonetheless identified. Specifically, instructors will give a number of small performance assignments leading up to the larger assessed performance in the course. These exercises may include telling a dramatic story, reciting a poem in a way that engages the audience and provokes an emotional response from them, and participating in in-class critiques and discussions of these shorter assignments. These exercises lay the groundwork for engaging an audience in a longer scripted piece. Additionally, prior to the final performance, students are given preliminary feedback and coaching from the instructors so that they can improve upon the scene’s dynamism.

Another area of improvement was noted in the category of elocution. In Spring 2019, 0% of students scored a 1 and 12.5%% scored a 2, 50% scored a 3, and 37.5% scored a 4. Although almost all students met this goal, it was clearly the weakest area in the overall assessment. Instructors currently employ some in-class exercises that invite students to explore the voice as an expressive tool, and will add some additional in-class activities that focus more on volume and diction in performance, in addition to emotional expressiveness. Further, prior to the final performance, students are given preliminary feedback and coaching from the instructors so that they can improve upon issues of volume and diction as they relate to the scripted performance.

Modern Languages set as its outcome for students to be able to function at the Novice High level of language proficiency according to the most recent American Council on the Teaching of Foreign Languages (ACTFL) Proficiency Guidelines (2012). The Modern Language Undergraduate Assessment Committee collects and analyzes a selection of anonymous GERM 101, FREN 101, and SPAN 101 final exams, and compares student scores on these exams against the goal of 80% or students scoring 70% or higher on tests deemed appropriate.

The department distinguishes individual elements of the ACTFL rubric for evaluation and uses this disaggregated data to make improvements. For example, in both the French and Spanish final exams, students scored above 70% on tasks that required them to extract relevant information from short, non-complex texts, an ACTFL standard described in the Modern Languages Assessment Report. They scored below 70% in tasks that required they derive meaning from reading context, part of the ACTFL Novice High sublevel also described in the assessment report. Modern Language faculty agreed to address this deficiency in 101 classes through more frequent use of realia and other materials from a wider variety of different contexts, followed by in-class practice in reading within different cultural and material contexts.

Additionally, 80% of the French, German, and Spanish students scored 70% or higher in essay writing, which evaluated their ability to express themselves in writing within a learned context. Students scored lower in tasks that required an ability to “meet limited basic writing needs,” like short messages, dialogue completions and simple notes. After discussion, the Modern Language faculty agreed to include short in-class writing assignments that were then reviewed in class so that students would have more practice with different writing tasks.

Finally, because the German Final Exams were not stringent enough to adequately test for all aspects of the Novice High sublevel standards, student scores exceeded the criterion established. Thus, these exams will be adjusted to be more appropriate to stated goals.

Literature and Humanities

In 2017-2018, Literature and Humanities assessed the “diverse literary and cultural texts” component of the objective. Because it was the first year of the new assessment protocol for the general education requirement, the Literature and Humanities assessment committee established a syllabus assessment in order to determine if students in literature courses were being expected to engage diverse literary and cultural texts. While this assessment did not directly measure student learning, it was necessary to provide a baseline of which courses were meeting the diversity component of general education and thus determine the review cycle going forward. The committee has now established a cycle for Literature and Humanities assessments that measures specific student learning outcomes and has carried out a student learning outcome assessment.

The committee’s Course Literary and Cultural Diversity Rubric gauged two competencies (diversity of perspective and diversity of forms/genres), using scores of 3 (exceeds expectations), 2 (meets expectations), and 1 (does not meet expectations). The goal was set as an average of 2.0 (meets expectations) for all courses assessed. In 2017-2018, all available syllabi (41 total) for ENGL 201, 201, 205, 206, 210, 211, and 212 courses taught in the spring 2018 semester were assessed. Neither of the two competencies met the criteria; the average scores across all courses was 1.6 in diversity of perspective and 1.6 in diversity of forms/genres.

Table 8.2.b – 5: Assessment Results for Literature

 

Based on the disaggregated results, the assessment committee noticed ENGL 202 had the lowest average in diversity of perspectives, and immediately worked to address this concern through its mentorship program and meetings with the Department’s Sophomore Literature Committee. The department held a peer-led workshop for all instructors who will or do teach these courses. The workshop addressed successful strategies for incorporating a diversity of literary perspectives, forms, and genres. Additionally, the Sophomore Literature Committee changed its protocols to require that all new 200-level General Education courses be reviewed by the committee and evaluated based on the assessment criteria.

During Spring 2019, the Department conducted direct assessment of critical reading and comprehension in historical survey courses. The Sophomore Literature Committee designed a quiz to test critical reading and comprehension about one reading passage particular to the course. The goal was for 75% of students in ENGL 201, 202, 205, and 206 to correctly answer 2 of the 3 quiz questions. The results indicated that 72% of the students answered 2 of the 3 questions correctly.

Table 8.2.b – 6: Assessment Results for Literature

 

After analyzing these results, the committee identified two areas of improvement. First, the Sophomore Literature Committee will more effectively communicate goals to instructors through email, Moodle, and face-to-face meetings, and require the student learning outcome to appear on the syllabus of every course meeting the General Education Literature requirement. This small but crucial step will improve course instruction in the area of reading. Additionally, this goal will be assessed next year in three additional courses (ENGL 210, 211, and 212) in order to compare between historical surveys and open-topic survey courses.

In Fall 2019, Literature and Humanities developed an assessment protocol in which four student learning components are assessed: reading, interpretation, writing, and diversity of texts. Quizzes will assess reading comprehension and the ability to analyze and interpret works of literature, while student writing will be reviewed using the Essay Assessment Rubric. This rubric will measure three competencies (critical interpretation through writing; paragraph organization; mechanics, usage, and vocabulary) with a three-part rating scale (exceeds expectations, meets expectations, and does not meet expectations).

Historical Perspectives

The department of History assesses students’ ability to compare and contrast different perspectives as demonstrated by the following outcomes: 1) identify a historical source as either a primary or secondary source; 2) recognize that a primary source has an author with a perspective; and 3) infer how the perspective or life circumstances of the author might influence the content of the source. The department used the disaggregated results to drive improvements in student learning.

The department generated two versions of a skills-based standardized exam that required no prior knowledge of historical content. On the exam, students were required to read an excerpt from a primary source, and then answer three multiple-choice questions. All instructors of HIST 100, 101, 102, 221, and 222 were provided the exam questions at the beginning of the Spring 2018 semester and were asked to distribute the quiz to all sections of these classes at the end of the semester. There was no identifying information on the quizzes; students and instructors remained anonymous. The assessment coordinator then graded a random sample of the submitted quizzes, approximately 10% of each version. The established target was an 80% pass rate, with “passing” equivalent to receiving 65% on the exam (two out of three questions correct).

The results indicated that 78.9% of the students passed by answering two of the three questions correctly, just below the established target.  But the disaggregated results offered more insight into success on individual competencies:

·         Competency 1 (“Identify a historical source…”): 70% of students passed

·         Competency 2 (“Recognize that a primary source…”): Not assessed that year 

·         Competency 3 (“Infer how the perspective…”): 77.5% of students passed

After considering these results, the department established guidance and training for all general education instructors to teach the difference between primary and secondary sources in their classes. The department also included an ungraded question on a quiz that asked students to explain qualitatively why the provided source was primary or secondary. Many of these answers correctly defined a primary source, and/or provided logical reasons for the choice (even if the choice was incorrect).

To streamline the assessment process and to encourage higher completion rates, the department is considering asking general education instructors to assess just one (rather than all) of their general education courses. For instructors who teach two or more general education classes, this could save valuable instruction time; for the assessment committee, this may streamline the process by collecting only those assessments that will be analyzed. Finally, the department is considering alternating the assessment of general education objectives, rather than assessing all each semester. A full calendar is proposed for Fall 2018, which has built-in communication with faculty throughout the semester.

Conclusion

UL Lafayette identifies and assesses outcomes and seeks improvement based on detailed, disaggregated data and analysis of results throughout its general education program.

 

sUPPORTING dOCUMENTS

·         Communications Assessment Report

·         Communication Rubric

·         History Assessment Report

·         Literature and Humanities: Assessment Protocol and Timeline

·         Literature and Humanities: Course Literary and Cultural Diversity Rubric

·         Literature and Humanities: Essay Assessment Rubric

·         Literature and Humanities: Literary and Cultural Diversity Assessment Report

·         Literature and Humanities: Reading Assessment Report

·         Literature and Humanities: Reading Quiz

·         Modern Languages Assessment Report

·         Natural Sciences Assessment Report

·         Theater Assessment Report

·         Theater Assessment Report Addendum