One of the most difficult parts of college-level instruction is assessing the effectiveness of your teaching. Many of the conventional methods for measuring student achievement are far from guaranteed indicators of course quality. Student course evaluations notoriously favor professors who water down course content, while rampant grade inflation makes the comparison of grades across courses and semesters completely unreliable in assessing instructional quality. Measuring the effectiveness of our courses is a priority for us at the Philosophy as a Way of Life Project. We want to know if our courses are helping students to live more philosophically, and how to introduce targeted interventions when we do not accomplish our course goals. In what follows, I will detail what our research has revealed about assessing the efficacy of college courses and how you can integrate these insights as you fashion your own classes for this coming semester.
Your Goal: Overall Student Satisfaction. How to Measure It: Student Course Evaluations.
Getting student input may not always be the best way to judge course quality. Not only do students give better overall reviews for inflated grades and easier classes, they are also influenced by factors that are irrelevant to effective instruction. With all of this in mind, should you bother looking at student course evaluations? Even those who are critical of student evaluations agree on one thing: Course evaluations are reliable indicators of student satisfaction. Although students are poor at evaluating their own learning, they are much better at judging how they feel upon course completion. Do they feel like they got a lot out of a course? Did the course content seem worthwhile? Would they recommend this course to others? Students are well-positioned to answer questions like these that capture how happy they are with their course experience, making student course evaluations an effective yardstick for measuring student satisfaction.
Your Goal: Rigorous Material.How to Measure It: Student Course Evaluations.
Of course, if you are just aiming for overall student satisfaction, you might be worried that you will sacrifice rigorous instruction at the altar of strong student reviews. Students can be satisfied with a course simply because it was an easy A or because they feel they accomplished something worthwhile. How can you tell if strong course evaluations are more attributable to the latter than the former? Fortunately for our purposes, student course evaluations are not only effective at measuring student satisfaction — they are also strong indicators of how challenging students find the course material. Students are quick to report a course’s workload as demanding or challenging, especially when the course is more difficult than they expected it to be, making student reviews an effective way to gauge a philosophy course’s rigor.
The primary difficulty with measuring both student satisfaction and course rigor via student course evaluations is that there is a tradeoff between the two. Even if students ultimately enjoyed a course, courses with lighter workloads and easier grading are generally ranked more highly. Because a challenging course might result in slightly lower rates of student satisfaction, you ultimately want to aim for a balance between them. If students report high rates of course satisfaction but spent little to no time on the course outside of class, then the next move might be to make the course content a bit more challenging. On the other hand, if students are overwhelmed with the amount of coursework, then it may be time to scale back and implement more course elements focused on student satisfaction.
Your Goal: Student Growth. How to Measure It: Efficient, Effective Feedback.
Students enter Philosophy as a Way of Life courses with a wide range of abilities. Some are just starting college and have very little experience structuring their study time or writing term papers, while others are further along in their college degree programs and better prepared to meet the demands of a rigorous philosophy course.Regardless of where students are in terms of academic achievement, our goal is to create classrooms designed for optimal academic and personal growth, courses that both meet students where they are at as well as challenge them to take the next step.
Though there is still some disagreement about the size of the effect, a sure-fire way to accelerate student growth in your courses is to give targeted, timely feedback. When students complete any task in your class, whether that is a daily journal entry, multiple choice question, or essay rough draft, this opens up a window of opportunity. Students want to know how they did: Did they get the multiple choice question right? Did they ace the rough draft? And if not, what was holding them back? We try to take full advantage of this window of opportunity, not only communicating correct answers to students, but also providing feedback for how they can improve on future assignments.
This feedback can take a number of different forms. In my course at Florida State, we review the answers to daily reading quizzes immediately, discussing which parts of the text students found difficult to understand. Rough draft essays are peer-reviewed within a week’s time, and all students are required to schedule a personal tutorial session at the virtual writing center. However you choose to give feedback, make it a priority to give that feedback efficiently. With timely feedback, students will be better positioned to learn from their mistakes and better able to incorporate what they learn in their future work.
Your Goal: Academic Achievement. How to Measure It: Common Assessment Items.
It might seem like we’re missing something. Thus far, I have focused on ways to measure how your class is doing in terms of course rigor, student growth, and overall satisfaction, but I haven’t touched on one of the core elements of a college class – how much students are learning! This is partially by design. If the structure of your course emphasizes student growth in the midst of challenging course content, your students will be learning far more than they would in an easy course without much instructor feedback.
Nevertheless, you might still want to check if your students are improving on particular learning objectives – whether they understand Kant’s Categorical Imperative or Berkeley’s Idealism for instance. We have already seen that grades might not always be very telling. If past classes were less challenging or had any grade inflation, creating a rigorous course may actually make course grades go down. In place of comparing student grades, the best strategy for judging whether students are making strides on particular content items is by including common assessments items in multiple courses. Common assessment items can include identical multiple choice or short answer questions, or even similar essay prompts. These common assignments will allow you to directly compare past and present student performance to see if students are making progress towards specific learning goals.
In order to use this method of measuring academic achievement, you do not need to copy all of the assignments from past courses in their entirety. Instead, just select particular assessment items that you particularly like from previous courses, keeping the best parts of previous iterations of the class, using those to compare how well you taught different content areas in the past and present versions of the course. This will allow you both to see how student understanding is improving as well as if there was anything you lost in your course redesign. Common assessment items are thus a strong, objective method for measuring how much students are learning from course to course.
I hope that you enjoyed our series on how to make a course relevant, engaging, resilient, and effective. For more pedagogical resources for creating engaging philosophy classrooms from the Philosophy as a Way of Life Project, check out our website or join our Instructor Community on Facebook!
Wes Siscoe
Wes Siscoe is a Dean’s Postdoctoral Fellow at Florida State University and the Mellon Course Design Coordinator for the Philosophy as a Way of Life Project. He received his PhD from the University of Arizona and has been a visiting researcher at Brown, Notre Dame, and Rutgers. His research revolves around several themes– rationality, language, and virtue – and their importance for accounts of human excellence and achievement.
