Welcome again to The Teaching Workshop, where your questions related to pedagogy are answered. Each post features questions submitted by readers, with answers from others within the profession. Have a question? Send it to PhilTeacherWorkshop@gmail.com.
*
Question: I work at a reasonably large state school. My introductory classes have so many students that multiple choice exams are my only real choice. But multiple choice philosophy exams are horrible. They are extremely difficult. Students aren’t prepared for the sort of careful reading and thinking that are central to figuring out the right answer. Making the questions easier isn’t the answer—the only way to do that is to make them so easy that I’m no longer testing them on anything philosophically significant. I want to work with my students to help them develop their reading and critical thinking skills so that they’re better prepared for the exam. But I’m struggling to figure out what sorts of activities or projects can be useful. What can I do to help students develop the skills associated with reading multiple choice philosophy questions and picking the right answer?
Answers:
Firstly, I was glad to see you recognized the challenge that multiple choice tests offer students. Rather than simply calling for “mere rote learning,” they can help students achieve clear understanding before they critique a theory. In a discipline such as philosophy, where we hear all too often,“There is no right or wrong answer, right?,” it can be important to convey to students that yes, there are right answers when it comes to whether you understand a philosopher’s claims and arguments, and a mastery of the basic ideas must precede reflective analysis.
Students often find these tests extremely challenging, so how can we help better prepare them? Here are some quick suggestions:
- Give frequent low-stakes quizzes that ask students to demonstrate that they know what a concept means or what a philosopher’s position is.
- Be sure to avoid assuming that students know words such as “objective,” “theism,” “argument,” “intrinsic,” etc. We live with these ideas, and too often the instructor assumes a basic vocabulary that is simply not there for many students. Here again, quizzes can help them see for themselves what they understand. Try pairing them up or having them work in groups to define key terms and explain important ideas.
- Consider using clickers as you lecture. Creating voting slides on a PowerPoint can engage students in paying attention and testing themselves on the reading and your lecture. You can go over a section of text or a theory and then put up some slides that ask them to choose the right answers. These votes can be anonymous, and students see immediately how they all voted. When wrong answers are chosen, you can give immediate feedback as to why they are mistaken.
- Be sure to go over multiple choice exams afterward and explain why an answer is right or wrong. You might consider giving them a do-over for an exam as a way to discourage them from simply giving up.
Try constructing a few comparable multiple choice questions for each of the chapters or sections of the material they are covering as they go along during the semester, possibly as part of students’ homework assignments. In addition, instructors should think about covering sample questions in a review session a week or so before the exam—even if it’s during the last week of classes (for a final exam).
Finally—and this is a bit tricky, I know—instructors should explain that while some questions might strike students as trick questions, really, what the questions are asking them to do is to demonstrate a mastery of complicated concepts in the material, as well as the ability to apply those concepts.
Take the following question (from critical reasoning) as an example:
- Which of the following may occur in a valid argument?
- true premises, true conclusion
- false premises, false conclusion
- false premises, true conclusion
- all of the above
Students must understand several things: the definition of “validity,” the relationship between the truth value of an argument’s premise(s) and conclusion and and its validity, and the scope of the question (“may” vs. “must” vs. “occur,” etc.).
In my experience, helping students prepare for multiple choice questions involves helping them understand logical operators. It’s hard for students to understand the available answers, especially when those answers have multiple parts.
Suppose this is a wrong answer: “A and B.” Let’s say A is false but B is true. Of course, that means the whole statement is false, but students have trouble with this move. Even when students thoroughly understand A’s and B’s truth values, they may not be comfortable with determining what this means for the statement as a whole. They might mark the answer correct just because they see B is true and don’t know what else to do with that information.
If you want students to be able to parse complicated answers like these, you will have to help train them to do so. Preparing for multiple choice questions involves practicing critical reading skills.
Note, “A and B” may seem like an overly simplified example, but even a conjunction like this can be extremely challenging for a student to parse during an exam. When stressed, one’s reasoning skills can sink very low. Many students need help with managing their emotions during an exam. We can’t really help them with this, but we can offer some advice. I like to remind my students that one of the best things they can do, when they confront a challenging question, is set theirs pencil down and take a deep breath. Help them develop their critical reading skills during in-class activities, and encourage them to find a calm mindset that will let them actually make use of those skills during the exam.
*
Suggested Additional Resources:
- Sterne, Jonathan, The Chronicle of Higher Education, Multiple Choice Exam Theory (Just In Time For The New Term), January 10th, 2013.
- Teaching Philosophy 101.
- Collins Peter, Examining Philosophy: “Choose the Best Answer,” Teaching Philosophy 16: 2, June 1993.
*
Do you have insights to add? Join the conversation in the comments below, or email us at PhilTeacherWorkshop@gmail.com. Remember, the best answers are constructive and specific.
One thing I’d suggest is to first re-think the idea that there’s no other way to go than to use multiple-choice exams. It’s always worth brainstorming to look for better options, which might involve questioning some basic pedagogical choices. But without knowing more about your individual situation (like whether there are TA’s available to do the grading), it’s hard to suggest those things from outside.
But going with multiple choice exams, one suggestion I have is to include as many questions as you possibly can, while still having it be a reasonable length for the time period. The more questions you have, the less penalty there is for each wrong answer, so there’s less pressure/anxiety per question. Along with this, include a wide variety of difficulty in the questions. Have lots of them be pretty easy for anyone who’s been in class and/or doing the reading, have some be of medium difficulty, and then have only a small portion be ones that really require some mastery of the content. This sort of exam can serve to reward attendance and attention, while also distributing grades in a reasonable way.
Also, consider different ways of grading multiple-choice questions. They don’t all have to be all-or-nothing. Suppose each question is worth 5 points. You might give 1 point just for answering a question, even if it’s dead wrong. After that, some questions might be appropriate to just be right or wrong in a cut-and-dry way, but others might reward wrong answers that aren’t terribly wrong. Maybe there’s one answer that’s clearly the best, so it gets 5 points, but another one that is not too unreasonable of a choice, so it gets 3 points, and the others only get 1 point.
Lastly, especially if the test is newly written, it’s good to be responsive to how well the students actually do on it when grading. If 75% of the students get a certain question wrong, make sure to re-examine that question super critically and maybe give more partial credit than you were planning, and then note that question as one you may want to rewrite later. You may decide it’s good as it stands and those 75% really shouldn’t have missed it, but you should at least take that data as feedback and re-examine it honestly.
One strategy I’ve learned about from colleagues in other disciplines is the following: Split the exam period into two parts. At the end of the first part, each student turns in a completed exam. Then you divide the class into small groups, and give each group another copy of the same exam. During the second part of the exam period, each group works together to fill out the exam (again). Where a group gets a question right, but a student in that group got that question wrong, you reward that student with some amount of partial credit. This strategy may indirectly help with the challenge you face, in that it will encourage students to figure out how to reason their way through a question, *together*.
A second strategy (one that I’ve tried, with fair success, when teaching intro logic) is to prepare two (or even more) versions of a given exam, so that students may retake it. (Here, assessing by means of exams actually provides an *advantage*, since having students write multiple papers on a given topic isn’t really feasible.) This has the advantage of explicitly rewarding students based *just* on how much of the material they’ve mastered, and not *when* they’ve mastered it. And my students have reported that it reduces stress in a way that makes it much easier, psychologically speaking, for them to learn the material.
I’m not sure whether you have the budget for it, but I’ve found IF-AT tests (http://www.epsteineducation.com/home/about/) to be very effective.
They’re tests with scratch off forms that require the students to continue scratching off options until they get to the right now. This gives them immediate feedback and allows you to give partial credit. The immediate feedback is helpful because they can see where they have to go back and reconsider (or reread). The partial credit option is helpful because you can ask the very hard questions and not doom the students to an all-or-nothing grading schema.
I’ve also done what Ned Hall has recommended (begin with an individual exam, end with small groups coming to consensus about that exam) to great success. It’s pretty fantastic to see students reasoning through the questions together.
I love the idea of individual-then-group test. I’m curious, though: anyone have an idea how to make that work for students with disabilities who take their exams elsewhere (so they can get extra time, etc)?