This post is part of a series of posts soliciting public comment on the APA’s new Good Practices Guide. In the first post in this series, I provided some background on how the APA Good Practices Guide came about and presented its preface and first section. For more on the guide and this series, go back and read that post.
In this fifth post of the series, I’m covering section 5 of the Good Practices Guide, titled “Countering Implicit Bias.” This section reviews research on implicit bias and outlines steps faculty and departments can take to reduce it and its impacts.
The section begins with a review of the definition of implicit bias—the unconscious activation of both positive and negative stereotypes—and how it operates. It invites readers to look at implicit bias as philosophers—“We are professionally involved in unearthing assumptions and values underlying ordinary thought and practice, and subjecting them to critical examination”—and as teachers and administrators— “We can make progress in dealing with implicit biases in ourselves and others if we can make the academic setting one in which members of diverse groups come together to work on shared projects.”
Section 5 goes on to identify ways that we can do this, including taking and encouraging others to take implicit bias self-assessments (including students, potentially as part of coursework) and putting in place structures for hiring and admissions that will limit the impact of implicit bias. (For more on these structures, see section 4 on best practices in interviewing.) The section also encourages critical engagement with implicit bias self-assessments—“Just as the tests and associated hypotheses afford a teaching opportunity, so do questions of the validity of the tests and hypotheses”—and includes several resources for further discussion of such tests.
The section ends with a list of informative resources on implicit bias, many of which are cited throughout the section.
Implicit bias is a hot topic, so I invite your feedback on this section, on the questions below and anything else you’d like to offer:
- Have you included the topic of implicit bias in your teaching? Have you encouraged or required students to take self-assessments? If so, do you have suggestions from your experience that might be useful for this section of the GPG?
- Has your department used implicit bias trainings and/or taken any of the other steps outlined in this section? How have those efforts been received? Based on your experience, is there anything you’d like to add to what’s provided in this section?
*
GPG Public Comment Series: 1 / 2 / 3 / 4 / 5 / 6 / 7 / 8
Amy Ferrer
Amy Ferrer has been Executive Director of the APA since 2012. She holds a bachelor's degree in women's studies and a master's degree in public policy and administration, both from the University of Massachusetts Amherst. She has spent more than 15 years in nonprofit management, having previously worked for national and regional organizations focusing on civil liberties, public health, and advocacy. Both her work and educational background have focused on diversity issues, communications, development, and program management. She currently serves on the boards of Delaware Humanities and the Academic Placement and Data Analysis project, and has previously served on the boards of the National Humanities Alliance, the American Council of Learned Societies, and the Association of American Colleges and Universities.
This section is straight up disinformation with a case in point being work on bias in letters of recommendation. This area of research should be notorious for its lack of consistent findings and faults in experimental design and interpretation. Nonetheless the Guide relies on just two studies in urging philosophers to adopt a “review process…encouraging greater awareness in faculty” about implicit bias when writing letters. Yet one of these, Morgan et al., didn’t have anything to do with that. Instead they found bias in the mostly female, undergraduate readers who were asked to evaluate prospective graduate students in psychology (though when asked to elaborate/justify there was no bias). Even taking this result at face value the relevance to philosophy is scarce, though note that “the disadvantaged group was men” — and before you point out this is still evidence of “reverse” gender bias, it is important to remember that the only systemic effect evidenced by this research is the failure to replicate results from one study to the next.
But leave all that to one side for now, and let’s start with why a study about bias in reading letters got reported as one about writing letters (this should have been clear just going off the abstract!). Call me crazy, but normative recommendations ought to be based on what studies find rather than what is found in the imaginations of philosophers.
Although the other study mentioned does concern writing letters, this time the Guide can be faulted for cherrypicking and failing to assess critically. Schmader et al. admitted their data “revealed more similarity than differences,” though a more honest conclusion would have been to say letters seem to be free of bias, and women can be reassured recommenders are not influenced or distracted by gender. This sounds incredible given the way Schmader et al.’s findings are constantly spun, and yet we know it is true since there was literally no significant evidence of bias among the dozen or so factors Schmader et al. examined, save for the sole finding that men received 17% more “standout” adjectives. Despite these anemic findings, instead of calling implicit bias into question, Schmader et al. propose it is taking on “subtle forms”! But what this “caloric in hiding” theory ignores is that the result could easily be due to chance (about 50/50, I gather, going off a 5% “significant” outcome hitting at least once over about twelve measurements: .95 x .95 x .95….) hence raising worries about the dodgy statistical practice known as P-fishing. A still better explanation takes into account that they also found men had 25% more publications.
And that’s it. This exhausts the “evidence” provided about letters of recommendation. As these kinds of errors and doubts go way beyond the standard “element of uncertainty,” and in light of the grip various myths about implicit bias have on many philosophers, deletion (as suggested by Professor Hales elsewhere on this blog) is appropriate. Another option would be to replace the section with this sentence: “Despite a growing concern about implicit bias among many philosophers, the scientific community continues to harbor strong doubts and no trustworthy recommendations about policy or practices should be made at this time.”
Have you considered that one role of philosophers when it comes to implicit bias might consist in dispassionately evaluating its conceptual foundations, its explanatory power, and its empirical support? Or is that only important for robustly replicated and broadly predictive concepts like general intelligence?
In the spirit of adding to the bibliography of papers about debiasing, here is a list of debiasing strategies and the evidence that motivates them: byrdnick.com/archives/7136
I am grateful to the APA for taking the initiative to engage with these very important issues. However, I share many of the concerns of the commenters above. These are complex empirical questions, and we will be doing a disservice to our students if we fail to give them an understanding of the present state of the debate.
Just to give one example, the APA guide says:
“The results of such tests [the IAT] have been found to have predictive value for biased behavior in a range of contexts — a value that is independent of, and sometimes greater than, measures of explicit bias.”
No one who has been following the recent empirical literature would think that this claim is now regarded as uncontroversially correct. On the contrary, a growing body of evidence specifically suggests that the IAT is not a good predictor of biased behavior.
Of course, reasonable people can disagree, and the authors of the guide might have a different take on the relevant empirical data. Still, my sense is that it would be pedagogically inappropriate to expose philosophy students to this sort of claim without also exposing them to the (very substantial) body of evidence that points in the opposite direction.
One of the items cited in the guide contains an excerpt that might capture the concern raised by Joshua Knobe:
“[Implicit Association Test] (IAT) measures have two properties that render them problematic to use to classify persons as likely to engage in discrimination. Those two properties are modest test–retest reliability (for the IAT, typically between r .5 and r .6) and small to moderate predictive validity effect sizes. Therefore, attempts to diagnostically use such measures for individuals risk undesirably high rates of erroneous classifications. These problems of limited test–retest reliability and small effect sizes are maximal when the sample consists of a single person (i.e., for individual diagnostic use), but they diminish substantially as sample size increases. Therefore, limited reliability and small to moderate effect sizes are not problematic in diagnosing system-level discrimination, for which analyses often involve large samples.” (Greenwald, Banaji, and Nosek 2015, p. 557).
The idea is that the IAT might (might) have predictive value in larger systems. But it’s far from clear that the IAT has much predictive value of individual behavior. So if the guide explicitly recommends “Taking the implicit attitude test (IAT) and encouraging other faculty to do so as well” (p. 49), then the guide might also include a word of caution about how to interpret one’s (or any individuals’) results.
Greenwald, A. G., Banaji, M. R., & Nosek, B. A. (2015). Statistically small effects of the Implicit Association Test can have societally large effects. Journal of Personality and Social Psychology, 108(4), 553–561. https://doi.org/10.1037/pspa0000016
Hi Nick,
Thanks for this very helpful comment. If students are encouraged to take the IAT themselves, they might well come away with the assumption that their IAT scores are diagnostic of their own personal propensity for biased behavior. But existing research suggests that this is not the case. Thus, if students are asked to take the IAT, they should presumably be told explicitly that biased responses on this implicit measure are not indicative of a propensity for actual biased behavior.
Also, I just wanted to say again that although I disagree with many of the recommendations in this part of the guide, I am very grateful to the authors for all of the work they put into this. What we face here is just an ordinary substantive disagreement — of the type that always arises in philosophy — but all the same, I have the utmost respect for the authors themselves and for this contribution.
As someone with a small (very small) leadership role in the APA (as the current chair of the Committee on Philosophy and Law) I’d like to strongly recommend taking Joshua Knobes’ comments very seriously. I don’t think the relevant material should be included, given its currently weak foundations. As a learned society, we should not be making recommendations on such grounds.
I am saddened by the fact that I continue to be embarrassed by my own professional organization. The way in which the IAT is uncritically used here — not to mention the enormous irony involved in the fact that it clearly is the result of the Association’s own biases in the direction of a very specific, very narrow conception of social justice, one that is at odds with the liberal tradition — is just the latest in what seem to me to be an ongoing parade of betrayals of our discipline and some of its most ancient and enduring values.
Not only do I not employ the IAT, but I will not. Not simply because meta-studies have revealed the test to be largely useless, but because in my view the manner in which the test is commonly being used is Orwellian and illiberal and serves little purpose but to “out” people as closet -ists of various sorts. In short, everything that we, as philosophers, should not be about.
For those who are interested, I published a short piece on the use of the IAT not too long ago. I was unable to link to it here, because the APA website marked it as “Spam” (lol), but the title is “Liberalism, Implicit Bias, and Thoughtcrime: On the Subject of the I.A.T.”
As a woman philosophy PhD student, I experience the comment section here as extremely troubling. Having a good practice guide out there that is designed to give recommendations for how to deal with issues such as sexism and racism in academic philosophy/ academia is extremely helpful and a great relief/ hope for relief for all those who have been suffering from the conditions and the problems pointed to in good practice guides in academia.
While I agree with your scientific concerns about the IAT, I strongly disagree with putting comments like the ones above out there.
These comments (all the more since they are all by men) unfortunately suggest that implicit bias does not exist and thus downplay and trivialize the importance of any good practice guide. I am sure this is not what all of you are thinking, however, this is the normative message that comes across. It would be helpful to make it clear in statements like the ones given above that you support good practice guides / that you do not mean to imply that sexism and racism are not an issue in academic philosophy/academia.
Hi Gina,
Thanks for your very helpful comment. I agree that it is important for those of us who disagree with the specific recommendations in this guide to make it clear that we support the broader mission, and I am happy to do precisely that.
I strongly agree that sexism and racism are serious problems in our profession, and I did not at all mean to suggest that implicit bias does not exist. Moreover, although I disagree with many of the recommendations in this specific guide, I am very grateful to see that philosophers are trying to make a difference and improve the climate in our discipline.
Again, thank you for making this comment. Many of us disagree with certain scientific claims that appear in this guide, but I hope we can all agree that the broader mission it represents is a deeply important one.
If we cannot have an open discussion on these sorts of topics, where people are able to honestly and freely express their considered views, then philosophy is in very bad shape. I wrote what I did carefully and with due consideration. I trivialized nothing, indeed quite the opposite. I said nothing about whether or not there is sexism or racism in philosophy or academia. All of that, you are reading into my comment.
My main concern is with basic, classical liberal principles, articulated in places like “On Liberty,” which in my view are threatened by this sort of use of an instrument like the IAT. I’m sorry that you find it “extremely troubling,” but surely it is still permissible in our discipline to espouse classical liberal views.
Thanks, Josh!
Daniel,
Please don’t rephrase what I wrote.
Of course we can have an open discussion. I strongly encourage criticisms on all the methodology and conclusions coming out of research out there – especially when it comes to normative topics. However, an open discussion does not exclude making clear that you agree that sexism and racism are serious problems in our profession (as Josh pointed out).
I simply tried to encourage you to also emphasize the importance of the APA guide and not only pointing to the small problematic aspects included in it.
Joshua: I’m curious. Are you including, in your thanks, Gina Eickers’ suggestion that I should not have voiced my views in a public forum whose purpose is for people to have an open discussion? That strikes me not only as antithetical to the entire tradition of our discipline, but to the liberal tradition on which our society is based.
@ Josh: Thanks!
@Daniel: Please do not rephrase what I wrote.
A quick follow-up: I never said you shouldn’t post. I said: all commenters here are male, and none had stated the value and importance of this guide. The IAT discussion is a sideshow (but I also said, I agree with your criticisms on IAT).
If this is an open discussion, it is open to my two cents on this as well, I suppose.
Now, for a (male) prof to lash out at a female PhD student for expressing legitimate feelings about the failure to acknowledge the value of this guide is totally inappropriate.
My reply was limited to this part of your post:
“While I agree with your scientific concerns about the IAT, I strongly disagree with putting comments like the ones above out there.”
Which I clearly did not misinterpret.
As for the rest, I think you are confused as to what constitutes “lashing out,” but I’ll leave it there.
Daniel – I see no confusion on Gina’s part as to the nature of the exchange. The liberal tradition to which you refer gives you every right to speak. But it does not require others to defer to your self-understanding of what you have to say.
Nothing I said could be reasonably construed as “lashing out.” Of course if one is determined to interpret people uncharitably and read all sorts of things into what they say, then, yes, you can turn a conversation into something it isn’t.
Hi Gina (if I may),
My recommendation is that as implicit bias conks out as a serious research topic in philosophy we need to enter a period of reflection: What kind of damage has been done? How can the profession do better? In what ways has ideology distorted research and publishing practices? What other factors led to the crisis: Fetishizing novelty? Neoliberal publishing models? Editorial cartels & echochambers? Inadequate training? How does philosophy’s recent history compare to other failed research programs? To what extent is implicit bias, micro aggressions, et al. failed science versus pseudoscience? Should philosophers avoid topics involving controversial/unsettled scientific research? What other empirical results are being abused? Does philosophy need its equivalent of the Society for the Improvement of Psychological Science? Alas, I see little interest in pursuing these questions.
It is not really relevant, but as it happens I am skeptical about whether guides such as these are a productive use of time and energy. (Or would philosophers take heed if the Guide urged them to engage with arguments instead of diverting attention to the persons making them?) Nevertheless, it is not my business if others feel differently and plunge ahead.
These are my thoughts,
Sean
Thank you so much for sharing this insightful blog post! I truly appreciate the effort and expertise that went into creating such valuable content. The information provided is not only informative but also presented in a clear and engaging manner. It has certainly added to my knowledge of unconscious bias. Your dedication to delivering quality content is evident, and I look forward to reading more from you in the future. Thanks again!
If you want to know further information related to unconscious bias visit https://americandiversityinitiative.com/courses/unconscious-bias-workplace-training/ website.