@dresstokilt @amydiehl this is useful to understand because these emerging judgments about competency based on assumptions of effort are shaping our world right now and when they are unevenly deployed against some identities and not others, it will create penalties for those groups.
@amydiehl what exactly is being tested here? That two identical resumes are perceived diffently based on the assumed gender of the name at the top?
If so, *why did AI* need to get involved?*
@dresstokilt @amydiehl you are misspecifying the study. The thing being manipulated between gender conditions is how people are perceived *for using AI*, and showing that there is a gender penalty in the attributions a reviewer makes about the competency of women that is steeper than the attribution made about men. This is useful because it reveals that there is a gender*competency beliefs effect, e.g. if it were only about AI regardless of user we would expect to see the same penalty
@grimalkina @amydiehl I'm not misspecifying, I was asking a legitimate question because the original post was a headline and a paywalled article.
@dresstokilt @amydiehl I'm explaining it to you because you asked. You are making an incorrect assumption about what's being measured. It's not a moral problem it's just a fact.
@dresstokilt @amydiehl this is useful to understand because these emerging judgments about competency based on assumptions of effort are shaping our world right now and when they are unevenly deployed against some identities and not others, it will create penalties for those groups.
@dresstokilt @amydiehl regardless of one's feelings about AI being appropriate for a task, it is useful to ask whether the same demonstrated behaviors are *described* by observers as having different meanings and being a signal of different aspects of a person. This is a classic gender effect: men's behavior is seen as more circumstantial, something explainable in the pressures of a moment, while women are more likely to be judged as innately flawed or lacking despite the same behavior.
@grimalkina @amydiehl "shaping our world right now" yes it's *only now* a bias problem.
"AI has the same bias as the people who programmed it" is less of a story than "dog bites man."
@amydiehl @mhoye I seriously wonder how people read resumes where the name is ambiguous.
(Eh very personal interest here - I’ve been misgendered based on my name nearly daily since I was a teenager. I’m a very cis male but daily get people asking if I’m using my wife’s card or being startled when I pick up for Shannon etc. I’ve long wondered if my resume gets penalized by bias like this study shows)
@mhoye @Rycaut @amydiehl
Unfortunately, anonymizing resumes isn't enough and sometimes even backfires.
https://journals.sagepub.com/doi/10.1177/01461672241304593
Intentionally building diverse teams is what works best.
@mayintoronto @mhoye @Rycaut @amydiehl
i just stumbled upon something where a researcher gave Claude a bunch of abstracts in genetics, to which the researcher had *randomly* assigned author names, and asked Claude to find patterns re research topics and author social identity. And Claude did. E.g., "Female authors" were more likely to produce research about "DEI, Ethics, and Social Dimensions of Genomics". It looks like there's no way to use these tools w/o *imposing cultural patterns*
@mhoye @amydiehl indeed I think that’s great but also hard once the process starts - I suspect however that most companies don’t do that work (and these days many use AI filters that likely have unknown biases to filter most applications before a human even sees them.)
It’s definitely a good idea if as a company you actually want to hire the best people and get past your own inevitable biases in that process. Hiring has been pretty broken however for a long time
Install bonfire.mavnn.eu
Get the full app experience
