Chatted with a data analyst at a coffee shop who changed my mind about AI bias

Was waiting for my latte and this guy next to me started talking about how he trains models for a hospital system. I mentioned I always thought bias in AI was just a coding problem, like fix the data and move on. He pointed out that even with perfect data, the questions you ask the model can be biased. Asked why that hit different for me. He said because it means the bias isn't just in the numbers, it's in the people deciding what the numbers mean. Never thought about it that way. Has anyone else had a conversation that totally flipped how you see this stuff?

3 comments

3 Comments

matthew8642mo ago

That is a really good point, I never thought of it that way either. I used to think bias was just a data problem, like if you scrubbed the training data clean everything would be fine. But the stuff about the questions being biased is what got me. Like asking a hiring model "does this candidate have the same traits as our best past employees" is a biased question if your best past employees were all one type of person. That honestly changed my whole view on it.

amy9751mo ago

your best past employees were all one type of person" is spot on but I think the bigger issue is even if you have diverse past employees the model still learns patterns from that specific time and place which might not hold up later. It's like training a model on 1990s hiring data and wondering why it misses things today.

miab872mo ago

Yeah a friend of mine said the same thing after talking to a researcher at a bar last month.