Your Mom's Cooking
Imagine you grew up eating only your mom's cooking. She makes amazing pasta, decent tacos, and pretty good stir-fry. That's your entire experience of food.
Then someone asks you: "What does Indian food taste like?" You have no idea. Or worse, you try to guess based on what you do know — and your guess is completely wrong. You're not trying to be wrong. You just don't have the right data.
Now imagine someone asks: "Rate every restaurant in the city." You'd rate Italian places highly (you understand that food), rate the taco truck fairly (you have some experience), and rate the Thai restaurant poorly — not because it's bad, but because you've never experienced anything like it and don't know how to evaluate it.
That's AI bias. And it's one of the biggest problems in artificial intelligence today. Not because AI is trying to be unfair, but because it can only learn from the data it's given — and that data almost always has blind spots.
Garbage In, Garbage Out
Remember from Article #1: neural networks learn by looking at training data. If the training data is incomplete, skewed, or reflects human prejudices, the AI will learn those same problems.
Here's a real example. A major tech company built an AI to screen job resumes. They trained it on 10 years of hiring data — the resumes of people who were actually hired. The problem? For those 10 years, the company had mostly hired men for technical roles (a common pattern in the tech industry).
So the AI learned: resumes that look like men's resumes = good. Resumes that mentioned women's colleges or women's organizations = less good. The AI wasn't programmed to be sexist. It learned sexism from the data, because the data reflected a biased reality.
The company scrapped the tool when they discovered the problem. But it illustrates the core issue: AI doesn't know what's fair. It only knows what's in the data.
The Training Data Mirror
Think of training data as a mirror reflecting the world. If the mirror is tilted, everything in the reflection is tilted too.
Some ways training data ends up biased:
Who's represented? If a facial recognition system is trained mostly on photos of lighter-skinned faces, it will be worse at recognizing darker-skinned faces. Not because of any intentional discrimination — simply because the data was imbalanced. This was documented in a famous study that found some commercial facial recognition systems had error rates 34 times higher for dark-skinned women compared to light-skinned men.
What's in the text? Language models (Article #3) are trained on text from the internet. If the internet contains stereotypes — "doctors are usually men," "nurses are usually women" — the AI absorbs those patterns. Ask it to generate a story about a doctor, and it's more likely to write about a male character. Ask about a nurse, and you get a female character.
Who labeled it? Many AI systems rely on humans to label training data ("this image shows a cat," "this text is positive/negative"). If the labelers bring their own biases to the task, those biases flow directly into the AI.
The Feedback Loop Trap
Here's where it gets even trickier. Biased AI can create feedback loops that make the problem worse over time.
Imagine a police department uses an AI to predict where crimes will happen. The AI is trained on historical arrest data. But arrest data isn't the same as crime data — it reflects where police chose to patrol and who they chose to arrest.
If certain neighborhoods were over-policed historically, the data shows more arrests there. The AI says "send more police to those neighborhoods." More police = more arrests. More arrests = more data that "confirms" those neighborhoods have more crime. The AI becomes more confident. The cycle reinforces itself.
The AI isn't discovering truth. It's amplifying a pattern that already existed in the data.
You've experienced a lighter version of this on social media. TikTok's algorithm shows you more of what you engage with. You watch one video about a topic, the algorithm shows you five more, you watch those, and suddenly your entire feed is about that one thing. You didn't choose that bubble — the feedback loop created it.
It's Not Just a Tech Problem
AI bias matters because AI is increasingly making decisions that affect people's lives:
- Loan applications: AI decides who gets approved for a mortgage or credit card
- College admissions: Some schools use AI to screen applications
- Criminal justice: AI tools predict recidivism risk, influencing bail and sentencing decisions
- Healthcare: AI prioritizes which patients receive care or follow-up
- Hiring: AI screens resumes and even conducts initial video interviews
If these systems are biased, the consequences are real: someone doesn't get a loan, a job, or adequate healthcare — not because of who they are, but because of patterns in historical data that reflected existing inequality.
What's Being Done About It
The good news: the AI community is actively working on solutions.
Diverse training data. The most straightforward fix: make sure your training data represents everyone. If your facial recognition system doesn't work well on certain groups, get more data from those groups.
Bias auditing. Test AI systems specifically for bias before deploying them. Check: does this system perform equally well across different genders, races, ages, and income levels? If not, fix it before it goes live.
Fairness constraints. Build mathematical fairness requirements directly into the AI's training. Instead of just optimizing for accuracy, optimize for accuracy and equal performance across groups.
Transparency. Require that AI systems explain their decisions. If an AI denies a loan, the applicant should be able to understand why — and challenge it if the reason reflects bias rather than legitimate risk factors.
Regulation. Governments are starting to pass laws requiring bias testing for AI systems used in high-stakes decisions like hiring, lending, and criminal justice.
Try It Yourself
Think about your own "training data" — the experiences that shape your assumptions. Where you grew up, the people you know, the media you consume, the schools you've attended.
Now think about a time when you made an assumption about someone or something that turned out to be wrong — because your experience was limited. Maybe you assumed everyone celebrates the same holidays, or that a food you'd never tried was gross, or that people who listen to certain music are a certain way.
That's the same thing that happens to AI. The fix is the same too: broader experience, more diverse perspectives, and the humility to check your assumptions.
The Big Takeaway
AI isn't inherently biased — but it inherits the biases in its training data. Since that data comes from a world with real inequalities and prejudices, AI systems can accidentally amplify those problems at massive scale.
The solution isn't to avoid using AI. It's to build AI systems carefully, with diverse data, rigorous testing, and ongoing monitoring. Just like humans need to check their assumptions, AI needs to be checked for bias — and the people building it need to be thoughtful about whose data is represented and whose isn't.
What's Next
In Article #8, we'll explore one of AI's best tricks: how it reuses knowledge from one task to master a completely different one. It's the same reason learning Spanish makes Italian easier — and it's called transfer learning. It's also the secret behind why ChatGPT can do so many different things despite never being specifically trained for most of them.
This is part of the AI from Scratch series — making AI and machine learning understandable for everyone, no PhD required. Follow along on Medium or at netcausal.ai/blog.