Every technology that makes decisions on behalf of people eventually runs into the same problem: the people who built it made choices they didn't realize were ethical.
The camera that identifies faces less accurately for darker skin tones wasn't designed to discriminate — it was trained on datasets that didn't represent the full range of human faces. The algorithm that scored Black patients as lower-risk than equally ill white patients wasn't designed to be racist — it was optimized for a cost metric that happened to proxy race without naming it.
These failures share a pattern. Technical decisions made without ethical awareness accumulate silently until they surface in a court case, a news investigation, or a patient outcome. By the time anyone calls it an ethics problem, the harm is already done.
This module gives you a framework for seeing those decisions before they become problems — and the vocabulary to demand accountability when they don't.
In 2016, a team at MIT launched an experiment called the Moral Machine. The premise was simple: a self-driving car's brakes have failed. It will hit someone. Who should it be?
Visitors were shown thousands of variations — passengers versus pedestrians, children versus the elderly, people following traffic rules versus those who weren't. Two million participants from 233 countries responded. The data revealed something uncomfortable: there was no consensus. Eastern and Western countries disagreed sharply on whose life to prioritize. Countries with stronger rule-of-law traditions were more likely to spare law-abiding pedestrians. The variation wasn't statistical noise. It was evidence that different communities held genuinely different moral frameworks — and that those differences mattered.
The results were published in Nature in 2018. The engineers building autonomous vehicles didn't wait for them. They had deadlines. They set default parameters. Assigned decision weights. Wrote config files. Most of them would have described that work as systems engineering, not ethics.
That gap — between "I'm making a technical decision" and "I'm making an ethical decision" — is where most harm in deployed AI systems originates. Not from malice. From the failure to recognize that when a system makes decisions about people, the person who designs the system makes those decisions first.
You are the engineer who wrote the config file. You did it six months ago. You forgot about it. Today someone is asking why the system made the choices it made — and the answer lives in decisions you encoded without realizing they were moral ones.
This module asks you to hold that responsibility seriously.
Every AI system that makes decisions encodes a value system. Not always intentionally — but always inevitably.
Amazon built a hiring tool trained on a decade of resumes. Most successful hires were men. The model learned to penalize resumes that included the word "women's" — as in, women's chess club, women's debate team. Amazon scrapped it in 2018 after internal audits surfaced the pattern, but it had been running in production for months. The engineers didn't set out to discriminate. They fed the model historical data without asking what that history contained.
Facebook optimized its recommendation algorithm for engagement. Internal researchers found the system drove users toward increasingly extreme content because outrage generated more clicks than nuance. That wasn't a bug — it was the intended behavior of a system doing exactly what it was built to do.
Values enter AI systems through three entry points.
Data — what was collected, excluded, and how it was labeled. Historical data contains historical bias. Hiring data captures who got hired, not who would have performed best. Policing data captures where officers patrolled, not where crime actually occurred.
Optimization targets — what the model is told to get better at. A model optimized for engagement will surface what provokes, not what informs. A model minimizing false negatives in cancer screening accepts more false positives. Every optimization tradeoff encodes a priority that someone chose.
Deployment decisions — who sees the output, how much weight it carries, whether a human reviews it, under what conditions it can be overridden. These choices are made in product meetings under deadline pressure — not in ethics seminars with unlimited reflection time.
The values were designed in — often by people who didn't realize they were making ethical decisions at all. Your job is to learn to see them before they cause harm, not after.
When you encounter an AI system making consequential decisions — about credit, healthcare, employment, bail, or risk — three questions help you read it with precision. You'll use all three in the lab.
Trace the decision back to the three doors: data, optimization target, deployment design. A recidivism model trained on arrest data inherits policing biases, not the reality of criminal behavior — that's a data problem. A content algorithm surfacing extreme material is performing as designed — that's an optimization problem. A medical diagnosis tool that gives physicians a score without explaining its reasoning concentrates accountability risk at the deployment level.
Ethical problems in AI are almost never technical inevitabilities. They are choices made by specific people with specific incentives at specific moments. Identifying who chose, when, and under what constraints is how you separate fixable problems from structural ones.
Not only what went wrong — but what oversight mechanism would have caught it, and who should have built that mechanism before deployment. This is where analysis becomes action.
These questions won't always give clean answers. But they give you a starting point that's more useful than outrage or resignation.