Algorithmic Bias

Algorithmic bias is when an automated decision process produces systematically unfair outcomes for some groups of people.

Bias can enter at data collection, feature selection, training, or deployment. If a score consistently favors one demographic, it often reflects hidden assumptions, historical inequity, or a shortcut in the pipeline. High-stakes decisions now run on these scores: loan approvals, medical diagnoses, job shortlists, parole recommendations.

Biased scores deny credit to qualified borrowers, misclassify patients, drop capable candidates, or extend prison time. Harm compounds as the same scores feed later decisions and public trust drops.

Fixes include better data practices, transparent design, and audits across subpopulations. Training data should match the people the system will score. Metrics should track fairness next to accuracy. Policymakers are starting to require bias assessments before deployment.

A biased training set, a proxy feature that encodes race or gender, a threshold tuned on one group, or a deployment context the data never saw: any of those can skew loan, diagnosis, hiring, or parole scores. Qualified borrowers lose credit. Patients get the wrong flag. Candidates never reach a human. Sentences get longer. Later systems train on those outcomes and repeat them.

Data audits, subgroup metrics, and pre-deployment bias assessments are the counter. Fairness has to be measured, not assumed from overall accuracy. Gender Shades (2018) showed commercial face systems failing more on darker-skinned women. Bias is measurable, not just a slogan.