You have no items in your shopping cart.

Programming a "Fairer" System: Assessing Bias in Enterprise AI Products (A)
Gentile, Mary; Krasniansky, Adriana Case OB-1340 / Published December 9, 2020 / 5 pages.
Format Price Quantity Select
PDF Download
$6.95
EPUB Download
$6.95
Printed Black & White Copy
$7.25

Product Overview

This case is part of the Giving Voice to Values (GVV) curriculum. To see other material in the GVV curriculum, please visit http://store.darden.virginia.edu/giving-voice-to-values. In this case, Timothy Brennan is the founder and CEO of technology company Northpointe, Inc. (Northpointe), and the creator of its flagship software program, COMPAS, an artificially intelligent software tool for US court systems that predicts a defendant's likelihood to reoffend and informs bail, parole, and probation sentencing decisions. Brennan originally created COMPAS in order to standardize decision-making within the criminal justice system and to reduce the likelihood of human error or bias impacting court rulings. However, years after COMPAS's public release and widespread adoption within the US court systems, an investigative journalism report claims that COMPAS is more likely to mislabel Black defendants as higher risk and White defendants as lower risk of recidivism. To complicate the matter, any coding adjustments that Northpointe would make to uncover or address the bias-causing programming might reduce the software's performance or reveal sensitive operational information to competitors. In this A case, Brennan's challenge is to organize a response to investigate bias within the COMPAS software, while still protecting the complexity and intellectual property of the product. In the B case, students read a synopsis of Brennan's actual response and review its implications for Northpointe and the US criminal justice system. They are encouraged to consider how Brennan could have responded more creatively and constructively.


Learning Objectives

This case is designed to help students learn how to (1) develop, implement, and engage with new and evolving technologies in a responsible manner; (2) organize and execute internal processes for continuous technological assessment and improvement; (3) acknowledge and respond to public-interest concerns, audits, or allegations; and (4) recognize relationships between day-to-day business operations and widespread social outcomes.

  • Videos List

  • Overview

    This case is part of the Giving Voice to Values (GVV) curriculum. To see other material in the GVV curriculum, please visit http://store.darden.virginia.edu/giving-voice-to-values. In this case, Timothy Brennan is the founder and CEO of technology company Northpointe, Inc. (Northpointe), and the creator of its flagship software program, COMPAS, an artificially intelligent software tool for US court systems that predicts a defendant's likelihood to reoffend and informs bail, parole, and probation sentencing decisions. Brennan originally created COMPAS in order to standardize decision-making within the criminal justice system and to reduce the likelihood of human error or bias impacting court rulings. However, years after COMPAS's public release and widespread adoption within the US court systems, an investigative journalism report claims that COMPAS is more likely to mislabel Black defendants as higher risk and White defendants as lower risk of recidivism. To complicate the matter, any coding adjustments that Northpointe would make to uncover or address the bias-causing programming might reduce the software's performance or reveal sensitive operational information to competitors. In this A case, Brennan's challenge is to organize a response to investigate bias within the COMPAS software, while still protecting the complexity and intellectual property of the product. In the B case, students read a synopsis of Brennan's actual response and review its implications for Northpointe and the US criminal justice system. They are encouraged to consider how Brennan could have responded more creatively and constructively.

  • Learning Objectives

    Learning Objectives

    This case is designed to help students learn how to (1) develop, implement, and engage with new and evolving technologies in a responsible manner; (2) organize and execute internal processes for continuous technological assessment and improvement; (3) acknowledge and respond to public-interest concerns, audits, or allegations; and (4) recognize relationships between day-to-day business operations and widespread social outcomes.