#@title Copyright 2024 Google LLC. Double-click for license information.
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
Note: For compatibility with the installed libraries (see Setup), this exercise uses a past version of the Colab runtime.
Colabs¶
Machine Learning Crash Course uses Colaboratory (Colab) notebooks for all programming exercises. Colab is Google's implementation of Jupyter Notebook. For more information about Colabs and how to use them, go to Welcome to Colaboratory.
Addressing Bias and Fairness Issues in ML Models¶
This notebook uses an updated and reconstructed version of the UCI Adult dataset called ACSIncome to:
- Train a model that predicts whether an individual's income is above $50,000 USD based on education, employment, and marital status, as well as sensitive attributes, such as age, sex, and race.
- Compute commonly-identified fairness metrics to evaluate how the model performs across demographic groups.
- Apply a model remediation technique to minimize the difference in error rates between demographic groups.
Learning Objectives¶
- Learn how to evaluate the performance of a trained model for fairness using TensorFlow Model Analysis and Fairness Indicators.
- Address bias and fairness issues in a trained model with TensorFlow Model Remediation.
Intended Use & Considerations¶
This notebook demonstrates how to mitigate undesirable biases in a trained model. In practice, the model and the prediction task in this notebook is not realistic. Rather, the emphasis here is the approach to evaluating the performance of a trained model and minimizing error rates between demographics groups.
With regard to the prediction task, there may circumstances where an individual's income may be used as a determining factor, such as obtaining a loan, acquiring insurance, applying for assistance programs, or possibly to advertise products. But in those cases, such businesses, institutions or organizations would not be looking to infer one's income using a machine learning model; they would instead obtain that information directly from the applicant, if possible, or rely on other signals to decide on the outcome.
With that said, this notebook is for educational purposes only and it is not advisable to use this model in any context outside of this exercise.
Setup¶
To run this exercise, TensorFlow Model Analysis, TensorFlow Model Remediation and Fairness Indicators will need to be installed.
Note: The following cell may take 10-15 minutes to run. If you encounter a prompt to restart the runtime after installing the necessary packages, please do so. Once the runtime has restarted, you can continue working with the subsequent cells without needing to reinstall the packages again.
!pip install \
tensorflow==2.15 \
tensorflow-model-remediation \
fairness-indicators==0.46.0 \
tensorflow-model-analysis==0.46.0 \
tensorflow-data-validation==1.15.1
With the libraries installed, all necessary components can now be imported — including MinDiff for addressing unfair bias in models and Fairness Indicators for evaluating and improving models for fairness concerns.
import pandas as pd
import tensorflow as tf
import tensorflow_model_analysis as tfma
from google.protobuf import text_format
from tensorflow_model_remediation import min_diff
About the ACSIncome Dataset¶
ACSIncome is one of several datasets created by Ding et al. as an alternative to UCI Adult. A few key details about ACSIncome:
- The dataset contains 1,664,500 datapoints pulled from the 2018 United States–wide American Community Survey (ACS) Public Use Microdata Sample (PUMS) data sample.
- All fifty US states and Puerto Rico are represented in this dataset.
- Each row represents a person described by various features, including age, race, and sex, which correspond to protected categories in different domains under US anti-discrimination laws.
- The dataset only includes individuals above 16 years old who worked at least 1 hour per week in the past year and had an income of at least $100 USD.
For more information on the dataset and how it was created to reconstruct UCI Adult, check out the following citations:
Ding, Frances, Moritz Hardt, John Miller, and Ludwig Schmidt. "Retiring adult: New datasets for fair machine learning." Advances in neural information processing systems 34 (2021): 6478-6490.
Sarah Flood, Miriam King, Renae Rodgers, Steven Ruggles, and J. Robert Warren (2020). Integrated Public Use Microdata Series, Current Population Survey: Version 8.0 [dataset]. Minneapolis, MN: IPUMS. https://doi.org/10.18128/D030.V8.0
# Import the dataset
acs_df = pd.read_csv(filepath_or_buffer="https://download.mlcc.google.com/mledu-datasets/acsincome_raw_2018.csv")
# Print five random rows of the pandas DataFrame.
acs_df.sample(5)
Features¶
After importing the dataset, five random samples appear in a table in the output cell. Each sample represents an individual, with each column representing an aspect of the invidiual, such as their age, occupation, place of birth, and so forth.
The following table describes each feature column:
| Feature | Description |
|---|---|
| AGEP | Age |
| COW | Class of worker (government employee, self-employed, private employee) |
| SCHL | Educational attainment (high school diploma, bachelor's degree, doctorate degree) |
| MAR | Marital status |
| OCCP | Occuptation |
| POBP | Place of birth |
| RELP | Relationship to householder (husband or wife, housemate or roommate, nursing home, group home, etc.) |
| WKHP | Usual hours worked per week in the past 12 months |
| SEX | Male or female |
| RAC1P | Recorded detailed race code |
| ST | US state code that represents the individual's location |
| PINCP | Total person's yearly income |
All of these features are represented numerically, though some of them correspond to a coded value. For example, for the COW (Class of worker) feature, 1.0 represents an employee of a private for-profit company or business, or of an individual, for wages, salary, or commissions and 2.0 represents an employee of a private not-for-profit, tax-exempt, or charitable organization. See the supplemental section of Ding et al. and the ACS PUMS 2018 Data Dictionary for the full mapping of codes.
Change Target Value to Binary¶
As stated earlier, the task is to predict whether the annual income of a US working adult is more than $50,000. The PINCP (total person's yearly income) column in the dataset represents the target variable; however, the value will need to be convereted into a binary. For each sample, an individual’s target label will be 1 if PINCP > 50000.0, otherwise 0.
LABEL_KEY = 'PINCP'
LABEL_THRESHOLD = 50000.0
acs_df[LABEL_KEY] = acs_df[LABEL_KEY].apply(
lambda income: 1 if income > LABEL_THRESHOLD else 0)
acs_df.sample(10)
Defining Base Model¶
For the purposes of this exercise, a simple, lightly-tuned keras.Model (using the Functional API for preprocessing) will be created and serve as the base model for this exercise. Note that while the focus for this exercise is the technique involved in addressing fairness concerns, the model architecture — as taught throughout MLCC — would be thoughfully chosen and hyperparameter tuning would be performed before attempting to address any fairness concerns that might arise in the model.
inputs = {}
features = acs_df.copy()
features.pop(LABEL_KEY)
# Instantiate a Keras input node for each column in the dataset.
for name, column in features.items():
if name != LABEL_KEY:
inputs[name] = tf.keras.Input(
shape=(1,), name=name, dtype=tf.float64)
# Stack the inputs as a dictionary and preprocess them.
def stack_dict(inputs, fun=tf.stack):
values = []
for key in sorted(inputs.keys()):
values.append(tf.cast(inputs[key], tf.float64))
return fun(values, axis=-1)
x = stack_dict(inputs, fun=tf.concat)
# Collect the features from the DataFrame, stack them together and normalize
# their values by passing them to the normalization layer.
normalizer = tf.keras.layers.Normalization(axis=-1)
normalizer.adapt(stack_dict(dict(features)))
# Build the main body of the model using a normalization layer, two dense
# rectified-linear layers, and a single output node for classification.
x = normalizer(x)
x = tf.keras.layers.Dense(64, activation='relu')(x)
x = tf.keras.layers.Dense(32, activation='relu')(x)
outputs = tf.keras.layers.Dense(1, activation='sigmoid')(x)
# Put it all together using the Keras Functional API
base_model = tf.keras.Model(inputs, outputs)
Configuring Base Model¶
Since this is a binary classification task, computing the cross-entropy loss between true labels and predicted labels will be sufficient for this exercise.
# Define the metrics used to monitor model performance while training.
METRICS = [
tf.keras.metrics.BinaryAccuracy(name='accuracy'),
tf.keras.metrics.AUC(name='auc'),
]
# Configure the model for training using a stochastic gradient descent
# optimizer, cross-entropy loss between true labels and predicted labels, and
# the metrics defined above to evaluate the base model during training.
base_model.compile(
optimizer='adam',
loss=tf.keras.losses.BinaryCrossentropy(),
metrics=METRICS)
Convert to tf.data.Dataset¶
Most of the exercises throughout MLCC use a pandas DataFrame directly as an input argument for Model.fit. But in this exercise, the dataset must be converted to tf.data.Dataset because that's the requirement for MinDiffModel, which will be introducted later. Fortunately, this conversion is simple to do, thanks to tf.convert_to_tensor.
The following helper function will be useful for preparing the dataset in subsequent code cells:
# Helper function to convert a pandas DataFrame into a tf.Data.dataset object
# necessary for the purposes of this exercise.
def dataframe_to_dataset(dataframe):
dataframe = dataframe.copy()
labels = dataframe.pop(LABEL_KEY)
dataset = tf.data.Dataset.from_tensor_slices(
((dict(dataframe), labels)))
return dataset
Finalize Training Set & Train Base Model¶
At this point, all that remains is splitting the dataset before training the base model.
NOTE: The following cell may take approximately 10—15 minutes to run.
RANDOM_STATE = 200
BATCH_SIZE = 100
EPOCHS = 10
# Use the sample() method in pandas to split the dataset into a training set
# that represents 80% of the original dataset, then convert it to a
# tf.data.Dataset object, and finally train the model using the
# converted training set.
acs_train_df = acs_df.sample(frac=0.8, random_state=RANDOM_STATE)
acs_train_ds = dataframe_to_dataset(acs_train_df)
acs_train_batches = acs_train_ds.batch(BATCH_SIZE)
base_model.fit(acs_train_batches, epochs=EPOCHS)
Evaluate Base Model¶
Consistent with Ding et al., the overall accuracy for the base model should be at around 80% with minimal tuning and a basic model architecture. The following code cell uses the test set to evaluate the performance of the base model:
# Use the indices from the training set to create the test set, which represents
# 20% of the original dataset; then convert it to a tf.data.Dataset object, and
# evaluate the base model using the converted test set.
acs_test_df = acs_df.drop(acs_train_df.index).sample(frac=1.0)
acs_test_ds = dataframe_to_dataset(acs_test_df)
acs_test_batches = acs_test_ds.batch(BATCH_SIZE)
base_model.evaluate(acs_test_batches, batch_size=BATCH_SIZE)
Evaluating for Fairness¶
With the base model trained, now would be a good opportunity to evaluate performance across demographic groups. For ease of analysis, Fairness Indicators will be used to compute fairness metrics across demographic groups and visualize results.
To begin, a column containing all the base model's predictions from the test set will be needed in order to configure Fairness Indicators. model.predict(test_set) will be used to generate the output predictions.
# Generate output predictions using the test set.
base_model_predictions = base_model.predict(
acs_test_batches, batch_size=BATCH_SIZE)
A Note on Sensitive Attributes¶
There are several features in ACSIncome that can be used to evaluate for fairness. For this exercise, the SEX attribute was chosen. This is in part to keep the config terse.
The SEX attribute used by the US Census Bureau surveys to construct ACSIncome was specifically intended to capture an individual's biological sex and not gender. As such, possible ambiguity of these concepts could have tampered with their intended data collection, which could result in some misrepresentation. Ideally, there would be a separate category in such surveys that allow for an individual to express their gender identity (male, female, non-binary, agender, and so forth). But for the purposes of this exercise, the SEX attribute available in ACSIncome will suffice for demonstrating how to perform fairness evaluations.
In practice, the recommended approach is to evaluate across any group that may be negatively impacted by the trained model and is accessible in the dataset, which can include race, ethnicity, working status, educational background, and even the US state that the individual is in.
SENSITIVE_ATTRIBUTE_VALUES = {1.0: "Male", 2.0: "Female"}
SENSITIVE_ATTRIBUTE_KEY = 'SEX'
PREDICTION_KEY = 'PRED'
# Make a copy of the test set, replace sensitive attribute values with
# categorial strings (for ease of visualization), and add predictions
# from the test set to the copied DataFrame as a separate column.
base_model_analysis = acs_test_df.copy()
base_model_analysis[SENSITIVE_ATTRIBUTE_KEY].replace(
SENSITIVE_ATTRIBUTE_VALUES, inplace=True)
base_model_analysis[PREDICTION_KEY] = base_model_predictions
# Show five random examples to ensure that it looks correct.
base_model_analysis.sample(5)
Configure Fairness Indicators¶
With a column of predictions now included in the test set, an eval_config must be created to use Fairness Indicators. This config must include: the names of the prediction and target label columns in the test set, a list of metrics to compute, and the sensitive attribute to designate how the metrics should be computed.
As far as metrics goes, there are several to choose from. For now, ConfusionMatrixPlot will provide everything needed to evaluate for fairness.
NOTE: The following cell may take 5—10 minutes to run.
# Specify Fairness Indicators using eval_config.
eval_config_pbtxt = """
model_specs {
prediction_key: "%s"
label_key: "%s" }
metrics_specs {
metrics { class_name: "ExampleCount" }
metrics { class_name: "BinaryAccuracy" }
metrics { class_name: "AUC" }
metrics { class_name: "ConfusionMatrixPlot" }
metrics {
class_name: "FairnessIndicators"
config: '{"thresholds": [0.50]}'
}
}
slicing_specs {
feature_keys: "%s"
}
slicing_specs {}
""" % (PREDICTION_KEY, LABEL_KEY, SENSITIVE_ATTRIBUTE_KEY)
eval_config = text_format.Parse(eval_config_pbtxt, tfma.EvalConfig())
# Run TensorFlow Model Analysis.
base_model_eval_result = tfma.analyze_raw_data(base_model_analysis, eval_config)
Task 1: Identify Fairness Concerns¶
Run the code cell below and take a moment to explore the results by selecting several metrics to display on the left pane. Individual graphs for each of the metrics selected will appear in the widget to the right.
For each individual graph, you should see a bar that represents the overall performance, followed by bars that correspond to a demographic group based on the sensitive attribute defined in the configuration.
After looking at performance across different metrics,
- Is there a metric that performed equally well across demographic groups?
- Is there a metric that was disproportionate across demographic groups, even though its overall performance seemed promising?
# Render Fairness Indicators.
tfma.addons.fairness.view.widget_view.render_fairness_indicator(
base_model_eval_result)
#@title Double-Click to View a Possible Answer { display-mode: "form" }
# 1. The overall AUC for the base model was around 0.88, with male and female
# groups performing just as well with 0.87 and 0.88, respectfully. A performance
# metric like the AUC would lead one to believe that the model performs well
# across groups.
#
# 2. However, when evaluting with respect to the false negative rate, the
# results show that performance is disappropriately favoring males, with female
# performance is nearly 27% worse than overall baseline. In fact, what the
# graphs reveal is that males perform better than the baseline by around 16%.
The Fairness Issue¶
Using Fairness Indicators to evaluate the base model performance across SEX, the false negative rates (FNR) between Male and Female groups revealed disproportionate outcomes. For context, the FNR represents the percentage of positive examples (individuals earning $50,000 USD or more annually) that are incorrectly predicted as negative (earning less than $50,000 USD annually). This relates to equality of opportunity.
In order for opportunities to be equal within a group, the goal when training the model should be to reduce the gap in the FNR between Male and Female groups. A tool like TensorFlow Model Remediation can be used at training time to intervene and minimize the error rates between the groups.
To set this up, a subset of the dataset must first be created with only positively labeled Female examples (which is referred to as the sensitive group, or the protected class to improve model performance on) and another with only positively labeled Male examples (which is referred to as the non-sensitive group, or any group that is not the protected class).
Task 2: Create Positively-Labeled Subsets¶
Using the training set:
- Add a line of code that creates a subset of the training set only containing positive
Femaleexamples. - Add another line of code that creates a subset of the training set but only containing positive
Maleexamples.
sensitive_group_pos = acs_train_df[ ? ] # Replace the ? with a way to filter
# positively labeled Female examples.
non_sensitive_group_pos = acs_train_df[ ? ] # Replace the ? with a way to filter
# positively labeled Male examples.
#@title Double-Click to View a Possible Answer { display-mode: "form" }
# A pandas DataFrame offers many approaches when it comes to indexing and
# selecting rows. One approach is by using boolean indexing
# (e.g., df[df['col'] == value]) as demonstrated in the following code:
sensitive_group_pos = acs_train_df[
(acs_train_df[SENSITIVE_ATTRIBUTE_KEY] == 2.0) & (acs_train_df[LABEL_KEY] == 1)]
non_sensitive_group_pos = acs_train_df[
(acs_train_df[SENSITIVE_ATTRIBUTE_KEY] == 1.0) & (acs_train_df[LABEL_KEY] == 1)]
# To learn more, visit: https://pandas.pydata.org/docs/user_guide/indexing.html
print(len(sensitive_group_pos),
'positively labeled sensitive group examples')
print(len(non_sensitive_group_pos),
'positively labeled non-sensitive group examples')
Preparing MinDiff Data¶
With the sensitive and non-sensitive subsets defined, attempts can now be made at equalizing the distributions between them using MinDiff. MinDiff is one of the TensorFlow Model Remediation techniques used to balance error rates (or, in this exercise, the FNR) between demographic groups (Male and Female) by penalizing distributional differences during training; hence, MinDiff, or minimizing the differences.
As specified earlier in this exercise, MinDiffModel requires tf.data.Dataset as the input. The following code cell will use that same helper function to convert the subsets into tf.data.Dataset.
# Convert sensitive and non-sensitive subsets into tf.data.Dataset.
MIN_DIFF_BATCH_SIZE = 50
sensitive_group_ds = dataframe_to_dataset(sensitive_group_pos)
non_sensitive_group_ds = dataframe_to_dataset(non_sensitive_group_pos)
# Batch the subsets.
sensitive_group_batches = sensitive_group_ds.batch(
MIN_DIFF_BATCH_SIZE, drop_remainder=True)
non_sensitive_group_batches = non_sensitive_group_ds.batch(
MIN_DIFF_BATCH_SIZE, drop_remainder=True)
Task 3: Packing the Datasets for MinDiff Model¶
Now that the subsets are prepared, they must be packed into a single dataset along with the original training set, which will then be passed along to the MinDiffModel for training.
To advance, add a line of code that uses min_diff.keras.utils.pack_min_diff_data() to pack the original dataset, the sensitive subset, and the nonsensitive subset.
Note: All datasets must be batched before packing them together to prevent errors downstream as a consequence of unintended input tensor shapes.
acs_train_min_diff_ds = min_diff.keras.utils.pack_min_diff_data(
original_dataset = ?, # Replace ? with the original training set
sensitive_group_dataset = ?, # Replace ? with the sensitive subset
nonsensitive_group_dataset = ?) # Replace ? with the non-sensitive subset
#@title Double-Click to View a Possible Answer { display-mode: "form" }
# All you have to do is include each of the three batched tf.data.Datasets into
# the arguments: the training set, the sensitive subset (female) and
# the non-sensitive subset (male).
acs_train_min_diff_ds = min_diff.keras.utils.pack_min_diff_data(
original_dataset = acs_train_batches,
sensitive_group_dataset = sensitive_group_batches,
nonsensitive_group_dataset = non_sensitive_group_batches)
Train MinDiff Model¶
With the data preparation complete, the base model can now be wrapped in the MinDiffModel and compiled. Configuring the MinDiff model is no different from how the base model was configured earlier.
# Wrap the original model in a MinDiffModel.
min_diff_model = min_diff.keras.MinDiffModel(
original_model=base_model,
loss=min_diff.losses.MMDLoss(),
loss_weight=1)
# Compile the model after wrapping the original model.
min_diff_model.compile(
optimizer='adam',
loss=tf.keras.losses.BinaryCrossentropy(from_logits=False),
metrics=METRICS)
As for training, just remember to pass in the packed dataset.
NOTE: The following cell may take approximately 5 minutes to run.
# Train MinDiff model using the packed dataset instead of the training set.
min_diff_model.fit(acs_train_min_diff_ds, epochs=EPOCHS)
Evaluate MinDiff Model¶
When it comes to overall performance, the results of the MinDiff model should be somewhat similar to the base model. Of course, the accuracy or AUC might slightly vary when compared to the base model.
min_diff_model.evaluate(acs_test_batches, batch_size=BATCH_SIZE)
Evaluating for Fairness Using Remediated Model¶
With the MinDiff model now trained, the same steps used to evaluate the base model with Fairness Indicators can be applied to the MinDiff model as well.
Begin by generating the predictions — this time passing the test set as an input argument to the MinDiff model.
# Generate MinDiff output predictions using the test set.
min_diff_model_predictions = min_diff_model.predict(
acs_test_batches, batch_size=BATCH_SIZE)
Same as before, append the predictions as a column onto the DataFrame, then pass it onto Fairness Indicators for evaluation. Note that the configuration used for the base model remains the same for the MinDiff model, which is why it is not being redefined below.
NOTE: The following cell may take 5—10 minutes to run.
# Make a copy of the test set, replace attribute with categorical values,
# and add the MinDiff test set predictions to the copied DataFrame as a separate
# column.
min_diff_model_analysis = acs_test_df.copy()
min_diff_model_analysis[SENSITIVE_ATTRIBUTE_KEY].replace(
SENSITIVE_ATTRIBUTE_VALUES, inplace=True)
min_diff_model_analysis[PREDICTION_KEY] = min_diff_model_predictions
# Run TensorFlow Model Analysis on the MinDiff model.
min_diff_model_eval_result = tfma.analyze_raw_data(
min_diff_model_analysis, eval_config)
Task 4: Reviewing MinDiff Results¶
Run the cell below to visualize the MinDiff results. Then click on AUC and False Negative Rate in the left pane to reveal their respective graphs in the widget to the right.
- Looking at the AUC, and compared to the base model, did model performance increase or decrease as a result of penalizing the model during training for differences in error rates?
- Looking at the false negative rate, are there any noticeable differences between the MinDiff model and the base model?
tfma.addons.fairness.view.widget_view.render_fairness_indicator(
min_diff_model_eval_result)
#@title Double-Click to View a Possible Answer
# 1. Though applying MinDiff may come with some performance tradeoffs in
# comparison to the original task, in this exercise, the MinDiff model performed
# nearly equally as well as the base model, at least in terms of AUC. What this
# is suggesting is that, in this context, MinDiff can be effective while not
# worsening overall performance.
#
# 2. Here is where we see the MinDiff performing better than the base model. Not
# only is the gap in error rates between male and female smaller, but the FNR
# for female went down drastically from nearly 35% all the way to 30%, while the
# overall performance still remains relatively the same (29% in MinDiff vs. 28%)
Considerations¶
This exercise demonstrated an approachable way to train and evalute a model for fairness by expanding on key concepts taught throughout MLCC. The tasks in this notebook provided an example of how a gap in a performance metric between two demographic groups could be a signal that the model may have unfair skews.
However, as discussed in other sections in MLCC, real-world production ML systems are large ecosystems — and the model is just a component of it. That means there are a lot of contributing factors (or confounds) that could affect model performance. Furthermore, there are numerous social and technical processes, known and unknown, that underpin and surround ML and AI technologies. Achieving equality on a particular metric alone does not ensure that the model itself is overall fair.
In practice, there is an assumption that the features used to compare performance across sensitive attributes are readily available, or can be accurately inferred. In actuality, datasets do not often include sensitive attributes, and it is generally not a good idea to impute them.