README.md 3.82 KB
Newer Older
ALVES Guilherme's avatar
ALVES Guilherme committed
1 2 3 4
# FixOut

FixOut addresses fairness issues of ML models based on decision outcomes, and shows how the simple idea of “feature dropout” followed by an “ensemble approach” can improve model fairness.

5
## Description
ALVES Guilherme's avatar
ALVES Guilherme committed
6

7
Originally, it was conceived to tackle process fairness of ML Models based on decision outcomes (see LimeOut [1]). For that it uses an explanation method to assess a model’s reliance on salient or sensitive features, that is integrated in a human-centered workflow: given a classifier M, a dataset D, a set F of sensitive features and an explanation method of choice, FIXOut outputs a competitive classifier M’ that improves in process fairness as well as in other fairness metrics.
ALVES Guilherme's avatar
ALVES Guilherme committed
8

9
Explainers available:
ALVES Guilherme's avatar
ALVES Guilherme committed
10
* LIME
ALVES Guilherme's avatar
ALVES Guilherme committed
11
* SHAP
ALVES Guilherme's avatar
ALVES Guilherme committed
12

13
## Installation
ALVES Guilherme's avatar
ALVES Guilherme committed
14

15 16 17
FixOut works on Python >= **3.6**.  
There is no proper installer at the moment, since the module is under construction.  
If you are on Linux, then install the `swig` package. For debian-based distributions:
ALVES Guilherme's avatar
ALVES Guilherme committed
18

19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56
```shell
$ sudo apt install swig
```

For all operating systems, install requirements:

```shell
$ pip install -r requirements.txt
```

## Usage example

### For tabular data

For a more complete example, see [examples/experimenter.py](examples/experimenter.py)

```python
from fixout.lime_tabular_global import TabularExplainer
from fixout.core_tabular import EnsembleOutTabular

lr = LogisticRegression()
model = make_pipeline(ct, lr)
model.fit(X_train, y_train)
print("Original score:", model.score(X_test, y_test))

# explain the original model
explainer_original = TabularExplainer(model.predict_proba, X_train,
                                      categorical_features=[1, 3, 5, 6, 7, 8, 9, 13]) # categorical features indexes
explainer_original.global_explanation(n_samples=500)

# print the explanation
for i, contrib in explainer_original.get_top_k(k=10) :
    print(features[i], '\t', contrib)

# make an ensemble
ensemble = EnsembleOutTabular(lr, ct, sensitive_features=(5, 8, 9))  # features which we want to lower the contribution
ensemble.fit(X_train, y_train)
print("Ensemble score:", ensemble.score(X_test, y_test))
ALVES Guilherme's avatar
ALVES Guilherme committed
57

58 59 60 61
# explain the ensemble
explainer_ensemble = TabularExplainer(ensemble.predict_proba, X_train,
                                      categorical_features=[1, 3, 5, 6, 7, 8, 9, 13])
explainer_ensemble.global_explanation(n_samples=500)
ALVES Guilherme's avatar
ALVES Guilherme committed
62

63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108
for i, contrib in explainer_ensemble.get_top_k(k=10) :
    print(features[i], '\t', contrib)
```

### For textual data

For a more complete example, see [examples/test_text.py](examples/test_text.py)

```python
from fixout.core_text import EnsembleOutText
from fixout.lime_text_global import TextExplainer

# creating a model pipeline and training it
vectorizer = TfidfVectorizer(lowercase=True)
lr = LogisticRegression()
model = make_pipeline(vectorizer, lr)
model.fit(X_train, y_train)

# evaluating our model
print("Accuracy:", model.score(X_test, y_test))

# explaining our model
explainer = TextExplainer(model.predict_proba)
explainer.global_explanation(X_test, n_samples=500)

for word, contrib in explainer.get_top_k(k=10) :
    print(word, '\t', contrib)

# correcting fairness if necessary
ensemble = EnsembleOutText(model, sensitive_words=[["host", "symposium"], ["desks", "edu"]])
ensemble.fit(X_train, y_train)
print("Ensemble accuracy:", ensemble.score(X_test, y_test))

# explaining the ensemble model
ensemble_explainer = TextExplainer(ensemble.predict_proba)
ensemble_explainer.global_explanation(X_test, n_samples=250)

for word, contrib in explainer.get_top_k(k=10) :
    print(word, '\t', contrib)
```


## References 
[1] Vaishnavi Bhargava, Miguel Couceiro, Amedeo Napoli. LimeOut: An Ensemble Approach To Improve Process Fairness. XKDD Workshop 2020. ⟨hal-02864059v2⟩

[2] Guilherme Alves, Vaishnavi Bhargava, Miguel Couceiro, Amedeo Napoli. Making ML models fairer through explanations: the case of LimeOut. AIST 2020. ⟨hal-02864059v5⟩