Large language models (LLMs) are increasingly used to translate technical explainable AI (XAI) artefacts such as SHAP into explanatory narratives (XAINs) for lay users.
In high-risk domains, these explanations are the main resource through which individuals understand and contest adverse decisions made by automated systems.
LLMs reproduce social bias, even from proxy variables alone. Fairness in XAINs was previously unstudied.
Since XAINs are bound to a ground-truth artefact, the concept of fairness not only concerns what the narrative says and how, but also whether it is equally faithful across social groups.
Proposed Framework
Recognizing this, we introduce fairness in XAINs as a new field of research and propose a framework that generates narratives under three conditions:
- Protected attribute disclosed
- Withheld
- Counterfactually swapped
We apply this approach to adversely classified instances from the credit scoring, education, and employment domains using a toy registry of 10+ datasets