Counterfactuals in Minds and Machines

"Had I started writing the paper earlier, I would have made it to the conference deadline." The ability to think about how things could have turned out differently from how they did in reality, often referred to as counterfactual reasoning, is a fundamental aspect of human cognition. Is counterfactual reasoning a human capacity that machines cannot have? Surprisingly, recent advances at the interface of psychology, causality and machine learning have demonstrated that it is possible to build machines that perform and benefit from counterfactual reasoning, in a way similarly as humans do. This tutorial aims to introduce students and researchers to the current state of research in this area, providing them both with insights from cognitive science (🧠) and with an overview of relevant technical advances in machine learning (🤖).

In the first part of the tutorial, we look into counterfactual reasoning from the perspective of psychology. We first discuss the functional roles of counterfactuals in human cognition, the factors that affect which counterfactuals we think about, and how we make counterfactual inferences using mental simulation. Then, we shift focus to machine learning. We start by introducing structural causal models (SCMs), a mathematical framework that allows us to formalize probabilistic counterfactual reasoning. We then discuss the identification of counterfactual quantities and present a range of areas where counterfactual reasoning has been successfully applied in machine learning. We conclude with an overview of recent advances at the intersection of counterfactual reasoning and large language models (LLMs) and a technical deep dive into using SCMs to enable counterfactual generation in LLMs.