A Review of Bayesian Networks and Structure Learning

This article reviews the topic of Bayesian networks. A Bayesian network  is a factorisation of a probability distribution along a directed acyclic graph. The relation between graphical d-separation and independence is described. A short article by Arthur Cayley (1853) [7] is discussed, which laid ideas later used in Bayesian networks: factorisation, the noisy `or' gate, applications of algebraic geometry to Bayesian networks. The ideas behind Pearl's intervention calculus when the DAG represents a causal dependence structure; the relation between the work of Cayley and Pearl is commented on. Most of the discussion is about structure learning, outlining the two main approaches; search and score versus constraint based. Constraint based algorithms often rely on the assumption of faithfulness, that the data to which the algorithm is applied is generated from distributions satisfying a faithfulness assumption where graphical d- separation and independence are equivalent. The article presents some considerations for constraint based algorithms based on recent data analysis, indicating a variety of situations where the faithfulness assumption does not hold.