tlooto

A Survey of Multi-Objective Sequential Decision-Making

Diederik Marijn Roijers, Peter Vamplew, Shimon Whiteson, Richard Dazeley

2013.10.1Journal of Artificial Intelligence Research

Paper details

Journal
Journal of Artificial Intelligence Research
Published
2013.10.1
Citations
877
References
145
Access
Open Access
Fields
Computer Science, Mathematics

tlooto Summary

This article surveys algorithms designed for sequential decision-making problems with multiple objectives and proposes a taxonomy that classifies multi-objective methods according to the applicable scenario, the nature of the scalarization function, and the type of policies considered.

Abstract

Sequential decision-making problems with multiple objectives arise naturally in practice and pose unique challenges for research in decision-theoretic planning and learning, which has largely focused on single-objective settings. This article surveys algorithms designed for sequential decision-making problems with multiple objectives. Though there is a growing body of literature on this subject, little of it makes explicit under what circumstances special methods are needed to solve multi-objective problems. Therefore, we identify three distinct scenarios in which converting such a problem to a single-objective one is impossible, infeasible, or undesirable. Furthermore, we propose a taxonomy that classifies multi-objective methods according to the applicable scenario, the nature of the scalarization function (which projects multi-objective values to scalar ones), and the type of policies considered. We show how these factors determine the nature of an optimal solution, which can be a single policy, a convex hull, or a Pareto front. Using this taxonomy, we survey the literature on multi-objective methods for planning and learning. Finally, we discuss key applications of such methods and outline opportunities for future work.

Citation format

ROIJERS, Diederik Marijn, et al. A survey of multi-objective sequential decision-making [preprint]. arXiv, 2013. arXiv:1402.0590.