A Survey of Multi-Objective Sequential Decision-Making
Diederik Marijn Roijers, Peter Vamplew, Shimon Whiteson, Richard Dazeley
Paper details
- Journal
- Journal of Artificial Intelligence Research
- Published
- 2013.10.1
- Citations
- 877
- References
- 145
- Access
- Open Access
- Fields
- Computer Science, Mathematics
tlooto Summary
This article surveys algorithms designed for sequential decision-making problems with multiple objectives and proposes a taxonomy that classifies multi-objective methods according to the applicable scenario, the nature of the scalarization function, and the type of policies considered.
Abstract
Sequential decision-making problems with multiple objectives arise naturally in practice and pose unique challenges for research in decision-theoretic planning and learning, which has largely focused on single-objective settings. This article surveys algorithms designed for sequential decision-making problems with multiple objectives. Though there is a growing body of literature on this subject, little of it makes explicit under what circumstances special methods are needed to solve multi-objective problems. Therefore, we identify three distinct scenarios in which converting such a problem to a single-objective one is impossible, infeasible, or undesirable. Furthermore, we propose a taxonomy that classifies multi-objective methods according to the applicable scenario, the nature of the scalarization function (which projects multi-objective values to scalar ones), and the type of policies considered. We show how these factors determine the nature of an optimal solution, which can be a single policy, a convex hull, or a Pareto front. Using this taxonomy, we survey the literature on multi-objective methods for planning and learning. Finally, we discuss key applications of such methods and outline opportunities for future work.
Citation format
ROIJERS, Diederik Marijn, et al. A survey of multi-objective sequential decision-making [preprint]. arXiv, 2013. arXiv:1402.0590.