Nonlinear Optimal Control Beyond the Hamilton-Jacobi-Bellman Equation

Within the framework of the Linear Quadratic Regulator it is well known that <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">“all roads lead to the Algebraic Riccati Equation”</i>. The deceptive veil of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">linearity</i> is torn herein by observing that such an algebraic equation generalizes to three different conditions in the nonlinear setting. The first one is obviously the celebrated Hamilton-Jacobi-Bellman partial differential equation arising by relying on Dynamic Programming arguments. However, it is shown that the optimal solution can be equivalently constructed also by characterizing a specific <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">invariant manifold</i> or an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">invariant distribution</i> of the associated Hamiltonian vector field. While these three strategies reduce to the Algebraic Riccati Equation in the linear case, they instead lead to distinct conditions in the nonlinear setting, with the latter two yielding quasi-linear and linear partial differential equations, respectively, in place of the quadratic Hamilton-Jacobi-Bellman equation.