Exercises - Optimal Control

Finite-horizon Riccati equation
Consider
$A= \begin{bmatrix} 0&1\\ 0&0 \end{bmatrix}, \qquad B= \begin{bmatrix} 0\\ 1 \end{bmatrix},$
(1)
with
$Q= \begin{bmatrix} q_1&0\\ 0&q_2 \end{bmatrix}, \qquad R=r.$
(2)
Assume
$S(t)= \begin{bmatrix} s_{11}&s_{12}\\ s_{12}&s_{22} \end{bmatrix}.$
(3)
(a) Derive the matrix Riccati differential equation.
(b) Derive the three independent scalar Riccati equations.
(c) State the terminal conditions.
Scalar finite-horizon LQR
Consider
$\dot{x}=ax+bu,$
(4)
with
$J= \frac12x^2(T)S_f + \frac12 \int_0^T (qx^2+ru^2)\,dt.$
(5)
Derive
(a) the Hamiltonian,
(b) the optimal control,
(c) the Riccati equation,
(d) the terminal condition,
(e) the optimal feedback law.
Costate decomposition
Assume
$\lambda=Sx+\beta.$
(6)
Starting from the state and costate equations,
(a) derive the Riccati equation,
(b) derive the differential equation for $\beta$ ,
(c) show that if
$\beta(T)=0,$
(7)
then
$\beta(t)\equiv0.$
(8)
Infinite-horizon scalar LQR
For
$\dot{x}=ax+bu,$
(9)
derive the algebraic Riccati equation.
(a) Solve analytically for both roots.
(b) Determine which solution gives a stabilizing controller.
(c) Compute the optimal feedback gain.
Hamiltonian matrix
Consider
$F= \begin{bmatrix} a&-b^2/r\\ -q&-a \end{bmatrix}.$
(10)
(a) Derive the characteristic polynomial.
(b) Compute the eigenvalues.
(c) Determine when they are real or imaginary.
(d) Explain the implication for closed-loop stability.
Time-varying LQR
Assume
$A(t), \quad B(t), \quad Q(t), \quad R(t)$
(11)
are time dependent.
Derive
(a) the optimal control law,
(b) the Riccati differential equation,
(c) the terminal condition,
(d) the closed-loop state equation.
Hamiltonian conservation
Starting from the canonical equations,
derive
$\frac{dH^*}{dt} = \frac{\partial H}{\partial t}.$
(12)
Then show that the optimal Hamiltonian is constant whenever the Hamiltonian has no explicit time dependence.
Symmetry of the Riccati equation
Assume
$S_f=S_f^{\mathsf T}, \qquad Q=Q^{\mathsf T}, \qquad R=R^{\mathsf T}.$
(13)
Show that the Riccati differential equation preserves symmetry by proving that
$S(t)=S^{\mathsf T}(t)$
(14)
for all $t$ .
Optimal feedback law
For the double integrator,
$\dot{x}_1=x_2, \qquad \dot{x}_2=u,$
(15)
with quadratic performance index,
(a) derive the optimal feedback law,
(b) write the closed-loop dynamics,
(c) express the gain matrix in terms of the Riccati solution,
(d) explain why the resulting controller is time varying over a finite horizon.
Backward–forward verification
Implement the finite-horizon LQR algorithm for any controllable second-order system.
(a) Integrate the Riccati equation backward.
(b) Simulate the closed-loop system forward.
(c) Verify numerically that
$\dot{\lambda} + Qx + A^{\mathsf T}\lambda \approx0,$
(16)
$Ru + B^{\mathsf T}\lambda \approx0,$
(17)
and
$\lambda(T)-S_fx(T) \approx0.$
(18)
Report the maximum residual in each equation.