Exercises - Optimal Control

Augmented performance index
Consider the optimal control problem
$\min \; J = \frac{1}{2} \int_0^T \left( x^2+u^2 \right)\,\mathrm{d}t,$
(1)
subject to
$\dot{x}=ax+bu, \qquad x(0)=x_0, \qquad x(T)=x_f.$
(2)
(a) Construct the augmented functional.
(b) Define the Hamiltonian.
(c) Identify the state, control, costate, and endpoint multiplier.
Pontryagin necessary conditions
For
$\dot{x}=x+u,$
(3)
with
$J= \frac12 \int_0^T \left( x^2+u^2 \right)\,\mathrm{d}t,$
(4)
derive
(a) the Hamiltonian,
(b) the state equation,
(c) the costate equation,
(d) the stationarity condition,
(e) the complete two-point boundary-value problem.
Linear-quadratic finite-horizon problem
Consider
$\dot{x}=-2x+u, \qquad x(0)=1, \qquad x(T)=0,$
(5)
with
$J= \frac12 \int_0^T u^2\,dt.$
(6)
Derive all necessary conditions and eliminate the control to obtain coupled state-costate differential equations.
Double integrator
Consider
$\dot{x}_1=x_2, \qquad \dot{x}_2=u,$
(7)
with
$J= \frac12 \int_0^T u^2\,dt.$
(8)
(a) Construct the Hamiltonian.
(b) Derive the costate equations.
(c) Derive the optimal control.
(d) Write the complete boundary-value problem.
Free terminal state
Consider
$J= \int_0^T \left( \frac12u^2+cx \right)\,dt,$
(9)
subject to
$\dot{x}=u, \qquad x(0)=0,$
(10)
where $x(T)$ is free.
Derive
(a) the Hamiltonian,
(b) all necessary conditions,
(c) the terminal transversality condition,
(d) the complete boundary-value problem.
Terminal Mayer cost
Consider
$J= \frac12 \left( x(T)-x_d \right)^2 + \frac12 \int_0^T u^2\,dt,$
(11)
subject to
$\dot{x}=u, \qquad x(0)=x_0.$
(12)
Derive
(a) the Hamiltonian,
(b) the costate equation,
(c) the stationarity condition,
(d) the terminal costate condition,
(e) the complete two-point boundary-value problem.
Bounded control
Suppose
$u\in[-2,2],$
(13)
and
$H= \lambda u + \frac12u^2.$
(14)
(a) Determine the unconstrained minimizing control.
(b) Determine the constrained minimizing control.
(c) Sketch the optimal control as a function of $\lambda$ .
Free final time
Consider
$J= \int_0^{t_f} \left( 1+\frac12u^2 \right)\,dt,$
(15)
subject to
$\dot{x}=u, \qquad x(0)=0, \qquad x(t_f)=x_f,$
(16)
where $t_f$ is free.
Derive
(a) the Hamiltonian,
(b) all necessary conditions,
(c) the free-final-time condition,
(d) the resulting boundary-value problem.
General endpoint constraints
Let
$\boldsymbol{\phi} = C_0\boldsymbol{x}(t_0) + C_f\boldsymbol{x}(t_f) + \boldsymbol{r}t_f - \boldsymbol{d}.$
(17)
(a) Compute
$\delta\boldsymbol{\phi}.$
(18)
(b) Derive
$\delta \left( -\boldsymbol{\nu}^{\mathsf T} \boldsymbol{\phi} \right).$
(19)
(c) Identify the coefficients multiplying
$\delta\boldsymbol{x}(t_0), \quad \delta\boldsymbol{x}(t_f), \quad \delta t_f.$
(20)
Complete derivation of Pontryagin’s conditions
Starting from the augmented functional
$J_a = \Phi - \boldsymbol{\nu}^{\mathsf T} \boldsymbol{\phi} + \int_{t_0}^{t_f} \left( H - \boldsymbol{\lambda}^{\mathsf T} \dot{\boldsymbol{x}} \right) dt,$
(21)
derive the complete first variation.
Use integration by parts to derive
(a) the state equations,
(b) the costate equations,
(c) the stationarity condition,
(d) the endpoint conditions,
(e) the free-final-time transversality condition.