Exercises - Optimal Control

Direct shooting with a polynomial control parameterization
Consider
$\dot{x}(t) = -x(t)+u(t), \qquad x(0)=x_0,$
(1)
with fixed final time $T$ , terminal constraint
$x(T)=x_f,$
(2)
and performance index
$J = \frac{1}{2} \int_0^T \left( x^2(t)+\rho u^2(t) \right) \,\mathrm{d}t.$
(3)
Approximate the control by
$u(t) = c_0+c_1t+c_2t^2.$
(4)
(a) Identify the finite-dimensional decision vector.
(b) Write the state obtained by integrating the dynamics for a given coefficient vector.
(c) Express the terminal constraint as a nonlinear algebraic constraint on $c_0,c_1,c_2$ .
(d) Introduce an accumulated-cost state and write the complete direct-shooting nonlinear program.
Forward and backward Euler stability
Apply forward Euler and backward Euler to
$\dot{x} = -5x, \qquad x(0)=1.$
(5)
(a) Derive the update equation and amplification factor for each method.
(b) Derive the forward-Euler stability restriction on $h$ .
(c) Show that backward Euler is stable for every $h>0$ .
(d) For a uniform mesh, derive explicit expressions for $x_k^{\mathrm{FE}}$ and $x_k^{\mathrm{BE}}$ .
(e) Compare both discrete solutions with
$x(t_k)=e^{-5t_k}.$
(6)
Generalized $\theta$ -method
Consider the convention
$x_{k+1} = x_k + h \left[ \theta f(t_k,x_k) + (1-\theta)f(t_{k+1},x_{k+1}) \right].$
(7)
(a) Identify the values of $\theta$ corresponding to forward Euler, backward Euler, and the trapezoidal method.
(b) Determine which value of $\theta$ produces an explicit method for a general nonlinear vector field.
(c) For the test equation
$\dot{x}=\lambda x, \qquad z=h\lambda,$
(8)
derive the stability function
$R_\theta(z).$
(9)
(d) Recover the stability functions of the three special cases.
(e) Compare the limits of the backward-Euler and trapezoidal stability functions as $z\rightarrow-\infty$ .
Runge–Kutta integration of controlled dynamics
Consider one mesh interval for
$\dot{x} = -x^3+u,$
(10)
where the control is constant on the interval:
$u(t)=u_k, \qquad t\in[t_k,t_{k+1}].$
(11)
(a) Write the four classical RK4 stage equations.
(b) Derive the RK4 update for $x_{k+1}$ .
(c) State the number of vector-field evaluations required per interval.
(d) Explain why RK4 is a one-step method despite using four stages.
(e) State its expected global order of accuracy.
Euler direct-collocation transcription
Transcribe
$\min_{x(\cdot),u(\cdot)} \; J = \frac{1}{2} \int_0^T \left( x^2+u^2 \right) \,\mathrm{d}t$
(12)
subject to
$\dot{x} = -x^3+u, \qquad x(0)=a, \qquad x(T)=b,$
(13)
using forward-Euler collocation on $K$ uniform intervals.
(a) Define the mesh and step size.
(b) Define the complete NLP decision vector.
(c) Derive all Euler defect constraints.
(d) Derive the left-rectangle discrete objective.
(e) Write the two endpoint constraints.
(f) Determine the number of decision variables, equality constraints, and remaining degrees of freedom.
Global residual equations and their Jacobian
For the scalar nonlinear initial-value problem
$\dot{x}=f(t,x), \qquad x(t_0)=x_0,$
(14)
on a mesh with $K$ intervals:
(a) Write all forward-Euler equations as one simultaneous algebraic residual system.
(b) Derive the nonzero Jacobian entries of a generic forward-Euler residual.
(c) Repeat parts (a) and (b) for backward Euler.
(d) Explain mathematically why the backward-Euler residual is implicit.
(e) State how Newton’s method would be applied to the global backward-Euler residual system.
Sparse Jacobian structure
For the scalar Euler-collocation problem in Exercise 5, use
$\boldsymbol{z} = \begin{bmatrix} x_0 & \cdots & x_K & u_0 & \cdots & u_{K-1} \end{bmatrix}^{\mathsf T}.$
(15)
(a) Derive the three nonzero partial derivatives in one defect row.
(b) Show that the complete equality-constraint Jacobian has dimensions
$(K+2)\times(2K+1).$
(16)
(c) Show that it has $3K+2$ nonzero entries.
(d) Derive its exact density.
(e) Compute the density for $K=10$ .
(f) Prove that the density is asymptotically
$\mathcal{O}\!\left(\frac{1}{K}\right).$
(17)
(g) Extend the variable count, constraint count, and maximum local nonzero count to $n_x$ states and $n_u$ controls.
Time normalization and free final time
Map
$t\in[t_0,t_f]$
(18)
to
$\tau\in[-1,1].$
(19)
(a) Derive the forward and inverse affine transformations.
(b) Derive $\mathrm{d}t/\mathrm{d}\tau$ .
(c) Transform
$\frac{\mathrm{d}\boldsymbol{x}}{\mathrm{d}t} = \boldsymbol{f}(t,\boldsymbol{x},\boldsymbol{u}).$
(20)
(d) Transform the Bolza objective
$J = \Phi + \int_{t_0}^{t_f} L(t,\boldsymbol{x},\boldsymbol{u}) \,\mathrm{d}t.$
(21)
(e) On a normalized mesh, derive the forward-Euler defect constraints.
(f) When $t_f$ is free, identify every part of the NLP in which it appears.
(g) Explain why the $t_f$ column is globally coupled while the remaining defect Jacobian is still sparse.
Piecewise-constant shooting with an unknown switching time
Consider
$\dot{x}=u, \qquad x(0)=0, \qquad x(T)=x_f,$
(22)
and parameterize the control as
$u(t) = \begin{cases} u_1, & 0\leq t<t_s,\\ u_2, & t_s\leq t\leq T, \end{cases}$
(23)
where $u_1$ , $u_2$ , and $t_s$ are decision variables.
(a) Integrate the dynamics analytically on both intervals.
(b) Derive the continuity condition at $t_s$ .
(c) Express the terminal constraint as an algebraic equation in $u_1,u_2,t_s$ .
(d) For
$J = \frac{1}{2} \int_0^T u^2(t)\,\mathrm{d}t,$
(24)
derive the finite-dimensional objective.
(e) Write the resulting direct-shooting NLP, including bounds
$u_{\min}\leq u_1,u_2\leq u_{\max}, \qquad 0\leq t_s\leq T.$
(25)
Analytical mesh-refinement study
Apply forward Euler to
$\dot{x}=-x, \qquad x(0)=1,$
(26)
on $[0,T]$ using $K$ uniform intervals.
(a) Show that
$x_K = \left( 1-\frac{T}{K} \right)^K.$
(27)
(b) Show that
$\lim_{K\rightarrow\infty}x_K=e^{-T}.$
(28)
(c) Use a logarithmic expansion to obtain the leading-order terminal error.
(d) Show that the terminal error is first order in $h=T/K$ .
(e) Derive the error ratio expected when the mesh size is doubled.
(f) State how this result can be used to verify a direct-collocation implementation.