Exercises - Optimal Control

LG nonlinear-programming Lagrangian and KKT equations
Consider the simplified Mayer problem on $[-1,1]$ ,
$\min_{\boldsymbol{x}(\cdot),\boldsymbol{u}(\cdot)} \; \Phi\!\left(\boldsymbol{x}(1)\right),$
(1)
subject to
$\dot{\boldsymbol{x}} = \boldsymbol{f}(\boldsymbol{x},\boldsymbol{u}), \qquad \boldsymbol{x}(-1)=\boldsymbol{x}_0.$
(2)
Use $N$ Legendre–Gauss points, the noncollocated initial state $\boldsymbol{X}_0$ , the collocated states $\boldsymbol{X}_{1:N}$ , and the recovered terminal state $\boldsymbol{X}_{N+1}$ .
(a) Write the LG defect equations.
(b) Write the terminal quadrature equation.
(c) Introduce KKT multipliers for the defect equations, terminal quadrature equation, and initial condition.
(d) Write the complete NLP Lagrangian using explicit summations.
(e) Differentiate with respect to $\boldsymbol{U}_i$ , $\boldsymbol{X}_j$ , $\boldsymbol{X}_{N+1}$ , and the multiplier of the initial condition.
(f) Identify the discrete equations corresponding to continuous control stationarity, costate dynamics, terminal transversality, and initial feasibility.
LG costate mapping
Let $\boldsymbol{\Lambda}_{1:N}$ denote the multipliers of the LG defect equations and let $\boldsymbol{\Lambda}_{N+1}$ denote the multiplier of the terminal quadrature equation.
Starting from the LG KKT equations, derive
$\boldsymbol{\lambda}_{N+1} = \boldsymbol{\Lambda}_{N+1},$
(3)
$\boldsymbol{\lambda}_{1:N} = \boldsymbol{W}^{-1} \boldsymbol{\Lambda}_{1:N} + \boldsymbol{1}\boldsymbol{\Lambda}_{N+1},$
(4)
and
$\boldsymbol{\lambda}_0 = \boldsymbol{\lambda}_{N+1} - \boldsymbol{d}_0^{\mathsf T} \boldsymbol{\Lambda}_{1:N}.$
(5)
(a) State the dimensions of every matrix and vector.
(b) Explain why the terminal quadrature multiplier appears in every interior costate value.
(c) Show that the mapped multipliers satisfy the discrete terminal transversality condition.
(d) Explain how the initial costate estimate can be checked against the multiplier of the initial condition.
LGR KKT equations and interior costate mapping
Consider a left-LGR transcription with collocation points $\tau_1=-1,\tau_2,\ldots,\tau_N$ and the noncollocated terminal state $\boldsymbol{X}_{N+1}$ .
The defect equations are written using the convention
$\boldsymbol{F}_{1:N} - \boldsymbol{D}\boldsymbol{X}_{1:N+1} = \boldsymbol{0}.$
(6)
(a) Write the NLP Lagrangian for the simplified Mayer problem.
(b) Differentiate with respect to each control vector $\boldsymbol{U}_i$ .
(c) Differentiate with respect to the interior state vectors $\boldsymbol{X}_j$ , $j=2,\ldots,N$ .
(d) Show that the control KKT condition can be written as
$\nabla_{\boldsymbol{U}_i} \left( \boldsymbol{\lambda}_i^{\mathsf T} \boldsymbol{F}_i \right) = \boldsymbol{0}.$
(7)
(e) Derive the interior costate mapping
$\boldsymbol{\lambda}_{1:N} = \boldsymbol{W}^{-1} \boldsymbol{\Lambda}_{1:N}.$
(8)
(f) Explain why this mapping is algebraically simpler than the LG mapping.
LGR endpoint costates
For the same LGR transcription, let
$\boldsymbol{d}_{N+1} = \boldsymbol{D}(:,N+1).$
(9)
(a) Differentiate the NLP Lagrangian with respect to $\boldsymbol{X}_{N+1}$ .
(b) Derive
$\nabla_{\boldsymbol{X}} \Phi(\boldsymbol{X}_{N+1}) = \boldsymbol{d}_{N+1}^{\mathsf T} \boldsymbol{\Lambda}_{1:N}.$
(10)
(c) Conclude that the terminal costate estimate is
$\boldsymbol{\lambda}_{N+1} = \boldsymbol{d}_{N+1}^{\mathsf T} \boldsymbol{\Lambda}_{1:N}.$
(11)
(d) Derive the KKT equation associated with $\boldsymbol{X}_1$ .
(e) Show how the initial-condition multiplier provides the consistency check
$\boldsymbol{\lambda}_1 \approx \boldsymbol{\mu}.$
(12)
LGL discrete adjoint system
Let $\boldsymbol{D}\in\mathbb{R}^{N\times N}$ be the LGL differentiation matrix and
$\boldsymbol{W} = \operatorname{diag}(w_1,\ldots,w_N).$
(13)
Define
$\boldsymbol{\lambda}_{1:N} = \boldsymbol{W}^{-1} \boldsymbol{\Lambda}_{1:N}.$
(14)
(a) Starting from the LGL KKT equations, derive the off-diagonal entries of the adjoint differentiation matrix
$D_{ij}^{\dagger} = - \frac{w_j}{w_i}D_{ji}, \qquad i\neq j.$
(15)
(b) Derive the modified endpoint diagonal entries.
(c) Show that the compact discrete adjoint equation is
$\boldsymbol{D}^{\dagger} \boldsymbol{\lambda}_{1:N} = - \nabla_{\boldsymbol{X}} \left\langle \boldsymbol{\lambda}_{1:N}, \boldsymbol{F}_{1:N} \right\rangle + \frac{\boldsymbol{e}_1}{w_1} \left( \boldsymbol{\mu} - \boldsymbol{\lambda}_1 \right) + \frac{\boldsymbol{e}_N}{w_N} \left[ \boldsymbol{\lambda}_N - \nabla_{\boldsymbol{X}} \Phi(\boldsymbol{X}_N) \right].$
(16)
(d) Explain why the two endpoint residuals remain coupled to the LGL costate dynamics.
(e) Relate this coupling to the singularity of the LGL differentiation matrix.
Comparison of LG, LGR, and LGL costate recovery
Construct a comparison for the three Legendre pseudospectral schemes.
For each method, determine:
(a) which endpoints are collocation points;
(b) whether a terminal quadrature equation is required;
(c) the mapping from defect multipliers to interior costates;
(d) the formula for the initial costate;
(e) the formula for the terminal costate;
(f) whether endpoint transversality residuals are decoupled from or coupled to the discrete costate dynamics;
(g) whether a full-rank square submatrix of the state differentiation matrix exists;
(h) which method has the simplest multiplier-to-costate mapping.
Then explain why accurate state and control trajectories do not automatically imply accurate recovered costates.
General LGR Bolza transcription
Consider
$J = \Phi \bigl( \boldsymbol{x}(t_0), t_0, \boldsymbol{x}(t_f), t_f \bigr) + \int_{t_0}^{t_f} L \bigl( \boldsymbol{x}(t), \boldsymbol{u}(t), t \bigr) \,\mathrm{d}t,$
(17)
subject to
$\dot{\boldsymbol{x}} = \boldsymbol{f} \bigl( \boldsymbol{x}, \boldsymbol{u}, t \bigr),$
(18)
endpoint constraints
$\boldsymbol{\phi} \bigl( \boldsymbol{x}(t_0), t_0, \boldsymbol{x}(t_f), t_f \bigr) = \boldsymbol{0},$
(19)
and path bounds
$\boldsymbol{c}_{\min} \leq \boldsymbol{c} \bigl( \boldsymbol{x}, \boldsymbol{u}, t \bigr) \leq \boldsymbol{c}_{\max}.$
(20)
(a) Derive the affine map from $[t_0,t_f]$ to $[-1,1]$ .
(b) Derive
$\frac{\mathrm{d}\boldsymbol{x}}{\mathrm{d}\tau} = \frac{t_f-t_0}{2} \boldsymbol{f}.$
(21)
(c) Derive the LGR quadrature approximation of the running cost.
(d) Write the LGR defect matrix.
(e) Write the discrete endpoint constraints.
(f) State precisely where the path constraints are imposed.
(g) Identify every expression in which a free $t_f$ appears.
(h) Assemble the complete finite-dimensional NLP.
Solver-ready decision vector and constraint assembly
Consider an LGR transcription with $N$ collocation points, $n_x$ states, $n_u$ controls, and free final time.
(a) Define one decision vector containing all state values, control values, and $t_f$ .
(b) Derive its total dimension.
(c) Give explicit reshape operations that recover the state and control arrays.
(d) Write the defect matrix for the linear dynamics
$\dot{\boldsymbol{x}} = \boldsymbol{A}\boldsymbol{x} + \boldsymbol{B}\boldsymbol{u}.$
(22)
(e) Show how to vectorize the defect matrix into one column vector.
(f) Add endpoint constraints and two-sided path constraints.
(g) Write the complete equality- and inequality-constraint vectors passed to an NLP solver.
(h) Explain how the ordering of constraint blocks determines the interpretation of the returned KKT multipliers.
Sign conventions and multiplier recovery
Suppose one implementation uses
$\boldsymbol{F} - \boldsymbol{D}\boldsymbol{X} = \boldsymbol{0},$
(23)
while another uses
$\boldsymbol{D}\boldsymbol{X} - \boldsymbol{F} = \boldsymbol{0}.$
(24)
(a) Write the NLP Lagrangian under each convention.
(b) Show how the defect multipliers are related.
(c) Determine how the recovered costates are affected if the same mapping formula is used without correcting the sign.
(d) Derive a practical sign check using the terminal transversality condition.
(e) Derive a second sign check using the control-stationarity residual.
(f) Explain why the exact solver convention must be documented before interpreting multiplier values physically.
Primal and dual verification study
Use an LGR transcription for
$\min \; J = \frac12x^2(t_f) + \frac12 \int_0^{t_f} \left( x^2+u^2 \right) \,\mathrm{d}t,$
(25)
subject to
$\dot{x} = -x+u, \qquad x(0)=x_0,$
(26)
with fixed $t_f$ .
(a) Write the complete LGR NLP.
(b) Recover the interior and terminal costates from the KKT multipliers.
(c) Compute the maximum dynamic-defect residual.
(d) Compute the maximum control-stationarity residual.
(e) Compute the maximum discrete-adjoint residual.
(f) Check the terminal transversality residual.
(g) Check the initial-costate consistency residual.
(h) Repeat the solution for increasing polynomial order.
(i) Compare convergence of the state, control, objective, and costate.
(j) Explain how primal convergence can occur before dual convergence.