Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

Path Constraints, Active Sets, and Augmented Hamiltonians

Real optimal-control problems almost always include restrictions that must hold throughout the trajectory. These are called path constraints.

This section develops:

General Path Constraints

A general path constraint may be written as

c(x(t),u(t),t)≤0.\boldsymbol{c}(\boldsymbol{x}(t),\boldsymbol{u}(t),t)\leq\boldsymbol{0}.
(1)

The constraint may involve:

Examples include:

v(t)≤vmax⁡,∣u(t)∣≤umax⁡,q(x,u,t)≤qmax⁡,T(x,u,t)≤Tmax⁡,gload(x,u,t)≤gmax⁡.\begin{aligned} v(t)&\leq v_{\max},\\ |u(t)|&\leq u_{\max},\\ q(\boldsymbol{x},\boldsymbol{u},t)&\leq q_{\max},\\ T(\boldsymbol{x},\boldsymbol{u},t)&\leq T_{\max},\\ g_{\mathrm{load}}(\boldsymbol{x},\boldsymbol{u},t)&\leq g_{\max}. \end{aligned}
(2)

Two-Sided Bounds

Suppose

cmin⁡≤c(x,u,t)≤cmax⁡.\boldsymbol{c}_{\min} \leq \boldsymbol{c}(\boldsymbol{x},\boldsymbol{u},t) \leq \boldsymbol{c}_{\max}.
(3)

This can be written as two one-sided inequalities:

c(x,u,t)−cmax⁡≤0,−c(x,u,t)+cmin⁡≤0.\begin{aligned} \boldsymbol{c}(\boldsymbol{x},\boldsymbol{u},t)-\boldsymbol{c}_{\max} &\leq\boldsymbol{0},\\ -\boldsymbol{c}(\boldsymbol{x},\boldsymbol{u},t)+\boldsymbol{c}_{\min} &\leq\boldsymbol{0}. \end{aligned}
(4)

Stacking them gives

k(x,u,t)=[c−cmax⁡−c+cmin⁡]≤0.\boxed{ \boldsymbol{k}(\boldsymbol{x},\boldsymbol{u},t) = \begin{bmatrix} \boldsymbol{c}-\boldsymbol{c}_{\max}\\ -\boldsymbol{c}+\boldsymbol{c}_{\min} \end{bmatrix} \leq\boldsymbol{0}. }
(5)

Thus, one-sided inequality form is sufficient for the theory.

Augmented Cost Functional

Consider a Bolza problem:

J=Φ+∫t0tfL(x,u,t) dtJ = \Phi + \int_{t_0}^{t_f} L(\boldsymbol{x},\boldsymbol{u},t)\,\mathrm{d} t
(6)

subject to

x˙=f(x,u,t),\dot{\boldsymbol{x}} = \boldsymbol{f}(\boldsymbol{x},\boldsymbol{u},t),
(7)

and

c(x,u,t)≤0.\boldsymbol{c}(\boldsymbol{x},\boldsymbol{u},t)\leq\boldsymbol{0}.
(8)

Introduce the multiplier function μ(t)\boldsymbol{\mu}(t) for the path constraint.

The augmented integrand becomes

L+λT(f−x˙)−μTc.L + \boldsymbol{\lambda}^{\mathsf{T}}(\boldsymbol{f}-\dot{\boldsymbol{x}}) - \boldsymbol{\mu}^{\mathsf{T}}\boldsymbol{c}.
(9)

The sign convention used here corresponds to the inequality

c≤0.\boldsymbol{c}\leq\boldsymbol{0}.
(10)

Augmented Hamiltonian

The ordinary Hamiltonian is

H=L+λTf.H = L+\boldsymbol{\lambda}^{\mathsf{T}}\boldsymbol{f}.
(11)

The augmented Hamiltonian is

Ha=H−μTc.\boxed{ H_a = H-\boldsymbol{\mu}^{\mathsf{T}}\boldsymbol{c}. }
(12)

Equivalently,

Ha=L+λTf−μTc.H_a = L+\boldsymbol{\lambda}^{\mathsf{T}}\boldsymbol{f}-\boldsymbol{\mu}^{\mathsf{T}}\boldsymbol{c}.
(13)

All first-order conditions must now be derived from HaH_a rather than HH.

Active and Inactive Constraints

For a scalar path constraint

c(x,u,t)≤0,c(\boldsymbol{x},\boldsymbol{u},t)\leq0,
(14)

there are two possibilities.

Inactive arc

If

c(x∗,u∗,t)<0,c(\boldsymbol{x}^*,\boldsymbol{u}^*,t)<0,
(15)

then the constraint does not affect the optimal solution locally.

Therefore,

μ∗(t)=0.\mu^*(t)=0.
(16)

Active arc

If

c(x∗,u∗,t)=0,c(\boldsymbol{x}^*,\boldsymbol{u}^*,t)=0,
(17)

then the trajectory lies on the constraint boundary.

The multiplier may be nonzero:

μ∗(t)≤0\mu^*(t)\leq0
(18)

under the sign convention adopted here.

Complementary Slackness

The active/inactive structure is summarized by

c(x∗,u∗,t)≤0,μ∗(t)≤0,μ∗(t)c(x∗,u∗,t)=0.\begin{aligned} c(\boldsymbol{x}^*,\boldsymbol{u}^*,t)&\leq0,\\ \mu^*(t)&\leq0,\\ \mu^*(t)c(\boldsymbol{x}^*,\boldsymbol{u}^*,t)&=0. \end{aligned}
(19)

In vector form:

μi∗(t)ci(x∗,u∗,t)=0∀i.\boxed{ \mu_i^*(t)c_i(\boldsymbol{x}^*,\boldsymbol{u}^*,t)=0 \quad \forall i. }
(20)

This is complementary slackness.

Typical Active-Set Structure

A path constraint may be inactive, become active, remain active over an interval, and later become inactive.

A typical sequence is

interior arc⟶boundary arc⟶interior arc.\text{interior arc} \longrightarrow \text{boundary arc} \longrightarrow \text{interior arc}.
(21)

The transition times are unknown and must be solved as part of the problem.

Boundary Arcs

On an active state path constraint,

c(x,t)=0.c(\boldsymbol{x},t)=0.
(22)

To remain on the boundary, one must also satisfy consistency conditions such as

 d dtc(x,t)=0.\frac{\,\mathrm{d}}{\,\mathrm{d} t}c(\boldsymbol{x},t)=0.
(23)

If the control does not appear, differentiate again:

 d2 dt2c(x,t)=0.\frac{\,\mathrm{d}^2}{\,\mathrm{d} t^2}c(\boldsymbol{x},t)=0.
(24)

Continue until the control appears. The resulting condition can determine the boundary control.

This resembles singular-control analysis, but the origin is different:

Modified Costate Dynamics

The costate equation becomes

λ˙=−(∂Ha∂x)T.\dot{\boldsymbol{\lambda}} = - \left( \frac{\partial H_a}{\partial\boldsymbol{x}} \right)^{\mathsf{T}}.
(25)

Therefore,

λ˙=−(∂H∂x−μT∂c∂x)T.\boxed{ \dot{\boldsymbol{\lambda}} = - \left( \frac{\partial H}{\partial\boldsymbol{x}} - \boldsymbol{\mu}^{\mathsf{T}} \frac{\partial\boldsymbol{c}}{\partial\boldsymbol{x}} \right)^{\mathsf{T}}. }
(26)

On inactive arcs,

μ=0,\boldsymbol{\mu}=\boldsymbol{0},
(27)

so the ordinary costate equation is recovered.

On active arcs, the multiplier modifies the costate dynamics.

Modified Control Optimality Condition

The control must minimize the augmented Hamiltonian:

u∗∈arg⁡min⁡u∈UHa(x∗,u,λ∗,μ∗,t).\boldsymbol{u}^* \in \arg\min_{\boldsymbol{u}\in\mathcal{U}} H_a(\boldsymbol{x}^*,\boldsymbol{u},\boldsymbol{\lambda}^*,\boldsymbol{\mu}^*,t).
(28)

If an interior stationarity condition applies,

∂Ha∂u=0.\boxed{ \frac{\partial H_a}{\partial\boldsymbol{u}} = \boldsymbol{0}. }
(29)

Thus,

∂H∂u−(∂c∂u)Tμ=0.\frac{\partial H}{\partial\boldsymbol{u}} - \left( \frac{\partial\boldsymbol{c}}{\partial\boldsymbol{u}} \right)^{\mathsf{T}} \boldsymbol{\mu} = \boldsymbol{0}.
(30)

Why Path Constraints Complicate Indirect Methods

Indirect methods require solving:

The active set is not known beforehand.

Therefore, the analyst may need to guess:

A wrong structural guess produces the wrong boundary-value problem.

Constraint Violation During Shooting

In a shooting method, an initial costate guess generates a trajectory.

If the trajectory violates

c(x,u,t)≤0,\boldsymbol{c}(\boldsymbol{x},\boldsymbol{u},t)\leq\boldsymbol{0},
(31)

then it is inadmissible.

However, the violation may occur only after integrating for some time. At that point, the method must determine whether to:

This makes indirect path-constrained shooting fragile.

Physical Examples

Vehicle motion

v(t)≤vmax⁡,amin⁡≤a(t)≤amax⁡,ymin⁡(x)≤y(t)≤ymax⁡(x).\begin{aligned} v(t)&\leq v_{\max},\\ a_{\min}&\leq a(t)\leq a_{\max},\\ y_{\min}(x)&\leq y(t)\leq y_{\max}(x). \end{aligned}
(32)

Aerospace systems

q(t)≤qmax⁡,Q˙(t)≤Q˙max⁡,n(t)≤nmax⁡,αmin⁡≤α(t)≤αmax⁡.\begin{aligned} q(t)&\leq q_{\max},\\ \dot{Q}(t)&\leq \dot{Q}_{\max},\\ n(t)&\leq n_{\max},\\ \alpha_{\min}&\leq\alpha(t)\leq\alpha_{\max}. \end{aligned}
(33)

Chemical processes

T(t)≤Tmax⁡,p(t)≤pmax⁡,0≤u(t)≤umax⁡,ci(t)≥0.\begin{aligned} T(t)&\leq T_{\max},\\ p(t)&\leq p_{\max},\\ 0&\leq u(t)\leq u_{\max},\\ c_i(t)&\geq0. \end{aligned}
(34)

Common Errors

  1. Treating every path constraint as active everywhere.

  2. Forgetting that inactive constraints have zero multipliers.

  3. Using the ordinary Hamiltonian instead of the augmented Hamiltonian.

  4. Ignoring complementary slackness.

  5. Guessing an active interval without checking admissibility.

  6. Confusing singular arcs with state-constraint boundary arcs.

  7. Assuming a numerical trajectory is valid even when it briefly violates a path constraint.

  8. Ignoring the unknown entry and exit times of active arcs.

Summary

  1. Path constraints restrict state, control, or mixed quantities throughout time.

  2. Two-sided bounds can be converted to stacked one-sided inequalities.

  3. Path constraints introduce multiplier functions.

  4. The augmented Hamiltonian is Ha=H−μTcH_a=H-\boldsymbol{\mu}^{\mathsf{T}}\boldsymbol{c}.

  5. Complementary slackness determines active and inactive behavior.

  6. Active constraints create boundary arcs.

  7. Unknown active-set structure makes indirect methods difficult.

  8. These difficulties motivate direct numerical methods.

Chapter 3: Linear–Quadratic, Indirect, and Singular Optimal Control
Coupled State–Costate Dynamics, Hamiltonian Matrices, and Shooting Instability
Chapter 3: Linear–Quadratic, Indirect, and Singular Optimal Control
Exercises