Function Spaces, Functionals, and the Calculus of Variations

Optimal control is fundamentally different from ordinary finite-dimensional optimization because the unknown is generally not a point. Instead, the unknown is an entire function of time. A control input

assigns a command at every instant, while the corresponding state trajectory

describes the evolution of the system over an interval of time.

This distinction requires a new mathematical framework. Ordinary calculus studies changes in numbers or vectors. The calculus of variations studies changes in entire functions. The purpose of this section is to establish the foundational distinctions among points, functions, functionals, differences, and variations, and then derive the first-order necessary condition for the classical fixed-endpoint problem in the calculus of variations.

The final result is the Euler–Lagrange equation:

Points and Functions¶

Points in finite-dimensional space¶

A point in $n$-dimensional Euclidean space is denoted by

In component form,

The object $\boldsymbol{x}$ is one point. It contains $n$ numerical components, but it is still a finite-dimensional object.

Vector-valued functions¶

A vector-valued function of time is written as

where $\Omega$ denotes an appropriate function space. Evaluating the function at a specific time $t$ gives a point:

A function space may include smoothness, continuity, endpoint, or integrability requirements. Examples include

Functions and Functionals¶

Ordinary functions¶

An ordinary function maps points to points. For example,

Given an input $\boldsymbol{x}\in\mathbb{R}^n$, the function returns

In the scalar case,

Functionals¶

A functional maps an entire function to a scalar:

Thus,

A fundamental example is the integral functional

The variable $\tau$ is a dummy variable of integration. Once the integration is complete, the result is a real number.

Why the Integration Variable Is Not an Output Variable¶

Consider

Although the integrand depends explicitly on $t$, the functional $J$ is not a function of the dummy variable $t$. The variable is integrated out over the interval $[t_0,t_f]$.

The result depends on the chosen function $x(\cdot)$, on the endpoints, and on the known integrand $L$, but not on the dummy variable remaining after integration.

Control as a Function¶

A control input is usually a function of time:

At each time $t$, the control specifies a command

For a vehicle, controls might include steering angle, throttle, and braking force. The defining feature of a control is that it can be selected, subject to allowable limits such as

The goal of optimal control is to determine an entire admissible function $\boldsymbol{u}^*(t)$ that guides the dynamical system toward a desired goal while optimizing a performance measure.

Differences and Variations¶

Difference between points¶

Let

and consider a second point

Then

is the difference or increment between two points.

Variation of a function¶

For a function $x(\cdot)$, a perturbed function is written

Here $\delta x(t)$ is called the variation of $x(t)$. It is itself a function.

Increments of Functions and Functionals¶

Increment of an ordinary function¶

Let

The increment in $f$ induced by $\Delta\boldsymbol{q}$ is

A first-order Taylor expansion gives

Increment of a functional¶

Let

The increment of $J$ produced by the perturbed function

is

The first-order portion of this increment is called the first variation:

Formally,

where $\eta(t)$ is an admissible direction and $\epsilon$ is a scalar perturbation parameter.

Equivalently,

Local and Global Minima¶

Minima of ordinary functions¶

A point $\boldsymbol{q}^*\in\mathbb{R}^n$ is a local minimum of $f$ if there exists a neighborhood of $\boldsymbol{q}^*$ such that

for all sufficiently small $\Delta\boldsymbol{q}$.

It is a global minimum if

Extrema of functionals¶

A function $x^*(\cdot)\in\Omega$ is a local minimum of $J$ if

for all sufficiently small admissible variations $\delta x(\cdot)$.

It is a global minimum if

More generally, minima and maxima of functionals are referred to as extrema.

Ordinary Calculus and the Calculus of Variations¶

Ordinary calculus determines extrema of functions by analyzing derivatives with respect to finite-dimensional variables.

For example, if

then an interior extremum satisfies

The calculus of variations performs an analogous task for functionals. An extremizing function $x^*(\cdot)$ must satisfy

for all admissible first-order variations.

The Classical Fixed-Endpoint Problem¶

Consider the functional

where $L$ is known and the function $x(t)$ is unknown.

Assume that

and that $t_0$, $t_f$, $x_0$, and $x_f$ are fixed.

The objective is to determine a function

that makes $J[x]$ stationary.

Admissible Variations¶

Let the perturbed path be

where $\eta(t)$ is an arbitrary smooth variation function.

Because the endpoint values are fixed, the perturbed path must satisfy

Substituting (49) gives

Since $x(t_0)=x_0$ and $x(t_f)=x_f$, it follows that

Equivalently,

Derivation of the First Variation¶

The first variation of (46) is

Because the integration limits are fixed, the variation can be moved inside the integral:

To first order,

There is no term involving a variation $\delta t$ because $t$ is the fixed independent variable in the present problem.

Substitution into (55) gives

Dependence of $\delta \dot{x}$ on $\delta x$¶

The variations $\delta x$ and $\delta\dot{x}$ are not independent because

Under standard smoothness assumptions, variation and differentiation commute:

Therefore,

Integration by Parts¶

Recall the integration-by-parts identity

Choose

Then

Substitution into (60) yields

Because the endpoints are fixed,

Hence the boundary term vanishes:

Thus,

The Fundamental Lemma of the Calculus of Variations¶

Applying the lemma to (67), and noting that $\delta x(t)$ is otherwise arbitrary, gives the necessary condition

Interpretation of the Euler–Lagrange Equation¶

The Euler–Lagrange equation is not the extremizing function itself. It is a differential equation that the extremizing function must satisfy.

The solution procedure is therefore:

1. compute

2. compute

3. differentiate the latter with respect to time:

4. form the Euler–Lagrange differential equation;

5. solve the differential equation subject to the endpoint conditions.

Worked Example: Minimizing the Integral of Squared Slope¶

Consider

subject to

Here,

Then

and

The Euler–Lagrange equation becomes

or

Integrating twice gives

Using the endpoint conditions,

Thus, the extremizing path is the straight line joining the two endpoints.

Worked Example: A Functional with a Trivial Minimizer¶

Consider

Because

for all $t$, it follows that

The lower bound is achieved by

which yields

The Euler–Lagrange equation confirms this result. Here

so

Therefore,

and hence

Vector Form of the Euler–Lagrange Equation¶

Let

and consider

The Euler–Lagrange equation becomes

Componentwise,

Connection to Lagrangian Mechanics¶

In analytical mechanics, the Lagrangian is often defined as

where $T$ is kinetic energy and $V$ is potential energy.

The action functional is

Hamilton’s principle states that the physically realized trajectory renders the action stationary:

The resulting Euler–Lagrange equations are

Thus, the same mathematical structure that determines stationary functionals also generates the equations of motion in classical mechanics.

Principle of Stationary Action¶

The phrase principle of least action is commonly used, but the more precise statement is that the physical path makes the action stationary. The path may correspond to a minimum, maximum, or saddle-type stationary value depending on the problem.

Therefore,

is the fundamental first-order condition.

Important Assumptions in the Derivation¶

The derivation of (71) relied on the following assumptions:

1. the endpoint times $t_0$ and $t_f$ are fixed;

2. the endpoint values $x(t_0)$ and $x(t_f)$ are fixed;

3. the admissible functions are sufficiently smooth;

4. the integrand $L(x,\dot{x},t)$ is differentiable in its arguments;

5. the variation and time differentiation commute;

6. only first-order variation terms are retained.

When endpoint values or endpoint times are free, additional boundary conditions arise. These are called natural boundary conditions or transversality conditions.

Chapter Summary¶

The principal ideas developed in this section are:

1. A point $\boldsymbol{x}\in\mathbb{R}^n$ and a function $\boldsymbol{x}(\cdot)\in\Omega$ are fundamentally different mathematical objects.

2. An ordinary function maps points to values:

3. A functional maps functions to real numbers:

4. A finite-dimensional increment is denoted by $\Delta\boldsymbol{x}$, while a functional variation is denoted by $\delta x(t)$.

5. The first variation $\delta J$ is the first-order change in a functional caused by a variation in its input function.

6. For the fixed-endpoint problem

the first-order stationarity condition is

7. Integration by parts converts the dependence on $\delta\dot{x}$ into dependence on $\delta x$.

8. Fixed endpoints imply

9. The resulting Euler–Lagrange equation is

10. The Euler–Lagrange equation is a necessary condition, not a guarantee of a minimum.

11. The same equation appears in classical mechanics through Hamilton’s principle of stationary action.

Connection. Once interior variations are understood, the next question is what changes when the endpoints are not fixed; endpoint variations provide that extension.