- Research
- Open Access

# A maximum principle for optimal control system with endpoint constraints

- Weifeng Wang
^{1}and - Bin Liu
^{1}Email author

**2012**:231

https://doi.org/10.1186/1029-242X-2012-231

© Wang and Liu; licensee Springer 2012

**Received:**30 May 2012**Accepted:**28 September 2012**Published:**12 October 2012

## Abstract

Pontryagin’s maximum principle for an optimal control system governed by an ordinary differential equation with endpoint constraints is proved under the assumption that the control domain has no linear structure. We also obtain the variational equation, adjoint equation and Hamilton system for our problem.

**MSC:**65K10, 34H05, 93C15.

## Keywords

- Pontryagin’s maximum principle
- optimal control
- Hamilton system
- transversality condition
- linear structure

## 1 Introduction

Optimal control problems have been studied for a long time and have a lot of practical applications in the fields such as physics, biology and economics, *etc.* Hamilton systems are derived from Pontryagin’s maximum principle, which is known as a necessary condition for optimality. Many results have been obtained both for finite and infinite dimensional control systems such as [1–5]. Regarding the state constraint problems, lots of results are also obtained. For example, readers can refer to [6, 7] and the references therein.

To our best knowledge, to derive the necessary conditions of Pontryagin’s maximum principle type for optimal control problems, there are two main perturbation methods. When the control domain is convex, we often use the convex perturbation. When the control domain is non-convex and does not have any linear structure, we usually use the spike perturbation. Many relevant results have been obtained; see [3, 6, 8–10] and the references therein. The two methods have their advantages and disadvantages. The convex variational needs the control domain being convex, but in reality it is not always satisfied. And the spike variational needs more regularity for the coefficients and the solutions to the state equations, especially in the stochastic case.

In 2010, Lou [7] introduced a new method to study the necessary and sufficient conditions of optimal control problems in the absence of linear structure for the deterministic case. The author gave a local linearization of the optimal control problem along the optimal control, and transformed the original problem into a new relaxed control problem. Moreover, he proved the equivalence of the two problems in some sense. Being directly inspired by [7], we are also interested in applying this method to an endpoint constraints optimal control system, which is also in the absence of linear structure. Also, Pontryagin’s maximum principle is obtained for our problem.

The rest of this paper is organized as follows. Section 2 begins with a general formulation of our state constraints optimal control problem and the local linearization of the problem is given. In Section 3, we give our main result and its proof. Moreover, we obtain the variational equation, adjoint equation and Hamilton system for our optimal control system.

## 2 Preliminaries

where *V* is a non-convex set in ${R}^{k}$.

where ${S}_{1}$ and ${S}_{2}$ are closed convex subsets of ${R}^{n}$. Let $S={S}_{1}\times {S}_{2}$.

Then the optimal control problem can be stated as follows.

**Problem 2.1**Find a pair $({\overline{x}}_{0},\overline{u}(\cdot ))\in {U}_{\mathrm{ad}}[0,T]$ such that

Any $({\overline{x}}_{0},\overline{u}(\cdot ))\in {U}_{\mathrm{ad}}[0,T]$ satisfying the above identity is called an optimal control, and the corresponding state $x(\cdot ;{\overline{x}}_{0},\overline{u}(\cdot ))\triangleq \overline{x}(\cdot )$ is called an optimal trajectory; $(\overline{x}(\cdot ),\overline{u}(\cdot ))$ is called an optimal pair.

Let the following hypotheses hold:

(${H}_{1}$) The metric space $(V,d)$ is separable, and *d* is the usual metric in ${R}^{k}$.

*t*, continuous in $(x,u)$ and continuously differentiable in

*x*, where ⊤ denotes the transpose of a matrix. Moreover, there exists a constant $L>0$ such that

where $|\cdot |$ denotes the usual Euclidean norm.

From the above conditions, it is easy to know that the state equation (2.1) has a unique solution $x(\cdot ;{x}_{0},u(\cdot ))$ for any $({x}_{0},u(\cdot ))\in U[0,T]$.

We can easily find that $x(\cdot ;{x}_{0},u(\cdot ))$ and $J({x}_{0},u(\cdot ))$ coincide with $x(\cdot ;{x}_{0},{\delta}_{u(\cdot )})$ and $J({x}_{0},{\delta}_{u(\cdot )})$ respectively. Thus, ${U}_{\mathrm{ad}}[0,T]$ can be viewed as a subset of ${M}_{\mathrm{ad}}[0,T]$ in the sense of identifying $({x}_{0},u(\cdot ))\in {U}_{\mathrm{ad}}[0,T]$ and $({x}_{0},{\delta}_{u(\cdot )})\in {M}_{\mathrm{ad}}[0,T]$. Because the elements of ${M}_{\mathrm{ad}}[0,T]$ are very simple, we need neither pose additional assumptions like that the control domain is compact as Warga [11] did nor introduce the relaxed control defined by finite-additive probability measure as Fattorini [12] did. Now, we can see that ${M}_{\mathrm{ad}}[0,T]$ already has a linear structure at $({\overline{x}}_{0},\overline{u}(\cdot ))$. First, we give some lemmas to show that $({\overline{x}}_{0},{\delta}_{\overline{u}(\cdot )})$ is a minimizer of $J({x}_{0},\sigma (\cdot ))$ over ${M}_{\mathrm{ad}}[0,T]$.

**Lemma 2.2** (Lou [7])

*Let*$({H}_{1})$-$({H}_{2})$

*hold*.

*Then there exists a positive constant*$C>0$,

*such that for any*$({x}_{0},\sigma (\cdot ))\in {M}_{\mathrm{ad}}[0,T]$

*and*${t}_{1},{t}_{2}\in [0,T]$,

**Lemma 2.3** (Lou [7])

*Let* $({H}_{1})$-$({H}_{2})$ *hold and* $({\overline{x}}_{0},\overline{u}(\cdot ))$ *be a minimizer of* $J({x}_{0},u(\cdot ))$ *over* ${U}_{\mathrm{ad}}[0,T]$. *Then* $({\overline{x}}_{0},{\delta}_{\overline{u}(\cdot )})$ *is a minimizer of* $J({x}_{0},\sigma (\cdot ))$ *over* ${M}_{\mathrm{ad}}[0,T]$.

**Remark 2.4** We can see that Lemma 2.3 shows the equivalence of the above two problems. And this lemma is essential for our following maximum principle.

## 3 Pontryagin’s maximum principle

In this section, we give our main result first and then prove it. Let ${f}_{x}$ denote the derivative of *f* on *x*, and others can be defined in the same way. $\u3008\cdot ,\cdot \u3009$ denotes the inner product in ${R}^{n}$.

**Theorem 3.1** (Pontryagin’s maximum principle)

*We assume*(${H}_{1}$)-(${H}_{2}$)

*hold*.

*Let*$(\overline{x}(\cdot ),\overline{u}(\cdot ))$

*be a solution of the optimal control problem*(2.1).

*Then there exists a nontrivial pair*$({\psi}^{0},p(\cdot ))\in R\times {C}^{1}([0,T];{R}^{n})$,

*i*.

*e*., $({\psi}^{0},p(\cdot ))\ne 0$

*such that*

*where for any*$(t,x,u,p,{\psi}^{0})\in [0,T]\times {R}^{n}\times V\times {R}^{n}\times R$,

*and we also have the following Hamilton system*:

*where* $\overline{\psi}$ *will be defined in the following part*.

We recall that under the conditions $({H}_{1})$-$({H}_{2})$, for any $({x}_{0},u(\cdot ))\in {R}^{n}\times U[0,T]$, the state equation (2.1) admits a unique solution $x(\cdot )$ with $x(0)={x}_{0}$. So, the cost functional is uniquely determined by $({x}_{0},u(\cdot ))$. In the sequel, we denote the unique solution of (2.1) with $x(0)={x}_{0}$ by $x(\cdot ;{x}_{0},u(\cdot ))$. Now, let $({\overline{x}}_{0},\overline{u}(\cdot ))\in {U}_{\mathrm{ad}}[0,T]$ be an optimal control. We denote $\overline{x}(0)={\overline{x}}_{0}$. Without loss of generality, we may assume that $J({\overline{x}}_{0},\overline{u}(\cdot ))=0$; otherwise, we may consider the optimal control problem with a cost functional of $J({x}_{0},u(\cdot ))-J({\overline{x}}_{0},\overline{u}(\cdot ))$.

Now, we give some definitions and lemmas for Theorem 3.1.

First, we define a penalty functional, via which, for convenience, we can transform the original problem to another one called the approximate problem, which has no endpoint constraint.

where $d({u}_{1}(\cdot ),{u}_{2}(\cdot ))$ denote the measure of $\{t\in [0,T]|{u}_{1}(t)\ne {u}_{2}(t)\}$ (it obviously is a metric).

where $\u3008\cdot ,\cdot \u3009$ denotes the inner product in ${R}^{n}$. For more properties of subdifferential $\partial {d}_{S}$, one can see p.146 in [6].

**Lemma 3.2**

*Let*$({H}_{1})$-$({H}_{2})$

*hold*.

*Then there exists a constant*$C>0$

*such that for all*$({x}_{0},u(\cdot )),({\stackrel{\u02c6}{x}}_{0},\stackrel{\u02c6}{u}(\cdot ))\in {R}^{n}\times U[0,T]$,

*Proof*Denote $x(\cdot )=x(t;{x}_{0},u(\cdot ))$ and $\stackrel{\u02c6}{x}(\cdot )=x(t;{\stackrel{\u02c6}{x}}_{0},\stackrel{\u02c6}{u}(\cdot ))$. From the state equation (2.1) and condition $({H}_{2})$, we have

*C*is independent of controls $u(\cdot )$ and $\stackrel{\u02c6}{u}(\cdot )$, and may be different at different places throughout this paper. Further, noting the definition of $\overline{d}$, we have

Taking supremum in the above inequality, the first inequality in (3.11) is obtained. The second inequality can be proved similarly. □

By the definition of ${J}_{\epsilon}({x}_{0},u(\cdot ))$ and Lemma 3.2, we can easily obtain the following result.

**Corollary 3.3** *The functional* ${J}_{\epsilon}({x}_{0},u(\cdot ))$ *is continuous on the space* $({R}^{n}\times U[0,T],\overline{d})$.

**Remark 3.4**By the definition of ${J}_{\epsilon}({x}_{0},u(\cdot ))$ (3.8) and Corollary 3.3, we can see that

The above implies that if we let ${x}^{\epsilon}(\cdot )=x(\cdot ;{x}_{0}^{\epsilon},{u}^{\epsilon}(\cdot ))$, then $({x}^{\epsilon}(\cdot ),{u}^{\epsilon}(\cdot ))$ is an optimal pair for the problem where the state equation is (2.1) and the cost functional is ${J}_{\epsilon}({x}_{0},u(\cdot ))$.

Now, we derive the necessary conditions for $({x}^{\epsilon}(\cdot ),{u}^{\epsilon}(\cdot ))$.

**Lemma 3.5**

*Let*$({x}^{\epsilon}(\cdot ),{u}^{\epsilon}(\cdot ))$

*be an optimal pair for the problem where the state equation is*(2.1)

*and the cost functional is*${J}_{\epsilon}({x}_{0},u(\cdot ))$,

*then there exists a nontrivial triple*$({\overline{{\psi}_{\epsilon}}}^{0},\overline{{\phi}_{\epsilon}},\overline{{\psi}_{\epsilon}})\in R\times {R}^{n}\times {R}^{n}$

*such that*

*and*

*where* *η*, ${X}_{\epsilon}(\cdot )$, ${Y}_{\epsilon}$ *will be defined in the following proof*.

*Proof* Similar to Section 2, we linearize $U[0,T]$ along ${u}^{\epsilon}(\cdot )$ and denote ${\sigma}^{\alpha ,\epsilon}(\cdot )=(1-\alpha ){\delta}_{{u}^{\epsilon}(\cdot )}+\alpha {\delta}_{u(\cdot )}$, ${x}^{\alpha ,\epsilon}(\cdot )=x(\cdot ;{x}_{0}^{\epsilon}+\alpha \eta ,{\sigma}^{\alpha ,\epsilon}(\cdot ))$, where $\eta \in {B}_{1}(0)\subset {R}^{n}$ (${B}_{1}(0)$ denotes the ball whose center is 0 and radius is 1). Recall that ${x}^{\epsilon}(\cdot )=x(\cdot ;{x}_{0}^{\epsilon},{u}^{\epsilon}(\cdot ))$, then we derive the variational equation.

*f*on

*x*. By virtue of $({H}_{2})$, and using the convergence of ${x}^{\alpha ,\epsilon}(t)\to {x}^{\epsilon}(t)$ (see the proof of Lemma 2.2 in [7]), we can easily obtain

*α*, we can define

□

**Remark 3.6**By the definition of subdifferential of the function ${d}_{S}(\cdot )$, for any $({x}_{1},{x}_{2})\in S$, we have

In order to pass to the limit as $\epsilon \to 0$, we give the following lemma first, which is necessary for the derivation of our maximum principle.

**Lemma 3.7**

*It holds that*

*Proof*By (3.13) and the definition of $\overline{d}$, it is easy to see that

Thus, the proof of this lemma is completed. □

**Remark 3.8**Now, we can let $\epsilon \to 0$. By (3.23), it is obvious that for any $({x}_{1},{x}_{2})\in S$, we have

Based on the above preparation, now we start to prove Theorem 3.1 by the duality relations.

*Proof of Theorem 3.1*Let $p(t)$ solve the adjoint equation

where ${\psi}^{0}=-{\overline{\psi}}^{0}$.

*H*is the Hamilton function defined in (3.5). Now, setting $\eta =0$ and $({x}_{1},{x}_{2})=({\overline{x}}_{0},\overline{x}(T))$ in (3.37), we obtain

*U*is separable and $({x}_{0},u(\cdot ))\in {U}_{\mathrm{ad}}[0,T]$ is arbitrary, the above inequality can be written as

Note that ${\psi}^{0}=0$ in this case, so this gives a contradiction to (3.22). The Hamilton system (3.6) is obvious. Then the proof of the maximum principle is completed. □

**Remark 3.9**We give some important special cases of our control problem.

- (i)The control problem with fixed endpoints. In this case, the constraint set is $S=\{({x}_{0},{x}_{T})\}$ and the endpoint constraint becomes of the following form:$x(0)={x}_{0},\phantom{\rule{2em}{0ex}}x(T)={x}_{T}.$
- (ii)The control problem with a terminal state constraint,
*i.e.*,$x(0)={x}_{0},\phantom{\rule{2em}{0ex}}x(T)\in {S}_{2}.$

## Declarations

### Acknowledgements

We would like to thank the referees for the useful suggestions which helped to improve the previous version of this paper. This work was partially supported by NNSF of China (Grant No. 11171122).

## Authors’ Affiliations

## References

- Barbu V:
*Optimal of Variational Inequalities*. Pitman, London; 1984.MATHGoogle Scholar - Barbu V, Da Prato G:
*Hamilton Jacobi Equations in Hilbert Spaces*. Pitman, London; 1983.MATHGoogle Scholar - Barbu V, Precupanu T:
*Convexity and Optimization in Banach Spaces*. Reidel, Dordrecht; 1986.MATHGoogle Scholar - Capuzzo-Dolcetta I, Evans LC: Optimal switching for ordinary differential equations.
*SIAM J. Control Optim.*1984, 22: 143–161. 10.1137/0322011MathSciNetView ArticleMATHGoogle Scholar - Pontryagin LS: The maximum principle in the theory of control processes.
*Proc. 1st. Congress IFAC, Moscow*1960.Google Scholar - Li XJ, Yong YM:
*Optimal Control Theory for Infinite Dimensional Systems*. Birkhäuser, Boston; 1995.View ArticleGoogle Scholar - Lou HW: Second-order necessary/sufficient conditions for optimal control problems in the absence of linear structure.
*Discrete Contin. Dyn. Syst., Ser. B*2010, 14: 1445–1464.MathSciNetView ArticleMATHGoogle Scholar - Casas E: Control of an elliptic problem with pointwise state constraints.
*SIAM J. Control Optim.*1986, 24: 1309–1318. 10.1137/0324078MathSciNetView ArticleMATHGoogle Scholar - Casas E, Fernández LA: Optimal control of semilinear elliptic equations with pointwise constraints on the gradient of the state.
*Appl. Math. Optim.*1993, 27: 35–56. 10.1007/BF01182597MathSciNetView ArticleMATHGoogle Scholar - Yong JM, Zhou XY:
*Stochastic Controls: Hamiltonian System and HJB Equations*. Springer, New York; 1999.View ArticleMATHGoogle Scholar - Warga J:
*Optimal Control of Differential and Functional Equations*. Academic Press, New York; 1972.MATHGoogle Scholar - Fattorini HO: Relaxed controls in infinite dimensional systems.
*Int. Ser. Numer. Math.*1991, 100: 115–128.MathSciNetView ArticleMATHGoogle Scholar

## Copyright

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.