 Research
 Open access
 Published:
Maximum principle for a stochastic delayed system involving terminal state constraints
Journal of Inequalities and Applications volume 2017, Article number: 103 (2017)
Abstract
We investigate a stochastic optimal control problem where the controlled system is depicted as a stochastic differential delayed equation; however, at the terminal time, the state is constrained in a convex set. We firstly introduce an equivalent backward delayed system depicted as a timedelayed backward stochastic differential equation. Then a stochastic maximum principle is obtained by virtue of Ekeland’s variational principle. Finally, applications to a state constrained stochastic delayed linearquadratic control model and a productionconsumption choice problem are studied to illustrate the main obtained result.
1 Introduction
In 1990, the nonlinear backward stochastic differential equation (BSDE in short) was introduced by Pardoux and Peng [1]. Until now, it has had applications in many fields, such as partial differential equation (see [2]), stochastic control (see [3, 4]) and mathematical finance (see [5]). Meanwhile, BSDE itself has been developed to many different branches, such as BSDE with jumps (see [6–8]), meanfield BSDE (see [9]), timedelayed BSDE (see [10–12]), anticipated BSDEs (see [13, 14]) and so on. A lot of works have been done for the control problem of such BSDEs. However, fewer works have been done on the control problems of stochastic delayed systems.
For a stochastic delayed system, Chen and Wu [15] obtained a stochastic maximum principle by virtue of a duality between stochastic differential delayed equations (SDDEs in short) and anticipated BSDEs. Øksendal, Sulem and Zhang [16] studied the optimal control problems for SDDEs with jumps. Yu [17] obtained a maximum principle for SDDEs with random coefficients. A maximum principle of optimal control of SDDEs on infinite horizon was proved in Agram, Haadem and Øksendal [18]. Some other recent developments on stochastic delayed system can be found in Huang, Li and Shi [19], Meng and Shen [20], etc.
To the authors’ knowledge, there has been no result concerning the control problem of a stochastic delayed system with state constraints until now. However, the state constraints of stochastic delayed systems indeed exist in reality. In this paper, the stochastic control problem of a forward delayed system with terminal state constraint is studied. The controlled system is depicted as the following SDDE:
where \(X(T)\in K\), a.s., \(K\in\mathbb{R}^{n}\) is a convex set. However, there are two (main) difficulties in this study. The first one is that the control system (1.1) is a delayed system, as stated in [15], which is more complex than the classical case. Another difficulty is the terminal state constraint, which is a samplewise constraint. As interpreted in Ji and Zhou [21], the stochastic control involving samplewise state constraints cannot be resolved by the classical theory.
Some recent developed results on state constraints (see [3, 21–26]) as well as the duality relation between timeadvanced stochastic differential equations (SDEs, for short) and timedelayed BSDEs (see [10]) may help us to overcome the above mentioned difficulties. Firstly, an equivalent backward formulation of stochastic delayed system (1.1) is introduced, where \(X(T)\) is judged as a control variable. Meanwhile, the state constraint turns out to be a control constraint. However, such a treatment brings us both the advantage and the disadvantage. The advantage is that, in the classical control theory, to manage control constraint is easier than to manage state constraint. The disadvantage is that the initial condition (\(X(0)=\eta(0)\)) now turns into an additional constraint. To deal with the additional initial constraint, Ekeland’s variational principle is used.
Note that the equivalent backward delayed system is described by a timedelayed BSDE, so the adjoint equation of the timedelayed BSDE via duality relation is an anticipated SDE. Therefore, both the delayed system and the anticipated system are needed in our study. As a routine, the variational procedure is made firstly. Then, by virtue of Ekeland’s variational principle, the variational inequality is got. At last, the necessary condition is derived by applying the duality relationship between the backward delayed controlled system and the anticipated forward adjoint system. There is a good thing that the theory of BSDE and our assumption allow us to make the inverse transformation, so that the optimal control process can be solved by the obtained optimal terminal control. To make our conclusions be directly perceived, we also study two applications. One of them is the stochastic delayed linear quadratic (LQ in short) control model. Moreover, a production and consumption choice optimization problem (see [27]) is also adapted to our case.
We organize this article as follows. Some preliminary results about timedelayed BSDE and anticipated SDE are presented in Section 2. In Section 3, the original control problem of a forward delayed controlled system with terminal state constraint is formulated. Then an equivalent transformation is made to get a backward delayed controlled system. Moreover, a stochastic maximum principle is derived, which presents the required condition of the optimal terminal control. In Section 4, two applications are given.
2 Preliminaries
Denote by \((\Omega,\mathcal{F},\mathbb{F},P)\) a probability space such that \(\mathcal{F}_{0}\) includes all Pnull elements of \(\mathcal{F}\) and assume the filtration \(\mathbb{F}=\{\mathcal{F}_{t},t\geq0\}\) is generated by a ddimensional standard Brownian motion \(W=\{W(t), t \geq0\}\). Let \(T>0\). And \(\delta> 0\) is a given finite time delay. We denote the following notations:

\(L^{2}(\mathcal{F}_{t};\mathbb{R}^{n})=\{\xi: \Omega\rightarrow\mathbb{R}^{n}\vert\xi\mbox{ is } \mathcal{F}_{t}\mbox{measurable}, E\vert \xi \vert ^{2}< \infty\}\);

\(L_{\mathbb{F}}^{2}(0,T;\mathbb{R}^{n})=\{\psi: \Omega\times[0,T]\rightarrow\mathbb{R}^{n}\vert\psi(\cdot)\mbox{ is } \mathbb{F}\mbox{measurable process}, E\int_{0}^{T} \vert \psi(t)\vert ^{2}\,dt< \infty\}\).
Similarly, we can define \(L_{\mathbb{F}}^{2}(0,T;\mathbb{R}^{n\times d})\), \(L_{\mathbb{F}}^{2}(\delta,T;\mathbb{R}^{n})\) and \(L_{ \mathbb{F}}^{2}(0,T+\delta;\mathbb{R}^{n})\).
Now we recall some useful results for the study of the following sections. Consider the following SDDE:
where η is a given continuous function, which represents the initial path of X, and \(b:[0,T]\times\mathbb{R}^{n}\times \mathbb{R}^{n}\rightarrow\mathbb{R}^{n}\) and \(\sigma:[0,T]\times \mathbb{R}^{n}\times\mathbb{R}^{n}\rightarrow\mathbb{R}^{n\times d}\) are given measurable functions satisfying the following:
 (H2.1):

There exists a constant \(D>0\) such that for all \(t\in[0,T]\), \(x,x',y,y'\in\mathbb{R}^{n}\),
$$\begin{aligned}& \bigl\vert b(t,x,y)  b \bigl(t,x',y' \bigr) \bigr\vert + \bigl\vert \sigma(t,x,y)  \sigma \bigl(t,x',y' \bigr) \bigr\vert \\& \quad \leq D \bigl( \bigl\vert xx' \bigr\vert + \bigl\vert yy' \bigr\vert \bigr); \\& \sup_{0\leq t\leq T} \bigl( \bigl\vert b(t,0,0)\bigr\vert + \bigl\vert \sigma(t,0,0)\bigr\vert \bigr)< +\infty. \end{aligned}$$
Then, from Theorem 2.2 in [15], under (H2.1), SDDE (2.1) has the unique adapted solution \(X(\cdot)\in L_{\mathbb{F}}^{2}(\delta,T;\mathbb{R}^{n})\).
For the timedelayed BSDE, we need the following assumption.
 (H2.2):

Assume that \(f:\Omega\times[0,T]\times \mathbb{R}^{n}\times\mathbb{R}^{n}\times\mathbb{R}^{n\times d} \rightarrow\mathbb{R}^{n}\) is \(\mathbb{F}\)adapted and for every \(y,y_{\delta},y',y_{\delta}'\in\mathbb{R}^{n}\), \(z,z'\in\mathbb{R} ^{n\times d}\),
$$\bigl\vert f(t,y,y_{\delta},z)f \bigl(t,y',y'_{\delta},z' \bigr) \bigr\vert ^{2}\leq C \bigl( \bigl\vert yy' \bigr\vert ^{2} + \bigl\vert y_{\delta}y_{\delta}' \bigr\vert ^{2} + \bigl\vert zz' \bigr\vert ^{2} \bigr), $$where \(C>0\) is a constant. Moreover, \(E\int_{0}^{T} \vert f(t,0,0,0)\vert ^{2}\,dt < +\infty\).
The following is the wellposedness of timedelayed BSDE.
Proposition 2.1
Suppose \(\xi\in L^{2}(\mathcal{F}_{T};\mathbb{R}^{n})\) and \(\varphi(\cdot)\) is a given continuous function. Then, under (H2.2), for sufficiently small \(\delta>0\), the following timedelayed BSDE
has the unique adapted solution \((Y(\cdot),Z(\cdot))\in L_{ \mathbb{F}}^{2}(\delta,T;\mathbb{R}^{n})\times L_{\mathbb{F}}^{2}(0,T; \mathbb{R}^{n\times d})\), and it satisfies the following estimate:
with \(C>0\). Furthermore, if \((Y'(\cdot),Z'(\cdot))\) is the solution to (2.2) with ξ replaced by \(\xi'\), then
As we stated in the part of Introduction, the anticipated SDE is necessary in our study. The following is the condition for anticipated SDE.
 (H2.3):

Suppose for each \(t\in[0,T]\), \(r\in[t,T+ \delta]\), \(b:\Omega\times\mathbb{R}^{n}\times L^{2}(\mathcal{F} _{r};\mathbb{R}^{n}) \rightarrow L^{2}(\mathcal{F}_{t};\mathbb{R}^{n})\), \(\sigma:\Omega\times\mathbb{R}^{n}\times L^{2}(\mathcal{F}_{r}; \mathbb{R}^{n}) \rightarrow L^{2}(\mathcal{F}_{t};\mathbb{R}^{n\times d})\) with
$$\begin{aligned}& \bigl\vert b(t,x, \varsigma_{t})  b \bigl(t,x', \varsigma'_{t} \bigr) \bigr\vert + \bigl\vert \sigma(t,x, \varsigma_{t})  \sigma \bigl(t,x',\varsigma '_{t} \bigr) \bigr\vert \\& \quad \leq C \bigl( \bigl\vert xx' \bigr\vert + E ^{\mathcal{F}_{t}} \bigl[ \bigl\vert \varsigma_{t}\varsigma'_{t} \bigr\vert \bigr] \bigr), \end{aligned}$$for every \(t\in[0,T]\), \(x,x'\in\mathbb{R}^{n}\), \(\varsigma(\cdot), \varsigma'(\cdot)\in L_{\mathbb{F}}^{2}(t,T+\delta;\mathbb{R}^{n})\), \(r\in[t,T+\delta]\) with \(C>0\). Moreover, \(\sup_{0\leq t \leq T} (\vert b(t,0,0)\vert + \vert \sigma(t,0,0)\vert )< +\infty\).
Proposition 2.2
Suppose \(x_{0}\in\mathbb{R}^{n}\) and \(\lambda(\cdot)\in L_{ \mathbb{F}}^{2}(T,T+\delta;\mathbb{R}^{n})\) is a given \(\mathbb{F}\)adapted process. Assume (H2.3) holds. Then, if δ is sufficiently small, the anticipated SDE
has the unique adapted solution \(X(\cdot)\in L_{\mathbb{F}}^{2}(0,T+ \delta;\mathbb{R}^{n})\).
The above results can be found in Delong and Imkeller [11] and Chen and Huang [10]. The following is the famous Ekeland’s variational principle.
Proposition 2.3
Suppose \((U,d(\cdot,\cdot))\) is a complete metric space with a function \(F(\cdot):U\rightarrow\mathbb{R}\) is proper lower semicontinuous. Then, for every \(v\in U\) and \(\epsilon>0\) such that \(F(v)\leq\inf_{u\in U} F(u) +\epsilon\), there is \(u_{\epsilon }\in U\) so that

(i)
\(F(v_{\epsilon})\leq F(v)\),

(ii)
\(d(v,v_{\epsilon})\leq\epsilon\),

(iii)
\(F(u) + \sqrt{\epsilon} d(u,v_{\epsilon})\geq F(v_{\epsilon })\), \(\forall u\in U\).
3 Main result
We study our main result in this part, i.e., a maximum principle about the optimal control of a stochastic delayed system involving terminal state constraint. It should be pointed out that the timedelayed state of the controlled system is different from the case without delay.
3.1 Problem formulation
Let
be the set of admissible controls. For every given \(u(\cdot)\), for the control system, we consider the pastdependent state \(X(\cdot)\) depicted as
where η is a given continuous function, \(b:[0,T]\times \mathbb{R}^{n}\times\mathbb{R}^{n}\times\mathbb{R}^{n\times d} \rightarrow\mathbb{R}^{n}\) and \(\sigma:[0,T]\times\mathbb{R}^{n} \times\mathbb{R}^{n}\times\mathbb{R}^{n\times d}\rightarrow \mathbb{R}^{n\times d}\) are given measurable functions. Define the cost function as follows:
where \(\widetilde{l}:[0,T]\times\mathbb{R}^{n}\times\mathbb{R}^{n \times d}\rightarrow\mathbb{R}^{n}\) and \(\phi:\mathbb {R}^{n}\rightarrow \mathbb{R}^{n}\) are given measurable functions. We give the following assumptions:
 (H3.1):

The functions b, σ, l̃, ϕ are all continuously differentiable in the arguments \((x,x',u)\), and their derivatives are all bounded.
 (H3.2):

Denote by \(C(1 + \vert x\vert + \vert u\vert )\) and \(C(1 + \vert x\vert )\) the bounds of derivatives of l̃ in its arguments \((x,u)\) and ϕ in its argument x, respectively.
Therefore, for every given \(u(\cdot)\in\mathcal{U}_{\mathrm{ad}}\), under assumptions (H3.1) and (H3.2), Eq. (3.1) admits the unique adapted solution \(X(\cdot)\in L_{\mathbb{F}}^{2}(\delta,T;\mathbb{R} ^{n})\).
Denote by \(K\in\mathbb{R}^{n}\) a given nonempty convex subset. The goal of our control problem is to solve
3.2 Timedelayed backward formulation
We now show an equivalent backward system of Problem A. In order to do this, one additional assumption is needed:
 (H3.3):

There exists \(\alpha>0\), and for each \(t\in[0,T]\), \(x,x'\in\mathbb{R}^{n}\) and \(u_{1},u_{2}\in\mathbb{R}^{n\times d}\),
$$\bigl\vert \sigma \bigl(t,x,x',u_{1} \bigr)  \sigma \bigl(t,x,x',u_{2} \bigr) \bigr\vert \geq \alpha \vert u_{1}u_{2}\vert . $$
Note that (H3.1) and (H3.3) imply, for every \((t,x,x')\in[0,T]\times \mathbb{R}^{n} \times\mathbb{R}^{n}\), that the following function
is a bijection on \(\mathbb{R}^{n\times d}\). Hence, by letting \(q\equiv\sigma(t,x,x',u)\), we obtain that there is the inverse \(\sigma^{1}\) satisfying \(u=\sigma^{1}(t,x,x',q)\). Then we can rewrite (3.1) as
where \(f(t,x,x',q)=b(t,x,x',\sigma^{1}(t,x,x',q))\).
Note that \(u \rightarrow\sigma(t,x,x',u)\) is a bijection, hence \(q(\cdot)\) could be regarded as the control, which is the crucial observation that encourages this method for working out Problem A. Furthermore, by virtue of the theory of BSDE, choosing the terminal state \(X(T)\) is equal to choosing \(q(\cdot)\). Therefore we innovate the following ‘controlled’ system, which essentially is a timedelayed BSDE:
where now ξ becomes the ‘control’ and belongs to the following set:
Moreover, here the equivalent cost function is
with \(l(t,x,q)=\widetilde{l}(t,x,\sigma^{1}(t,x,x',q))\).
Hence, the original Problem A is equivalent to the following Problem B:
where \(X^{\xi}(0)=a\) (we denote \(a=\eta(0)\) in the following for simplicity) is the solution to Eq. (3.2) at the initial time 0 under ξ.
In control theory, it is well known that to solve the control constraint is easer than to solve the state constraint. From now on, since Problem A is equivalent to Problem B, we concentrate on dealing with Problem B. The benefit is that by virtue of ξ becoming a control variable now, a control constraint in Problem B replaces the state constraint in Problem A.
Definition 3.1
For \(\xi\in U\) and \(a\in\mathbb{R}^{n}\), if the solution to (3.2) suits \(X^{\xi}(0)=a\), then we call the random variable ξ feasible. For any given a, the collection of every feasible ξ is denoted by \(\mathcal{N}(a)\). Moreover, if \(\xi^{*}\in U\) gets the minimum value of \(J(\xi)\) over \(\mathcal{N}(a)\), we call \(\xi^{*}\) optimal.
3.3 Variational equation
In the following subsections, we denote the following notations for simplicity:
where \(f_{\varphi}\) denotes the partial derivative of f at φ with \(\varphi=x,x_{\delta},q\), respectively. In U, for \(\xi^{1},\xi^{2}\in U\), define a metric by
Apparently, \((U,d(\cdot,\cdot))\) becomes a complete metric space. Suppose \(\xi^{*}\) is optimal and, associated with \(\xi^{*}\), the pair \((X^{*}(\cdot),q^{*}(\cdot))\) is the corresponding state processes of Eq. (3.2). Because U is convex, for every ξ, the following variational control \(\xi^{\rho}\) is also in U:
Denote the solution to Eq. (3.2) associated with \(\xi=\xi^{ \rho}\) by \((X^{\rho}(\cdot),q^{\rho}(\cdot))\). And denote by \((\widehat{X}(\cdot),\widehat{q}(\cdot))\) the solution of the following variational equation:
Remark 3.2
It is easy to know that (3.4) is a linear timedelayed BSDE. By Proposition 2.1, under conditions (H3.1)(H3.3), Eq. (3.4) has a unique adapted solution in \(L_{\mathbb{F}}^{2}(\delta,T; \mathbb{R}^{n})\times L_{\mathbb{F}}^{2}(0,T;\mathbb{R}^{n\times d})\).
Lemma 3.3
Under assumptions (H3.1)(H3.3), one has
where
Remark 3.4
Since the proof of Lemma 3.3 above is the same as that of Lemma 3.1 in Chen and Huang [10], for simplicity of presentation, we only present the main result and omit the detailed proof. In fact, it is straightforward to prove Lemma 3.3 by applying Proposition 2.1, Taylor expansion and the Lebesgue dominated convergence theorem.
3.4 Variational inequality
We solve the initial constraint \(X^{\xi}(0)=a\) and obtain a variational inequality in this subsection.
Given the optimal \(\xi^{*}\), for a constant \(\varepsilon>0\), define \(F\varepsilon(\cdot):U\rightarrow\mathbb{R}\) as follows:
Remark 3.5
One can test that the functions \(\vert X^{\xi}(0)a\vert ^{2}\) and \(\phi( \xi)\) are both continuous in their argument ξ. Hence, \(F_{\varepsilon}\), defined on U, is also a continuous function in its argument ξ.
Theorem 3.6
Under assumptions (H3.1)(H3.3), suppose that \(\xi^{*}\) is an optimal solution to Problem B, so we have \(h_{0}\in\mathbb{R}^{+}\) and \(h_{1}\in\mathbb{R}^{n}\) satisfying \(\vert h_{0}\vert +\vert h_{1}\vert \neq0\), so that for every \(\xi\in U\), we have the following variational inequality:
where \(\widehat{X}(0)\) is the solution to Eq. (3.4) at time 0.
Proof
We can check the following properties by the definition:
Therefore, from Proposition 2.3 (Ekeland’s variational principle), there exists \(\xi^{\varepsilon}\in U\) satisfying:

(i)
\(F_{\varepsilon} (\xi^{\varepsilon} ) \leq F_{\varepsilon} (\xi ^{*} )\);

(ii)
\(d (\xi^{\varepsilon},\xi^{*} )\leq\sqrt{ \varepsilon}\);

(iii)
\(F_{\varepsilon}(\xi)+\sqrt{\varepsilon}d (\xi, \xi^{\varepsilon} )\geq F_{\varepsilon} (\xi^{\varepsilon} )\), \(\forall\xi\in U\).
For every \(\xi\in U\), denote \(\xi_{\rho}^{\varepsilon}:= \xi^{\varepsilon}+ \rho(\xi \xi^{\varepsilon}), 0\leq\rho \leq1\). Let \((X_{\rho}^{\varepsilon}(\cdot),q_{\rho}^{\varepsilon }(\cdot))\) (resp. \(X^{\varepsilon}(\cdot)\), \(q^{\varepsilon}(\cdot)\)) be the solution to (3.2) under \(\xi_{\rho}^{\varepsilon}\) (resp. \(\xi^{\varepsilon}\)), and let \((\widehat{X}^{\varepsilon}(\cdot), \widehat{q}^{\varepsilon}(\cdot))\) be the solution to (3.4) when \(\xi^{\varepsilon}\) is replaced by \(\xi^{*}\). Hence, applying the item (iii) above, one obtains
On the other hand, similar to Lemma 3.3, one concludes
Thus
which leads to the following expansion:
Moreover,
In the next, we study the following two cases for given \(\varepsilon >0\).
Case 1: There is \(\rho_{0}>0\) so that for every \(\rho\in(0,\rho_{0})\),
We see that
Now, using ρ to split (3.6) and letting ρ to 0, one has
where
Case 2: There is a positive series \(\{\rho_{n}\}\) satisfying \(\rho_{n}\rightarrow0\), so that
From the definition of \(F_{\varepsilon}\), for large n, \(F_{\varepsilon }(\xi_{\rho_{n}}^{\varepsilon})= \{ \vert X_{\rho _{n}}^{\varepsilon}(0)a\vert ^{2} \} ^{\frac{1}{2}}\). Owing to the continuity of \(F_{\varepsilon}(\cdot)\), one has \(F_{\varepsilon}(\xi^{\varepsilon })= \{ \vert X^{\varepsilon}(0)a\vert ^{2} \} ^{\frac{1}{2}}\).
Moreover,
From (3.6), the same as in Case 1,
where
For both cases, in summary, from the definition of \(F_{\varepsilon}( \cdot)\), one has \(h_{0}^{\varepsilon}\geq0\) and
Therefore, there exists a convergent subsequence of \((h_{0}^{\varepsilon },h_{1}^{\varepsilon})\) whose limit is denoted by \((h_{0},h_{1})\).
Due to \(d(\xi^{\varepsilon},\xi^{*})\leq\sqrt{\varepsilon}\), we have \(\xi^{\varepsilon}\rightarrow\xi^{*}\), as \(\varepsilon \rightarrow0\). Then, from the estimate of Proposition 2.1, we see that \(\widehat{X}^{\varepsilon}(0)\rightarrow\widehat{X}(0)\) as \(\varepsilon\rightarrow0\). Thus (3.5) holds. The desired result is proved now. □
By using similar analysis, when \(l(t,x,q)\neq0\), the following variational inequality can be obtained.
Theorem 3.7
Let (H3.1)(H3.3) hold. Suppose that \(\xi^{*}\) is an optimal solution of Problem B. Then we have \(h_{0}\in\mathbb{R}^{+}\), \(h_{1}\in \mathbb{R}^{n}\) satisfying \(\vert h_{0}\vert +\vert h_{1}\vert \neq0\), so that for every \(\xi\in U\), we have the variational inequality:
where \(l_{\varphi}^{*}(t)=l_{\varphi}(t,X^{*}(t),q^{*}(t))\) denotes the partial derivative of \(l^{*}\) at φ with \(\varphi= x,q\), respectively, and \((\widehat{X}(\cdot),\widehat{q}(\cdot))\) is the solution to variation equation (3.4).
3.5 Maximum principle
For the sake of establishing the maximum principle, in this part, as the dual equation of Eq. (3.4), the following equation is introduced:
Remark 3.8
In Eq. (3.10), \(f^{*}_{x_{\delta}}\vert_{t+\delta}\) represents the value of \(f^{*}_{x_{\delta}}\) when t is replaced by \(t+\delta\).
Remark 3.9
It is easy to see that (3.10) is a linear timeadvanced SDE. By Proposition 2.2, under conditions (H3.1)(H3.3), Eq. (3.10) admits the unique adapted solution in \(L_{\mathbb{F}}^{2}(0, T+\delta;\mathbb{R}^{n})\).
Theorem 3.10
Let (H3.1)(H3.3) hold. If \(\xi^{*}\) is optimal to Problem B with \((X^{*}(\cdot),q^{*}(\cdot))\) being the corresponding state of Eq. (3.2), then we have \(h_{0}\in\mathbb{R}^{+}\) and \(h_{1}\in \mathbb{R}^{n}\) satisfying \(\vert h_{0}\vert +\vert h_{1}\vert \neq0\), so that for every \(\eta\in U\),
where \(m(\cdot)\) is the solution of Eq. (3.10).
Proof
By using Itô’s formula to \(\langle m(t),\widehat{X}(t) \rangle \), we obtain
Therefore,
with
Paying attention to the terminal and initial conditions, one derives
Hence
From the arbitrariness of \(\xi\in U\), for every \(\eta\in U\), we have
□
Now, we let ∂K represent the boundary of K and denote
According to Theorem 3.10, we directly deduce the following result.
Corollary 3.11
Assume that the assumptions in Theorem 3.10 hold, then for each \(\eta\in K\), we have
Remark 3.12
By the above study, for the optimal terminal control \(\xi^{*}\), we obtain the necessary condition. Note that the previous transformation process and (H3.3) allow us to make the inverse transformation. Therefore, the characterization of the optimal control process \(u^{*}(\cdot)\) can be derived by the obtained stochastic maximum principle of the optimal terminal control \(\xi^{*}\).
4 Applications of the main result
As stated in the section of Introduction, we study two applications of the main result established above in this section.
4.1 Stochastic delayed LQ control involving terminal state constraints
Stochastic delayed LQ control problem involving terminal state constraints is considered in this subsection. In order to simplify the presentation, we focus on the case \(d=n=1\). For the higher dimensional situation, one can deal with it in a similar method without substantial difficulty.
Consider the following state equation:
with \(A_{i},B_{i}\in\mathbb{R}\), \(i=1,2,3\).
Next, we investigate the cost function independent of the running cost without loss of generality. Therefore, subject to \(u(\cdot)\in \mathcal{U}_{\mathrm{ad}}\), \(X(T)\in\mathbb{R}^{+}\), a.s., the goal is to minimize the following cost function:
Without doubt, question (4.2) is an extraordinary example of Problem A with
Now we give the backward formulation of problem (4.2). Denote
Then Eq. (4.1) becomes
and we can rewrite problem (4.2) as follows:
where
By Theorem 3.10, if \(\xi^{*}\) is optimal, we have \(h_{0}\in \mathbb{R}^{+}\) and \(h_{1}\in\mathbb{R}\) satisfying \(\vert h_{0}\vert +\vert h_{1}\vert \neq0\), so that \(\forall \eta\in U\),
in which \(m(\cdot)\) is the solution to the following adjoint equation:
Denote \(\Omega_{0}:=\{ \omega\in\Omega\vert \xi^{*}(\omega)=0 \}\). Now, the following necessary condition can be deduced owing to the arbitrariness of ξ.
where \(m(\cdot)\) is the solution of Eq. (4.6).
4.2 Productionconsumption choice optimization problem
By applying the maximum principle established before, we investigate a type of productionconsumption choice optimization problem in this subsection. The shape for this issue, as in [15], originates from Ivanov and Swishchuk [27]. For the sake of completeness, let us present the model at length.
We assume an investor is going to invest his money to invent goods, and he could obtain benefits from the goods. We mark the capital of investor, the labor at time t and the rate of consumption by \(X(t)\), \(A(t)\) and \(c(t)\geq0\), respectively. Based upon the assumption that earning of the production is a function of the total sum of the capital and labor, in order to depict this system, Ramsey [28] introduced the following shape:
Since in the procedure of investment, in reality, there is some risk and delay, Chen and Wu [15] generalized the model in (4.7) to the following case:
where η is a given continuous function.
However, the rationality of the shape has been questioned as there is no constraint for the terminal capital \(X(T)\) on the basis of the hypothesis. In fact, in real situations, sometimes the investor will set a goal (constraint) for the terminal capital \(X(T)\) in the investment. Hence we believe that in a concordant model of the production and consumption some constraints for the terminal capital \(X(T)\) should be considered, i.e., \(X(T)\in Q\), where \(Q\in\mathbb{R}^{n}\).
Let us consider the following model which modified (4.7) and (4.8):
For simplicity, let \(n=d=1\). Consider the following hypotheses:

(1)
The function \(f(X(t\delta),A(t))=KX^{\alpha}(t\delta)A ^{\beta}(t)\), where K, α, β are some suitable constants. Moreover, let \(\alpha=\beta=1\) and \(A(t)\equiv y\) be a constant.

(2)
The terminal constraint \(Q\in\mathbb{R}\) is a given convex set.
Under the above assumptions, we can rewrite our shape as follows:
By electing the hypothesis rate \(c(t)\geq0\) under the terminal constraint \(X(T)\in Q\), the purpose is to maximize the following desired function:
where r represents the bond rate, \(\gamma\in(0,1)\), and \(1\gamma\) represents the investor’s risk aversion.
Clearly, this is a special case of Problem A when
with
Now, let \(q\equiv\sigma(x',c)\) and \(f(x',q)=Ky\cdot x' + \widetilde{\sigma}(x',q)\), where σ̃ is the inverse function of σ w.r.t. c, i.e., \(c=\widetilde{\sigma}:= \widetilde{\sigma}(x',q)\). Then one can rewrite (4.9) as follows:
Define \(U=\{ \xi \vert E\vert \xi\vert^{2}<\infty, \xi\in Q, \mbox{a.s.} \}\) and consider the following performance function:
Then problem (4.10) is equivalent to the following problem:
Consider the adjoint equation
where \(h_{1}\in\mathbb{R}\) is a parameter. Denote \(\Omega_{0}:=\{ \omega\in\Omega\vert \xi^{*}(\omega)=\partial Q \}\). Therefore, by using Theorem 3.10, one obtains the result below.
Theorem 4.1
Suppose \((X^{*}(\cdot),c^{*}(\cdot))\) is an optimal pair to problem (4.10), then we have \(h_{0}\in\mathbb{R}^{+}\) and \(h_{1}\in \mathbb{R}\) satisfying \(\vert h_{0}\vert +\vert h_{1}\vert \neq 0\) so that \(\xi^{*}\equiv X^{*}(T)\), we have
where \(m(\cdot)\) is the solution to Eq. (4.14) with parameter \(h_{1}\).
5 Conclusions
In this content, we study a stochastic optimal control problem for stochastic differential delayed equation with terminal state constraint (at the terminal time, the state is constrained in a convex set). However, the control problem with terminal nonconvex state constraint is still open. We will focus on the open problem in the future study.
References
Pardoux, E, Peng, S: Adapted solution of a backward stochastic differential equation. Syst. Control Lett. 4, 5561 (1990)
Peng, S: Probabilistic interpretations for systems of quasilinear parabolic partial differential equation. Stoch. Stoch. Rep. 37, 6174 (1991)
El Karoui, N, Peng, S, Quenez, MC: A dynamic maximum principle for the optimization of recursive utilities under constraints. Ann. Appl. Probab. 11, 664693 (2001)
Yong, J, Zhou, X: Stochastic Controls: Hamiltonian Systems and HJB Equations. Springer, New York (1999)
El Karoui, N, Peng, S, Quenez, MC: Backward stochastic differential equations in finance. Math. Finance 7(1), 171 (1997)
Barles, G, Buckdahn, R, Pardoux, E: Backward stochastic differential equations and integralpartial differential equations. Stoch. Stoch. Rep. 60, 5783 (1997)
Li, J, Wei, Q: \(L^{p}\) estimates for fully coupled FBSDEs with jumps. Stoch. Process. Appl. 124(4), 15821611 (2014)
Li, J, Wei, Q: Stochastic differential games for fully coupled FBSDEs with jumps. Appl. Math. Optim. 71, 411448 (2015)
Buckdahn, R, Li, J, Peng, S: Meanfield backward stochastic differential equations and related partial differential equations. Stoch. Process. Appl. 119(10), 31333154 (2009)
Chen, L, Huang, J: Stochastic maximum principle for controlled backward delayed system via advanced stochastic differential equation. J. Optim. Theory Appl. 167(3), 11121135 (2015)
Delong, Ł, Imkeller, P: Backward stochastic differential equations with time delayed generators results and counterexamples. Ann. Appl. Probab. 20, 15121536 (2010)
Delong, Ł, Imkeller, P: On Malliavin’s differentiability of BSDE with time delayed generators driven by Brownian motions and Poisson random measures. Stoch. Process. Appl. 120, 17481775 (2010)
Peng, S, Yang, Z: Anticipated backward stochastic differential equations. Ann. Probab. 37, 877902 (2009)
Wen, J, Shi, Y: Anticipative backward stochastic differential equations driven by fractional Brownian motion. Stat. Probab. Lett. 122, 118127 (2017)
Chen, L, Wu, Z: Maximum principle for the stochastic optimal control problem with delay and application. Automatica 46, 10741080 (2010)
Øksendal, B, Sulem, A, Zhang, T: Optimal control of stochastic delay equations and timeadvanced backward stochastic differential equations. Adv. Appl. Probab. 43(2), 572596 (2011)
Yu, Z: The stochastic maximum principle for optimal control problems of delay systems involving continuous and impulse controls. Automatica 48(10), 24202432 (2012)
Agram, N, Haadem, S, Øksendal, B, Proske, F: A maximum principle for infinite horizon delay equations. SIAM J. Math. Anal. 45(4), 24992522 (2013)
Huang, J, Li, X, Shi, J: Forwardbackward linear quadratic stochastic optimal control problem with delay. Syst. Control Lett. 61(5), 623630 (2012)
Meng, Q, Shen, Y: Optimal control of meanfield jumpdiffusion systems with delay: a stochastic maximum principle approach. J. Comput. Appl. Math. 279, 1330 (2015)
Ji, S, Zhou, X: A maximum principle for stochastic optimal control with terminal state constraints and its applications. Commun. Inf. Syst. 6, 321338 (2006). A special issue dedicated Tyrone Duncan on the occasion of his 65th birthday
Ji, S, Peng, S: Terminal perturbation method for the backward approach to continuous time meanvariance portfolio selection. Stoch. Process. Appl. 118(6), 952967 (2008)
Ji, S, Zhou, X: A generalized NeymanPearson lemma under gprobabilities. Probab. Theory Relat. Fields 148, 645669 (2010)
Ji, S, Wei, Q: A maximum principle for fully coupled forwardbackward stochastic control systems with terminal state constraints. J. Math. Anal. Appl. 407, 200210 (2013)
Aghayeva, C: Stochastic linear quadratic control problem of switching systems with constraints. J. Inequal. Appl. 2016, 100 (2016)
Wei, Q: Stochastic maximum principle for meanfield forwardbackward stochastic control system with terminal state constraints. Sci. China Math. 59(4), 809822 (2016)
Ivanov, A, Swishchuk, A: Optimal control of stochastic differential delay equations with application in economics. In: Conference on Stochastic Modelling of Complex Systems, Australia (2005)
Ramsey, F: A mathematical theory of savings. Econ. J. 38, 543559 (1928)
Acknowledgements
The first author would like to appreciate the Department of Mathematics of University of Central Florida, USA, for its hospitality, and express the gratitude to Prof. Qingmeng Wei for her careful reading of this paper and helpful comments.
The first and second authors are supported by NNSF of China (Grant Nos. 11371226, 11071145, 11526205, 11626247 and 11231005), the Foundation for Innovative Research Groups of National Natural Science Foundation of China (Grant No. 11221061) and the 111 Project (Grant No. B12023).
Author information
Authors and Affiliations
Corresponding author
Additional information
Competing interests
The first and second authors announce to have no competing interests.
Authors’ contributions
The first and second authors contributed equally to the writing of this paper. Moreover, the first and second authors read and approved the final manuscript.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Wen, J., Shi, Y. Maximum principle for a stochastic delayed system involving terminal state constraints. J Inequal Appl 2017, 103 (2017). https://doi.org/10.1186/s136600171378z
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s136600171378z