Maximum principle for a stochastic delayed system involving terminal state constraints
 Jiaqiang Wen^{1} and
 Yufeng Shi^{1, 2}Email author
https://doi.org/10.1186/s136600171378z
© The Author(s) 2017
Received: 24 January 2017
Accepted: 21 April 2017
Published: 5 May 2017
Abstract
We investigate a stochastic optimal control problem where the controlled system is depicted as a stochastic differential delayed equation; however, at the terminal time, the state is constrained in a convex set. We firstly introduce an equivalent backward delayed system depicted as a timedelayed backward stochastic differential equation. Then a stochastic maximum principle is obtained by virtue of Ekeland’s variational principle. Finally, applications to a state constrained stochastic delayed linearquadratic control model and a productionconsumption choice problem are studied to illustrate the main obtained result.
Keywords
MSC
1 Introduction
In 1990, the nonlinear backward stochastic differential equation (BSDE in short) was introduced by Pardoux and Peng [1]. Until now, it has had applications in many fields, such as partial differential equation (see [2]), stochastic control (see [3, 4]) and mathematical finance (see [5]). Meanwhile, BSDE itself has been developed to many different branches, such as BSDE with jumps (see [6–8]), meanfield BSDE (see [9]), timedelayed BSDE (see [10–12]), anticipated BSDEs (see [13, 14]) and so on. A lot of works have been done for the control problem of such BSDEs. However, fewer works have been done on the control problems of stochastic delayed systems.
For a stochastic delayed system, Chen and Wu [15] obtained a stochastic maximum principle by virtue of a duality between stochastic differential delayed equations (SDDEs in short) and anticipated BSDEs. Øksendal, Sulem and Zhang [16] studied the optimal control problems for SDDEs with jumps. Yu [17] obtained a maximum principle for SDDEs with random coefficients. A maximum principle of optimal control of SDDEs on infinite horizon was proved in Agram, Haadem and Øksendal [18]. Some other recent developments on stochastic delayed system can be found in Huang, Li and Shi [19], Meng and Shen [20], etc.
Some recent developed results on state constraints (see [3, 21–26]) as well as the duality relation between timeadvanced stochastic differential equations (SDEs, for short) and timedelayed BSDEs (see [10]) may help us to overcome the above mentioned difficulties. Firstly, an equivalent backward formulation of stochastic delayed system (1.1) is introduced, where \(X(T)\) is judged as a control variable. Meanwhile, the state constraint turns out to be a control constraint. However, such a treatment brings us both the advantage and the disadvantage. The advantage is that, in the classical control theory, to manage control constraint is easier than to manage state constraint. The disadvantage is that the initial condition (\(X(0)=\eta(0)\)) now turns into an additional constraint. To deal with the additional initial constraint, Ekeland’s variational principle is used.
Note that the equivalent backward delayed system is described by a timedelayed BSDE, so the adjoint equation of the timedelayed BSDE via duality relation is an anticipated SDE. Therefore, both the delayed system and the anticipated system are needed in our study. As a routine, the variational procedure is made firstly. Then, by virtue of Ekeland’s variational principle, the variational inequality is got. At last, the necessary condition is derived by applying the duality relationship between the backward delayed controlled system and the anticipated forward adjoint system. There is a good thing that the theory of BSDE and our assumption allow us to make the inverse transformation, so that the optimal control process can be solved by the obtained optimal terminal control. To make our conclusions be directly perceived, we also study two applications. One of them is the stochastic delayed linear quadratic (LQ in short) control model. Moreover, a production and consumption choice optimization problem (see [27]) is also adapted to our case.
We organize this article as follows. Some preliminary results about timedelayed BSDE and anticipated SDE are presented in Section 2. In Section 3, the original control problem of a forward delayed controlled system with terminal state constraint is formulated. Then an equivalent transformation is made to get a backward delayed controlled system. Moreover, a stochastic maximum principle is derived, which presents the required condition of the optimal terminal control. In Section 4, two applications are given.
2 Preliminaries

\(L^{2}(\mathcal{F}_{t};\mathbb{R}^{n})=\{\xi: \Omega\rightarrow\mathbb{R}^{n}\vert\xi\mbox{ is } \mathcal{F}_{t}\mbox{measurable}, E\vert \xi \vert ^{2}< \infty\}\);

\(L_{\mathbb{F}}^{2}(0,T;\mathbb{R}^{n})=\{\psi: \Omega\times[0,T]\rightarrow\mathbb{R}^{n}\vert\psi(\cdot)\mbox{ is } \mathbb{F}\mbox{measurable process}, E\int_{0}^{T} \vert \psi(t)\vert ^{2}\,dt< \infty\}\).
 (H2.1):

There exists a constant \(D>0\) such that for all \(t\in[0,T]\), \(x,x',y,y'\in\mathbb{R}^{n}\),$$\begin{aligned}& \bigl\vert b(t,x,y)  b \bigl(t,x',y' \bigr) \bigr\vert + \bigl\vert \sigma(t,x,y)  \sigma \bigl(t,x',y' \bigr) \bigr\vert \\& \quad \leq D \bigl( \bigl\vert xx' \bigr\vert + \bigl\vert yy' \bigr\vert \bigr); \\& \sup_{0\leq t\leq T} \bigl( \bigl\vert b(t,0,0)\bigr\vert + \bigl\vert \sigma(t,0,0)\bigr\vert \bigr)< +\infty. \end{aligned}$$
Then, from Theorem 2.2 in [15], under (H2.1), SDDE (2.1) has the unique adapted solution \(X(\cdot)\in L_{\mathbb{F}}^{2}(\delta,T;\mathbb{R}^{n})\).
 (H2.2):

Assume that \(f:\Omega\times[0,T]\times \mathbb{R}^{n}\times\mathbb{R}^{n}\times\mathbb{R}^{n\times d} \rightarrow\mathbb{R}^{n}\) is \(\mathbb{F}\)adapted and for every \(y,y_{\delta},y',y_{\delta}'\in\mathbb{R}^{n}\), \(z,z'\in\mathbb{R} ^{n\times d}\),where \(C>0\) is a constant. Moreover, \(E\int_{0}^{T} \vert f(t,0,0,0)\vert ^{2}\,dt < +\infty\).$$\bigl\vert f(t,y,y_{\delta},z)f \bigl(t,y',y'_{\delta},z' \bigr) \bigr\vert ^{2}\leq C \bigl( \bigl\vert yy' \bigr\vert ^{2} + \bigl\vert y_{\delta}y_{\delta}' \bigr\vert ^{2} + \bigl\vert zz' \bigr\vert ^{2} \bigr), $$
The following is the wellposedness of timedelayed BSDE.
Proposition 2.1
 (H2.3):

Suppose for each \(t\in[0,T]\), \(r\in[t,T+ \delta]\), \(b:\Omega\times\mathbb{R}^{n}\times L^{2}(\mathcal{F} _{r};\mathbb{R}^{n}) \rightarrow L^{2}(\mathcal{F}_{t};\mathbb{R}^{n})\), \(\sigma:\Omega\times\mathbb{R}^{n}\times L^{2}(\mathcal{F}_{r}; \mathbb{R}^{n}) \rightarrow L^{2}(\mathcal{F}_{t};\mathbb{R}^{n\times d})\) withfor every \(t\in[0,T]\), \(x,x'\in\mathbb{R}^{n}\), \(\varsigma(\cdot), \varsigma'(\cdot)\in L_{\mathbb{F}}^{2}(t,T+\delta;\mathbb{R}^{n})\), \(r\in[t,T+\delta]\) with \(C>0\). Moreover, \(\sup_{0\leq t \leq T} (\vert b(t,0,0)\vert + \vert \sigma(t,0,0)\vert )< +\infty\).$$\begin{aligned}& \bigl\vert b(t,x, \varsigma_{t})  b \bigl(t,x', \varsigma'_{t} \bigr) \bigr\vert + \bigl\vert \sigma(t,x, \varsigma_{t})  \sigma \bigl(t,x',\varsigma '_{t} \bigr) \bigr\vert \\& \quad \leq C \bigl( \bigl\vert xx' \bigr\vert + E ^{\mathcal{F}_{t}} \bigl[ \bigl\vert \varsigma_{t}\varsigma'_{t} \bigr\vert \bigr] \bigr), \end{aligned}$$
Proposition 2.2
The above results can be found in Delong and Imkeller [11] and Chen and Huang [10]. The following is the famous Ekeland’s variational principle.
Proposition 2.3
 (i)
\(F(v_{\epsilon})\leq F(v)\),
 (ii)
\(d(v,v_{\epsilon})\leq\epsilon\),
 (iii)
\(F(u) + \sqrt{\epsilon} d(u,v_{\epsilon})\geq F(v_{\epsilon })\), \(\forall u\in U\).
3 Main result
We study our main result in this part, i.e., a maximum principle about the optimal control of a stochastic delayed system involving terminal state constraint. It should be pointed out that the timedelayed state of the controlled system is different from the case without delay.
3.1 Problem formulation
 (H3.1):

The functions b, σ, l̃, ϕ are all continuously differentiable in the arguments \((x,x',u)\), and their derivatives are all bounded.
 (H3.2):

Denote by \(C(1 + \vert x\vert + \vert u\vert )\) and \(C(1 + \vert x\vert )\) the bounds of derivatives of l̃ in its arguments \((x,u)\) and ϕ in its argument x, respectively.
3.2 Timedelayed backward formulation
 (H3.3):

There exists \(\alpha>0\), and for each \(t\in[0,T]\), \(x,x'\in\mathbb{R}^{n}\) and \(u_{1},u_{2}\in\mathbb{R}^{n\times d}\),$$\bigl\vert \sigma \bigl(t,x,x',u_{1} \bigr)  \sigma \bigl(t,x,x',u_{2} \bigr) \bigr\vert \geq \alpha \vert u_{1}u_{2}\vert . $$
In control theory, it is well known that to solve the control constraint is easer than to solve the state constraint. From now on, since Problem A is equivalent to Problem B, we concentrate on dealing with Problem B. The benefit is that by virtue of ξ becoming a control variable now, a control constraint in Problem B replaces the state constraint in Problem A.
Definition 3.1
For \(\xi\in U\) and \(a\in\mathbb{R}^{n}\), if the solution to (3.2) suits \(X^{\xi}(0)=a\), then we call the random variable ξ feasible. For any given a, the collection of every feasible ξ is denoted by \(\mathcal{N}(a)\). Moreover, if \(\xi^{*}\in U\) gets the minimum value of \(J(\xi)\) over \(\mathcal{N}(a)\), we call \(\xi^{*}\) optimal.
3.3 Variational equation
Remark 3.2
It is easy to know that (3.4) is a linear timedelayed BSDE. By Proposition 2.1, under conditions (H3.1)(H3.3), Eq. (3.4) has a unique adapted solution in \(L_{\mathbb{F}}^{2}(\delta,T; \mathbb{R}^{n})\times L_{\mathbb{F}}^{2}(0,T;\mathbb{R}^{n\times d})\).
Lemma 3.3
Remark 3.4
Since the proof of Lemma 3.3 above is the same as that of Lemma 3.1 in Chen and Huang [10], for simplicity of presentation, we only present the main result and omit the detailed proof. In fact, it is straightforward to prove Lemma 3.3 by applying Proposition 2.1, Taylor expansion and the Lebesgue dominated convergence theorem.
3.4 Variational inequality
We solve the initial constraint \(X^{\xi}(0)=a\) and obtain a variational inequality in this subsection.
Remark 3.5
One can test that the functions \(\vert X^{\xi}(0)a\vert ^{2}\) and \(\phi( \xi)\) are both continuous in their argument ξ. Hence, \(F_{\varepsilon}\), defined on U, is also a continuous function in its argument ξ.
Theorem 3.6
Proof
 (i)
\(F_{\varepsilon} (\xi^{\varepsilon} ) \leq F_{\varepsilon} (\xi ^{*} )\);
 (ii)
\(d (\xi^{\varepsilon},\xi^{*} )\leq\sqrt{ \varepsilon}\);
 (iii)
\(F_{\varepsilon}(\xi)+\sqrt{\varepsilon}d (\xi, \xi^{\varepsilon} )\geq F_{\varepsilon} (\xi^{\varepsilon} )\), \(\forall\xi\in U\).
Due to \(d(\xi^{\varepsilon},\xi^{*})\leq\sqrt{\varepsilon}\), we have \(\xi^{\varepsilon}\rightarrow\xi^{*}\), as \(\varepsilon \rightarrow0\). Then, from the estimate of Proposition 2.1, we see that \(\widehat{X}^{\varepsilon}(0)\rightarrow\widehat{X}(0)\) as \(\varepsilon\rightarrow0\). Thus (3.5) holds. The desired result is proved now. □
By using similar analysis, when \(l(t,x,q)\neq0\), the following variational inequality can be obtained.
Theorem 3.7
3.5 Maximum principle
Remark 3.8
In Eq. (3.10), \(f^{*}_{x_{\delta}}\vert_{t+\delta}\) represents the value of \(f^{*}_{x_{\delta}}\) when t is replaced by \(t+\delta\).
Remark 3.9
It is easy to see that (3.10) is a linear timeadvanced SDE. By Proposition 2.2, under conditions (H3.1)(H3.3), Eq. (3.10) admits the unique adapted solution in \(L_{\mathbb{F}}^{2}(0, T+\delta;\mathbb{R}^{n})\).
Theorem 3.10
Proof
Corollary 3.11
Remark 3.12
By the above study, for the optimal terminal control \(\xi^{*}\), we obtain the necessary condition. Note that the previous transformation process and (H3.3) allow us to make the inverse transformation. Therefore, the characterization of the optimal control process \(u^{*}(\cdot)\) can be derived by the obtained stochastic maximum principle of the optimal terminal control \(\xi^{*}\).
4 Applications of the main result
As stated in the section of Introduction, we study two applications of the main result established above in this section.
4.1 Stochastic delayed LQ control involving terminal state constraints
Stochastic delayed LQ control problem involving terminal state constraints is considered in this subsection. In order to simplify the presentation, we focus on the case \(d=n=1\). For the higher dimensional situation, one can deal with it in a similar method without substantial difficulty.
4.2 Productionconsumption choice optimization problem
By applying the maximum principle established before, we investigate a type of productionconsumption choice optimization problem in this subsection. The shape for this issue, as in [15], originates from Ivanov and Swishchuk [27]. For the sake of completeness, let us present the model at length.
However, the rationality of the shape has been questioned as there is no constraint for the terminal capital \(X(T)\) on the basis of the hypothesis. In fact, in real situations, sometimes the investor will set a goal (constraint) for the terminal capital \(X(T)\) in the investment. Hence we believe that in a concordant model of the production and consumption some constraints for the terminal capital \(X(T)\) should be considered, i.e., \(X(T)\in Q\), where \(Q\in\mathbb{R}^{n}\).
 (1)
The function \(f(X(t\delta),A(t))=KX^{\alpha}(t\delta)A ^{\beta}(t)\), where K, α, β are some suitable constants. Moreover, let \(\alpha=\beta=1\) and \(A(t)\equiv y\) be a constant.
 (2)
The terminal constraint \(Q\in\mathbb{R}\) is a given convex set.
Theorem 4.1
5 Conclusions
In this content, we study a stochastic optimal control problem for stochastic differential delayed equation with terminal state constraint (at the terminal time, the state is constrained in a convex set). However, the control problem with terminal nonconvex state constraint is still open. We will focus on the open problem in the future study.
Declarations
Acknowledgements
The first author would like to appreciate the Department of Mathematics of University of Central Florida, USA, for its hospitality, and express the gratitude to Prof. Qingmeng Wei for her careful reading of this paper and helpful comments.
The first and second authors are supported by NNSF of China (Grant Nos. 11371226, 11071145, 11526205, 11626247 and 11231005), the Foundation for Innovative Research Groups of National Natural Science Foundation of China (Grant No. 11221061) and the 111 Project (Grant No. B12023).
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Authors’ Affiliations
References
 Pardoux, E, Peng, S: Adapted solution of a backward stochastic differential equation. Syst. Control Lett. 4, 5561 (1990) MathSciNetView ArticleMATHGoogle Scholar
 Peng, S: Probabilistic interpretations for systems of quasilinear parabolic partial differential equation. Stoch. Stoch. Rep. 37, 6174 (1991) MathSciNetView ArticleMATHGoogle Scholar
 El Karoui, N, Peng, S, Quenez, MC: A dynamic maximum principle for the optimization of recursive utilities under constraints. Ann. Appl. Probab. 11, 664693 (2001) MathSciNetView ArticleMATHGoogle Scholar
 Yong, J, Zhou, X: Stochastic Controls: Hamiltonian Systems and HJB Equations. Springer, New York (1999) View ArticleMATHGoogle Scholar
 El Karoui, N, Peng, S, Quenez, MC: Backward stochastic differential equations in finance. Math. Finance 7(1), 171 (1997) MathSciNetView ArticleMATHGoogle Scholar
 Barles, G, Buckdahn, R, Pardoux, E: Backward stochastic differential equations and integralpartial differential equations. Stoch. Stoch. Rep. 60, 5783 (1997) MathSciNetView ArticleMATHGoogle Scholar
 Li, J, Wei, Q: \(L^{p}\) estimates for fully coupled FBSDEs with jumps. Stoch. Process. Appl. 124(4), 15821611 (2014) MathSciNetView ArticleMATHGoogle Scholar
 Li, J, Wei, Q: Stochastic differential games for fully coupled FBSDEs with jumps. Appl. Math. Optim. 71, 411448 (2015) MathSciNetView ArticleMATHGoogle Scholar
 Buckdahn, R, Li, J, Peng, S: Meanfield backward stochastic differential equations and related partial differential equations. Stoch. Process. Appl. 119(10), 31333154 (2009) MathSciNetView ArticleMATHGoogle Scholar
 Chen, L, Huang, J: Stochastic maximum principle for controlled backward delayed system via advanced stochastic differential equation. J. Optim. Theory Appl. 167(3), 11121135 (2015) MathSciNetView ArticleMATHGoogle Scholar
 Delong, Ł, Imkeller, P: Backward stochastic differential equations with time delayed generators results and counterexamples. Ann. Appl. Probab. 20, 15121536 (2010) MathSciNetView ArticleMATHGoogle Scholar
 Delong, Ł, Imkeller, P: On Malliavin’s differentiability of BSDE with time delayed generators driven by Brownian motions and Poisson random measures. Stoch. Process. Appl. 120, 17481775 (2010) MathSciNetView ArticleMATHGoogle Scholar
 Peng, S, Yang, Z: Anticipated backward stochastic differential equations. Ann. Probab. 37, 877902 (2009) MathSciNetView ArticleMATHGoogle Scholar
 Wen, J, Shi, Y: Anticipative backward stochastic differential equations driven by fractional Brownian motion. Stat. Probab. Lett. 122, 118127 (2017) MathSciNetView ArticleMATHGoogle Scholar
 Chen, L, Wu, Z: Maximum principle for the stochastic optimal control problem with delay and application. Automatica 46, 10741080 (2010) MathSciNetView ArticleMATHGoogle Scholar
 Øksendal, B, Sulem, A, Zhang, T: Optimal control of stochastic delay equations and timeadvanced backward stochastic differential equations. Adv. Appl. Probab. 43(2), 572596 (2011) MathSciNetView ArticleMATHGoogle Scholar
 Yu, Z: The stochastic maximum principle for optimal control problems of delay systems involving continuous and impulse controls. Automatica 48(10), 24202432 (2012) MathSciNetView ArticleMATHGoogle Scholar
 Agram, N, Haadem, S, Øksendal, B, Proske, F: A maximum principle for infinite horizon delay equations. SIAM J. Math. Anal. 45(4), 24992522 (2013) MathSciNetView ArticleMATHGoogle Scholar
 Huang, J, Li, X, Shi, J: Forwardbackward linear quadratic stochastic optimal control problem with delay. Syst. Control Lett. 61(5), 623630 (2012) MathSciNetView ArticleMATHGoogle Scholar
 Meng, Q, Shen, Y: Optimal control of meanfield jumpdiffusion systems with delay: a stochastic maximum principle approach. J. Comput. Appl. Math. 279, 1330 (2015) MathSciNetView ArticleMATHGoogle Scholar
 Ji, S, Zhou, X: A maximum principle for stochastic optimal control with terminal state constraints and its applications. Commun. Inf. Syst. 6, 321338 (2006). A special issue dedicated Tyrone Duncan on the occasion of his 65th birthday MathSciNetMATHGoogle Scholar
 Ji, S, Peng, S: Terminal perturbation method for the backward approach to continuous time meanvariance portfolio selection. Stoch. Process. Appl. 118(6), 952967 (2008) MathSciNetView ArticleMATHGoogle Scholar
 Ji, S, Zhou, X: A generalized NeymanPearson lemma under gprobabilities. Probab. Theory Relat. Fields 148, 645669 (2010) View ArticleMATHGoogle Scholar
 Ji, S, Wei, Q: A maximum principle for fully coupled forwardbackward stochastic control systems with terminal state constraints. J. Math. Anal. Appl. 407, 200210 (2013) MathSciNetView ArticleMATHGoogle Scholar
 Aghayeva, C: Stochastic linear quadratic control problem of switching systems with constraints. J. Inequal. Appl. 2016, 100 (2016) MathSciNetView ArticleMATHGoogle Scholar
 Wei, Q: Stochastic maximum principle for meanfield forwardbackward stochastic control system with terminal state constraints. Sci. China Math. 59(4), 809822 (2016) MathSciNetView ArticleMATHGoogle Scholar
 Ivanov, A, Swishchuk, A: Optimal control of stochastic differential delay equations with application in economics. In: Conference on Stochastic Modelling of Complex Systems, Australia (2005) Google Scholar
 Ramsey, F: A mathematical theory of savings. Econ. J. 38, 543559 (1928) View ArticleGoogle Scholar