Open Access

Stochastic singular optimal control problem of switching systems with constraints

Journal of Inequalities and Applications20162016:13

https://doi.org/10.1186/s13660-015-0944-5

Received: 4 July 2015

Accepted: 13 December 2015

Published: 7 January 2016

Abstract

This paper is devoted to the optimal control problem of switching system in which constraints on the state variable are given by inclusions. Using Ekeland’s variational principle, second-order necessary condition of optimality for stochastic switching systems with constraints is obtained, and transversality conditions for switching law are established.

Keywords

stochastic optimal control necessary condition of optimality singular switching systems transversality conditions

1 Introduction

Stochastic differential equations have provided a lot of interest for problems of nuclear fission, communication systems, self-oscillating systems, etc., where the influences of random disturbances cannot be ignored [1, 2].

Hybrid systems, including special class-switching systems, are widely use for representation of noninvariant phenomena. Switching systems have the benefit of modeling dynamic processes with continuous law of movement. For general theory of stochastic switching systems, we refer to [35]. Recently, optimization problems of hybrid systems and, in particular, optimization problems of switching systems have attracted a lot of theoretical and practical interest [3, 610].

Stochastic control problems have a variety of practical applications in fields such as physics, biology, economics, management sciences, etc. [11, 12]. A classical approach for optimization and particularly for control problems is to derive necessary conditions satisfied by an optimal solution. The modern stochastic optimal control theory has been developed along the lines of Pontryagin’s maximum principle and Bellman’s dynamic programming [13, 14]. The stochastic maximum principle has been first considered by Kushner [15]. The earliest results on the extension of Pontryagin’s maximum principle to stochastic control problems are obtained in [1619]. A general theory of the stochastic maximum principle based on random convex analysis was given by Bismut [20]. Modern presentations of stochastic maximum principle with backward stochastic differential equations are considered in [2123]. First-order necessary conditions of optimality for stochastic switching systems have been studied by the author in [2428].

It is well known that the first-order necessary conditions provide a basic tool to study the properties of optimal controls. However, in some cases, the Hamilton function vanishes almost everywhere, and therefore, first-order necessary conditions cannot provide enough information to find the desired optimal controls. The above-mentioned cases are called singular, and to investigate these situations, an additional test in terms of high-order necessary conditions is required. Deterministic optimization problems for singular controls were intensively used by [29, 30]. Such kind problems for stochastic systems have been investigated in [31, 32].

In this paper, the singular optimal control problem of stochastic switching systems with uncontrolled diffusion coefficients is considered. Second-order necessary conditions of optimality and transversality conditions are obtained in [33] for uncontrolled switching systems. Ekeland’s variational principle [34] has been widely used in various areas of analysis such as fixed point analysis, optimization, and optimal control theory. In this paper, backward stochastic differential equations and Ekeland’s variational principle is used to establish a singular maximum principle for stochastic optimal control problems of constrained switching systems.

2 Preliminaries and formulation of problem

Throughout this paper, we use the following notation. By N we denote some positive constant, \(R^{n} \) denotes the n-dimensional real vector space, \(|\cdot| \) denotes the Euclidean norm in \(R^{n}\), and E represents the mathematical expectation. Assume that \(w^{1}(t) ,w^{2}(t) ,\ldots,w^{r}(t) \) are independent Wiener processes that generate the filtration \(F_{t}^{l} = \bar{\sigma}(w^{l}(t) ,t_{l - 1}, t_{l} )\), \(l = \overline{1,r} \), \(0 = t_{0} < t_{1} < \cdots < t_{r} = T\). Let \((\Omega,F,P)\) be a probability space with filtration \(\{F_{t} ,t \in [ {0,T} ] \}\), where \(F_{t} = \bigcup_{l = 1}^{r} {F_{t}^{l} } \); \(L_{F}^{2} ( {a,b;R^{n}} ) \) denotes the space of all predictable processes \(x(t, \omega)\equiv x(t)\) such that \(E\int _{a}^{b} {\vert {x_{t} ( \omega )} \vert ^{2}\,dt < + \infty} \); \(R^{m\times n}\) is the space of linear transformations from \(R^{m}\) to \(R^{n}\). Let \(O_{l} \subset R^{n_{l} }\), \(Q_{l} \subset R^{m_{l} }\), \(l = \overline{1,r} \), be open sets. Unless specified otherwise, we use the following notation: \(\mathbf{t}=(t_{0},t_{1},\ldots,t_{r})\), \(\mathbf{u}=(u^{1},u^{2},\ldots,u^{r})\), \(\mathbf {x}=(x^{1},x^{2},\ldots,x^{r})\).

Consider the following stochastic control system:
$$\begin{aligned}& dx^{l}(t) = g^{l} \bigl( {x^{l}(t) ,u^{l}(t) ,t} \bigr)\,dt + f^{l} \bigl( {x^{l}(t),t} \bigr)\,dw^{l}(t),\quad t \in( {t_{l - 1} ,t_{l} } ] , l = \overline{1,r} , \end{aligned}$$
(1)
$$\begin{aligned}& x^{l}(t_{l - 1}) = \Phi^{l - 1} \bigl( {x^{l - 1}(t_{l - 1}) ,t_{l - 1}} \bigr),\quad l = \overline{2,r } ;x_{t_{0} }^{1} = x_{0} , \end{aligned}$$
(2)
$$\begin{aligned}& u^{l}(t) \in U_{\partial}^{l} \equiv \bigl( {u^{l} ( {\cdot,\cdot} )\in \mathrm{L}_{F^{l}}^{2}| u^{l} ( {t,\cdot} )\in U^{l}\subset\mathrm{R}^{m_{l} }, \mathrm{a.c.}} \bigr). \end{aligned}$$
(3)
Elements of \(U_{\partial}^{l} \) are called admissible controls. The problem is to find an optimal solution \(\mathbf{(}x,u)= ( {x^{1},x^{2},\ldots,x^{r},u^{1},u^{2},\ldots,u^{r}} )\) and a switching sequence \(\mathbf{t}=(t_{1} ,t_{2} ,\ldots,t_{r}) \) such that the cost functional
$$ J(u) = \sum_{l = 1}^{r} {E \biggl[ {\varphi^{l} \bigl( {x^{l}(t_{l}) } \bigr) + \int _{t_{l-1} }^{t_{l} } {p^{l} \bigl( {x^{l}(t) ,u^{l}(t) ,t} \bigr)\,dt} } \biggr]} $$
(4)
is minimized on the decisions of system (1)-(3) generated by all admissible controls \(U = U^{1}\times U^{2}\times\cdots\times U^{r}\) under the conditions
$$ E q^{l}\bigl({x^{l}(t_{l})} \bigr)\in G^{l},\quad l = \overline{1,r }, $$
(5)
\(G^{l}\) are a closed convex sets in \(R^{1}\).
Assume that the following requirements are satisfied:
  1. (AI)

    The functions \(g^{l}\), \(f^{l}\), \(p^{l}\), \(l=\overline{1,r} \), and their derivatives are continuous in \(( {x,u,t} )\).

     
  2. (AII)

    The derivatives of \(g^{l}\), \(f^{l}\), \(p^{l}\), \(l=\overline {1,r} \), are bounded by \(N( {1+\vert x \vert })\).

     
  3. (AIII)

    The functions \(\varphi^{l}(x)\), \(l=\overline {1,r}\), and the functions \(\Phi^{l}(x,t)\), \(l=\overline {1,r-1}\), are continuously differentiable, and their derivatives are bounded by \(N( {1+\vert x \vert })\).

     
  4. (AIV)

    The functions \(q^{l}(x)\), \(l=\overline {1,r}\), are continuously differentiable, and their derivatives are bounded by \(N( {1+\vert x \vert })\).

     
Furthermore, the following requirements are assumed.
  1. (BI)

    The unctions \(g^{l}\), \(f^{l}\), \(p^{l}\), \(l= \overline{1,r} \), are twice continuously differentiable with respect to x.

     
  2. (BII)

    The functions \(g^{l}\), \(f^{l}\), \(p^{l}\), \(l = \overline{1,r} \), and all their derivatives are continuous in \(( {x,u} )\); \(g_{x}^{l}\), \(g_{xx}^{l}\), \(f_{x}^{l}\), \(f_{xx}^{l}\), \(p_{xx}^{l} \) are bounded.

     
  3. (BIII)

    The functions \(\varphi^{l}(x):R^{n_{l}} \to R^{1}\), \(l = \overline{1,r}\), and the functions \(\Phi^{l}(x,t):R^{n_{l} }\times{\mathrm {T}} \to R^{1}\), \(l =\overline{1,r - 1} \), are twice continuously bounded differentiable.

     
  4. (BIV)

    The functions \(q^{l}(x):R^{n_{l}} \to R^{1}\), \(l = \overline{1,r} \), are twice continuously bounded differentiable.

     
Consider the sets:
$$A_{i} = \mathrm{T}^{i + 1}\times\prod _{j = 1}^{i} {O_{j} } \times\prod _{j = 1}^{i} {Q_{j} } , \quad i = \overline{1,r}, $$
with the elements
$$\pi^{i} = \bigl(t_{0} ,t_{1} ,t_{i} ,x^{1}(t) ,x^{2}(t) ,\ldots,x^{i}(t),u^{1},u^{2}, \ldots,u^{i}\bigr). $$

Definition 1

The set of functions \(\lbrace{x^{l}(t) = x^{l} ({t,\pi^{l}} ) }, t \in [ {t_{l - 1} ,t_{l} } ], l = \overline{1,r}\rbrace\) is said to be a solution of equations (1)-(2) with variable structure corresponding to an element \(\pi^{r} \in{\mathrm {A}}_{r} \) if the function \(x^{l}(t) \in O_{l} \) on the interval \([ {t_{l - 1} ,t_{l} } ]\) satisfies condition (2) at point \(t_{l}\), is absolutely continuous on the interval \([ {t_{l - 1} ,t_{l} } ]\) with probability 1, and satisfies equation (1) almost everywhere.

Definition 2

The element \(\pi^{r} \in A_{r} \) is said to be admissible if the pairs \(( {x^{l}(t) , u^{l}(t) } )\), \(t \in [ {t_{l - 1} ,t_{l} } ]\), \(l = \overline{1,r} \), are solutions of system (1)-(3) and satisfy the constraints (5).

Definition 3

Let \(A_{r}^{0} \) be the set of admissible elements. The element \(\bar{\pi}^{r} \in A_{r}^{0} \) is said to be an optimal solution of problem (1)-(5) if there exist admissible controls \(\bar {u}^{l}(t)\), \(t \in [ {t_{l - 1} ,t_{l} } ]\), \(l = \overline {1,r} \), and the corresponding solutions \(\bar {x}^{l}(t)\), \(t \in [ {t_{l - 1} ,t_{l} } ]\), \(l = \overline {1,r} \), of system (1)-(2) such that the pairs \(( {\bar{x}^{l}(t) ,\bar{u}^{l}(t)} )\), \(l = \overline{1,r} \), minimize the functional (4).

3 Maximum principle

The following is a revised and improved version of Theorem 1 in [25].

Theorem 1

Suppose that, conditions (AI)-(AIII) hold and
$$\pi^{r}=\bigl(t_{0} ,t_{1}, t_{r} ,x^{1}(t) ,x^{2}(t) ,\ldots, x^{r}(t), u^{1}, u^{2}, \ldots, u^{r}\bigr) $$
is an optimal solution of problem (1)-(4). Then
  1. (a)
    there exist random processes \((\psi^{l}(t) ,\beta^{l}(t))\in{L}_{F}^{2} (t_{l-1} ,t_{l} ;{R}^{n_{l} })\times{L}_{F}^{2} (t_{l-1} ,t_{l} ;{R}^{n_{l}\times n_{l}})\) that are the solutions of the following adjoint equations:
    $$ \left \{ \textstyle\begin{array}{@{}l} d\psi^{l}(t) =-H_{x}^{l} (\psi^{l}(t) ,x^{l}(t) ,u^{l}(t) ,t)\,dt+\beta^{l}(t)\,dw^{l}(t), \quad t_{l-1} \le t< t_{l} , \\ \psi^{l}(t_{l}) =-\varphi_{x}^{l} (x^{l}(t_{l}))+\psi^{l+1}(t_{l}) \Phi_{x}^{l} (x^{l}(t_{l}) ,t_{l} ),\quad l=\overline{1,r-1} , \\ \psi^{r}(t_{r}) =-\varphi_{x}^{r} (x^{r}(t_{r})); \end{array}\displaystyle \right . $$
    (6)
     
  2. (b)
    \(\forall \bar{u}^{l}\in U^{l}\), \(l=\overline{1,r}\), a.c. fulfills the maximum principle:
    $$ H^{l}\bigl(\psi^{l}(\theta) ,x^{l}( \theta) ,\bar{u}^{l},\theta\bigr)-H^{l}\bigl( \psi^{l}(\theta) ,x^{l}(\theta) ,u^{l}(\theta) , \theta\bigr)\le0, \quad\textit{a.e. } \theta\in[t_{l-1} , t_{l} ] ; $$
    (7)
     
  3. (c)
    the following transversality conditions hold:
    $$\psi^{l+1}(t_{l})\Phi_{t}^{l} \bigl( {x^{l}(t_{l}), t_{l}} \bigr)=0, \quad \textit{a.c.}, l=\overline{1,r-1} , $$
    (8)
    where, for \(t\in [ {t_{l-1} ,t_{l} } ] \),
    $$H^{l} \bigl( {\psi(t), x(t), u(t), t} \bigr)=\psi(t) g^{l} \bigl( {x(t), u(t), t} \bigr)+\beta(t) f^{l} \bigl( {x(t), t} \bigr)-p^{l} \bigl( {x(t), u(t), t} \bigr). $$
     

The following result can be proved according to the scheme in the proof of Theorem 2 in [25]. In order to investigate the optimal control problem with constraints (5), we use Ekeland’s variational principle.

Theorem 2

Suppose that conditions (AI)-(AIV) hold and
$$\pi^{r}=\bigl(t_{0} ,t_{1}, t_{r} ,x^{1}(t) ,x^{2}(t) ,\ldots, x^{r}(t), u^{1}, u^{2}, \ldots, u^{r}\bigr) $$
is an optimal solution of problem (1)-(5). Then
  1. (a)
    there exist random processes \((\psi^{l}(t) ,\beta^{l}(t))\in{L}_{F}^{2} (t_{l-1} ,t_{l} ;{R}^{n_{l} })\times{L}_{F}^{2} (t_{l-1} ,t_{l} ;{R}^{n_{l}\times n_{l}})\) that are solutions of the following adjoint equations:
    $$ \left \{ \textstyle\begin{array}{@{}l} d\psi^{l}(t) =-H_{x}^{l} (\psi^{l}(t) ,x^{l}(t) ,u^{l}(t) ,t)\,dt+\beta^{l}(t)\,dw^{l}(t), \quad t_{l-1} \le t< t_{l} , \\ \psi^{l}(t_{l}) =- \lambda_{0}^{l} q_{x}^{l} (x^{l}(t_{l}) )-\lambda_{1}^{l}\varphi_{x}^{l} (x^{l}(t_{l}))+\psi^{l+1}(t_{l}) \Phi_{x}^{l} (x^{l}(t_{l}) ,t_{l} ), \quad l=\overline{1,r-1} , \\ \psi^{r}(t_{r}) =- \lambda_{0}^{r} q_{x}^{r} (x^{r}(t_{r}) )-\lambda_{1}^{r}\varphi_{x}^{r} (x^{r}(t_{r})); \end{array}\displaystyle \right . $$
    (9)
     
  2. (b)
    \(\forall\bar{u}^{l}\in U^{l}\), \(l=\overline{1,r}\), a.c. fulfills the maximum principle:
    $$ H^{l}\bigl(\psi^{l}(\theta) ,x^{l}( \theta) ,\bar{u}^{l},\theta\bigr)-H^{l}\bigl( \psi^{l}(\theta) ,x^{l}(\theta) ,u^{l}(\theta) , \theta\bigr)\le0, \quad\textit{a.e. } \theta\in[t_{l-1} , t_{l} ] ; $$
    (10)
     
  3. (c)
    the following transversality conditions hold:
    $$\psi^{l+1}(t_{l})\Phi_{t}^{l} \bigl( {x^{l}(t_{l}), t_{l}} \bigr)=0, \quad \textit{a.c.}, l=\overline{1,r-1} , $$
    (11)
    where, for \(t\in [ {t_{l-1} ,t_{l} } ] \),
    $$H^{l} \bigl( {\psi(t), x(t), u(t), t} \bigr)=\psi(t) g^{l} \bigl( {x(t), u(t), t} \bigr)+\beta(t) f^{l} \bigl( {x(t), t} \bigr)-p^{l} \bigl( {x(t), u(t), t} \bigr). $$
     

4 The second-order necessary condition

In order to review in detail the stochastic singular control switching systems, we need to introduce the definition of singular controls in the maximum principle sense.

Definition 4

Admissible controls \(u^{l}(t)\), \(l=1,\ldots,r\), are said to be singular on control regions \(V^{l}\) if each \(V^{l}\subset U^{l}\) is nonempty and, for a.e. \(t \in[t_{l - 1} ,t_{l} ) \), we have
$$H_{x}^{l} \bigl(\psi^{l}(t) ,x^{l}(t) ,u^{l}(t) ,t\bigr)=H_{x}^{l} \bigl( \psi^{l}(t) ,x^{l}(t) ,v^{l} ,t\bigr),\quad \forall v^{l}\in V^{l}. $$

To state the main result of this paper, we will use the following theorem, the full proof of which can be found in [33]. However, for convenience of the reader, we present a brief proof.

Theorem 3

Suppose that conditions (AI)-(AIII) and (BI)-(BIII) hold. Let \(\pi^{r} = (t_{0} ,t_{1} ,\ldots, t_{r} ,x(t_{1}) ,x(t_{2}) ,\ldots,x(t_{r}), u^{1},u^{2},\ldots,u^{r})\) be an optimal solution of problem (1)-(4), and \(\mathbf{u} = ( u^{1},u^{2},\ldots,u^{r})\) be singular on the control region \(\mathbf{V} = ( V^{1},V^{2},\ldots,V^{r})\). Then
  1. (a)
    there exist random processes \((\psi^{l}(t) ,\beta^{l}(t)) \in L_{F^{l}}^{2}(t_{l - 1} ,t_{l} ;R^{n_{l} })\times L_{F^{l}}^{2} (t_{l - 1}, t_{l} ;R^{n_{l}\times n_{l} })\) and \((\Psi^{l}(t) ,\mathrm{K}^{l}(t) ) \in L_{F^{l}}^{2} (t_{l - 1} ,t_{l} ;R^{n_{l} })\times L_{F^{l}}^{2} (t_{l - 1} ,t_{l} ;R^{n_{l} \times n_{l} })\) that are solutions of the following adjoint equations:
    $$\begin{aligned}& \left \{ \textstyle\begin{array}{@{}l} d\psi^{l}(t) = - H_{x}^{l} (\psi^{l}(t) ,x^{l}(t) ,u^{l}(t) ,t)\,dt + \beta^{l}(t)\,dw(t) ,\quad t_{l - 1}\leq t < t_{l} ,\\ \psi^{l}(t_{l} ) = - \varphi_{x}^{l} (x^{l}(t_{l}) ) + \psi^{l + 1}(t_{l} ) \Phi _{x}^{l} (x^{l}(t_{l}) ,t_{l} ),\quad l = \overline{1,r - 1} , \\ \psi^{r}(t_{r} ) = - \varphi_{x}^{r} (x^{r}(t_{r})) ; \end{array}\displaystyle \right . \end{aligned}$$
    (12)
    $$\begin{aligned}& \left \{ \textstyle\begin{array}{@{}l} d\Psi^{l}(t) = -[\mathbf{{H}}_{x}^{l} ( {\Psi_{t} ^{l},x^{l}(t) ,u^{l}(t) ,t} ) + H_{xx}^{l} (\psi^{l}(t) ,x^{l}(t) ,u^{l}(t) ,t)\\ \hphantom{d\Psi^{l}(t) =}{}+ f_{x}^{l\ast} (x^{l}(t) ,t)\Psi^{l}(t) f_{x}^{l} (x^{l}(t) ,t) ]\,dt + \mathrm{K}^{l}(t)\,dw^{l}(t) ,\quad t \in[t_{l - 1} ,t_{l} ), \\ \Psi^{l}(t_{l}) = - \varphi_{xx}^{l} (x^{l}(t_{l}) ) + \Psi^{l + 1}(t_{l}) \Phi_{xx}^{l} (x^{l}(t_{l}) ,t_{l} ),\quad l = \overline{1,r - 1} , \\ \Psi^{r}(t_{r}) = - \varphi_{xx}^{r}(x^{r}(t_{r})); \end{array}\displaystyle \right . \end{aligned}$$
    (13)
     
  2. (b)
    for a.e. \(\theta\in[t_{l - 1} ,t_{l} ] \) and \(\forall \bar{u}^{l} \in V^{l}\), \(l = \overline{1,r} \), a.c. the second-order maximum conditions fulfill
    $$\begin{aligned} &\Delta_{\bar{u}^{l}} H_{x}^{l}\bigl( \psi^{l}(\theta),x^{l}(\theta),u^{l},\theta\bigr) \Delta _{\bar{u}^{l}} g^{l\ast}\bigl(x^{l}(\theta) ,u^{l}(\theta),\theta\bigr) \\ &\quad{} +\Delta_{\bar{u}^{l}} g^{l\ast}\bigl(x^{l}(\theta) ,u^{l}(\theta),\theta \bigr)\Psi^{l}(\theta) \Delta_{\bar{u}^{l}} g^{l}\bigl(x^{l}(\theta) ,u^{l}(\theta),\theta \bigr) \le0; \end{aligned}$$
    (14)
     
  3. (c)
    the following transversality conditions hold:
    $$\begin{aligned} &\psi^{l + 1}(t_{l}) \Phi_{t}^{l} \bigl( {x^{l}(t_{l}) ,t_{l}} \bigr)+0.5\Psi ^{{l + 1}^{*}}(t_{l}) \Phi_{tt}^{l} \bigl( {x^{l}(t_{l}) ,t_{l} } \bigr)\Psi^{l + 1}(t_{l}) = 0 , \\ &\quad l = \overline{1,r - 1} , \textit{a.c.} \end{aligned}$$
    (15)
    Here
    $$\begin{aligned} \mathbf{H} ^{l}\bigl( {\Psi(t),x(t),u(t),t} \bigr) ={}& \Psi(t) g^{l}\bigl({x(t) ,u(t) ,t}\bigr) + g^{l\ast} \bigl( {x(t) ,u(t),t} \bigr)\Psi(t) \\ &{}+\mathrm{K }(t) f^{l} \bigl( {x(t) ,t} \bigr)+ f^{l\ast} \bigl( {x(t) ,t} \bigr)\mathrm{K }(t). \end{aligned}$$
     

Proof

Let \(\bar{u}^{l}(t)= u^{l}(t) + \Delta\bar{u}^{l}(t)\), \(l = \overline {1,r}\), be some admissible controls and \(\bar{x}^{l}(t)= x^{l}(t) + \Delta\bar {x}^{l}(t)\), \(l = \overline{1,r}\), be the corresponding trajectories of system (1)-(3). Let \(0 = t_{0} < t_{1} < \cdots < t_{r} = T\) be a switching sequence. Then we obtain the following identities for some sequence \(0 = \bar{t}_{0} < \bar{t}_{1} < \cdots < \bar {t}_{r} =T\):
$$ \left \{ \textstyle\begin{array}{@{}l} d\Delta\bar{x}^{l}(t) = [\Delta_{\bar{u}^{l}} g^{l}(x^{l}(t) ,u^{l}(t) ,t) + g_{x}^{l} (x^{l}(t) ,u^{l}(t) ,t)\Delta\bar{x}^{l}(t) \\ \hphantom{d\Delta\bar{x}^{l}(t) =}{}+0.5\Delta x^{\ast}(t)g_{xx}^{l} (x^{l}(t) ,u^{l}(t) ,t)\Delta x(t) ]\,dt \\ \hphantom{d\Delta\bar{x}^{l}(t) =}{}+ [f_{x}^{l} (x^{l}(t) ,t)\Delta\bar{x}^{l}(t) + 0.5\Delta x^{\ast}(t)f_{xx}^{l} (x^{l}(t) ,t)\Delta x(t) ]\,dw^{l}(t) + \eta_{t}^{1} , \\ \Delta\bar{x}^{1}(t_{0}) = 0, \\ \Delta\bar{x}^{l}(t_{l - 1}) = \Phi^{l - 1} ( {\bar{x} ^{l - 1}(t_{l - 1}) ,\bar{t}_{l - 1} } ) - \Phi^{l - 1} (x ^{l - 1}(t_{l - 1}) ,t_{l - 1} ),\quad l = \overline{2,r}, \end{array}\displaystyle \right . $$
(16)
where
$$\begin{aligned} \eta_{t}^{1} ={}& \int _{0}^{1} { \bigl[ {g_{x}^{l\ast} \bigl(x^{l}(t) + \mu \Delta\bar{x}^{l}(t) , \bar{u}_{t}^{l} ,t\bigr) - g_{x}^{l\ast} \bigl(x^{l}(t) ,u^{l}(t) ,t\bigr)} \bigr]} \Delta \bar{x}^{l}(t)\,d\mu \,dt \\ &{}+0.5 \int _{0}^{1} \Delta\bar{x}^{l\ast}(t) \bigl[ g_{xx}^{l\ast} \bigl(x^{l}(t) + \mu\Delta \bar{x}^{l}(t) ,u^{l}(t) ,t\bigr) - {g_{xx}^{\ast}\bigl(x^{l}(t) ,u^{l}(t) ,t\bigr)} \bigr] {\Delta\bar {x}^{l}(t)\,d\mu}\,dt \\ &{}+ \int _{0}^{1} { \bigl[ {f_{x}^{l\ast} \bigl(x^{l}(t) + \mu\Delta\bar {x}^{l}(t),t\bigr) - f_{x}^{l\ast} \bigl(x^{l}(t) ,t\bigr)} \bigr]} \Delta \bar{x}^{l}(t)\,d\mu \,dw^{l}(t) \\ &{}+0.5 \int _{0}^{1} \Delta\bar{x}^{l\ast}(t) \bigl[ f_{xx}^{l\ast} \bigl(x^{l}(t)+\mu\Delta \bar{x}^{l}(t) ,t\bigr) - {f_{xx}^{\ast}\bigl(x^{l}(t) ,t\bigr)} \bigr] { \Delta\bar {x}^{l}(t)\,d\mu}\,dw^{l}(t) . \end{aligned}$$

In order to establish the existence and uniqueness of a solution of adjoint stochastic differential equations, it suffices to follow the method described in the article [17] and to use the independence of Wiener processes \({w}^{1}(t),\ldots,{w}^{r}(t)\).

The stochastic processes \(\psi^{l}(t) \), \(l=\overline{1,r} \), at the points \(t_{1} ,t_{2} ,\ldots,t_{r} \) are defined as
$$ \psi^{l}(t_{l}) = - \varphi_{x}^{l} \bigl( {x^{l}(t_{l}) } \bigr) + \psi^{l + 1}(t_{l}) \Phi_{x}^{l} \bigl({x^{l}(t_{l} ) ,t_{l} } \bigr),\qquad \psi^{r}(t_{r})= - \varphi_{x}^{r} \bigl( {x(t_{r})^{r}} \bigr) $$
(17)
and
$$ \Psi^{l}(t_{l}) = - \varphi_{xx}^{l} \bigl( {x^{l}(t_{l}) } \bigr) + \Psi^{l + 1}(t_{l}) \Phi_{xx}^{l} \bigl( {x^{l}(t_{l}) ,t_{l} } \bigr),\qquad \Psi^{r}(t_{r}) = - \varphi_{xx}^{r} \bigl( {x^{r}(t_{r})} \bigr). $$
(18)
According to Itô’s formula [12], the expression of an increment of the cost functional (4) takes the following form:
$$\begin{aligned} \Delta J ( u )=&{}\sum_{l=1}^{r} E \biggl\{ {\varphi ^{l} \bigl( {\bar{x}^{l} (t_{l})} \bigr)-\varphi^{l} \bigl( {x^{l}(t_{l}) } \bigr)+ \int _{t_{l-1} }^{t_{l} } { \bigl[ {p^{l} \bigl( { \bar{x}^{l}(t) ,\bar {u}_{t}^{l} ,t} \bigr) -p^{l} \bigl( {x^{l}(t) ,u^{l}(t) ,t} \bigr)} \bigr]\,dt} } \biggr\} \\ ={}& {-}\sum_{l=1}^{r} E \int _{t_{-1} }^{t_{l} } \bigl[ {\Delta _{\bar{u}^{l}}H^{l} \bigl( {\psi^{l}(t) ,x^{l}(t) ,u^{l}(t) ,t} \bigr) +H_{x}^{l} \bigl( {\psi ^{l}(t) ,x^{l}(t) ,u^{l}(t) ,t} \bigr)}\Delta\bar{x}^{l} (t) \\ &{}-0.5\Delta\bar{x}^{l\ast}(t) f_{x}^{l\ast} \bigl( {x^{l}(t) ,t} \bigr) \Psi^{l}(t) f_{x}^{l} \bigl( {x^{l}(t) ,t} \bigr)\Delta\bar{x}^{l} (t) \\ &{}+\Delta\bar{x}^{l\ast}(t) \Delta _{\bar{u}^{l}} g^{l} \bigl({x^{l}(t) ,u^{l}(t),t} \bigr) \Psi^{l}(t) \Delta\bar {x}^{l}(t) \\ &{}-\Delta\bar{x}^{l\ast}(t) g_{x}^{l} \bigl(x^{l}(t) ,u^{l}(t) ,t \bigr) \Psi^{l}(t) \Delta\bar{x}^{l} (t) - \Delta\bar{x}^{l\ast}(t) f_{x}^{l} \bigl( x^{l}(t) ,t \bigr) K_{t}^{l} \Delta\bar{x}^{l} (t) \\ &{}+\psi_{t}^{l *} \Delta_{\bar{u}^{l}} g_{x}^{l} \bigl(x^{l}(t) ,u^{l}(t) ,t \bigr)\Delta\bar{x}^{l} (t) - \Delta _{\bar{u}^{l}} p_{x}^{l} \bigl(x^{l}(t) ,u^{l}(t) ,t \bigr) \Delta\bar {x}^{l} (t) \bigr] \Delta\bar{t}_{l} \,dt \\ &{}+\sum_{l=1}^{r-1} {\psi^{l+1}(t_{l}) \Phi_{t} \bigl( {x^{l}(t_{l}) ,t_{l} } \bigr)} +\sum_{l=1}^{r} \eta_{t_{l-1} }^{t_{l} }, \end{aligned}$$
(19)
where
$$\begin{aligned} \eta_{t_{l - 1}}^{t_{l}} ={}& E \int _{0}^{1} ( {1 - \mu} ) \bigl[ { \varphi_{x}^{l*} \bigl( {\bar{x}^{l} (t_{l})} \bigr) - \varphi_{x}^{*} \bigl( {x^{l}(t_{l}) } \bigr)} \bigr] \Delta\bar{x}^{l} (t_{l})\,d\mu \\ &{}-E \int _{t_{l - 1} }^{t_{l} } \int _{0}^{1} ({1 - \mu} ) \bigl[ H_{x}^{l} \bigl( \psi^{l}(t) ,{ \bar{x}^{l} (t_{l})},u^{l}(t) ,t \bigr) \\ &{}- H_{x}^{l} \bigl( {\psi^{l}(t) ,x^{l}(t) u^{l}(t) ,t} \bigr) \bigr]\Delta\bar{x}^{l}(t) \Delta \bar{t}_{l} \,d\mu\, dt \\ &{}- E \int _{0}^{1} { ( {1 - \mu} )\psi_{t_{l} }^{l + 1 } \bigl[ {\Phi_{x}^{l} \bigl( {\bar{x}^{l} (t_{l})} \bigr) - } } \Phi_{x}^{l} \bigl( {x^{l}(t) ,t_{l} } \bigr)\bigr]\Delta x^{l}(t) \Delta \bar{t}_{l} \,d\mu \\ &{}+E \int _{0}^{1} { ( {1 - \mu} )\Delta \bar{x}_{t_{l} }^{l * } \bigl[ {\varphi_{xx}^{l * } \bigl( {\bar{x}^{l} (t_{l})} \bigr) - \varphi_{xx}^{*} \bigl( {x^{l}(t_{l}) } \bigr)} \bigr]} \Delta \bar{x}^{l} (t_{l})\,d\mu \\ &{}+ E \int _{t_{l - 1} }^{t_{l} } \int _{0}^{1} ( {1 - \mu} )\Delta \bar{x}_{t}^{l * } \bigl[ H_{xx}^{l} \bigl( {\psi^{l}(t) ,{\bar{x}^{l} (t_{l})} ,u^{l}(t) ,t} \bigr) \\ &{}- {H_{xx}^{l} \bigl( { \psi^{l}(t) ,x^{l}(t) u^{l}(t) ,t} \bigr)} \bigr] \Delta\bar{x}^{l}(t) \Delta\bar{t}_{l} \,d\mu\, dt \\ &{}+ E \int _{0}^{1} ({1 - \mu} )\Delta\bar{x}_{t_{l} }^{l * } \Psi^{l+1}(t_{l}) \bigl[ {\Phi_{xx}^{l} \bigl( {\bar{x}^{l} (t_{l})},t_{l} } \bigr) - \Phi_{xx}^{l} \bigl( {x^{l}(t) ,t_{l} } \bigr) \bigr] \Delta\bar{x}^{l} (t_{l})\Delta \bar{t}_{l} \,d\mu. \end{aligned}$$
(20)
By assumptions (AIII) and (BIII), using identities (17) and (18) from expressions (16), (19), and (20), according to the fact that the coefficients of the independent increments \(\Delta{x}^{l} (t)\), Δ equal zero, we obtain that (15) is true. Taking into consideration (12), (13), (17), and (18), through the simple expression (19) of transformations, can be written as
$$\begin{aligned} \Delta J ( u ) ={}& \sum_{l = 1}^{s} \Delta J^{l} \bigl( {u^{l}} \bigr) \\ ={}& {-} \sum _{l = 1}^{s} E \int _{t_{l - 1} }^{t_{l} } \bigl[ {\Delta_{\overline{u} ^{l}} H^{l}} \bigl( {\psi^{l}(t) ,x^{l}(t) ,u^{l}(t) ,t} \bigr) + \Delta_{\overline{u} ^{l}} H_{x^{l}}^{l} \bigl( {\psi^{l}(t) ,x^{l}(t) ,u^{l}(t) ,t} \bigr) \Delta\bar{x}^{l}(t) \\ &{}- 0.5\Delta\bar{x}_{t_{l} }^{l\ast} f^{l\ast} \bigl( {x^{l}(t) ,t} \bigr)\Psi^{l}(t) f_{x}^{l} \bigl( {x^{l}(t) ,t} \bigr)\Delta\bar{x}^{l} (t_{l}) \\ &{}+ \Delta\bar{x}^{l\ast}(t_{l} ) \Delta_{\bar{u}^{l}} g^{l} \bigl( {x^{l}(t) ,u^{l}(t),t} \bigr) \Psi^{l}(t) \Delta\bar{x}^{l} (t_{l}) \\ &{}-\Delta\bar {x}^{l\ast}(t_{l} ) g_{x}^{l} \bigl( {x^{l}(t) ,u^{l}(t) ,t} \bigr) \Psi^{l}(t) \Delta \bar{x}^{l} (t_{l}) - \Delta\bar{x}^{l\ast}(t_{l} ) f_{x}^{l} \bigl( {x^{l}(t) ,t} \bigr) K^{l}(t) \Delta {\bar{x}^{l} (t_{l})} \bigr] \Delta\bar{t}_{l} \,dt \\ &{}+ \sum_{l = 1}^{r} \eta_{t_{l - 1} }^{t_{l}}. \end{aligned}$$
(21)
Consider the following spike variations:
$$\Delta u^{l}(t) =\Delta u_{t,\varepsilon^{l}}^{\theta_{l}} = \left \{ \textstyle\begin{array}{@{}l@{\quad}l} 0, & t\notin[ {\theta_{l} ,\theta_{l} +\varepsilon_{l} } ),\varepsilon_{l} >0, \theta_{l} \in [ {t_{l-1} ,t_{l} } ), \\ \bar{u}^{l}- {u}_{t}^{l} ,& t\in [ {\theta_{l} ,\theta_{l} +\varepsilon_{l} } ),\bar{u}^{l}\in L^{2} ( {\Omega ,F^{\theta_{l} },P;R^{m}} ) , \end{array}\displaystyle \right . $$
where \(\varepsilon_{l}>0 \) are small enough.

The following lemma will be used in estimation of increment (21).

Lemma 1

Assume that conditions (AI), (AII), (BI), (BII) are satisfied. Then
$$\lim _{\varepsilon_{l} \to0} E\bigl\vert {x_{\varepsilon_{l} }^{\theta_{l} }(t)- x^{l}(t) } \bigr\vert ^{2} \leq N\varepsilon_{l}^{2} \quad\textit{a.e. in } [ {t_{l - 1} ,t_{l} } ), l = \overline{1,r} . $$

Here \(x_{\varepsilon_{l} }^{\theta_{l} }(t) \) are the trajectories of system (1)-(2) corresponding to the controls \(u_{\varepsilon _{l} }^{\theta_{l} }(t) = u^{l}(t) + \Delta u_{\varepsilon_{l} }^{\theta_{l}}(t) \).

Proof

See Lemma 1 in [33]. □

By invoking expression (20) and using Lemma 1 the following estimation is implied:
$$\eta_{\theta_{l} }^{\theta_{l} +\varepsilon_{l} } =o \bigl( {\varepsilon_{l}^{2}} \bigr),\quad l=\overline{1,r} . $$
According to the singularity of controls \(\bar{u}^{l}(t)\), \(l=\overline {1,r} \), in the maximum sense for given special variation from (21), it follows that, for each l,
$$\begin{aligned} \Delta_{\theta^{l}} J(u) ={}& {-}\varepsilon_{l} E \bigl[ \Delta_{\bar{u}^{l}} H_{x}\bigl(\psi^{l}( \theta_{l}), x^{l}(\theta_{l}) ,u^{l}( \theta_{l}),\theta_{l}\bigr) \Delta_{\bar{u}^{l}} g^{l}\bigl(x^{l}(\theta_{l}) ,u^{l}( \theta_{l}) ,\theta_{l} \bigr) \\ &{} +\Delta_{\bar{u}^{l}} g^{l\ast}\bigl(x^{l}( \theta_{l}) ,u^{l}(\theta_{l}) ,\theta_{l} \bigr) \Psi^{l}(\theta_{l}) \Delta_{\bar{u}^{l}} g^{l} \bigl(x_{\theta _{l} }^{l} ,u^{l}(\theta_{l}) , \theta_{l} \bigr) \Delta\bar{t}_{l} \bigr] + o\bigl( \varepsilon_{l}^{2} \bigr) \geq0. \end{aligned}$$

Finally, due to the smallness and arbitrariness of \(\varepsilon_{l} \), (14) is fulfilled. Theorem 3 is proved. □

5 Necessary condition of optimality for singular controls

This section is devoted to investigation of singular optimal control problems for stochastic switching systems with constraints. We obtain a necessary condition of optimality for stochastic control problem of switching systems (1)-(5). Using Ekeland’s variational principle, the given problem is converted into a sequence of unconstrained problems. Based on Theorem 3, we establish the maximum principle and transversality conditions for the sequence of switching systems. Then we obtain a necessary condition of optimality in the case where endpoint constraints are imposed.

Theorem 4

Suppose that conditions (AI)-(AIV) and (BI)-(BIV) hold. Let \(\pi^{r} = (t_{0} ,t_{1} ,\ldots, t_{r} ,x(t_{1}) ,x(t_{2}) , \ldots,x(t_{r}), u^{1},u^{2},\ldots,u^{r})\) be an optimal solution of problem (1)-(4), and \(\mathbf{u} = ( u^{1},u^{2},\ldots,u^{r})\) be singular on the control region \(\mathbf{V} = ( V^{1},V^{2},\ldots,V^{r})\). Then
  1. (a)
    there exist random processes \((\psi^{l}(t) ,\beta^{l}(t)) \in L_{F^{l}}^{2}(t_{l - 1} ,t_{l} ;R^{n_{l} })\times L_{F^{l}}^{2} (t_{l - 1} ,t_{l} ;R^{n_{l}\times n_{l} })\) and \((\Psi^{l}(t) ,\mathrm{K}^{l}(t) ) \in L_{F^{l}}^{2} (t_{l - 1} ,t_{l} ;R^{n_{l} })\times L_{F^{l}}^{2} (t_{l - 1} ,t_{l} ;R^{n_{l} \times n_{l} })\) that are solutions of the following adjoint equations:
    $$\begin{aligned}& \left \{ \textstyle\begin{array}{@{}l} d\psi^{l}(t) = - H_{x}^{l} (\psi^{l}(t) ,x^{l}(t) ,u^{l}(t) ,t)\,dt + \beta^{l}(t)\,dw(t), \quad t_{l - 1}\leq t < t_{l} ,\\ \psi^{l}(t_{l} ) = - \lambda_{0}^{l} q_{x}^{l} (x^{l}(t_{l}) )- \lambda_{1}^{l}\varphi _{x}^{l} (x^{l}(t_{l}) ) + \psi^{l + 1}(t_{l} ) \Phi _{x}^{l} (x^{l}(t_{l}) ,t_{l} ),\quad l = \overline{1,r - 1} , \\ \psi^{r}(t_{r} ) = - \lambda_{0}^{r} q_{x}^{r}(x^{r}(t_{r}))-\lambda_{1}^{r} \varphi_{x}^{r} (x^{r}(t_{r})) ; \end{array}\displaystyle \right . \end{aligned}$$
    (22)
    $$\begin{aligned}& \left \{ \textstyle\begin{array}{@{}l} d\Psi^{l}(t) = - [\mathbf{{H}} _{x}^{l} ( {\Psi^{l}(t),x^{l}(t) ,u^{l}(t) ,t} ) + H_{xx}^{l} (\psi^{l}(t) ,x^{l}(t) ,u^{l}(t) ,t)\\ \hphantom{d\Psi^{l}(t) =}{}+ f_{x}^{l\ast} (x^{l}(t) ,t)\Psi^{l}(t) f_{x}^{l} (x^{l}(t) ,t) ]\,dt + \mathrm{K}^{l}(t)\,dw^{l}(t) ,\quad t \in[t_{l - 1} ,t_{l} ), \\ \Psi^{l}(t_{l}) = - \lambda_{0}^{l} q_{xx}^{l} (x^{l}(t_{l}) )- \lambda_{1}^{l}\varphi _{xx}^{l} (x^{l}(t_{l}) ) \\ \hphantom{\Psi^{l}(t_{l}) =}{}+ \Psi^{l + 1}(t_{l}) \Phi_{xx}^{l} (x^{l}(t_{l}) ,t_{l} ), \quad l = \overline{1,r - 1} , \\ \Psi^{r}(t_{r}) = - \lambda_{0}^{r} q_{xx}^{r} (x^{r}(t_{r}) )- \lambda_{1}^{r} \varphi _{xx}^{r}(x^{r}(t_{r})); \end{array}\displaystyle \right . \end{aligned}$$
    (23)
     
  2. (b)

    a.e. \(\theta\in[t_{l - 1} ,t_{l} ] \) and \(\forall \bar{u}^{l} \in V^{l}\), \(l = \overline{1,r} \), a.c. the second-order maximum conditions (14) are satisfied;

     
  3. (c)

    the transversality conditions (15) hold.

     

Proof

For any natural j, let us introduce the following approximating functional for each \(l = \overline{1,r} \):
$$\begin{aligned} I_{j} ( {\mathbf{u}} ) &= S_{j} \Biggl\langle E\sum _{l=1}^{r} \biggl[\varphi ^{l} ( {t_{l} } ) + \int _{t_{l - 1} }^{t_{l} } {p^{l} \bigl( {x ^{l}(t) ,u ^{l}(t) ,t} \bigr)\,dt}\biggr] ,E\sum _{l=1}^{r} q^{l}\bigl(x^{l}(t_{l}) \bigr)\Biggr\rangle \\ &=\min_{(c,y^{l}) \in\varepsilon}\sqrt{\bigl\vert c - 1 / j - EM ( \mathbf{x} ,\mathbf{u} ,\mathbf{t} )\bigr\vert ^{2} + \sum _{l=1}^{r} \bigl\vert y^{l}- {Eq^{l}\bigl(x^{l}(t_{l})\bigr)} \bigr\vert ^{2}} , \end{aligned}$$
where \(M ( \mathbf{x} ,\mathbf{u} ,\mathbf{t} )=\sum_{i=1}^{r} [ {\varphi^{l} (x ^{l}(t_{l}) ) + \int _{t_{l - 1} }^{t_{l} } {p(x ^{l}(t) ,u ^{l}(t) ,t)\,dt} } ] \), \(\varepsilon= \{c:c \le J^{0},y^{l}\in G^{l} \}\), \(c=c^{1}+\cdots+c^{r}\), and \(J^{0}\) is the minimal value of the functional in problem (1)-(5).

Let \(V^{l} \equiv(U_{\partial}^{l} ,d)\) be the space of controls obtained by means of the following metric: \(d(u^{l},v^{l}) = ( {l \otimes P} ) \{ { ( {t,\omega} ) \in [ {t_{l - 1} ,t_{l} } ]\times\Omega:\nu_{t}^{l} \ne u ^{l}(t) } \}\). For each \(l = \overline{1,r} \), \(V^{l}\) is a complete metric space [34].

It is easy to prove the following fact.

Lemma 2

Assume that conditions (AI)-(AIV) hold, \(u^{l,n}(t)\), \(l = \overline{1,r} \), is a sequence of admissible controls from \(V^{l}\), and \(x^{l,n}(t) \) is the sequence of corresponding trajectories of system (1)-(3). If \(d(u^{l,n}(t) ,u ^{l}(t) ) \to0\), then, \(\lim_{n \to\infty} \{ \sup_{t_{l - 1} \leq t \leq t_{l} } E\vert {x^{l,n}(t) - x ^{l}(t) } \vert ^{2} \} = 0\), where \(x ^{l}(t) \) is a trajectory corresponding to the admissible controls \(u ^{l}(t)\), \(l = \overline{1,r}\).

Due to the continuity of the functionals \(I_{j}^{l} :V^{l} \to R^{n_{l} }\), according to Ekeland’s variational principle, there are controls such that \(u^{l,j}(t) :d(u^{l,j}(t) ,u ^{l}(t) ) \le\sqrt{\varepsilon_{j}^{l} } \) and for \(\forall u ^{l}(t) \in V^{l}\), the following is achieved: \(I_{j}^{l} (u^{l,j}) \le I_{j}^{l} (u^{l}) + \sqrt{\varepsilon_{j}^{l} } d(u^{l,j},u^{l})\), \(\varepsilon_{j}^{l} = \frac{1}{j}\).

This inequality means that \((t_{1} ,\ldots,t_{r} ,x^{1,j}(t) ,\ldots,x^{r,j}(t), u^{1,j}(t) ,\ldots,u^{r,j}(t) ) \) for each \(t \in (t_{l - 1} ,t_{l} ] \) is a solution of the following problem:
$$ \left \{ \textstyle\begin{array}{@{}l} J_{j}(u) = \sum_{l = 1}^{r} ( {I_{j}^{l} (u^{l}) + \sqrt{\varepsilon _{j}^{l} } E\int _{t_{l - 1} }^{t_{l} } {\delta(u ^{l}(t) ,u^{l,j}(t) )\,dt} } ) \to\min, \\ dx ^{l}(t) = g^{l}(x ^{l}(t) ,u ^{l}(t) ,t)\,dt + f^{l}(x ^{l}(t) ,t)\,dw(t),\quad l = \overline{1,r}, \\ x^{l + 1}(t_{l}) = \Phi^{l} ( {x ^{l}(t_{l}) ,t_{l} } ),\quad l = \overline{1,r - 1 } ;x^{1}(t_{0} ) = x_{0} , \\ u ^{l}(t) \in U_{\partial}^{l}. \end{array}\displaystyle \right . $$
(24)
The function \(\delta(u,v)\) is determined in the following way:
$$\delta (u,v) = \left \{ \textstyle\begin{array}{@{}l@{\quad}l} 0,&u = v, \\ 1,&u \ne v. \end{array}\displaystyle \right . $$

Then according to Theorem 3, we have that if \((x^{1,j}(t) ,\ldots,x^{r,j}(t),u^{1,j}(t) ,\ldots,u^{r,j}(t))\) is an optimal solution of problem (24), then

(1) there exist random processes \((\psi^{l,j}(t), \beta^{l,j}(t)) \in L_{F^{l}}^{2} (t_{l - 1} ,t_{l} ;R^{n_{l} })\times L_{F^{l}}^{2} (t_{l - 1} ,t_{l} ;R^{n_{l} \times n_{l} })\) that are solutions of the following system:
$$ \left \{ \textstyle\begin{array}{@{}l} d\psi^{l,j}(t) = - H_{x}^{l} ( {\psi^{l,j}(t) ,x^{l,j}(t) ,u^{l,j}(t) ,t} )\,dt + \beta^{l,j}(t)\,dw^{l}(t) , \quad t \in [ {t_{l - 1} ,t_{l} } ), l = \overline{1,r}, \\ \psi^{l,j}(t_{l}) = - \lambda_{0}^{l,j}\varphi_{x}^{l} ( {x^{l,j}(t_{l})} ) - \lambda_{1}^{l,j} q_{x}^{l} ( {x^{l,j}(t_{l})} ) +\psi _{t_{l + 1} }^{l} \Phi_{x}^{l} (x^{l,j}(t_{l}) ,t_{l} ),\quad l= \overline{1,r - 1}, \\ \psi^{r}(t_{r}) = - \lambda_{0}^{r,j} \varphi_{x}^{r} ( {x^{r,j}(t_{r}} ) - \lambda_{1}^{r,j} q_{x}^{r} ( {x^{r,j}(t_{r})} ) , \end{array}\displaystyle \right . $$
(25)
and the random processes \(\Psi^{l,j}(t) \in L_{F^{l}}^{2} (t_{l - 1} ,t_{l} ;R^{n_{l} })\), \(\mathrm{K}^{l,j}(t) \in L_{F^{l}}^{2} (t_{l - 1} ,t_{l} ;R^{n_{l} \times n_{l} })\) that are solutions of the following system:
$$ \left \{ \textstyle\begin{array}{@{}l} d\Psi^{l,j}(t)= - [\mathbf{{H}}_{x}^{l} ( {\Psi^{l,j}(t),x^{l,j}(t) ,u^{l,j}(t) ,t} ) + H_{xx}^{l} (\psi^{l,j}(t),x^{l,j}(t) ,u^{l,j}(t),t)\\ \hphantom{d\Psi^{l,j}(t)=}{}+ f_{x}^{l\ast} (x^{l,j}(t),u^{l,j}(t) ,t)\Psi^{l,j}(t)f_{x}^{l} (x^{l,j}(t),u^{l,j}(t),t) ]\,dt + \mathrm{K}^{l,j}(t)\,dw^{l}(t), \\ \Psi^{l,j}(t_{l}) = -\lambda_{0}^{l,j} \varphi_{xx}^{l} (x^{l,j}(t_{l}) ) - {\lambda_{1}^{l,j} } q_{xx}^{l} (x^{l,j}(t_{l})+ \Psi_{t_{l + 1} }^{l,j} \Phi_{xx}^{l} (x^{l,j}(t_{l}) ,t_{l} ), \quad l = \overline{1,r - 1} ,\\ \Psi^{r,j}(t_{r}) = - \lambda_{0}^{r,j} \varphi_{xx}^{r} (x^{r,j}(t_{r}) ) - {\lambda_{1}^{r,j} } q_{xx}^{r} (x^{r,j}(t_{r}), \end{array}\displaystyle \right . $$
(26)
where nonzero \((\lambda_{0}^{l,j} ,\lambda_{1}^{l,j} ) \in R^{k + 1}\), \(l = \overline{1,r} \), satisfy the following requirement:
$$ \bigl( \lambda_{0}^{l,j},\lambda_{1}^{l,j} \bigr)= \biggl[-c^{l}+1/j+ {\varphi ^{l}\bigl(x ^{l}(t_{l}) \bigr) + \int _{t_{l - 1} }^{t_{l} } {p\bigl(x ^{l}(t) ,u ^{l}(t) ,t\bigr)\,dt} } ,-y^{l}+ Eq^{l} \bigl({x^{l,j}(t_{l})}\bigr) \biggr]\Big/J_{j}^{0} $$
(27)
with
$$J_{j}^{0} = \Biggl( \sum_{l = 1}^{r} \bigl\vert y^{l}-{Eq^{l}\bigl(x^{l,j}(t_{l}) \bigr)} \bigr\vert ^{2} + { \Biggl\vert {c - 1 / j - E\sum _{l = 1}^{r} \biggl[ {\varphi ^{l}\bigl(x ^{l}(t_{l}) \bigr) + \int _{t_{l - 1} }^{t_{l} } {p\bigl(x ^{l}(t) ,u ^{l}(t) ,t\bigr)\,dt} } \biggr]} \Biggr\vert ^{2} } \Biggr)^{1/2} ; $$
(2) for a.e. \(t \in[t_{l - 1} ,t_{l} ] \) and \(\forall \tilde{u}^{l} \in V^{l}\), \(l = \overline{1,r} \), a.c. is satisfied:
$$\begin{aligned} &\Delta_{\bar{u}^{l,j}} H_{x}^{l}\bigl( \psi^{l,j}(\theta),x^{l,j}(\theta ),u^{l,j},\theta \bigr)\Delta_{\bar{u}^{l,j}} g^{l\ast}\bigl(x^{l,j}(\theta) ,u^{l,j}(\theta),\theta\bigr) \\ &\quad{}+\Delta_{\bar{u}^{l,j}} g^{l\ast}\bigl(x^{l,j}(\theta) ,u^{l,j}(\theta ),\theta\bigr)\Psi^{l,j}(\theta) \Delta_{\bar{u}^{l}} g^{l}\bigl(x^{l,j}(\theta) ,u^{l,j}(\theta),\theta\bigr) \leq0; \end{aligned}$$
(28)
(3) the following transversality conditions hold:
$$ \psi^{l+1,j}(t_{l}) \Phi_{t}^{l,j} \bigl( {x^{l,j}(t_{l}) ,t_{l}} \bigr)+0.5 \Psi^{{l + 1,j}^{*}}(t_{l}) \Phi_{tt}^{l,j} \bigl( {x^{l,j}(t_{l}) ,t_{l} } \bigr)\Psi^{l + 1,j}(t_{l}) = 0 ,\quad l = \overline{1,r - 1} , \textit{a.c.} $$
(29)

According to conditions (BI)-(BIV), we have that \((\lambda_{0}^{j} ,\lambda_{1}^{j} ) \to(\lambda_{0} ,\lambda_{1} )\) as \(j \to \infty\).

The proof of item (a) of Theorem 4 is based on the following lemmas.

Lemma 3

Let \(\psi^{l}(t_{l} ) \) be a solution of system (22), and \(\psi^{l,j}(t_{l} ) \) be a solution of system (25). If \(d(u^{l,j}(t) ,u ^{l}(t) ) \to0\), \(j \to\infty\), then
$$E \int _{t_{l - 1} }^{t_{l} } \bigl|\psi^{l,j}(t) - \psi^{l}(t) \bigr|^{2}\,dt + E \int _{t_{l - 1} }^{t_{l} } {\bigl|\beta^{l,j}(t) - \beta^{l}(t) \bigr|^{2}\,dt \to0} ,\quad l = \overline{1,r.} $$

Proof

It is clear that \(\forall t \in[t_{l - 1} ,t_{l} ]\),
$$\begin{aligned} &d \bigl( {\psi^{l,j}(t) - \psi^{l}(t) } \bigr) \\ &\quad= - \bigl[ {H_{x}^{l} \bigl( {\psi _{t}^{l,j} ,x^{l,j}(t) ,u^{l,j}(t) ,t} \bigr) - H_{x}^{l} \bigl( {\psi^{l}(t) ,x ^{l}(t) ,u ^{l}(t) ,t} \bigr)} \bigr]\,dt+ \bigl( {\beta^{l,j}(t) - \beta^{l}(t) } \bigr)\,dw(t) \\ &\quad = - \bigl[ {\psi^{l,j}(t) } g_{x}^{l} \bigl( {x^{l,j}(t) ,u^{l,j}(t) ,t} \bigr) + \beta^{l,j}(t) f_{x}^{l} \bigl( {x^{l,j}(t) ,t} \bigr) \\ &\qquad{}- p_{x}^{l} \bigl( {x^{l,j}(t) ,u^{l,j}(t) ,t} \bigr) - \psi^{l}(t) g_{x}^{l} \bigl( {x ^{l}(t) ,u ^{l}(t) ,t} \bigr) \\ &\qquad{}- \beta^{l}(t) f_{x}^{l} \bigl( {x ^{l}(t) ,t} \bigr) + {p_{x}^{l} \bigl( {x ^{l}(t) ,u ^{l}(t) ,t} \bigr)} \bigr]\,dt + \bigl( { \beta^{l,j}(t) - \beta^{l}(t) } \bigr)\,dw(t). \end{aligned}$$
Let us square both sides of the last equation. Then, according to Itô’s formula, \(\forall s \in[t_{r - 1} ,T]\),
$$\begin{aligned} &E\bigl(\psi^{l,j}(t) -\psi^{l}(t)\bigr) ^{2}-E \bigl(\psi^{l,j}(s) -\psi^{l}(s) \bigr)^{2} \\ &\quad= 2E \int _{s}^{t_{r} } \bigl[\psi^{l,j}(t) - \psi^{l}(t) \bigr] \bigl[\bigl(g_{x}^{l\ast} \bigl(x^{l,j}(t), u^{l,j}(t), t\bigr)-g_{x}^{l\ast} \bigl(x ^{l}(t), u ^{l}(t), t\bigr)\bigr)\psi ^{l,j}(t) \\ &\qquad{}+g_{x}^{l\ast} \bigl(x ^{l}(t) ,u ^{l}(t), t\bigr) \bigl(\psi^{l,j}(t) -\psi^{l}(t) \bigr)+\bigl(f_{x}^{l\ast} \bigl(x^{l,j}(t), t \bigr)-f_{x}^{l\ast} \bigl(x ^{l}(t),t\bigr)\bigr) \beta ^{l,j}(t) \\ &\qquad{}+f_{x}^{l\ast} \bigl(x ^{l}(t), t\bigr)\bigl(\beta^{l,j}(t) -\beta^{l}(t) \bigr)-p^{l} \bigl(x^{l,j}(t), u^{l,j}(t), t \bigr)+p_{x}^{l} \bigl(x ^{l}(t), u ^{l}(t), t\bigr)\bigr]\,dt\\ &\qquad{}+E \int _{s}^{t_{r} } {\bigl( \beta^{l,j}(t) - \beta^{l}(t) \bigr)^{2}\,dt}. \end{aligned}$$
Now, due to assumptions (AI)-(AIII), we get:
$$\begin{aligned} &E \int _{s}^{t_{r}} \bigl|\beta^{l,j}(t) - \beta^{l}(t) \bigr|^{2}\,dt+E \bigl|\psi^{l,j}(s) - \psi^{l}(s) \bigr|^{2} \\ &\quad\le EN \int _{s}^{t_{r} } {\bigl|\psi ^{l,j}(t) - \psi^{l}(t) \bigr|^{2}\,dt} +EN\varepsilon \int _{s}^{t_{r} } {\bigl|\beta^{l,j}(t) - \beta^{l}(t) \bigr| ^{2}\,dt+E\bigl\vert {\psi^{l,j}({t_{r}}) -\psi^{l}(_{t_{r} })} \bigr\vert ^{2}.} \end{aligned}$$
Hence, by the Gronwall inequality [13] we obtain that
$$ E\bigl|\psi^{l,j}(s) -\psi^{l}(s) \bigr|^{2} \le De^{N(t_{r} -s)} \quad\mbox{a.e. in } [t_{r-1} ,t_{r} ], $$
(30)
where \(D=E|\psi^{l,j}(t_{r})- \psi^{l}( {t_{r} }) |^{2}\). Hence, it follows from (22) and (25) that \(\psi^{l,j}(t_{r}) \to\psi^{l}(t_{r})\), which leads to \(D\to0\). Consequently, it follows that \(\psi^{l,j}(s) \to\psi^{l}(s) \) in \(L_{F}^{2} (t_{r-1}, t_{r} ;R^{n_{l} })\), and thus \(\beta ^{l,j}(s) \to\beta^{l}(s) \) in \(L_{F}^{2} (t_{r-1}, t_{r}; R^{n_{l} \times n_{l} })\).
Then from the expression
$$\begin{aligned} &E\bigl|\psi^{l,j}(t_{l}) - \psi^{l}(t_{l} ) \bigr|^{2} - E\bigl|\psi^{l,j}(s) - \psi_{s}^{l} \bigr|^{2} \\ &\quad= 2E \int _{s}^{t_{l} } \bigl(\psi^{l,j}(t) - \psi^{l}(t) \bigr)\bigl[\bigl(g_{x}^{l\ast} \bigl(x^{l,j}(t) , u^{l,j}(t) ,t\bigr) - g_{x}^{l\ast} \bigl(x ^{l}(t) ,u ^{l}(t) ,t\bigr)\bigr) \psi^{l,j}(t) \\ &\qquad{}+g_{x}^{l\ast} \bigl(x ^{l}(t) ,u ^{l}(t) ,t\bigr) \bigl(\psi^{l,j}(t) - \psi^{l}(t) \bigr) + \bigl(f_{x}^{l\ast} \bigl(x^{l,j}(t),t\bigr) - f_{x}^{l\ast} \bigl(x ^{l}(t) ,t\bigr)\bigr) \beta_{t}^{j} \\ &{}\qquad{}+ f_{x}^{l\ast} \bigl(x ^{l}(t) ,t\bigr) \bigl(\beta^{l,j}(t) - \beta^{l,j}(t) \bigr) + p_{x}^{l} \bigl(x ^{l}(t) ,u ^{l}(t) ,t\bigr) - p_{x}^{l} \bigl(x^{l,j}(t) ,u^{l,j}(t) ,t\bigr)\bigr]\,dt \\ &\qquad{}+ E \int _{s}^{t_{l} } {\bigl|\beta^{l,j}(t) - \beta^{l}(t) \bigr|^{2}\,dt}, \end{aligned}$$
using simple transformations, in view of assumptions (AI)-(AIV), we obtain:
$$\begin{aligned} &E \int _{s}^{t_{l} } \bigl|\beta^{l,j}(t) - \beta^{l}(t) \bigr|^{2}\,dt + E\bigl| \psi^{l,j}(s) - \psi_{s}^{l} \bigr|^{2} \\ &\quad\leq EN \int _{s}^{t_{l} } {\bigl\vert {\psi^{l,j}(t) - \psi^{l}(t) } \bigr\vert ^{2}\,dt } + EN\varepsilon \int _{s}^{t_{l} } {\bigl|\beta^{l,j}(t) - \beta^{l}(t) \bigr|^{2}\,dt} + E\bigl|\psi^{l,j}(t_{l}) - \psi^{l}(t_{l} ) \bigr|^{2}. \end{aligned}$$
Hence, according to the Gronwall inequality, we have:
$$E\bigl|\psi^{l,j}(s) - \psi^{l}(s) \bigr|^{2} \leq De^{N(t{ }_{l} - s)}\quad \mbox{a.e. in } [t_{l - 1} ,t_{l} ], l = \overline{1,r-1}, $$
where the constant D is determined as \(D = E|\psi^{l,j}(t_{l}) - \psi^{l}(t_{l} ) |^{2}\), and \(D \to0\). Then from (30) it follows that \(\psi^{l,j}(s) \to\psi^{l}(s) \) in \(L_{F^{l}}^{2} (t_{l - 1} ,t_{l} ;R^{n})\) and \(\beta^{l,j}(s) \to\beta^{l}(s) \) in \(L_{F^{l}}^{2} (t_{l - 1} ,t_{l} ;R^{n\times n})\). Lemma 3 is proved. □

Lemma 4

Let \(\Psi^{l,j}(t_{l} ) \) be a solution of system (23), and \(\Psi^{l}(t_{l} ) \) be a solution of system (26). Then
$$E \int _{t_{l - 1} }^{t_{l} } \bigl|\Psi^{l,j}(t) - \Psi^{l}(t) \bigr|^{2}\,dt + E \int _{t_{l - 1} }^{t_{l} } {\bigl|\mathrm{K}^{l,j}(t) - \mathrm{K}^{l}(t) \bigr|^{2}\,dt \to0},\quad l = \overline{1,r} , \textit{as }j \to\infty. $$

Proof

Due to Itô’s formula, from expressions (23) and (26) for all \(s \in[t_{l - 1} ,t_{l} )\) we have:
$$\begin{aligned} d\bigl(\Psi^{l,j}(t) - \Psi^{l}(t) \bigr) ={}&{-} \bigl\{ \bigl(g_{x}^{l\ast} \bigl(x^{l,j}(t) ,u^{l,j}(t) ,t\bigr)\Psi^{l,j}(t) - g_{x}^{l\ast} \bigl(x_{t}^{j} ,u_{t}^{j} ,t\bigr) \Psi^{l}(t) \bigr) \\ &{}+\bigl(\Psi _{t}^{l,j} g_{x}^{l} \bigl(x^{l,j}(t) ,u^{l,j}(t) ,t\bigr) - \Psi^{l}(t) g_{x}^{l} \bigl(x_{t}^{j} ,u_{t}^{j} ,t\bigr)\bigr) \\ &{}+ \bigl( {f_{x}^{l\ast} \bigl(x^{l,j}(t) ,t\bigr)\mathrm{K}^{l,j}(t) - f_{x}^{l\ast} \bigl(x ^{l}(t),t\bigr) \mathrm{K}^{l}(t) } \bigr) \\ &{}+ H_{xx}^{l} \bigl( {\psi^{l,j}(t) ,x^{l,j}(t) ,u^{l,j}(t) ,t} \bigr) - {H_{xx}^{l} \bigl( {\psi^{l}(t) ,x ^{l}(t) ,u ^{l}(t) ,t} \bigr)} \bigr\} \,dt \\ &{}+ \bigl(\mathrm{K}^{l,j}(t) - \mathrm{K}^{l}(t) \bigr)\,dw^{l}(t). \end{aligned}$$
Based on Itô’s formula, from the previous expression we get:
$$\begin{aligned} &E\bigl|\Psi^{l,j}(t_{l}) - \Psi^{l}(t_{l} ) \bigr|^{2} - E\bigl|\Psi^{l,j}(s) - \Psi^{l}(s) \bigr|^{2} \\ &\quad\leq 2E \int _{s}^{t_{l}} \bigl[\Psi^{l,j}(t) - \Psi _{t}^{l}\bigr] \bigl[\bigl(g_{x}^{l\ast} \bigl(x^{l,j}(t) ,u^{l,j}(t) ,t\bigr) - g_{x}^{l\ast} \bigl(x_{t}^{j} ,u_{t}^{j} ,t\bigr) \bigr)\Psi^{l,j}(t) \\ &\qquad{}+g_{x}^{l,\ast} \bigl(x_{t}^{j},u_{t}^{j} ,t\bigr) \bigl(\Psi^{l,j}(t) - \Psi^{l}(t) \bigr) + \bigl(f_{x}^{l\ast} \bigl(x^{l,j}(t),t\bigr) - f_{x}^{l\ast} \bigl(x ^{l}(t),t\bigr)\bigr) {\mathrm {K}}^{l,j}(t) \\ &\qquad{}+ H_{xx}^{l} \bigl( {\psi^{l,j}(t) ,x^{l,j}(t) ,u^{l,j}(t) ,t} \bigr) - H_{xx}^{l} \bigl( {\psi_{t}^{j} ,x ^{l}(t) ,u ^{l}(t) ,t} \bigr) \\ &\qquad{}+H_{xx}^{l} \bigl( {\psi^{l,j}(t) ,x ^{l}(t) ,u ^{l}(t) ,t} \bigr) -H_{xx}^{l} { \bigl( {\psi^{l}(t) ,x ^{l}(t) ,u ^{l}(t) ,t} \bigr)} \bigr]\,dt \\ &\qquad{}+E \int _{s}^{t_{l} } {\bigl|\mathrm{K}^{l,j}(t) - \mathrm{K}^{l}(t) \bigr|^{2}\,dt.} \end{aligned}$$
Then by simple transformations we obtain:
$$\begin{aligned} &E \int _{s}^{t_{l} } {\bigl|\mathrm{K}^{l,j}(t) - \mathrm{K}^{l}(t) \bigr|^{2}\,dt + E} \bigl|\Psi^{l,j}(t) - \Psi^{l}(t) \bigr|^{2} \\ &\quad\leq EN \int _{s}^{t_{l} } \bigl|\Psi^{l,j}(t) - \Psi^{l}(t) \bigr|^{2}\,dt + EN\varepsilon \int _{s}^{t_{1} } {\bigl|\mathrm{K}^{l,j}(t) - \mathrm{K}^{l}(t) \bigr|^{2}\,dt } + E\bigl|\Psi^{l,j}(t_{l}) - \Psi^{l}(t_{l} ) \bigr|^{2}. \end{aligned}$$
According to the Gronwall inequality, a.e. in \([t_{l - 1} ,t_{l} ) \) we have:
$$E\bigl|\Psi^{l,j}(s) - \Psi^{l}(s) \bigr|^{2} \le De^{ - N(t_{l} - s)}, $$
where the constant D is defined as
$$D = E\bigl|\Psi^{l,j}(t_{l}) - \Psi^{l}(t_{l}) \bigr|^{2} + EN\varepsilon \int _{s}^{t_{l} } {\bigl|\mathrm{K}^{l,j}(t) - \mathrm{K}^{l}(t) \bigr|^{2}\,dt,} $$
so that \(\Psi^{l,j}(t_{r}) \to\Psi^{l}(t_{r}) \), and hence, according to assumptions (BI)-(BIV) and expressions (23) and (26), we obtain \(\Psi^{l,j}(t) \to\Psi^{l}(t) \) in \(L_{F^{r}}^{2} (t_{r - 1} ,t_{r} ;R^{n})\) as \(j \to\infty\).

Then, according to sufficient smallness of ε, it follows that \(D \to 0\). Consequently, \(\Psi^{l,j}(t) \to\Psi^{l}(t) \) in \(L_{F^{l}}^{2} (t_{l - 1} ,t_{l}; R^{n})\) and \({\mathrm {K}}^{l,j}(t) \to\mathrm{K}^{l}(t) \) in \(L_{F^{l}}^{2} (t_{l - 1} ,t_{l} ;R^{n\times n})\), \(l = \overline{1,r-1} \). Lemma 4 is proved. □

Based on Lemma 3 and Lemma 4, we can pass to the limit in systems (25) and (26), and the fulfilment of (22) and (23) is derived. Following a similar scheme and taking the limits in (28) and (29), we can obtain the fulfilment of second-order necessary conditions and transversality conditions. Theorem 4 is proved. □

6 Conclusions

This paper is motivated by the increased interest of the research on switching systems. Many theoretical and numerical advances have recently been realized in the field of stochastic optimization. The stochastic control problem of switching systems in which the endpoint restrictions are defined with the help of convex closed sets is considered. The results developed in this study can be viewed as stochastic analogues of the problems formulated in [7, 8] and an extension of the results for switching systems given in [31, 32]. The main result, Theorem 4, is a natural evolution of Theorem 3 in [33].

Declarations

Acknowledgements

This work was supported by Scientific Research Project No. 150F202 of Anadolu University, Turkey.

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors’ Affiliations

(1)
Department of Industrial Engineering, Anadolu University
(2)
Institute of Control Systems, ANAS

References

  1. Arnold, L: Stochastic Differential Equations: Theory and Applications. Wiley, New York (1974) MATHGoogle Scholar
  2. Kolmanovsky, VB, Myshkis, AD: Applied Theory of Functional Differential Equations. Kluwer Academic, Dordrecht (1992) View ArticleGoogle Scholar
  3. Antsaklis, PJ, Nerode, A: Hybrid control systems: an introductory discussion to the special issue. IEEE Trans. Autom. Control 43(4), 457-460 (1998) MathSciNetView ArticleGoogle Scholar
  4. Boukas, E: Stochastic Switching Systems: Analysis and Design. Birkhäuser Boston, Boston (2006) Google Scholar
  5. Liberzon, D: Switching in Systems and Control. Birkhäuser Boston, Boston (2003) MATHView ArticleGoogle Scholar
  6. Baklacioglu, T, Turan, O, Aydin, H: Dynamic modeling of exergy efficiency of turboprop engine components using hybrid genetic algorithm-artificial neural networks. Energy 86, 709-721 (2015) View ArticleGoogle Scholar
  7. Bengea, SC, Raymond, AC: Optimal control of switching systems. Automatica 41, 11-27 (2005) MATHGoogle Scholar
  8. Capuzzo, DI, Evans, LC: Optimal switching for ordinary differential equations. SIAM J. Control Optim. 22, 143-161 (1984) MATHMathSciNetView ArticleGoogle Scholar
  9. Seidmann, TI: Optimal control for switching systems. In: Proceedings of the 21st Annual Conference on Information Science and Systems, pp. 485-489 (1987) Google Scholar
  10. Xu, X, Antsaklis, PJ: Results and perspectives on computational methods for optimal control of switched systems. In: Maler, O, Pnueli, A (eds.) Hybrid Systems: Computation and Control. Lecture Notes in Computer Science, vol. 2623, pp. 540-556 (2003) View ArticleGoogle Scholar
  11. Afanasyev, VN, Kolmanovsky, VB, Nosov, VP: Mathematical Theory for Synthesizing Control Systems. Higher School, Moscow (2003) Google Scholar
  12. Gikhman, II, Skorokhod, AV: Controlled Stochastic Processes. Springer, New York (1979) MATHView ArticleGoogle Scholar
  13. Fleming, WH, Rishel, RW: Deterministic and Stochastic Optimal Control. Springer, Berlin (1975) MATHView ArticleGoogle Scholar
  14. Yong, J, Zhou, XY: Stochastic Controls: Hamiltonian Systems and HJB Equations. Springer, New York (1999) MATHView ArticleGoogle Scholar
  15. Kushner, HJ: Necessary conditions for continuous parameter stochastic optimization problems. SIAM J. Control 10, 550-565 (1976) MathSciNetView ArticleGoogle Scholar
  16. Arkin, VI, Saksonov, MT: The necessary conditions of optimality in control problems of stochastic differential equations. Dokl. Akad. Nauk SSSR 244(4), 11-16 (1979) MathSciNetGoogle Scholar
  17. Bensoussan, A: Lectures on Stochastic Control in Nonlinear Filtering and Stochastic Control. Lectures Notes in Mathematics. Springer, Berlin (1982) Google Scholar
  18. Bismut, JM: Linear quadratic optimal stochastic control with random coefficients. SIAM J. Control 14, 419-444 (1976) MATHMathSciNetView ArticleGoogle Scholar
  19. Haussman, UG: General necessary conditions for optimal control of stochastic systems. Math. Program. Stud. 6, 30-48 (1976) View ArticleGoogle Scholar
  20. Bismut, JM: An introductory approach to duality in optimal stochastic control. SIAM Rev. 20, 62-78 (1978) MATHMathSciNetView ArticleGoogle Scholar
  21. Borkar, V: Controlled diffusion processes. Probab. Surv. 2(4), 213-244 (2005) MATHMathSciNetGoogle Scholar
  22. Makhmudov, NI: General necessary optimality conditions for stochastic systems with controllable diffusion. In: Statistics and Control of Stochastic Processes, pp. 135-138. Nauka, Moscow (1989) Google Scholar
  23. Peng, S: A general stochastic maximum principle for optimal control problem. SIAM J. Control Optim. 28, 966-979 (1990) MATHMathSciNetView ArticleGoogle Scholar
  24. Aghayeva, CA, Abushov, QU: The maximum principle for the nonlinear stochastic optimal control problem of switching systems. J. Glob. Optim. 56(2), 341-352 (2013). doi:https://doi.org/10.1007/s10898-011-9825-8 MATHMathSciNetView ArticleGoogle Scholar
  25. Abushov, Q, Aghayeva, C: Stochastic maximum principle for the nonlinear optimal control problem of switching systems. J. Comput. Appl. Math. 259, 371-376 (2014). doi:https://doi.org/10.1016/j.cam.2013.06.010 MathSciNetView ArticleGoogle Scholar
  26. Aghayeva, C, Abushov, Q: The maximum principle for some nonlinear stochastic optimal control problem of switching system. Trans. Acad. Sci. Azerb. Ser. Phys.-Tech. Math. Sci. 31(3), 74-81 (2011) MathSciNetGoogle Scholar
  27. Aghayeva, C, Abushov, Q: Stochastic maximum principle for switching systems. In: Proceedings of the 4th International Conference ‘Problem of Cybernetics and Informatics’, vol. 3, pp. 198-201 (2012) Google Scholar
  28. Aghayeva, C, Abushov, Q: Stochastic optimal control problem for switching system with controlled diffusion coefficients. In: Proceedings Book of the World Congress on Engineering, vol. I, pp. 202-207 (2013) Google Scholar
  29. Gabasov, R, Kirillova, FM: Singular Optimal Control. Plenum, New York (1982) Google Scholar
  30. Mordukhovich, B: Approximation Methods in Problems of Optimization and Control. Nauka, Moscow (1988) MATHGoogle Scholar
  31. Makhmudov, NI, Bashirov, AE: First order and second order necessary conditions of optimality for stochastic systems. In: Proceedings of Steklov Mathematical Institute Seminar, Statistics and Control of Stochastic Processes, pp. 283-295 (1997) Google Scholar
  32. Tang, S: A second-order maximum principle for singular optimal stochastic control. Discrete Contin. Dyn. Syst., Ser. B 14(4), 1581-1599 (2010) MATHMathSciNetView ArticleGoogle Scholar
  33. Aghayeva, C, Alpaslan, M: Singular optimal control problem of stochastic switching systems. In: Proceedings of the World Congress on Engineering and Computer Science, vol. II, pp. 855-859 (2014) Google Scholar
  34. Ekeland, I: On the variational principle. J. Math. Anal. Appl. 47, 324-353 (1974) MATHMathSciNetView ArticleGoogle Scholar

Copyright

© Aghayeva 2016