Stochastic singular optimal control problem of switching systems with constraints

Abstract

This paper is devoted to the optimal control problem of switching system in which constraints on the state variable are given by inclusions. Using Ekeland’s variational principle, second-order necessary condition of optimality for stochastic switching systems with constraints is obtained, and transversality conditions for switching law are established.

1 Introduction

Stochastic differential equations have provided a lot of interest for problems of nuclear fission, communication systems, self-oscillating systems, etc., where the influences of random disturbances cannot be ignored [1, 2].

Hybrid systems, including special class-switching systems, are widely use for representation of noninvariant phenomena. Switching systems have the benefit of modeling dynamic processes with continuous law of movement. For general theory of stochastic switching systems, we refer to [3–5]. Recently, optimization problems of hybrid systems and, in particular, optimization problems of switching systems have attracted a lot of theoretical and practical interest [3, 6–10].

Stochastic control problems have a variety of practical applications in fields such as physics, biology, economics, management sciences, etc. [11, 12]. A classical approach for optimization and particularly for control problems is to derive necessary conditions satisfied by an optimal solution. The modern stochastic optimal control theory has been developed along the lines of Pontryagin’s maximum principle and Bellman’s dynamic programming [13, 14]. The stochastic maximum principle has been first considered by Kushner [15]. The earliest results on the extension of Pontryagin’s maximum principle to stochastic control problems are obtained in [16–19]. A general theory of the stochastic maximum principle based on random convex analysis was given by Bismut [20]. Modern presentations of stochastic maximum principle with backward stochastic differential equations are considered in [21–23]. First-order necessary conditions of optimality for stochastic switching systems have been studied by the author in [24–28].

It is well known that the first-order necessary conditions provide a basic tool to study the properties of optimal controls. However, in some cases, the Hamilton function vanishes almost everywhere, and therefore, first-order necessary conditions cannot provide enough information to find the desired optimal controls. The above-mentioned cases are called singular, and to investigate these situations, an additional test in terms of high-order necessary conditions is required. Deterministic optimization problems for singular controls were intensively used by [29, 30]. Such kind problems for stochastic systems have been investigated in [31, 32].

In this paper, the singular optimal control problem of stochastic switching systems with uncontrolled diffusion coefficients is considered. Second-order necessary conditions of optimality and transversality conditions are obtained in [33] for uncontrolled switching systems. Ekeland’s variational principle [34] has been widely used in various areas of analysis such as fixed point analysis, optimization, and optimal control theory. In this paper, backward stochastic differential equations and Ekeland’s variational principle is used to establish a singular maximum principle for stochastic optimal control problems of constrained switching systems.

2 Preliminaries and formulation of problem

Throughout this paper, we use the following notation. By N we denote some positive constant, $R^{n}$ denotes the n-dimensional real vector space, $|\cdot|$ denotes the Euclidean norm in $R^{n}$, and E represents the mathematical expectation. Assume that $w^{1}(t) ,w^{2}(t) ,\ldots,w^{r}(t)$ are independent Wiener processes that generate the filtration $F_{t}^{l} = \bar{\sigma}(w^{l}(t) ,t_{l - 1}, t_{l} )$, $l = \overline{1,r}$, $0 = t_{0} < t_{1} < \cdots < t_{r} = T$. Let $(\Omega,F,P)$ be a probability space with filtration $\{F_{t} ,t \in [ {0,T} ] \}$, where $F_{t} = \bigcup_{l = 1}^{r} {F_{t}^{l} }$; $L_{F}^{2} ( {a,b;R^{n}} )$ denotes the space of all predictable processes $x(t, \omega)\equiv x(t)$ such that $E\int _{a}^{b} {\vert {x_{t} ( \omega )} \vert ^{2}\,dt < + \infty}$; $R^{m\times n}$ is the space of linear transformations from $R^{m}$ to $R^{n}$. Let $O_{l} \subset R^{n_{l} }$, $Q_{l} \subset R^{m_{l} }$, $l = \overline{1,r}$, be open sets. Unless specified otherwise, we use the following notation: $\mathbf{t}=(t_{0},t_{1},\ldots,t_{r})$, $\mathbf{u}=(u^{1},u^{2},\ldots,u^{r})$, $\mathbf {x}=(x^{1},x^{2},\ldots,x^{r})$.

Consider the following stochastic control system:

$$\begin{aligned}& dx^{l}(t) = g^{l} \bigl( {x^{l}(t) ,u^{l}(t) ,t} \bigr)\,dt + f^{l} \bigl( {x^{l}(t),t} \bigr)\,dw^{l}(t),\quad t \in( {t_{l - 1} ,t_{l} } ] , l = \overline{1,r} , \end{aligned}$$

(1)

$$\begin{aligned}& x^{l}(t_{l - 1}) = \Phi^{l - 1} \bigl( {x^{l - 1}(t_{l - 1}) ,t_{l - 1}} \bigr),\quad l = \overline{2,r } ;x_{t_{0} }^{1} = x_{0} , \end{aligned}$$

(2)

$$\begin{aligned}& u^{l}(t) \in U_{\partial}^{l} \equiv \bigl( {u^{l} ( {\cdot,\cdot} )\in \mathrm{L}_{F^{l}}^{2}| u^{l} ( {t,\cdot} )\in U^{l}\subset\mathrm{R}^{m_{l} }, \mathrm{a.c.}} \bigr). \end{aligned}$$

(3)

Elements of $U_{\partial}^{l}$ are called admissible controls. The problem is to find an optimal solution $\mathbf{(}x,u)= ( {x^{1},x^{2},\ldots,x^{r},u^{1},u^{2},\ldots,u^{r}} )$ and a switching sequence $\mathbf{t}=(t_{1} ,t_{2} ,\ldots,t_{r})$ such that the cost functional

$$ J(u) = \sum_{l = 1}^{r} {E \biggl[ {\varphi^{l} \bigl( {x^{l}(t_{l}) } \bigr) + \int _{t_{l-1} }^{t_{l} } {p^{l} \bigl( {x^{l}(t) ,u^{l}(t) ,t} \bigr)\,dt} } \biggr]} $$

(4)

is minimized on the decisions of system (1)-(3) generated by all admissible controls $U = U^{1}\times U^{2}\times\cdots\times U^{r}$ under the conditions

$$ E q^{l}\bigl({x^{l}(t_{l})} \bigr)\in G^{l},\quad l = \overline{1,r }, $$

(5)

$G^{l}$ are a closed convex sets in $R^{1}$.

Assume that the following requirements are satisfied:

• (AI) The functions $g^{l}$, $f^{l}$, $p^{l}$, $l=\overline{1,r}$, and their derivatives are continuous in $( {x,u,t} )$.

• (AII) The derivatives of $g^{l}$, $f^{l}$, $p^{l}$, $l=\overline {1,r}$, are bounded by $N( {1+\vert x \vert })$.

• (AIII) The functions $\varphi^{l}(x)$, $l=\overline {1,r}$, and the functions $\Phi^{l}(x,t)$, $l=\overline {1,r-1}$, are continuously differentiable, and their derivatives are bounded by $N( {1+\vert x \vert })$.

• (AIV) The functions $q^{l}(x)$, $l=\overline {1,r}$, are continuously differentiable, and their derivatives are bounded by $N( {1+\vert x \vert })$.

Furthermore, the following requirements are assumed.

• (BI) The unctions $g^{l}$, $f^{l}$, $p^{l}$, $l= \overline{1,r}$, are twice continuously differentiable with respect to x.

• (BII) The functions $g^{l}$, $f^{l}$, $p^{l}$, $l = \overline{1,r}$, and all their derivatives are continuous in $( {x,u} )$; $g_{x}^{l}$, $g_{xx}^{l}$, $f_{x}^{l}$, $f_{xx}^{l}$, $p_{xx}^{l}$ are bounded.

• (BIII) The functions $\varphi^{l}(x):R^{n_{l}} \to R^{1}$, $l = \overline{1,r}$, and the functions $\Phi^{l}(x,t):R^{n_{l} }\times{\mathrm {T}} \to R^{1}$, $l =\overline{1,r - 1}$, are twice continuously bounded differentiable.

• (BIV) The functions $q^{l}(x):R^{n_{l}} \to R^{1}$, $l = \overline{1,r}$, are twice continuously bounded differentiable.

Consider the sets:

$$A_{i} = \mathrm{T}^{i + 1}\times\prod _{j = 1}^{i} {O_{j} } \times\prod _{j = 1}^{i} {Q_{j} } , \quad i = \overline{1,r}, $$

with the elements

$$\pi^{i} = \bigl(t_{0} ,t_{1} ,t_{i} ,x^{1}(t) ,x^{2}(t) ,\ldots,x^{i}(t),u^{1},u^{2}, \ldots,u^{i}\bigr). $$

Definition 1

The set of functions $\lbrace{x^{l}(t) = x^{l} ({t,\pi^{l}} ) }, t \in [ {t_{l - 1} ,t_{l} } ], l = \overline{1,r}\rbrace$ is said to be a solution of equations (1)-(2) with variable structure corresponding to an element $\pi^{r} \in{\mathrm {A}}_{r}$ if the function $x^{l}(t) \in O_{l}$ on the interval $[ {t_{l - 1} ,t_{l} } ]$ satisfies condition (2) at point $t_{l}$, is absolutely continuous on the interval $[ {t_{l - 1} ,t_{l} } ]$ with probability 1, and satisfies equation (1) almost everywhere.

Definition 2

The element $\pi^{r} \in A_{r}$ is said to be admissible if the pairs $( {x^{l}(t) , u^{l}(t) } )$, $t \in [ {t_{l - 1} ,t_{l} } ]$, $l = \overline{1,r}$, are solutions of system (1)-(3) and satisfy the constraints (5).

Definition 3

Let $A_{r}^{0}$ be the set of admissible elements. The element $\bar{\pi}^{r} \in A_{r}^{0}$ is said to be an optimal solution of problem (1)-(5) if there exist admissible controls $\bar {u}^{l}(t)$, $t \in [ {t_{l - 1} ,t_{l} } ]$, $l = \overline {1,r}$, and the corresponding solutions $\bar {x}^{l}(t)$, $t \in [ {t_{l - 1} ,t_{l} } ]$, $l = \overline {1,r}$, of system (1)-(2) such that the pairs $( {\bar{x}^{l}(t) ,\bar{u}^{l}(t)} )$, $l = \overline{1,r}$, minimize the functional (4).

3 Maximum principle

The following is a revised and improved version of Theorem 1 in [25].

Theorem 1

Suppose that, conditions (AI)-(AIII) hold and

$$\pi^{r}=\bigl(t_{0} ,t_{1}, t_{r} ,x^{1}(t) ,x^{2}(t) ,\ldots, x^{r}(t), u^{1}, u^{2}, \ldots, u^{r}\bigr) $$

is an optimal solution of problem (1)-(4). Then

• (a) there exist random processes $(\psi^{l}(t) ,\beta^{l}(t))\in{L}_{F}^{2} (t_{l-1} ,t_{l} ;{R}^{n_{l} })\times{L}_{F}^{2} (t_{l-1} ,t_{l} ;{R}^{n_{l}\times n_{l}})$ that are the solutions of the following adjoint equations:

  $$ \left \{ \textstyle\begin{array}{@{}l} d\psi^{l}(t) =-H_{x}^{l} (\psi^{l}(t) ,x^{l}(t) ,u^{l}(t) ,t)\,dt+\beta^{l}(t)\,dw^{l}(t), \quad t_{l-1} \le t< t_{l} , \\ \psi^{l}(t_{l}) =-\varphi_{x}^{l} (x^{l}(t_{l}))+\psi^{l+1}(t_{l}) \Phi_{x}^{l} (x^{l}(t_{l}) ,t_{l} ),\quad l=\overline{1,r-1} , \\ \psi^{r}(t_{r}) =-\varphi_{x}^{r} (x^{r}(t_{r})); \end{array}\displaystyle \right . $$

  (6)

• (b) $\forall \bar{u}^{l}\in U^{l}$, $l=\overline{1,r}$, a.c. fulfills the maximum principle:

  $$ H^{l}\bigl(\psi^{l}(\theta) ,x^{l}( \theta) ,\bar{u}^{l},\theta\bigr)-H^{l}\bigl( \psi^{l}(\theta) ,x^{l}(\theta) ,u^{l}(\theta) , \theta\bigr)\le0, \quad\textit{a.e. } \theta\in[t_{l-1} , t_{l} ] ; $$

  (7)

• (c) the following transversality conditions hold:

  $$\psi^{l+1}(t_{l})\Phi_{t}^{l} \bigl( {x^{l}(t_{l}), t_{l}} \bigr)=0, \quad \textit{a.c.}, l=\overline{1,r-1} , $$

  (8)

  where, for $t\in [ {t_{l-1} ,t_{l} } ]$,

  $$H^{l} \bigl( {\psi(t), x(t), u(t), t} \bigr)=\psi(t) g^{l} \bigl( {x(t), u(t), t} \bigr)+\beta(t) f^{l} \bigl( {x(t), t} \bigr)-p^{l} \bigl( {x(t), u(t), t} \bigr). $$

The following result can be proved according to the scheme in the proof of Theorem 2 in [25]. In order to investigate the optimal control problem with constraints (5), we use Ekeland’s variational principle.

Theorem 2

Suppose that conditions (AI)-(AIV) hold and

$$\pi^{r}=\bigl(t_{0} ,t_{1}, t_{r} ,x^{1}(t) ,x^{2}(t) ,\ldots, x^{r}(t), u^{1}, u^{2}, \ldots, u^{r}\bigr) $$

is an optimal solution of problem (1)-(5). Then

• (a) there exist random processes $(\psi^{l}(t) ,\beta^{l}(t))\in{L}_{F}^{2} (t_{l-1} ,t_{l} ;{R}^{n_{l} })\times{L}_{F}^{2} (t_{l-1} ,t_{l} ;{R}^{n_{l}\times n_{l}})$ that are solutions of the following adjoint equations:

  $$ \left \{ \textstyle\begin{array}{@{}l} d\psi^{l}(t) =-H_{x}^{l} (\psi^{l}(t) ,x^{l}(t) ,u^{l}(t) ,t)\,dt+\beta^{l}(t)\,dw^{l}(t), \quad t_{l-1} \le t< t_{l} , \\ \psi^{l}(t_{l}) =- \lambda_{0}^{l} q_{x}^{l} (x^{l}(t_{l}) )-\lambda_{1}^{l}\varphi_{x}^{l} (x^{l}(t_{l}))+\psi^{l+1}(t_{l}) \Phi_{x}^{l} (x^{l}(t_{l}) ,t_{l} ), \quad l=\overline{1,r-1} , \\ \psi^{r}(t_{r}) =- \lambda_{0}^{r} q_{x}^{r} (x^{r}(t_{r}) )-\lambda_{1}^{r}\varphi_{x}^{r} (x^{r}(t_{r})); \end{array}\displaystyle \right . $$

  (9)

• (b) $\forall\bar{u}^{l}\in U^{l}$, $l=\overline{1,r}$, a.c. fulfills the maximum principle:

  $$ H^{l}\bigl(\psi^{l}(\theta) ,x^{l}( \theta) ,\bar{u}^{l},\theta\bigr)-H^{l}\bigl( \psi^{l}(\theta) ,x^{l}(\theta) ,u^{l}(\theta) , \theta\bigr)\le0, \quad\textit{a.e. } \theta\in[t_{l-1} , t_{l} ] ; $$

  (10)

• (c) the following transversality conditions hold:

  $$\psi^{l+1}(t_{l})\Phi_{t}^{l} \bigl( {x^{l}(t_{l}), t_{l}} \bigr)=0, \quad \textit{a.c.}, l=\overline{1,r-1} , $$

  (11)

  where, for $t\in [ {t_{l-1} ,t_{l} } ]$,

  $$H^{l} \bigl( {\psi(t), x(t), u(t), t} \bigr)=\psi(t) g^{l} \bigl( {x(t), u(t), t} \bigr)+\beta(t) f^{l} \bigl( {x(t), t} \bigr)-p^{l} \bigl( {x(t), u(t), t} \bigr). $$

4 The second-order necessary condition

In order to review in detail the stochastic singular control switching systems, we need to introduce the definition of singular controls in the maximum principle sense.

Definition 4

Admissible controls $u^{l}(t)$, $l=1,\ldots,r$, are said to be singular on control regions $V^{l}$ if each $V^{l}\subset U^{l}$ is nonempty and, for a.e. $t \in[t_{l - 1} ,t_{l} )$, we have

$$H_{x}^{l} \bigl(\psi^{l}(t) ,x^{l}(t) ,u^{l}(t) ,t\bigr)=H_{x}^{l} \bigl( \psi^{l}(t) ,x^{l}(t) ,v^{l} ,t\bigr),\quad \forall v^{l}\in V^{l}. $$

To state the main result of this paper, we will use the following theorem, the full proof of which can be found in [33]. However, for convenience of the reader, we present a brief proof.

Theorem 3

Suppose that conditions (AI)-(AIII) and (BI)-(BIII) hold. Let $\pi^{r} = (t_{0} ,t_{1} ,\ldots, t_{r} ,x(t_{1}) ,x(t_{2}) ,\ldots,x(t_{r}), u^{1},u^{2},\ldots,u^{r})$ be an optimal solution of problem (1)-(4), and $\mathbf{u} = ( u^{1},u^{2},\ldots,u^{r})$ be singular on the control region $\mathbf{V} = ( V^{1},V^{2},\ldots,V^{r})$. Then

• (a) there exist random processes $(\psi^{l}(t) ,\beta^{l}(t)) \in L_{F^{l}}^{2}(t_{l - 1} ,t_{l} ;R^{n_{l} })\times L_{F^{l}}^{2} (t_{l - 1}, t_{l} ;R^{n_{l}\times n_{l} })$ and $(\Psi^{l}(t) ,\mathrm{K}^{l}(t) ) \in L_{F^{l}}^{2} (t_{l - 1} ,t_{l} ;R^{n_{l} })\times L_{F^{l}}^{2} (t_{l - 1} ,t_{l} ;R^{n_{l} \times n_{l} })$ that are solutions of the following adjoint equations:

  $$\begin{aligned}& \left \{ \textstyle\begin{array}{@{}l} d\psi^{l}(t) = - H_{x}^{l} (\psi^{l}(t) ,x^{l}(t) ,u^{l}(t) ,t)\,dt + \beta^{l}(t)\,dw(t) ,\quad t_{l - 1}\leq t < t_{l} ,\\ \psi^{l}(t_{l} ) = - \varphi_{x}^{l} (x^{l}(t_{l}) ) + \psi^{l + 1}(t_{l} ) \Phi _{x}^{l} (x^{l}(t_{l}) ,t_{l} ),\quad l = \overline{1,r - 1} , \\ \psi^{r}(t_{r} ) = - \varphi_{x}^{r} (x^{r}(t_{r})) ; \end{array}\displaystyle \right . \end{aligned}$$

  (12)
  $$\begin{aligned}& \left \{ \textstyle\begin{array}{@{}l} d\Psi^{l}(t) = -[\mathbf{{H}}_{x}^{l} ( {\Psi_{t} ^{l},x^{l}(t) ,u^{l}(t) ,t} ) + H_{xx}^{l} (\psi^{l}(t) ,x^{l}(t) ,u^{l}(t) ,t)\\ \hphantom{d\Psi^{l}(t) =}{}+ f_{x}^{l\ast} (x^{l}(t) ,t)\Psi^{l}(t) f_{x}^{l} (x^{l}(t) ,t) ]\,dt + \mathrm{K}^{l}(t)\,dw^{l}(t) ,\quad t \in[t_{l - 1} ,t_{l} ), \\ \Psi^{l}(t_{l}) = - \varphi_{xx}^{l} (x^{l}(t_{l}) ) + \Psi^{l + 1}(t_{l}) \Phi_{xx}^{l} (x^{l}(t_{l}) ,t_{l} ),\quad l = \overline{1,r - 1} , \\ \Psi^{r}(t_{r}) = - \varphi_{xx}^{r}(x^{r}(t_{r})); \end{array}\displaystyle \right . \end{aligned}$$

  (13)

• (b) for a.e. $\theta\in[t_{l - 1} ,t_{l} ]$ and $\forall \bar{u}^{l} \in V^{l}$, $l = \overline{1,r}$, a.c. the second-order maximum conditions fulfill

  $$\begin{aligned} &\Delta_{\bar{u}^{l}} H_{x}^{l}\bigl( \psi^{l}(\theta),x^{l}(\theta),u^{l},\theta\bigr) \Delta _{\bar{u}^{l}} g^{l\ast}\bigl(x^{l}(\theta) ,u^{l}(\theta),\theta\bigr) \\ &\quad{} +\Delta_{\bar{u}^{l}} g^{l\ast}\bigl(x^{l}(\theta) ,u^{l}(\theta),\theta \bigr)\Psi^{l}(\theta) \Delta_{\bar{u}^{l}} g^{l}\bigl(x^{l}(\theta) ,u^{l}(\theta),\theta \bigr) \le0; \end{aligned}$$

  (14)

• (c) the following transversality conditions hold:

  $$\begin{aligned} &\psi^{l + 1}(t_{l}) \Phi_{t}^{l} \bigl( {x^{l}(t_{l}) ,t_{l}} \bigr)+0.5\Psi ^{{l + 1}^{*}}(t_{l}) \Phi_{tt}^{l} \bigl( {x^{l}(t_{l}) ,t_{l} } \bigr)\Psi^{l + 1}(t_{l}) = 0 , \\ &\quad l = \overline{1,r - 1} , \textit{a.c.} \end{aligned}$$

  (15)

  Here

  $$\begin{aligned} \mathbf{H} ^{l}\bigl( {\Psi(t),x(t),u(t),t} \bigr) ={}& \Psi(t) g^{l}\bigl({x(t) ,u(t) ,t}\bigr) + g^{l\ast} \bigl( {x(t) ,u(t),t} \bigr)\Psi(t) \\ &{}+\mathrm{K }(t) f^{l} \bigl( {x(t) ,t} \bigr)+ f^{l\ast} \bigl( {x(t) ,t} \bigr)\mathrm{K }(t). \end{aligned}$$

Proof

Let $\bar{u}^{l}(t)= u^{l}(t) + \Delta\bar{u}^{l}(t)$, $l = \overline {1,r}$, be some admissible controls and $\bar{x}^{l}(t)= x^{l}(t) + \Delta\bar {x}^{l}(t)$, $l = \overline{1,r}$, be the corresponding trajectories of system (1)-(3). Let $0 = t_{0} < t_{1} < \cdots < t_{r} = T$ be a switching sequence. Then we obtain the following identities for some sequence $0 = \bar{t}_{0} < \bar{t}_{1} < \cdots < \bar {t}_{r} =T$:

$$ \left \{ \textstyle\begin{array}{@{}l} d\Delta\bar{x}^{l}(t) = [\Delta_{\bar{u}^{l}} g^{l}(x^{l}(t) ,u^{l}(t) ,t) + g_{x}^{l} (x^{l}(t) ,u^{l}(t) ,t)\Delta\bar{x}^{l}(t) \\ \hphantom{d\Delta\bar{x}^{l}(t) =}{}+0.5\Delta x^{\ast}(t)g_{xx}^{l} (x^{l}(t) ,u^{l}(t) ,t)\Delta x(t) ]\,dt \\ \hphantom{d\Delta\bar{x}^{l}(t) =}{}+ [f_{x}^{l} (x^{l}(t) ,t)\Delta\bar{x}^{l}(t) + 0.5\Delta x^{\ast}(t)f_{xx}^{l} (x^{l}(t) ,t)\Delta x(t) ]\,dw^{l}(t) + \eta_{t}^{1} , \\ \Delta\bar{x}^{1}(t_{0}) = 0, \\ \Delta\bar{x}^{l}(t_{l - 1}) = \Phi^{l - 1} ( {\bar{x} ^{l - 1}(t_{l - 1}) ,\bar{t}_{l - 1} } ) - \Phi^{l - 1} (x ^{l - 1}(t_{l - 1}) ,t_{l - 1} ),\quad l = \overline{2,r}, \end{array}\displaystyle \right . $$

(16)

where

$$\begin{aligned} \eta_{t}^{1} ={}& \int _{0}^{1} { \bigl[ {g_{x}^{l\ast} \bigl(x^{l}(t) + \mu \Delta\bar{x}^{l}(t) , \bar{u}_{t}^{l} ,t\bigr) - g_{x}^{l\ast} \bigl(x^{l}(t) ,u^{l}(t) ,t\bigr)} \bigr]} \Delta \bar{x}^{l}(t)\,d\mu \,dt \\ &{}+0.5 \int _{0}^{1} \Delta\bar{x}^{l\ast}(t) \bigl[ g_{xx}^{l\ast} \bigl(x^{l}(t) + \mu\Delta \bar{x}^{l}(t) ,u^{l}(t) ,t\bigr) - {g_{xx}^{\ast}\bigl(x^{l}(t) ,u^{l}(t) ,t\bigr)} \bigr] {\Delta\bar {x}^{l}(t)\,d\mu}\,dt \\ &{}+ \int _{0}^{1} { \bigl[ {f_{x}^{l\ast} \bigl(x^{l}(t) + \mu\Delta\bar {x}^{l}(t),t\bigr) - f_{x}^{l\ast} \bigl(x^{l}(t) ,t\bigr)} \bigr]} \Delta \bar{x}^{l}(t)\,d\mu \,dw^{l}(t) \\ &{}+0.5 \int _{0}^{1} \Delta\bar{x}^{l\ast}(t) \bigl[ f_{xx}^{l\ast} \bigl(x^{l}(t)+\mu\Delta \bar{x}^{l}(t) ,t\bigr) - {f_{xx}^{\ast}\bigl(x^{l}(t) ,t\bigr)} \bigr] { \Delta\bar {x}^{l}(t)\,d\mu}\,dw^{l}(t) . \end{aligned}$$

In order to establish the existence and uniqueness of a solution of adjoint stochastic differential equations, it suffices to follow the method described in the article [17] and to use the independence of Wiener processes ${w}^{1}(t),\ldots,{w}^{r}(t)$.

The stochastic processes $\psi^{l}(t)$, $l=\overline{1,r}$, at the points $t_{1} ,t_{2} ,\ldots,t_{r}$ are defined as

$$ \psi^{l}(t_{l}) = - \varphi_{x}^{l} \bigl( {x^{l}(t_{l}) } \bigr) + \psi^{l + 1}(t_{l}) \Phi_{x}^{l} \bigl({x^{l}(t_{l} ) ,t_{l} } \bigr),\qquad \psi^{r}(t_{r})= - \varphi_{x}^{r} \bigl( {x(t_{r})^{r}} \bigr) $$

(17)

and

$$ \Psi^{l}(t_{l}) = - \varphi_{xx}^{l} \bigl( {x^{l}(t_{l}) } \bigr) + \Psi^{l + 1}(t_{l}) \Phi_{xx}^{l} \bigl( {x^{l}(t_{l}) ,t_{l} } \bigr),\qquad \Psi^{r}(t_{r}) = - \varphi_{xx}^{r} \bigl( {x^{r}(t_{r})} \bigr). $$

(18)

According to Itô’s formula [12], the expression of an increment of the cost functional (4) takes the following form:

$$\begin{aligned} \Delta J ( u )=&{}\sum_{l=1}^{r} E \biggl\{ {\varphi ^{l} \bigl( {\bar{x}^{l} (t_{l})} \bigr)-\varphi^{l} \bigl( {x^{l}(t_{l}) } \bigr)+ \int _{t_{l-1} }^{t_{l} } { \bigl[ {p^{l} \bigl( { \bar{x}^{l}(t) ,\bar {u}_{t}^{l} ,t} \bigr) -p^{l} \bigl( {x^{l}(t) ,u^{l}(t) ,t} \bigr)} \bigr]\,dt} } \biggr\} \\ ={}& {-}\sum_{l=1}^{r} E \int _{t_{-1} }^{t_{l} } \bigl[ {\Delta _{\bar{u}^{l}}H^{l} \bigl( {\psi^{l}(t) ,x^{l}(t) ,u^{l}(t) ,t} \bigr) +H_{x}^{l} \bigl( {\psi ^{l}(t) ,x^{l}(t) ,u^{l}(t) ,t} \bigr)}\Delta\bar{x}^{l} (t) \\ &{}-0.5\Delta\bar{x}^{l\ast}(t) f_{x}^{l\ast} \bigl( {x^{l}(t) ,t} \bigr) \Psi^{l}(t) f_{x}^{l} \bigl( {x^{l}(t) ,t} \bigr)\Delta\bar{x}^{l} (t) \\ &{}+\Delta\bar{x}^{l\ast}(t) \Delta _{\bar{u}^{l}} g^{l} \bigl({x^{l}(t) ,u^{l}(t),t} \bigr) \Psi^{l}(t) \Delta\bar {x}^{l}(t) \\ &{}-\Delta\bar{x}^{l\ast}(t) g_{x}^{l} \bigl(x^{l}(t) ,u^{l}(t) ,t \bigr) \Psi^{l}(t) \Delta\bar{x}^{l} (t) - \Delta\bar{x}^{l\ast}(t) f_{x}^{l} \bigl( x^{l}(t) ,t \bigr) K_{t}^{l} \Delta\bar{x}^{l} (t) \\ &{}+\psi_{t}^{l *} \Delta_{\bar{u}^{l}} g_{x}^{l} \bigl(x^{l}(t) ,u^{l}(t) ,t \bigr)\Delta\bar{x}^{l} (t) - \Delta _{\bar{u}^{l}} p_{x}^{l} \bigl(x^{l}(t) ,u^{l}(t) ,t \bigr) \Delta\bar {x}^{l} (t) \bigr] \Delta\bar{t}_{l} \,dt \\ &{}+\sum_{l=1}^{r-1} {\psi^{l+1}(t_{l}) \Phi_{t} \bigl( {x^{l}(t_{l}) ,t_{l} } \bigr)} +\sum_{l=1}^{r} \eta_{t_{l-1} }^{t_{l} }, \end{aligned}$$

(19)

where

$$\begin{aligned} \eta_{t_{l - 1}}^{t_{l}} ={}& E \int _{0}^{1} ( {1 - \mu} ) \bigl[ { \varphi_{x}^{l*} \bigl( {\bar{x}^{l} (t_{l})} \bigr) - \varphi_{x}^{*} \bigl( {x^{l}(t_{l}) } \bigr)} \bigr] \Delta\bar{x}^{l} (t_{l})\,d\mu \\ &{}-E \int _{t_{l - 1} }^{t_{l} } \int _{0}^{1} ({1 - \mu} ) \bigl[ H_{x}^{l} \bigl( \psi^{l}(t) ,{ \bar{x}^{l} (t_{l})},u^{l}(t) ,t \bigr) \\ &{}- H_{x}^{l} \bigl( {\psi^{l}(t) ,x^{l}(t) u^{l}(t) ,t} \bigr) \bigr]\Delta\bar{x}^{l}(t) \Delta \bar{t}_{l} \,d\mu\, dt \\ &{}- E \int _{0}^{1} { ( {1 - \mu} )\psi_{t_{l} }^{l + 1 } \bigl[ {\Phi_{x}^{l} \bigl( {\bar{x}^{l} (t_{l})} \bigr) - } } \Phi_{x}^{l} \bigl( {x^{l}(t) ,t_{l} } \bigr)\bigr]\Delta x^{l}(t) \Delta \bar{t}_{l} \,d\mu \\ &{}+E \int _{0}^{1} { ( {1 - \mu} )\Delta \bar{x}_{t_{l} }^{l * } \bigl[ {\varphi_{xx}^{l * } \bigl( {\bar{x}^{l} (t_{l})} \bigr) - \varphi_{xx}^{*} \bigl( {x^{l}(t_{l}) } \bigr)} \bigr]} \Delta \bar{x}^{l} (t_{l})\,d\mu \\ &{}+ E \int _{t_{l - 1} }^{t_{l} } \int _{0}^{1} ( {1 - \mu} )\Delta \bar{x}_{t}^{l * } \bigl[ H_{xx}^{l} \bigl( {\psi^{l}(t) ,{\bar{x}^{l} (t_{l})} ,u^{l}(t) ,t} \bigr) \\ &{}- {H_{xx}^{l} \bigl( { \psi^{l}(t) ,x^{l}(t) u^{l}(t) ,t} \bigr)} \bigr] \Delta\bar{x}^{l}(t) \Delta\bar{t}_{l} \,d\mu\, dt \\ &{}+ E \int _{0}^{1} ({1 - \mu} )\Delta\bar{x}_{t_{l} }^{l * } \Psi^{l+1}(t_{l}) \bigl[ {\Phi_{xx}^{l} \bigl( {\bar{x}^{l} (t_{l})},t_{l} } \bigr) - \Phi_{xx}^{l} \bigl( {x^{l}(t) ,t_{l} } \bigr) \bigr] \Delta\bar{x}^{l} (t_{l})\Delta \bar{t}_{l} \,d\mu. \end{aligned}$$

(20)

By assumptions (AIII) and (BIII), using identities (17) and (18) from expressions (16), (19), and (20), according to the fact that the coefficients of the independent increments $\Delta{x}^{l} (t)$, Δt̄ equal zero, we obtain that (15) is true. Taking into consideration (12), (13), (17), and (18), through the simple expression (19) of transformations, can be written as

$$\begin{aligned} \Delta J ( u ) ={}& \sum_{l = 1}^{s} \Delta J^{l} \bigl( {u^{l}} \bigr) \\ ={}& {-} \sum _{l = 1}^{s} E \int _{t_{l - 1} }^{t_{l} } \bigl[ {\Delta_{\overline{u} ^{l}} H^{l}} \bigl( {\psi^{l}(t) ,x^{l}(t) ,u^{l}(t) ,t} \bigr) + \Delta_{\overline{u} ^{l}} H_{x^{l}}^{l} \bigl( {\psi^{l}(t) ,x^{l}(t) ,u^{l}(t) ,t} \bigr) \Delta\bar{x}^{l}(t) \\ &{}- 0.5\Delta\bar{x}_{t_{l} }^{l\ast} f^{l\ast} \bigl( {x^{l}(t) ,t} \bigr)\Psi^{l}(t) f_{x}^{l} \bigl( {x^{l}(t) ,t} \bigr)\Delta\bar{x}^{l} (t_{l}) \\ &{}+ \Delta\bar{x}^{l\ast}(t_{l} ) \Delta_{\bar{u}^{l}} g^{l} \bigl( {x^{l}(t) ,u^{l}(t),t} \bigr) \Psi^{l}(t) \Delta\bar{x}^{l} (t_{l}) \\ &{}-\Delta\bar {x}^{l\ast}(t_{l} ) g_{x}^{l} \bigl( {x^{l}(t) ,u^{l}(t) ,t} \bigr) \Psi^{l}(t) \Delta \bar{x}^{l} (t_{l}) - \Delta\bar{x}^{l\ast}(t_{l} ) f_{x}^{l} \bigl( {x^{l}(t) ,t} \bigr) K^{l}(t) \Delta {\bar{x}^{l} (t_{l})} \bigr] \Delta\bar{t}_{l} \,dt \\ &{}+ \sum_{l = 1}^{r} \eta_{t_{l - 1} }^{t_{l}}. \end{aligned}$$

(21)

Consider the following spike variations:

$$\Delta u^{l}(t) =\Delta u_{t,\varepsilon^{l}}^{\theta_{l}} = \left \{ \textstyle\begin{array}{@{}l@{\quad}l} 0, & t\notin[ {\theta_{l} ,\theta_{l} +\varepsilon_{l} } ),\varepsilon_{l} >0, \theta_{l} \in [ {t_{l-1} ,t_{l} } ), \\ \bar{u}^{l}- {u}_{t}^{l} ,& t\in [ {\theta_{l} ,\theta_{l} +\varepsilon_{l} } ),\bar{u}^{l}\in L^{2} ( {\Omega ,F^{\theta_{l} },P;R^{m}} ) , \end{array}\displaystyle \right . $$

where $\varepsilon_{l}>0$ are small enough.

The following lemma will be used in estimation of increment (21).

Lemma 1

Assume that conditions (AI), (AII), (BI), (BII) are satisfied. Then

$$\lim _{\varepsilon_{l} \to0} E\bigl\vert {x_{\varepsilon_{l} }^{\theta_{l} }(t)- x^{l}(t) } \bigr\vert ^{2} \leq N\varepsilon_{l}^{2} \quad\textit{a.e. in } [ {t_{l - 1} ,t_{l} } ), l = \overline{1,r} . $$

Here $x_{\varepsilon_{l} }^{\theta_{l} }(t)$ are the trajectories of system (1)-(2) corresponding to the controls $u_{\varepsilon _{l} }^{\theta_{l} }(t) = u^{l}(t) + \Delta u_{\varepsilon_{l} }^{\theta_{l}}(t)$.

Proof

See Lemma 1 in [33]. □

By invoking expression (20) and using Lemma 1 the following estimation is implied:

$$\eta_{\theta_{l} }^{\theta_{l} +\varepsilon_{l} } =o \bigl( {\varepsilon_{l}^{2}} \bigr),\quad l=\overline{1,r} . $$

According to the singularity of controls $\bar{u}^{l}(t)$, $l=\overline {1,r}$, in the maximum sense for given special variation from (21), it follows that, for each l,

$$\begin{aligned} \Delta_{\theta^{l}} J(u) ={}& {-}\varepsilon_{l} E \bigl[ \Delta_{\bar{u}^{l}} H_{x}\bigl(\psi^{l}( \theta_{l}), x^{l}(\theta_{l}) ,u^{l}( \theta_{l}),\theta_{l}\bigr) \Delta_{\bar{u}^{l}} g^{l}\bigl(x^{l}(\theta_{l}) ,u^{l}( \theta_{l}) ,\theta_{l} \bigr) \\ &{} +\Delta_{\bar{u}^{l}} g^{l\ast}\bigl(x^{l}( \theta_{l}) ,u^{l}(\theta_{l}) ,\theta_{l} \bigr) \Psi^{l}(\theta_{l}) \Delta_{\bar{u}^{l}} g^{l} \bigl(x_{\theta _{l} }^{l} ,u^{l}(\theta_{l}) , \theta_{l} \bigr) \Delta\bar{t}_{l} \bigr] + o\bigl( \varepsilon_{l}^{2} \bigr) \geq0. \end{aligned}$$

Finally, due to the smallness and arbitrariness of $\varepsilon_{l}$, (14) is fulfilled. Theorem 3 is proved. □

5 Necessary condition of optimality for singular controls

This section is devoted to investigation of singular optimal control problems for stochastic switching systems with constraints. We obtain a necessary condition of optimality for stochastic control problem of switching systems (1)-(5). Using Ekeland’s variational principle, the given problem is converted into a sequence of unconstrained problems. Based on Theorem 3, we establish the maximum principle and transversality conditions for the sequence of switching systems. Then we obtain a necessary condition of optimality in the case where endpoint constraints are imposed.

Theorem 4

Suppose that conditions (AI)-(AIV) and (BI)-(BIV) hold. Let $\pi^{r} = (t_{0} ,t_{1} ,\ldots, t_{r} ,x(t_{1}) ,x(t_{2}) , \ldots,x(t_{r}), u^{1},u^{2},\ldots,u^{r})$ be an optimal solution of problem (1)-(4), and $\mathbf{u} = ( u^{1},u^{2},\ldots,u^{r})$ be singular on the control region $\mathbf{V} = ( V^{1},V^{2},\ldots,V^{r})$. Then

• (a) there exist random processes $(\psi^{l}(t) ,\beta^{l}(t)) \in L_{F^{l}}^{2}(t_{l - 1} ,t_{l} ;R^{n_{l} })\times L_{F^{l}}^{2} (t_{l - 1} ,t_{l} ;R^{n_{l}\times n_{l} })$ and $(\Psi^{l}(t) ,\mathrm{K}^{l}(t) ) \in L_{F^{l}}^{2} (t_{l - 1} ,t_{l} ;R^{n_{l} })\times L_{F^{l}}^{2} (t_{l - 1} ,t_{l} ;R^{n_{l} \times n_{l} })$ that are solutions of the following adjoint equations:

  $$\begin{aligned}& \left \{ \textstyle\begin{array}{@{}l} d\psi^{l}(t) = - H_{x}^{l} (\psi^{l}(t) ,x^{l}(t) ,u^{l}(t) ,t)\,dt + \beta^{l}(t)\,dw(t), \quad t_{l - 1}\leq t < t_{l} ,\\ \psi^{l}(t_{l} ) = - \lambda_{0}^{l} q_{x}^{l} (x^{l}(t_{l}) )- \lambda_{1}^{l}\varphi _{x}^{l} (x^{l}(t_{l}) ) + \psi^{l + 1}(t_{l} ) \Phi _{x}^{l} (x^{l}(t_{l}) ,t_{l} ),\quad l = \overline{1,r - 1} , \\ \psi^{r}(t_{r} ) = - \lambda_{0}^{r} q_{x}^{r}(x^{r}(t_{r}))-\lambda_{1}^{r} \varphi_{x}^{r} (x^{r}(t_{r})) ; \end{array}\displaystyle \right . \end{aligned}$$

  (22)
  $$\begin{aligned}& \left \{ \textstyle\begin{array}{@{}l} d\Psi^{l}(t) = - [\mathbf{{H}} _{x}^{l} ( {\Psi^{l}(t),x^{l}(t) ,u^{l}(t) ,t} ) + H_{xx}^{l} (\psi^{l}(t) ,x^{l}(t) ,u^{l}(t) ,t)\\ \hphantom{d\Psi^{l}(t) =}{}+ f_{x}^{l\ast} (x^{l}(t) ,t)\Psi^{l}(t) f_{x}^{l} (x^{l}(t) ,t) ]\,dt + \mathrm{K}^{l}(t)\,dw^{l}(t) ,\quad t \in[t_{l - 1} ,t_{l} ), \\ \Psi^{l}(t_{l}) = - \lambda_{0}^{l} q_{xx}^{l} (x^{l}(t_{l}) )- \lambda_{1}^{l}\varphi _{xx}^{l} (x^{l}(t_{l}) ) \\ \hphantom{\Psi^{l}(t_{l}) =}{}+ \Psi^{l + 1}(t_{l}) \Phi_{xx}^{l} (x^{l}(t_{l}) ,t_{l} ), \quad l = \overline{1,r - 1} , \\ \Psi^{r}(t_{r}) = - \lambda_{0}^{r} q_{xx}^{r} (x^{r}(t_{r}) )- \lambda_{1}^{r} \varphi _{xx}^{r}(x^{r}(t_{r})); \end{array}\displaystyle \right . \end{aligned}$$

  (23)

• (b) a.e. $\theta\in[t_{l - 1} ,t_{l} ]$ and $\forall \bar{u}^{l} \in V^{l}$, $l = \overline{1,r}$, a.c. the second-order maximum conditions (14) are satisfied;

• (c) the transversality conditions (15) hold.

Proof

For any natural j, let us introduce the following approximating functional for each $l = \overline{1,r}$:

$$\begin{aligned} I_{j} ( {\mathbf{u}} ) &= S_{j} \Biggl\langle E\sum _{l=1}^{r} \biggl[\varphi ^{l} ( {t_{l} } ) + \int _{t_{l - 1} }^{t_{l} } {p^{l} \bigl( {x ^{l}(t) ,u ^{l}(t) ,t} \bigr)\,dt}\biggr] ,E\sum _{l=1}^{r} q^{l}\bigl(x^{l}(t_{l}) \bigr)\Biggr\rangle \\ &=\min_{(c,y^{l}) \in\varepsilon}\sqrt{\bigl\vert c - 1 / j - EM ( \mathbf{x} ,\mathbf{u} ,\mathbf{t} )\bigr\vert ^{2} + \sum _{l=1}^{r} \bigl\vert y^{l}- {Eq^{l}\bigl(x^{l}(t_{l})\bigr)} \bigr\vert ^{2}} , \end{aligned}$$

where $M ( \mathbf{x} ,\mathbf{u} ,\mathbf{t} )=\sum_{i=1}^{r} [ {\varphi^{l} (x ^{l}(t_{l}) ) + \int _{t_{l - 1} }^{t_{l} } {p(x ^{l}(t) ,u ^{l}(t) ,t)\,dt} } ]$, $\varepsilon= \{c:c \le J^{0},y^{l}\in G^{l} \}$, $c=c^{1}+\cdots+c^{r}$, and $J^{0}$ is the minimal value of the functional in problem (1)-(5).

Let $V^{l} \equiv(U_{\partial}^{l} ,d)$ be the space of controls obtained by means of the following metric: $d(u^{l},v^{l}) = ( {l \otimes P} ) \{ { ( {t,\omega} ) \in [ {t_{l - 1} ,t_{l} } ]\times\Omega:\nu_{t}^{l} \ne u ^{l}(t) } \}$. For each $l = \overline{1,r}$, $V^{l}$ is a complete metric space [34].

It is easy to prove the following fact.

Lemma 2

Assume that conditions (AI)-(AIV) hold, $u^{l,n}(t)$, $l = \overline{1,r}$, is a sequence of admissible controls from $V^{l}$, and $x^{l,n}(t)$ is the sequence of corresponding trajectories of system (1)-(3). If $d(u^{l,n}(t) ,u ^{l}(t) ) \to0$, then, $\lim_{n \to\infty} \{ \sup_{t_{l - 1} \leq t \leq t_{l} } E\vert {x^{l,n}(t) - x ^{l}(t) } \vert ^{2} \} = 0$, where $x ^{l}(t)$ is a trajectory corresponding to the admissible controls $u ^{l}(t)$, $l = \overline{1,r}$.

Due to the continuity of the functionals $I_{j}^{l} :V^{l} \to R^{n_{l} }$, according to Ekeland’s variational principle, there are controls such that $u^{l,j}(t) :d(u^{l,j}(t) ,u ^{l}(t) ) \le\sqrt{\varepsilon_{j}^{l} }$ and for $\forall u ^{l}(t) \in V^{l}$, the following is achieved: $I_{j}^{l} (u^{l,j}) \le I_{j}^{l} (u^{l}) + \sqrt{\varepsilon_{j}^{l} } d(u^{l,j},u^{l})$, $\varepsilon_{j}^{l} = \frac{1}{j}$.

This inequality means that $(t_{1} ,\ldots,t_{r} ,x^{1,j}(t) ,\ldots,x^{r,j}(t), u^{1,j}(t) ,\ldots,u^{r,j}(t) )$ for each $t \in (t_{l - 1} ,t_{l} ]$ is a solution of the following problem:

$$ \left \{ \textstyle\begin{array}{@{}l} J_{j}(u) = \sum_{l = 1}^{r} ( {I_{j}^{l} (u^{l}) + \sqrt{\varepsilon _{j}^{l} } E\int _{t_{l - 1} }^{t_{l} } {\delta(u ^{l}(t) ,u^{l,j}(t) )\,dt} } ) \to\min, \\ dx ^{l}(t) = g^{l}(x ^{l}(t) ,u ^{l}(t) ,t)\,dt + f^{l}(x ^{l}(t) ,t)\,dw(t),\quad l = \overline{1,r}, \\ x^{l + 1}(t_{l}) = \Phi^{l} ( {x ^{l}(t_{l}) ,t_{l} } ),\quad l = \overline{1,r - 1 } ;x^{1}(t_{0} ) = x_{0} , \\ u ^{l}(t) \in U_{\partial}^{l}. \end{array}\displaystyle \right . $$

(24)

The function $\delta(u,v)$ is determined in the following way:

$$\delta (u,v) = \left \{ \textstyle\begin{array}{@{}l@{\quad}l} 0,&u = v, \\ 1,&u \ne v. \end{array}\displaystyle \right . $$

Then according to Theorem 3, we have that if $(x^{1,j}(t) ,\ldots,x^{r,j}(t),u^{1,j}(t) ,\ldots,u^{r,j}(t))$ is an optimal solution of problem (24), then

(1) there exist random processes $(\psi^{l,j}(t), \beta^{l,j}(t)) \in L_{F^{l}}^{2} (t_{l - 1} ,t_{l} ;R^{n_{l} })\times L_{F^{l}}^{2} (t_{l - 1} ,t_{l} ;R^{n_{l} \times n_{l} })$ that are solutions of the following system:

$$ \left \{ \textstyle\begin{array}{@{}l} d\psi^{l,j}(t) = - H_{x}^{l} ( {\psi^{l,j}(t) ,x^{l,j}(t) ,u^{l,j}(t) ,t} )\,dt + \beta^{l,j}(t)\,dw^{l}(t) , \quad t \in [ {t_{l - 1} ,t_{l} } ), l = \overline{1,r}, \\ \psi^{l,j}(t_{l}) = - \lambda_{0}^{l,j}\varphi_{x}^{l} ( {x^{l,j}(t_{l})} ) - \lambda_{1}^{l,j} q_{x}^{l} ( {x^{l,j}(t_{l})} ) +\psi _{t_{l + 1} }^{l} \Phi_{x}^{l} (x^{l,j}(t_{l}) ,t_{l} ),\quad l= \overline{1,r - 1}, \\ \psi^{r}(t_{r}) = - \lambda_{0}^{r,j} \varphi_{x}^{r} ( {x^{r,j}(t_{r}} ) - \lambda_{1}^{r,j} q_{x}^{r} ( {x^{r,j}(t_{r})} ) , \end{array}\displaystyle \right . $$

(25)

and the random processes $\Psi^{l,j}(t) \in L_{F^{l}}^{2} (t_{l - 1} ,t_{l} ;R^{n_{l} })$, $\mathrm{K}^{l,j}(t) \in L_{F^{l}}^{2} (t_{l - 1} ,t_{l} ;R^{n_{l} \times n_{l} })$ that are solutions of the following system:

$$ \left \{ \textstyle\begin{array}{@{}l} d\Psi^{l,j}(t)= - [\mathbf{{H}}_{x}^{l} ( {\Psi^{l,j}(t),x^{l,j}(t) ,u^{l,j}(t) ,t} ) + H_{xx}^{l} (\psi^{l,j}(t),x^{l,j}(t) ,u^{l,j}(t),t)\\ \hphantom{d\Psi^{l,j}(t)=}{}+ f_{x}^{l\ast} (x^{l,j}(t),u^{l,j}(t) ,t)\Psi^{l,j}(t)f_{x}^{l} (x^{l,j}(t),u^{l,j}(t),t) ]\,dt + \mathrm{K}^{l,j}(t)\,dw^{l}(t), \\ \Psi^{l,j}(t_{l}) = -\lambda_{0}^{l,j} \varphi_{xx}^{l} (x^{l,j}(t_{l}) ) - {\lambda_{1}^{l,j} } q_{xx}^{l} (x^{l,j}(t_{l})+ \Psi_{t_{l + 1} }^{l,j} \Phi_{xx}^{l} (x^{l,j}(t_{l}) ,t_{l} ), \quad l = \overline{1,r - 1} ,\\ \Psi^{r,j}(t_{r}) = - \lambda_{0}^{r,j} \varphi_{xx}^{r} (x^{r,j}(t_{r}) ) - {\lambda_{1}^{r,j} } q_{xx}^{r} (x^{r,j}(t_{r}), \end{array}\displaystyle \right . $$

(26)

where nonzero $(\lambda_{0}^{l,j} ,\lambda_{1}^{l,j} ) \in R^{k + 1}$, $l = \overline{1,r}$, satisfy the following requirement:

$$ \bigl( \lambda_{0}^{l,j},\lambda_{1}^{l,j} \bigr)= \biggl[-c^{l}+1/j+ {\varphi ^{l}\bigl(x ^{l}(t_{l}) \bigr) + \int _{t_{l - 1} }^{t_{l} } {p\bigl(x ^{l}(t) ,u ^{l}(t) ,t\bigr)\,dt} } ,-y^{l}+ Eq^{l} \bigl({x^{l,j}(t_{l})}\bigr) \biggr]\Big/J_{j}^{0} $$

(27)

with

$$J_{j}^{0} = \Biggl( \sum_{l = 1}^{r} \bigl\vert y^{l}-{Eq^{l}\bigl(x^{l,j}(t_{l}) \bigr)} \bigr\vert ^{2} + { \Biggl\vert {c - 1 / j - E\sum _{l = 1}^{r} \biggl[ {\varphi ^{l}\bigl(x ^{l}(t_{l}) \bigr) + \int _{t_{l - 1} }^{t_{l} } {p\bigl(x ^{l}(t) ,u ^{l}(t) ,t\bigr)\,dt} } \biggr]} \Biggr\vert ^{2} } \Biggr)^{1/2} ; $$

(2) for a.e. $t \in[t_{l - 1} ,t_{l} ]$ and $\forall \tilde{u}^{l} \in V^{l}$, $l = \overline{1,r}$, a.c. is satisfied:

$$\begin{aligned} &\Delta_{\bar{u}^{l,j}} H_{x}^{l}\bigl( \psi^{l,j}(\theta),x^{l,j}(\theta ),u^{l,j},\theta \bigr)\Delta_{\bar{u}^{l,j}} g^{l\ast}\bigl(x^{l,j}(\theta) ,u^{l,j}(\theta),\theta\bigr) \\ &\quad{}+\Delta_{\bar{u}^{l,j}} g^{l\ast}\bigl(x^{l,j}(\theta) ,u^{l,j}(\theta ),\theta\bigr)\Psi^{l,j}(\theta) \Delta_{\bar{u}^{l}} g^{l}\bigl(x^{l,j}(\theta) ,u^{l,j}(\theta),\theta\bigr) \leq0; \end{aligned}$$

(28)

(3) the following transversality conditions hold:

$$ \psi^{l+1,j}(t_{l}) \Phi_{t}^{l,j} \bigl( {x^{l,j}(t_{l}) ,t_{l}} \bigr)+0.5 \Psi^{{l + 1,j}^{*}}(t_{l}) \Phi_{tt}^{l,j} \bigl( {x^{l,j}(t_{l}) ,t_{l} } \bigr)\Psi^{l + 1,j}(t_{l}) = 0 ,\quad l = \overline{1,r - 1} , \textit{a.c.} $$

(29)

According to conditions (BI)-(BIV), we have that $(\lambda_{0}^{j} ,\lambda_{1}^{j} ) \to(\lambda_{0} ,\lambda_{1} )$ as $j \to \infty$.

The proof of item (a) of Theorem 4 is based on the following lemmas.

Lemma 3

Let $\psi^{l}(t_{l} )$ be a solution of system (22), and $\psi^{l,j}(t_{l} )$ be a solution of system (25). If $d(u^{l,j}(t) ,u ^{l}(t) ) \to0$, $j \to\infty$, then

$$E \int _{t_{l - 1} }^{t_{l} } \bigl|\psi^{l,j}(t) - \psi^{l}(t) \bigr|^{2}\,dt + E \int _{t_{l - 1} }^{t_{l} } {\bigl|\beta^{l,j}(t) - \beta^{l}(t) \bigr|^{2}\,dt \to0} ,\quad l = \overline{1,r.} $$

Proof

It is clear that $\forall t \in[t_{l - 1} ,t_{l} ]$,

$$\begin{aligned} &d \bigl( {\psi^{l,j}(t) - \psi^{l}(t) } \bigr) \\ &\quad= - \bigl[ {H_{x}^{l} \bigl( {\psi _{t}^{l,j} ,x^{l,j}(t) ,u^{l,j}(t) ,t} \bigr) - H_{x}^{l} \bigl( {\psi^{l}(t) ,x ^{l}(t) ,u ^{l}(t) ,t} \bigr)} \bigr]\,dt+ \bigl( {\beta^{l,j}(t) - \beta^{l}(t) } \bigr)\,dw(t) \\ &\quad = - \bigl[ {\psi^{l,j}(t) } g_{x}^{l} \bigl( {x^{l,j}(t) ,u^{l,j}(t) ,t} \bigr) + \beta^{l,j}(t) f_{x}^{l} \bigl( {x^{l,j}(t) ,t} \bigr) \\ &\qquad{}- p_{x}^{l} \bigl( {x^{l,j}(t) ,u^{l,j}(t) ,t} \bigr) - \psi^{l}(t) g_{x}^{l} \bigl( {x ^{l}(t) ,u ^{l}(t) ,t} \bigr) \\ &\qquad{}- \beta^{l}(t) f_{x}^{l} \bigl( {x ^{l}(t) ,t} \bigr) + {p_{x}^{l} \bigl( {x ^{l}(t) ,u ^{l}(t) ,t} \bigr)} \bigr]\,dt + \bigl( { \beta^{l,j}(t) - \beta^{l}(t) } \bigr)\,dw(t). \end{aligned}$$

Let us square both sides of the last equation. Then, according to Itô’s formula, $\forall s \in[t_{r - 1} ,T]$,

$$\begin{aligned} &E\bigl(\psi^{l,j}(t) -\psi^{l}(t)\bigr) ^{2}-E \bigl(\psi^{l,j}(s) -\psi^{l}(s) \bigr)^{2} \\ &\quad= 2E \int _{s}^{t_{r} } \bigl[\psi^{l,j}(t) - \psi^{l}(t) \bigr] \bigl[\bigl(g_{x}^{l\ast} \bigl(x^{l,j}(t), u^{l,j}(t), t\bigr)-g_{x}^{l\ast} \bigl(x ^{l}(t), u ^{l}(t), t\bigr)\bigr)\psi ^{l,j}(t) \\ &\qquad{}+g_{x}^{l\ast} \bigl(x ^{l}(t) ,u ^{l}(t), t\bigr) \bigl(\psi^{l,j}(t) -\psi^{l}(t) \bigr)+\bigl(f_{x}^{l\ast} \bigl(x^{l,j}(t), t \bigr)-f_{x}^{l\ast} \bigl(x ^{l}(t),t\bigr)\bigr) \beta ^{l,j}(t) \\ &\qquad{}+f_{x}^{l\ast} \bigl(x ^{l}(t), t\bigr)\bigl(\beta^{l,j}(t) -\beta^{l}(t) \bigr)-p^{l} \bigl(x^{l,j}(t), u^{l,j}(t), t \bigr)+p_{x}^{l} \bigl(x ^{l}(t), u ^{l}(t), t\bigr)\bigr]\,dt\\ &\qquad{}+E \int _{s}^{t_{r} } {\bigl( \beta^{l,j}(t) - \beta^{l}(t) \bigr)^{2}\,dt}. \end{aligned}$$

Now, due to assumptions (AI)-(AIII), we get:

$$\begin{aligned} &E \int _{s}^{t_{r}} \bigl|\beta^{l,j}(t) - \beta^{l}(t) \bigr|^{2}\,dt+E \bigl|\psi^{l,j}(s) - \psi^{l}(s) \bigr|^{2} \\ &\quad\le EN \int _{s}^{t_{r} } {\bigl|\psi ^{l,j}(t) - \psi^{l}(t) \bigr|^{2}\,dt} +EN\varepsilon \int _{s}^{t_{r} } {\bigl|\beta^{l,j}(t) - \beta^{l}(t) \bigr| ^{2}\,dt+E\bigl\vert {\psi^{l,j}({t_{r}}) -\psi^{l}(_{t_{r} })} \bigr\vert ^{2}.} \end{aligned}$$

Hence, by the Gronwall inequality [13] we obtain that

$$ E\bigl|\psi^{l,j}(s) -\psi^{l}(s) \bigr|^{2} \le De^{N(t_{r} -s)} \quad\mbox{a.e. in } [t_{r-1} ,t_{r} ], $$

(30)

where $D=E|\psi^{l,j}(t_{r})- \psi^{l}( {t_{r} }) |^{2}$. Hence, it follows from (22) and (25) that $\psi^{l,j}(t_{r}) \to\psi^{l}(t_{r})$, which leads to $D\to0$. Consequently, it follows that $\psi^{l,j}(s) \to\psi^{l}(s)$ in $L_{F}^{2} (t_{r-1}, t_{r} ;R^{n_{l} })$, and thus $\beta ^{l,j}(s) \to\beta^{l}(s)$ in $L_{F}^{2} (t_{r-1}, t_{r}; R^{n_{l} \times n_{l} })$.

Then from the expression

$$\begin{aligned} &E\bigl|\psi^{l,j}(t_{l}) - \psi^{l}(t_{l} ) \bigr|^{2} - E\bigl|\psi^{l,j}(s) - \psi_{s}^{l} \bigr|^{2} \\ &\quad= 2E \int _{s}^{t_{l} } \bigl(\psi^{l,j}(t) - \psi^{l}(t) \bigr)\bigl[\bigl(g_{x}^{l\ast} \bigl(x^{l,j}(t) , u^{l,j}(t) ,t\bigr) - g_{x}^{l\ast} \bigl(x ^{l}(t) ,u ^{l}(t) ,t\bigr)\bigr) \psi^{l,j}(t) \\ &\qquad{}+g_{x}^{l\ast} \bigl(x ^{l}(t) ,u ^{l}(t) ,t\bigr) \bigl(\psi^{l,j}(t) - \psi^{l}(t) \bigr) + \bigl(f_{x}^{l\ast} \bigl(x^{l,j}(t),t\bigr) - f_{x}^{l\ast} \bigl(x ^{l}(t) ,t\bigr)\bigr) \beta_{t}^{j} \\ &{}\qquad{}+ f_{x}^{l\ast} \bigl(x ^{l}(t) ,t\bigr) \bigl(\beta^{l,j}(t) - \beta^{l,j}(t) \bigr) + p_{x}^{l} \bigl(x ^{l}(t) ,u ^{l}(t) ,t\bigr) - p_{x}^{l} \bigl(x^{l,j}(t) ,u^{l,j}(t) ,t\bigr)\bigr]\,dt \\ &\qquad{}+ E \int _{s}^{t_{l} } {\bigl|\beta^{l,j}(t) - \beta^{l}(t) \bigr|^{2}\,dt}, \end{aligned}$$

using simple transformations, in view of assumptions (AI)-(AIV), we obtain:

$$\begin{aligned} &E \int _{s}^{t_{l} } \bigl|\beta^{l,j}(t) - \beta^{l}(t) \bigr|^{2}\,dt + E\bigl| \psi^{l,j}(s) - \psi_{s}^{l} \bigr|^{2} \\ &\quad\leq EN \int _{s}^{t_{l} } {\bigl\vert {\psi^{l,j}(t) - \psi^{l}(t) } \bigr\vert ^{2}\,dt } + EN\varepsilon \int _{s}^{t_{l} } {\bigl|\beta^{l,j}(t) - \beta^{l}(t) \bigr|^{2}\,dt} + E\bigl|\psi^{l,j}(t_{l}) - \psi^{l}(t_{l} ) \bigr|^{2}. \end{aligned}$$

Hence, according to the Gronwall inequality, we have:

$$E\bigl|\psi^{l,j}(s) - \psi^{l}(s) \bigr|^{2} \leq De^{N(t{ }_{l} - s)}\quad \mbox{a.e. in } [t_{l - 1} ,t_{l} ], l = \overline{1,r-1}, $$

where the constant D is determined as $D = E|\psi^{l,j}(t_{l}) - \psi^{l}(t_{l} ) |^{2}$, and $D \to0$. Then from (30) it follows that $\psi^{l,j}(s) \to\psi^{l}(s)$ in $L_{F^{l}}^{2} (t_{l - 1} ,t_{l} ;R^{n})$ and $\beta^{l,j}(s) \to\beta^{l}(s)$ in $L_{F^{l}}^{2} (t_{l - 1} ,t_{l} ;R^{n\times n})$. Lemma 3 is proved. □

Lemma 4

Let $\Psi^{l,j}(t_{l} )$ be a solution of system (23), and $\Psi^{l}(t_{l} )$ be a solution of system (26). Then

$$E \int _{t_{l - 1} }^{t_{l} } \bigl|\Psi^{l,j}(t) - \Psi^{l}(t) \bigr|^{2}\,dt + E \int _{t_{l - 1} }^{t_{l} } {\bigl|\mathrm{K}^{l,j}(t) - \mathrm{K}^{l}(t) \bigr|^{2}\,dt \to0},\quad l = \overline{1,r} , \textit{as }j \to\infty. $$

Proof

Due to Itô’s formula, from expressions (23) and (26) for all $s \in[t_{l - 1} ,t_{l} )$ we have:

$$\begin{aligned} d\bigl(\Psi^{l,j}(t) - \Psi^{l}(t) \bigr) ={}&{-} \bigl\{ \bigl(g_{x}^{l\ast} \bigl(x^{l,j}(t) ,u^{l,j}(t) ,t\bigr)\Psi^{l,j}(t) - g_{x}^{l\ast} \bigl(x_{t}^{j} ,u_{t}^{j} ,t\bigr) \Psi^{l}(t) \bigr) \\ &{}+\bigl(\Psi _{t}^{l,j} g_{x}^{l} \bigl(x^{l,j}(t) ,u^{l,j}(t) ,t\bigr) - \Psi^{l}(t) g_{x}^{l} \bigl(x_{t}^{j} ,u_{t}^{j} ,t\bigr)\bigr) \\ &{}+ \bigl( {f_{x}^{l\ast} \bigl(x^{l,j}(t) ,t\bigr)\mathrm{K}^{l,j}(t) - f_{x}^{l\ast} \bigl(x ^{l}(t),t\bigr) \mathrm{K}^{l}(t) } \bigr) \\ &{}+ H_{xx}^{l} \bigl( {\psi^{l,j}(t) ,x^{l,j}(t) ,u^{l,j}(t) ,t} \bigr) - {H_{xx}^{l} \bigl( {\psi^{l}(t) ,x ^{l}(t) ,u ^{l}(t) ,t} \bigr)} \bigr\} \,dt \\ &{}+ \bigl(\mathrm{K}^{l,j}(t) - \mathrm{K}^{l}(t) \bigr)\,dw^{l}(t). \end{aligned}$$

Based on Itô’s formula, from the previous expression we get:

$$\begin{aligned} &E\bigl|\Psi^{l,j}(t_{l}) - \Psi^{l}(t_{l} ) \bigr|^{2} - E\bigl|\Psi^{l,j}(s) - \Psi^{l}(s) \bigr|^{2} \\ &\quad\leq 2E \int _{s}^{t_{l}} \bigl[\Psi^{l,j}(t) - \Psi _{t}^{l}\bigr] \bigl[\bigl(g_{x}^{l\ast} \bigl(x^{l,j}(t) ,u^{l,j}(t) ,t\bigr) - g_{x}^{l\ast} \bigl(x_{t}^{j} ,u_{t}^{j} ,t\bigr) \bigr)\Psi^{l,j}(t) \\ &\qquad{}+g_{x}^{l,\ast} \bigl(x_{t}^{j},u_{t}^{j} ,t\bigr) \bigl(\Psi^{l,j}(t) - \Psi^{l}(t) \bigr) + \bigl(f_{x}^{l\ast} \bigl(x^{l,j}(t),t\bigr) - f_{x}^{l\ast} \bigl(x ^{l}(t),t\bigr)\bigr) {\mathrm {K}}^{l,j}(t) \\ &\qquad{}+ H_{xx}^{l} \bigl( {\psi^{l,j}(t) ,x^{l,j}(t) ,u^{l,j}(t) ,t} \bigr) - H_{xx}^{l} \bigl( {\psi_{t}^{j} ,x ^{l}(t) ,u ^{l}(t) ,t} \bigr) \\ &\qquad{}+H_{xx}^{l} \bigl( {\psi^{l,j}(t) ,x ^{l}(t) ,u ^{l}(t) ,t} \bigr) -H_{xx}^{l} { \bigl( {\psi^{l}(t) ,x ^{l}(t) ,u ^{l}(t) ,t} \bigr)} \bigr]\,dt \\ &\qquad{}+E \int _{s}^{t_{l} } {\bigl|\mathrm{K}^{l,j}(t) - \mathrm{K}^{l}(t) \bigr|^{2}\,dt.} \end{aligned}$$

Then by simple transformations we obtain:

$$\begin{aligned} &E \int _{s}^{t_{l} } {\bigl|\mathrm{K}^{l,j}(t) - \mathrm{K}^{l}(t) \bigr|^{2}\,dt + E} \bigl|\Psi^{l,j}(t) - \Psi^{l}(t) \bigr|^{2} \\ &\quad\leq EN \int _{s}^{t_{l} } \bigl|\Psi^{l,j}(t) - \Psi^{l}(t) \bigr|^{2}\,dt + EN\varepsilon \int _{s}^{t_{1} } {\bigl|\mathrm{K}^{l,j}(t) - \mathrm{K}^{l}(t) \bigr|^{2}\,dt } + E\bigl|\Psi^{l,j}(t_{l}) - \Psi^{l}(t_{l} ) \bigr|^{2}. \end{aligned}$$

According to the Gronwall inequality, a.e. in $[t_{l - 1} ,t_{l} )$ we have:

$$E\bigl|\Psi^{l,j}(s) - \Psi^{l}(s) \bigr|^{2} \le De^{ - N(t_{l} - s)}, $$

where the constant D is defined as

$$D = E\bigl|\Psi^{l,j}(t_{l}) - \Psi^{l}(t_{l}) \bigr|^{2} + EN\varepsilon \int _{s}^{t_{l} } {\bigl|\mathrm{K}^{l,j}(t) - \mathrm{K}^{l}(t) \bigr|^{2}\,dt,} $$

so that $\Psi^{l,j}(t_{r}) \to\Psi^{l}(t_{r})$, and hence, according to assumptions (BI)-(BIV) and expressions (23) and (26), we obtain $\Psi^{l,j}(t) \to\Psi^{l}(t)$ in $L_{F^{r}}^{2} (t_{r - 1} ,t_{r} ;R^{n})$ as $j \to\infty$.

Then, according to sufficient smallness of ε, it follows that $D \to 0$. Consequently, $\Psi^{l,j}(t) \to\Psi^{l}(t)$ in $L_{F^{l}}^{2} (t_{l - 1} ,t_{l}; R^{n})$ and ${\mathrm {K}}^{l,j}(t) \to\mathrm{K}^{l}(t)$ in $L_{F^{l}}^{2} (t_{l - 1} ,t_{l} ;R^{n\times n})$, $l = \overline{1,r-1}$. Lemma 4 is proved. □

Based on Lemma 3 and Lemma 4, we can pass to the limit in systems (25) and (26), and the fulfilment of (22) and (23) is derived. Following a similar scheme and taking the limits in (28) and (29), we can obtain the fulfilment of second-order necessary conditions and transversality conditions. Theorem 4 is proved. □

6 Conclusions

This paper is motivated by the increased interest of the research on switching systems. Many theoretical and numerical advances have recently been realized in the field of stochastic optimization. The stochastic control problem of switching systems in which the endpoint restrictions are defined with the help of convex closed sets is considered. The results developed in this study can be viewed as stochastic analogues of the problems formulated in [7, 8] and an extension of the results for switching systems given in [31, 32]. The main result, Theorem 4, is a natural evolution of Theorem 3 in [33].