Stochastic linear quadratic control problem of switching systems with constraints
- Charkaz Aghayeva^{1, 2}Email author
https://doi.org/10.1186/s13660-016-1046-8
© Aghayeva 2016
Received: 26 January 2016
Accepted: 18 March 2016
Published: 31 March 2016
Abstract
This paper is devoted to the optimal control problem for stochastic linear switching systems with a quadratic cost functional. A necessary and sufficient condition of optimality for mentioned linear control systems under endpoint constraints is obtained. A linear quadratic controller is simply constructed via a set of stochastic backward Riccati equations.
Keywords
stochastic linear system conditions of optimality switching systems transversality conditions1 Introduction
The Linear Quadratic (LQ) problem was mathematically formulated and solved, as well as the filtering one, in the 1960s by Kalman [1]. An important advantage of the LQ theory is the existence of explicit feedback forms for optimal state control and the optimal cost value through the Riccati equations. The deterministic Riccati equation was essentially solved by Wonham [2] by applying Bellman’s principle of quasilinearization [3]. A detailed research of stochastic LQ control problems has been performed by Bismut [4]. The existence of a unique solution for the associated Riccati equations was studied in [5].
Switching systems are more advantageous models to describe the noninvariant phenomena with the continuous law of movement and they have gained considerable attention in science and engineering. Examples of these systems include many evolutionary processes, robotics, integrated circuit design, multimedia, manufacturing, power electronics, chaos generators, and air traffic management systems [6, 7]. Optimization problems have also received growing interest among the researchers of deterministic and for stochastic switching control systems [8–14].
Manifold problems of stochastic optimal control theory have been considered in [15–22]. Optimal control problems of switching systems have attracted considerable attention, due to the advantages, for instance, in modeling and improving the transient response on highly complex systems and systems with large uncertainties. The stochastic maximum principle via backward stochastic differential equations is derived in [23–27]. The necessary conditions of optimality for stochastic switching systems earlier have been obtained in [28–30]. In [31] the linear quadratic control problem has been investigated for a special type of stochastic systems.
In this paper, the LQ problem of stochastic switching systems with restrictions is considered. Ekeland’s variational principle [32] has been used to establish the necessary and sufficient conditions of optimality for a given problem.
2 Statement of main problem
Unless specified otherwise, throughout the paper we use the same notations as in [30].
\(A_{i}\) represents the set of elements \(\pi^{i} = (t_{0} ,t_{1} ,t_{i} ,x^{1}(t) ,x^{2}(t) ,\ldots,x^{i}(t),u^{1},u^{2},\ldots,u^{i})\) for each \(i=1,\ldots,r \). To describe the main result we need to introduce some concepts, such as a solution of linear switching systems, admissible element of control problem and optimal solution for LQ problem of stochastic switching systems. For a detailed account we refer the reader to [29, 30].
3 Stochastic LQ problem of switching systems
This section is devoted to the investigation of optimal control problems for linear stochastic switching systems with constraints. The LQ problem belongs to a special class of convex control problems for which the maximum principle is a necessary as well as sufficient condition of optimality. The next theorem provides necessary and sufficient conditions of the optimality of stochastic linear switching systems.
Theorem 1
- (a)there exist random processes \((\psi^{l}(t) ,\beta^{l}(t))\in{L}_{F}^{2} (t_{l-1} ,t_{l} ;{R}^{n_{l} })\times{L}_{F}^{2} (t_{l-1} ,t_{l} ;{R}^{n_{l}\times n_{l}})\) which are the solutions of the following stochastic backward equations:$$ \left \{ \textstyle\begin{array}{@{}l} d\psi^{l}(t) =- [A^{l*}(t)\psi^{l}(t)+C^{l*}(t)\beta ^{l}(t)-M^{l}(t)x(t) ] \\ \hphantom{d\psi^{l}(t) =}{}+\beta^{l}(t)\,dw^{l}(t) ,\quad t_{l-1} \le t< t_{l} , \\ \psi^{l}(t_{l}) =-\lambda^{l}_{0}G^{l} x^{l}(t_{l})-\lambda^{l}_{1}q^{l} +\psi ^{l+1}(t_{l}) \Phi^{l} (t_{l} ),\quad l=1,\ldots,r-1 , \\ \psi^{r}(t_{r}) =-\lambda^{r}_{0}G^{r} x^{r}(t_{r})-\lambda^{r}_{1}q^{r} ; \end{array}\displaystyle \right . $$(6)
- (b)the candidate optimal controls \({u}^{l}\in U^{l}\), \(l=\overline{1,r}\), are defined by$$ N^{l*}(t)u^{l}(t)=B^{l*}(t) \psi^{l}(t)+D^{l*}(t)\beta^{l}(t), \quad\textit{a.e. }\theta\in[t_{l-1} , t_{l} ] ; $$(7)
- (c)the following transversality conditions hold:$$ \psi^{l+1}(t_{l}) \bigl(\Phi_{t}^{l*} \bigl(t^{l}\bigr) x^{l}(t_{l})+K_{t}^{l*}(t_{l}) \bigr)=0, \quad \textit{a.c.}, l=1,\ldots,r-1 . $$(8)
Proof
It is well known that a necessary and sufficient condition of optimality for the convex functional is given by \(J^{\prime}(\mathbf{u})=0\). The validity of (7) and (8), hence the necessary conditions of optimality for the considered unrestricted problem (1)-(4) follows from equations (11) and (13). At last, according to the independence of the increments \(\Delta\bar{x}^{l}(t)\), \(\Delta\bar {u}^{l}(t)\), \(\Delta\bar{t_{l}}\), sufficiency follows from equation (12).
To construct the optimality condition of LQ problem (1)-(4) with the right endpoint constraints (5), the above mentioned problem by using Ekeland’s variational principle [32] is converted into a sequence of unconstrained problems. Based on the results already obtained for problem (1)-(4), necessary and sufficient conditions for the sequence of switching systems are established.
For the following fact it is significant that we can provide a relation between the sequence of controls from the metric space \(V^{l}\) and the sequence of corresponding trajectories of system (1)-(2).
Lemma 1
([30], Lemma 4.3)
Since \(\sum_{l = 1}^{r} |\lambda_{0}^{l,j}|^{2}+|\lambda_{1}^{l,j} |^{2} = 1 \) exists by (16) \((\lambda_{0}^{l,j} ,\lambda_{1}^{l,j} ) \to(\lambda_{0}^{l} ,\lambda _{1}^{l} )\) if \(j \to\infty\).
The truth of (6) is based upon the following lemma, which can be proved by the same method as the proof of Lemma 4 [33].
Lemma 2
Based on Lemma 2, we can pass to the weak limit in system (15) and obtain the fulfillment of (6). Following a similar scheme, we take the limits in (17) and (18), and justifications of (7), (8) are derived. Theorem 1 is proved.
4 Riccati equations for switching systems
To determine the stochastic processes \(p^{l}(t)\) we introduce the following theorem.
Theorem 2
Proof
Finally, the feedback design for LQ problem (1)-(5) is obtained by means of the next theorem.
Theorem 3
Proof
Therefore, the assertion of the theorem is true. □
5 Conclusion
There are a lot relevant applications of LQ problems in fields such as aerospace, biology, economics, management sciences, etc. [34–38].
Switching systems provide a natural and convenient theoretical account for mathematical modeling of many complex real phenomena and practical applications. A broad spectrum of the latest research is concerned with optimal control problems of stochastic switching systems [39–41].
The LQ problem of switching systems in which the endpoint restrictions are defined with the help of convex closed sets has been investigated. The objective of the present research is to give an explicit solution to the LQ problem of stochastic switching systems of which drift and diffusion coefficients comprise non-homogeneous terms. The results developed in this study can be viewed as an extension of the problems formulated in [42, 43] for stochastic switching systems.
Declarations
Acknowledgements
The author thanks the two anonymous reviewers whose comments and suggestions helped improve this manuscript. The research underlying this paper is supported by the Scientific Research Project No. 1505F202 of Anadolu University, Turkey.
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Authors’ Affiliations
References
- Kalman, RE: Contributions to the theory of optimal control. Bol. Soc. Mat. Mexicana 5, 102-119 (1960) MathSciNetMATHGoogle Scholar
- Wonham, WM: On a matrix Riccati equation of stochastic control. SIAM J. Control Optim. 6, 312-326 (1968) MathSciNetView ArticleMATHGoogle Scholar
- Bellman, R: Functional equations in the theory of dynamic programming, positivity and quasilinearity. Proc. Natl. Acad. Sci. USA 41, 743-746 (1955) View ArticleMATHGoogle Scholar
- Bismut, JM: Linear quadratic optimal stochastic control with random coefficients. SIAM J. Control 14, 419-444 (1976) MathSciNetView ArticleMATHGoogle Scholar
- Bismut, JM: An introductory approach to duality in optimal stochastic control. SIAM Rev. 20, 62-78 (1978) MathSciNetView ArticleMATHGoogle Scholar
- Boukas, E: Stochastic Switching Systems: Analysis and Design. Birkhäuser, Basel (2006) MATHGoogle Scholar
- Liberzon, D: Switching in Systems and Control. Birkhäuser, Basel (2003) View ArticleMATHGoogle Scholar
- Bengea, SC, Raymond, AC: Optimal control of switching systems. Automatica 41, 11-27 (2005) MathSciNetMATHGoogle Scholar
- Capuzzo, DI, Evans, LC: Optimal switching for ordinary differential equations. SIAM J. Control Optim. 22, 143-161 (1984) MathSciNetView ArticleMATHGoogle Scholar
- Seidmann, TI: Optimal control for switching systems. In: Proceedings of the 21st Annual Conference on in Formations Science and Systems, pp. 485-489 (1987) Google Scholar
- Hoek, J, Elliott, RJ: American option prices in a Markov chain model. Appl. Stoch. Models Bus. Ind. 28, 35-39 (2012) MathSciNetView ArticleMATHGoogle Scholar
- Azevedo, N, Pinheiro, D, Weber, G-W: Dynamic programming for a Markov-switching jump-diffusion. J. Comput. Appl. Math. 267, 1-19 (2014) MathSciNetView ArticleMATHGoogle Scholar
- Temocin, B, Weber, G-W: Optimal control of stochastic hybrid system with jumps: a numerical approximation. J. Comput. Appl. Math. 259, 443-451 (2014) MathSciNetView ArticleMATHGoogle Scholar
- Xu, X, Antsaklis, PJ: Results and perspectives on computational methods for optimal control of switched systems. In: Maler, O, Pnueli, A (eds.) Hybrid Systems: Computation and Control. Lecture Notes in Computer Science, vol. 2623, pp. 540-556 (2003) View ArticleGoogle Scholar
- Haussman, UG: General necessary conditions for optimal control of stochastic systems. In: Stochastic Systems: Modeling, Identification and Optimization, II. Mathematical Programming Studies, vol. 6, pp. 30-48 (1976) View ArticleGoogle Scholar
- Kek, SL, Teo, LK, Ismail, AM: An integrated optimal control algorithm for discrete-time nonlinear stochastic system. Int. J. Control 83(12), 2536-2545 (2010) MathSciNetView ArticleMATHGoogle Scholar
- Kushner, HJ: Necessary conditions for continuous parameter stochastic optimization problems. SIAM J. Control 10, 550-565 (1976) MathSciNetView ArticleMATHGoogle Scholar
- Pham, H: On some recent aspects of stochastic control and their applications. Probab. Surv. 2, 506-549 (2005) MathSciNetView ArticleMATHGoogle Scholar
- Hafayed, M, Abbas, S, Abba, A: On mean-field partial information maximum principle of optimal control for stochastic systems with Lévy processes. J. Optim. Theory Appl. 167, 1051-1069 (2015) MathSciNetView ArticleMATHGoogle Scholar
- Shen, Y, Siu, TK: The maximum principle for a jump-diffusion mean-field model and its application to the mean-variance problem. Nonlinear Anal. 86, 58-73 (2013) MathSciNetView ArticleMATHGoogle Scholar
- Wang, G, Zhang, C, Zhang, W: Stochastic maximum principle for mean-field type optimal control under partial information. IEEE Trans. Autom. Control 59(2), 522-528 (2014) MathSciNetView ArticleGoogle Scholar
- Yong, J, Zhou, XY: Stochastic Controls: Hamiltonian Systems and HJB Equations. Springer, New York (1999) View ArticleMATHGoogle Scholar
- Borkar, V: Controlled diffusion processes. Probab. Surv. 2(4), 213-244 (2005) MathSciNetMATHGoogle Scholar
- Makhmudov, NI: General necessary optimality conditions for stochastic systems with controllable diffusion. In: Statistics and Control of Random Processes, pp. 135-138. Nauka, Moscow (1989) Google Scholar
- Peng, S: A general stochastic maximum principle for optimal control problem. SIAM J. Control Optim. 28, 966-979 (1990) MathSciNetView ArticleMATHGoogle Scholar
- Hafayed, M, Veverka, P, Abbas, S: On near-optimal necessary and sufficient conditions for forward-backward stochastic systems with jumps, with applications to finance. Appl. Math. 59(4), 407-440 (2014) MathSciNetView ArticleMATHGoogle Scholar
- Hafayed, M, Abba, A, Boukaf, S: On Zhou’s maximum principle for near-optimal control of mean-field forward-backward stochastic systems with jumps and its applications. Int. J. Model. Identif. Control 25(1), 1-16 (2016) View ArticleGoogle Scholar
- Aghayeva, C, Abushov, Q: The maximum principle for the nonlinear stochastic optimal control problem of switching systems. J. Glob. Optim. 56(2), 341-352 (2013) MathSciNetView ArticleMATHGoogle Scholar
- Abushov, Q, Aghayeva, C: Stochastic maximum principle for the nonlinear optimal control problem of switching systems. J. Comput. Appl. Math. 259, 371-376 (2014) MathSciNetView ArticleMATHGoogle Scholar
- Aghayeva, C: Necessary conditions of optimality for stochastic switching control systems. Dyn. Syst. Appl. 24(3), 243-258 (2015) Google Scholar
- Aghayeva, C, Abushov, Q: Linear-square stochastic optimal control problem with variable delay on control and state. Transactions ANAS, math.- ph. series, Informatics and Control Problems 25(3), 204-208 (2005) Google Scholar
- Ekeland, I: On the variational principle. J. Math. Anal. Appl. 47, 324-353 (1974) MathSciNetView ArticleMATHGoogle Scholar
- Aghayeva, C, Abushov, Q: The maximum principle for some nonlinear stochastic control system with variable structure. Theory Stoch. Process. 32(1), 1-11 (2010) MathSciNetMATHGoogle Scholar
- Bensoussan, A, Delfour, MC, Mitter, SK: The linear quadratic optimal control problem for infinite dimensional systems over an infinite horizon; survey and examples. In: Proceedings of the IEEE Conference on Decision and Control and the 15th Symposium on Adaptive Processes, pp. 746-751. Inst. Electr. Electron. Engrs., New York (1976) Google Scholar
- Kohlmann, M, Zhou, X: Relationship between backward stochastic differential equations and stochastic controls: a linear-quadratic approach. SIAM J. Control Optim. 38, 1392-1407 (2000) MathSciNetView ArticleMATHGoogle Scholar
- Turan, O: An exergy way to quantify sustainability metrics for a high bypass turbofan engine. Energy 86, 722-736 (2015) View ArticleGoogle Scholar
- Yuan, C, Wu, F: Hybrid control for switched linear systems with average dwell time. IEEE Trans. Autom. Control 60(1), 240-245 (2015) MathSciNetView ArticleGoogle Scholar
- Sahin, O: Optimum arrival routes for flight efficiency. J. Power Energy Eng. 3, 449-452 (2015) View ArticleGoogle Scholar
- Azhmyakov, V, Attia, SA, Raisch, J: On the maximum principle for impulsive hybrid system. Lect. Notes Comput. Sci. 4981, 30-42 (2008) MathSciNetView ArticleMATHGoogle Scholar
- Ghomanjani, F, HadiFarahi, M: Optimal control of switched systems based on Bezier control points. Int. J. Intell. Syst. Appl. 7, 16-22 (2012) Google Scholar
- Wang, S, Wu, Z: Maximum principle for optimal control problems of forward-backward regime-switching systems involving impulse controls. Math. Probl. Eng. 2015, 892304 (2015) MathSciNetGoogle Scholar
- Debrabant, K, Jakobsen, ER: Semi-Lagrangian schemes for linear and fully nonlinear diffusion equations. Math. Comput. 82, 1433-1462 (2013) MathSciNetView ArticleMATHGoogle Scholar
- Su, Y, Huang, J: Stability of a class of linear switching systems with applications to two consensus problems. IEEE Trans. Autom. Control 57(6), 1420-1430 (2012) MathSciNetView ArticleGoogle Scholar