- Research
- Open access
- Published:
A smoothing and regularization predictor-corrector method for nonlinear inequalities
Journal of Inequalities and Applications volume 2012, Article number: 214 (2012)
Abstract
For a system of nonlinear inequalities, we approximate it by a family of parameterized smooth equations via a new smoothing function. We present a new smoothing and regularization predictor-corrector algorithm. The global and local superlinear convergence of the algorithm is established. In addition, the smoothing parameter μ and the regularization parameter ε in our algorithm are viewed as different independent variables. Preliminary numerical results show the efficiency of the algorithm.
MSC:90C33, 90C30, 15A06.
1 Introduction
Consider the following system of nonlinear inequalities:
where and is a continuously differentiable function for . This problem finds applications in data analysis, set separation problems, computer-aided design problems and image reconstructions [1–3]. Among various solution methods for the inequality problems [4–10], the smoothing-type methods receive much attention [8–10] which first transform the problem as a system of nonsmooth equations and approximate it by a smooth equation and then solve it by the smoothing Newton methods. Since the derivative of the underlying mapping may be seriously ill-conditioned, which may prevent the smoothing methods from converging to a solution of the problem, a perturbed regularization technique is introduced to overcome this drawback [9, 11, 12]. In 2003, Huang et al. proposed a predictor-corrector smoothing Newton method for nonlinear complementarity problem with a function based on the perturbed minimum function [13]. The method was shown to be locally superlinear convergent under the assumptions that all are nonsingular and is locally Lipschitz continuous around .
In this paper, motivated by the smoothed penalty function for constrained optimization [14], we construct a new smoothing function for nonlinear inequalities, and thus we can approximate the nonsmooth system of transformed equations by a system of smooth equations. We develop a regularization smoothing predictor-corrector method for solving the problem by modifying and extending the method in [13]. Besides choosing an arbitrarily starting point, the presented algorithm is simpler than the predictor-corrector noninterior continuation methods developed by Burke and Xu [15].
The rest of this paper is organized as follows. In Section 2, we review some preliminaries to be used in the subsequent analysis and introduce a new smoothing function and its properties. In Section 3, we present a smoothing and regularization predictor-corrector method for solving the nonlinear inequalities and establish the global and local convergence of the proposed algorithm. Preliminary numerical experiments are reported to show the efficiency of the algorithm in Section 5.
To end this section, we introduce some notations used in this paper. The set of matrices with real entries is denoted by , () denotes the nonnegative (positive) orthant in . The superscript ⊤ denotes the transpose of a matrix or a vector. Define , and for any vector , we let denote the diagonal matrix whose i-th diagonal element is . denotes the 2-norm of a vector . For a continuously differentiable function , we denote the Jacobian of f at by .
2 Smooth reformulation of nonlinear inequalities
In this section, we first review some definitions and basic results, and then introduce a new smoothing function and show its properties.
Definition 2.1 A matrix is said to be a -matrix if every principle minor of M is nonnegative.
Definition 2.2 A function is said to be a -function if for all with , there exists an index such that
For a -matrix, the following conclusion holds [16].
Lemma 2.1 If is a -matrix, then every matrix of the form
is nonsingular for all positive definite diagonal matrices .
Definition 2.3 Suppose that is a locally Lipschitz function. G is said to be semi-smooth at x if G is directionally differentiable at x and
exists for any , where denotes the generalized derivative in [17].
The concept of semi-smoothness was originally introduced by Mifflin for functions [18]. Qi and Sun extended the definition of a semi-smooth function to vector-valued functions [19]. Convex functions, smooth functions, piecewise linear functions, convex and concave functions, and sub-smooth functions are examples of semi-smooth functions. A function is semi-smooth at x if and only if all its component functions are. The composition of semi-smooth functions is still a semi-smooth function.
Lemma 2.2 [19]
Suppose that is a locally Lipschitz function semi-smooth at x. Then
-
(a)
for any , ,
-
(b)
for any ,
For problem (1.1), based on the function
it can be transformed into the following system of equations [8]:
Since problem (2.1) is a nonsmooth equation, the classical Newton methods cannot be used to solve it. Following the ideas in [14, 20, 21], we adopt the following smoothing function to approximate the nonsmooth equation
where a smoothing parameter and .
This new smoothing function has the following properties.
Lemma 2.3 For any , it holds that
-
(1)
is continuously differentiable at any .
-
(2)
Let , then .
-
(3)
at any .
Proof (1) is straightforward, so we only prove (2) and (3).
For (2), following the ideas in [20], we have
Furthermore, one can obtain the estimate [20]
Then .
For (3), by a simple calculation, we can show
then at any . We complete the proof. □
Let and
where
Define a merit function
Then the inequalities (1.1) can be reformulated as the following nonlinear equations:
Theorem 2.1 Let be defined as (2.3). Then
-
(a)
is continuously differentiable at any with its Jacobian
(2.7)
where
-
(b)
If f is a -function, then is nonsingular at any .
Proof (a) is straightforward, so we only prove (b). For (b), we only need to show is nonsingular. In fact, since f is a -function, then is a -matrix for all by Theorem 3.3 in [22]. We also note that and εI are positive diagonal matrices, we know that is nonsingular by Lemma 2.1, which implies that is also nonsingular. This completes the proof. □
3 Algorithm and convergence
In this section, we first describe our algorithm and then we reveal the global convergence analysis of the algorithm. Now, we are at a position to give the description of our smoothing predictor-corrector algorithm.
Algorithm 3.1
Step 0. Take , . Let and is an arbitrary point. Choose and parameter such that , . Set .
Step 1. If , then stop. Otherwise, let where is defined by .
Step 2. Predictor step. If , set and go to Step 3. Otherwise, compute by
If
then set . Otherwise, set .
Step 3. Corrector step. If , then stop. Otherwise, compute by
Let be the smallest nonnegative integer l such that
Set and .
Step 4. Set and return to Step 1.
Remark 3.1 If , then Algorithm 3.1 solves only one linear system of equations at each iteration. Otherwise, it solves two linear systems of equations at each iteration. However, the coefficient matrices of these two systems are identical when (3.2) is not satisfied. There are the same points as the algorithm in [13], the neighborhood of the path does not appear in the algorithm, thus, it does not need a few additional computations which keep the iteration sequence staying in the given neighborhood.
To prove the convergence of Algorithm 3.1, first, define the set
The following lemmas show that Algorithm 3.1 is well defined and generates an infinite sequence with some good features.
Lemma 3.1 If f is a continuously differentiable -function, then Algorithm 3.1 is well defined. In addition, , and for any .
Proof Since f is a continuously differentiable function, then it follows from Theorem 2.1 that the matrix is nonsingular for , . Since , by the choice of an initial point, we may assume, without loss of generality, that , , we show that , . If the predictor step is accepted, then by (3.1),
otherwise, we have , which means , . Thus, we obtain , . Furthermore, and are nonsingular which means that (3.1) and (3.3) are well defined.
Given , for any , let
Noting is continuously differentiable, we obtain . Then, it follows from (3.3) and (3.7) that
Therefore, from (3.8), it shows that there exists a positive number such that for all and ,
holds, which implies
That is, the nonnegative satisfying (3.4) can be found, which demonstrates that (3.4) is well defined.
For , since , we know . Assuming now that is true for , we show that it continues to hold for . If the predictor step is accepted, then it follows from (3.5), (3.6) and (3.2) that
and
which implies
Otherwise, from and the inductive assumption, we obtain that (3.12) also holds. Noting (3.3), we have
In addition, from (3.9) we know that there exists such that
Therefore, it follows from (3.12), (3.13) and (3.15) that
Similarly, we can obtain . Thus, .
Since , , we may assume that , for any given . From , , it follows from (3.13) that , . Hence, , for any . □
Lemma 3.2 Suppose that the infinite sequence is generated by Algorithm 3.1, then and the sequence is monotonically decreasing.
Proof For any , it follows from (3.12), (3.13) and (3.14) that
If the predictor step (Step 2) is not accepted at the k-th iterate, then (3.17) and (3.18) show the desired result. Otherwise, from (3.5), (3.6), and , one has
Thus, we obtain that , hold for any .
If the predictor step (Step 2) is not accepted at the k-th iterate, then (3.15) implies that
and the desired result has been obtained. Otherwise, it follows from (3.2) and that
Hence, for any , we obtain
which means the sequence is monotonically decreasing. □
Lemma 3.3 Assume that f is a -function and , , , are given positive numbers satisfying , . Then, H defined by (2.3) has the property
for any sequence such that , for any k and as .
Proof We outline the proof by contradiction. Suppose that the lemma is not true. Then there exists a sequence such that , , but . Since the sequence is unbounded, the index set is nonempty. Without loss of generality, we can assume that for all . Then the following sequence is bounded which is defined by
Since f is a -function, by Definition 2.2, we have
where is one of the indices for which the max is attained, and is assumed, without loss of generality, to be independent of k. Since , one has as . We now break up the proof into two cases.
Case 1. If as . In this case, since is bounded, we deduce from (3.21) that .
If , for , , letting yields
is bounded and
Thus, as .
If , for , , we have
and
Thus, as .
Case 2. as . In this case, since is bounded, we deduce from (3.21) that .
If , for , , we have
is bounded and
Thus, as .
If , for , , we have
is bounded and
Thus, as .
In summary, we obtain as , which contradicts , and the proof is completed. □
Under the assumption of f being a -function, Lemma 3.2 and Lemma 3.3 indicate that the level set defined by
is bounded.
To obtain the global convergence of Algorithm 3.1, we need the following assumption.
Assumption 3.1 The solution of (1.1) is nonempty and bounded.
Note that Assumption 3.1 seems to be the weakest condition used in the previous literature to ensure the bound of iteration sequences (see [23]).
Theorem 3.1 Assume that the infinite sequence is generated by Algorithm 3.1. Then
-
(a)
The sequences , and converge to zero as , and hence any accumulation point of is a solution of (1.1).
-
(b)
If Assumption 4.1 is satisfied, then the sequence is bounded, hence there exists at least one accumulation point with and .
Proof By Lemma 3.2, we know that converges to as . Suppose that does not converge to zero. Then, and is bounded by Lemma 3.2 and Lemma 3.3. Assume that is an accumulation point of . Without loss of generality, we assume that converges to . Then, by the continuity of H and the definition of , we know that , and converge to , , respectively and that . Therefore, by (3.4), we have
On the one hand, from Step 3 in Algorithm 3.1, we get
which implies that
Letting , we have
On the other hand, by (3.3), we have
i.e.,
Combining (3.24) and (3.25), we deduce that
which means
Since , then
i.e.,
This contradicts the fact that and . Hence, we have , . Thus, , that is, is a solution of (1.1).
Next we prove (b). It follows from (a) that as . By (2.3), one has
Therefore, by the famous mountain pass theorem (Theorem 5.4 in [24]) and along the lines of the proof of Theorem 3.1 in [23], we obtain that is bounded and hence is. Thus, has at least one accumulation point . By (a), we have and , , . □
Next, we show the local superlinear convergence of Algorithm 3.1.
Theorem 3.2 Suppose that f is a continuously differentiable -function, Assumption 3.1 is satisfied and is an accumulation point of the iteration sequence generated by Algorithm 3.1. If all are nonsingular and is locally Lipschitz continuous around , then the whole sequence superlinearly converges to , i.e.,
and
Proof First, from Theorem 3.1, we know that is a solution of . Then since all are nonsingular, it follows from [19, 25, 27] that for all sufficiently close to , we have
where is a constant.
Then, since is semi-smooth at , is locally Lipschitz continuous near , for all sufficiently close to ,
For all sufficiently close to , we have
Thus, for sufficiently close to , we obtain
Hence, for sufficiently close to , we have . By (3.28), we prove that
holds.
Next, when k is sufficiently large, then , so
and
Hence, for all k sufficiently large,
which, together with (3.29), yields
and
This means that , and the desired result follows. □
4 Numerical experiments
In this section, we test our algorithm for solving the systems of inequalities. In our implementation, we adopt the strategy in [8], the function H defined by (2.3) is replaced by
where c is a constant. It is easy to see that such a change does not destroy any theoretical results obtained in Section 3.
In our numerical experiments, the parameters used in the algorithm are chosen as follows: , , , . The algorithm terminates when . In the tables of test results, st denotes the starting point of , ic denotes the corrector iteration numbers in Step 3 followed directly from Step 1, ip denotes the predictor iteration numbers, denotes the iteration numbers of smoothing method (in [8]), denotes the CPU time for solving the underlying problems in seconds, and denotes a solution of the test problem. In the following, we reveal a detailed description of the tested problems.
In the following, we reveal a detailed description of the tested problems. For Example 4.1, 4.2 and 4.3, we compare the results obtained by our method with which obtained by smoothing method [8]. The results are summarized in Table 1, Table 2 and Table 3.
Consider (1.1), where with and
Consider (1.1), where with and
Example 4.3 [26]
Consider (1.1), where with and
5 Conclusion
In this paper, we present a new smoothing and regularization predictor-corrector algorithm to solve the nonlinear inequalities, the global and local convergence are obtained. Furthermore, the smoothing parameter μ and the regularization parameter ε in our algorithm are viewed as independent variables. Preliminary numerical results show the efficiency of the algorithm.
References
Zakian V, Nail UA: Design of dynamical and control systems by the method of inequalities. Proc. Inst. Electr. Eng. 1973, 120: 1421–1472.
Dennis JE, Schnabel RB: Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Prentice-Hall, Englewood Cliffs; 1983.
Neumaier A: Interval Methods for System of Equations. Cambridge University Press, Cambridge; 1990.
Daniel JW: Newtons method for nonlinear inequalities. Numer. Math. 1973, 21: 381–387.
Mayne DQ, Polak E, Heunis AJ: Solving nonlinear inequalities in a finite number of iterations. J. Optim. Theory Appl. 1981, 33: 207–221.
Sahba M: On the solution of nonlinear inequalities in a finite number of iterations. Numer. Math. 1985, 46: 229–236.
Yin HX, Huang ZH, Qi L: The convergence of a Levenberg-Marquardt method for nonlinear inequalities. Numer. Funct. Anal. Optim. 2008, 29: 687–716.
Huang ZH, Zhang Y, Wu W: A smoothing-type algorithm for solving system of inequalities. J. Comput. Appl. Math. 2008, 220: 355–363.
Zhu JG, Liu HW, Li XL:A regularized smoothing-type algorithm for solving a system of inequalities with a -function. J. Comput. Appl. Math. 2010, 233: 2611–2619.
He C, Ma CF: A smoothing self-adaptive Levenberg-Marquardt algorithm for solving system of nonlinear inequalities. Appl. Math. Comput. 2010, 216: 3056–3063.
Huang ZH, Qi L, Sun D:Sub-quadratic convergence of a smoothing Newton algorithm for the and monotone LCP. Math. Program. 2004, 99: 423–441.
Zhao N, Huang ZH:A nonmonotone smoothing Newton algorithm for solving box constrained variational inequalities with a function. J. Ind. Manag. Optim. 2011, 7(2):467–482.
Huang ZH, Han J, Chen Z:Predictor-corrector smoothing Newton method, based on a new smoothing function, for solving the nonlinear complementarity with a function. J. Optim. Theory Appl. 2003, 117: 39–68.
Herty M, Klar A, Singh AK, Spellucci P: Smoothed penalty algorithms for optimization of nonlinear models. Comput. Optim. Appl. 2007, 37: 157–176.
Burke J, Xu S: A noninterior predictor-corrector path-following algorithm for the monotone linear complementarity problem. Math. Program. 2000, 87: 113–130.
Luca TD, Facchinei F, Kanzow C: A semismooth equation approach to the solution of nonlinear complementarity problems. Math. Program. 1996, 75: 407–439.
Clarke FH: Optimization and Nonsmooth Analysis. Wiley, New York; 1983.
Mifflin R: Semismooth and semiconvex functions in constrained optimization. SIAM J. Control Optim. 1977, 15(6):957–972.
Qi L, Sun J: A nonsmooth version of Newton’s method. Math. Program. 1993, 58(3):353–367.
Lee YJ, Mangasarin OL: SSVM: A smooth support vector machine for classification. Comput. Optim. Appl. 2001, 20: 5–22.
Che H, Li M: A smoothing and regularization Broyden-like method for nonlinear inequalities. J. Appl. Math. Comput. 2012. doi:10.1007/s12190–012–0588–2
Chen B, Harker PT: A non-interior-point continuation method for linear complementarity problems. SIAM J. Matrix Anal. Appl. 1993, 14: 1168–1190.
Huang ZH, Han J, Xu D, Zhang L: The non-linear continuation methods for solving the P0-functions non-linear complementarity problem. Sci. China 2001, 44(2):1107–1114.
Facchinei F, Kanzow C: Beyond monotonicity in regularization methods for nonlinear complementarity problems. SIAM J. Control Optim. 1999, 37(2):1150–1161.
Qi L: Convergence analysis of some algorithms for solving nonsmooth equations. Math. Oper. Res. 1993, 18: 227–244.
Zhang Y, Huang ZH: A nonmonotone smoothing-type algorithm for solving a system of equalities and inequalities. J. Comput. Appl. Math. 2010, 233: 2312–2321.
Qi L, Sun DF, Zhou GL: A new look at smoothing Newton methods for nonlinear complementarity problems and box constrained variational inequalities. Math. Program., Ser. A 2000, 87: 1–35.
Acknowledgements
This research was supported by the Natural Science Foundation of China (Grant Nos. 11171180, 11171193, 11126233, 10901096) and the fund of Natural Science of Shandong Province (Grant Nos. ZR2009AL019, ZR2011AM016). The authors are in debt to the anonymous referees for their numerous insightful comments and constructive suggestions which help improve the presentation of the article. The authors thank Prof. Yiju Wang for his careful reading of the manuscript.
Author information
Authors and Affiliations
Corresponding author
Additional information
Competing interests
We declare that we have no competing interests.
Authors’ contributions
The author carried out the proof. The author conceived of the study and participated in its design and coordination. The author read and approved the final manuscript.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License (https://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
About this article
Cite this article
Che, H. A smoothing and regularization predictor-corrector method for nonlinear inequalities. J Inequal Appl 2012, 214 (2012). https://doi.org/10.1186/1029-242X-2012-214
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/1029-242X-2012-214