A modified exact smooth penalty function for nonlinear constrained optimization
© Bingzhuang and Wenling; licensee Springer 2012
Received: 13 February 2012
Accepted: 26 July 2012
Published: 6 August 2012
In this paper, a modified simple penalty function is proposed for a constrained nonlinear programming problem by augmenting the dimension of the program with a variable that controls the weight of the penalty terms. This penalty function enjoys improved smoothness. Under mild conditions, it can be proved to be exact in the sense that local minimizers of the original constrained problem are precisely the local minimizers of the associated penalty problem.
MSC:47H20, 35K55, 90C30.
Merit function has always taken an important role in optimization problem. It is traditionally constructed to solve nonlinear programs by augmenting the objective function or a corresponding Lagrange function some penalty or barrier terms with respect of the constraints. Then it can be optimized by some unconstrained or bounded constrained optimization softwares or sequential quadratic programming (SQP) techniques. No matter what kind of techniques are involved, the merit function always depends on a small parameter ε or large parameter . As , the minimizer of a merit function such as a barrier function or the quadratic penalty function, converges to a minimizer of the original problem. By using some exact penalty function such as penalty function (see [1, 9, 20–22]), the minimizer of the corresponding penalty problem must be a minimizer of the original problem when ε is sufficiently small. There are some nonsmooth penalty functions for nonsmooth optimization problems, such as the exact penalty function using the distance function for the nonsmooth variational inequality problem in Hilbert spaces  and the one in .
The traditional exact penalty functions  are always nonsmooth. When it is used as a merit function to accept a new iterate in an SQP method, it may cause the Maratos effect . On the other hand, a traditional smooth penalty function like the quadratic penalty function cannot be an exact one. So we must compute a sequence of minimization subproblem as . At that time, ill-conditioning may occur when the penalty parameter is too large or small, which also brings difficulty of computation. In  and , some kinds of augmented Lagrangian penalty functions have been proposed with improved exactness under strong conditions. In , exact penalty functions via regularized gap function for variational inequalities have also been given. All these functions enjoy some smoothness, but at the very beginning, to use this smoothness we need second-order or third-order derivative information of the problem function that is difficult to estimate in practice. Besides, all the above kinds of penalty functions (see [2–4, 7, 16] for summary) may be unbounded below even when the constrained problem is bounded, which may make it difficult to locate a minimizer.
In the paper , a new penalty function is proposed for the constrained optimization problem. By augmenting the dimension of the program with an additional variable ε that controls the weight of the penalty terms, this new penalty function enjoys properties of smoothness and exactness, and remains bounded below under reasonable conditions. Its important new idea is that the penalty function is considered as a function of variable x and the additional variable ε simultaneously. Under proper assumptions, the minimizer of the merit function satisfies , and is a minimizer of the original problem. However, the penalty function given in  is not smooth in a small neighborhood of , where the minimizer of the original constrained problem lies. In this paper, we give a penalty function which enjoys the properties of the penalty function given in  and has improved smoothness.
The rest of this paper is organized as follows. In Section 2, a penalty function is introduced for a smooth nonlinear optimization problem with equality constraints and bounded constraints. The smoothness of this penalty function is discussed, as well as other properties, including being bounded below under mild assumptions. Section 3 shows the exactness of our penalty function in the sense that under certain conditions, local minimizer of our penalty function has the form with and is a local minimizer of the original problem, and a converse result holds.
where the lower bound and the upper bound are vectors containing proper or infinite bounds on the components of x and is referred to an n-dimensional box.
2 New penalty function
where, in addition, and , are all fixed numbers, is a penalty parameter and is a Euclidean norm, with for any vector x.
The main difference between (2.3) and the penalty function given in  is that in (2.3), , which does not have the property that , as .
2.1 Boundedness of the penalty function
whenever is bounded below on the set . This is a reasonable condition since it usually holds when f is bounded below on the feasible set, is small enough, and q is large enough.
The denominator is included since it forces the level sets of to remain in the set , hence in some sense does not go far away from the feasible set of (P).
3 Exactness of the penalty function
In this section, we show that our penalty function is exact in the sense that under certain conditions, local minimizer of our penalty function has the form in which and is a local minimizer of the original problem and a converse proposition holds.
Theorem 3.1 Assume that the set defined in Section 2 is bounded, and the Mangasarian-Fromovitz condition holds at each . Let in (). If σ is sufficiently large, then there is no Kuhn-Tucker point of () with .
where is the gradient of with respect to . The assertion of the theorem is proved by contradiction. □
Now similarly as above, it yields , which is a contradiction with . Thus such a sequence cannot exist, and for sufficiently large , all Kuhn-Tucker points of () are of the form .
Theorem 3.2 Assume that is a local minimizer of minimizer of () with finite , where is sufficiently large. If the hypotheses of Theorem 3.1 are fulfilled, then is a local minimizer of (P).
Proof Now let be a local minimizer of () with finite and is sufficiently large. If , then must be a Kuhn-Tucker point of (), which is a contradiction with Theorem 3.1. Therefore, , and since is finite, . It implies that , and by (2.5) there is a neighborhood of where for feasible x. Therefore, is a local minimizer of (P). □
We now show a converse result of Theorem 3.2, which will use the following lemmas.
so , for and , .
and this complete the proof. □
where is defined in Lemma 3.1.
where the last inequality holds from (3.10).
which complete the proof. □
In particular, is a local minimizer of .
where is defined in Lemma 3.1.
For , where and , we distinguish two cases.
where the second inequality is by (3.11).
The last inequality holds since .
From Case 1 and Case 2, we obtain the conclusion. □
4 Conclusion remarks
In this paper, a modified exact penalty function for equality constrained nonlinear programming problem is constructed by augmenting a new variable that controls the constraint violence. This function enjoys smoothness, and with very mild conditions it is proved to be an exact penalty function.
Since in practice, a lot of applied problems are nonsmooth, it is a meaningful work to extend the results in this paper to the nonsmooth case. By using the limiting subgradients that is presented in two books written by Mordukhovich [14, 15], as well as Clarke’s generalized gradients in , we can extend the penalty function with the mentioned good properties to nonsmooth optimization problems, just as that has been done in [17–19]. That will be our future research direction.
The authors wish to thank the anonymous referees for their endeavors and valuable comments. The authors would also like to thank Professor Zhang Liansheng for some very helpful comments on a preliminary version of this paper. This research was supported by the National Natural Science Foundation of China under Grants 10971118, 11101248, Natural Science Foundation of Shandong Province under Grants ZR2012AM016, and the foundation 4041-409012 of Shandong University of Technology.
- Antczak T: Exact penalty functions method for mathematical programming problems involving invex functions. Eur. J. Oper. Res. 2009, 198: 29–36. 10.1016/j.ejor.2008.07.031MathSciNetView ArticleGoogle Scholar
- Bazaraa MS, Sherali HD, Shetty CM: Nonlinear Optimization Theory and Algorithms. 2nd edition. Wiley, New York; 1993.Google Scholar
- Bertsekas DP: Constrained Optimization and Lagrange Multiplier Methods. Academic Press, New York; 1982.Google Scholar
- Boukari D, Fiacco AV: Survey of penalty, exact-penalty and multiplier methods from 1968 to 1993. Optimization 1995, 32: 301–334. 10.1080/02331939508844053MathSciNetView ArticleGoogle Scholar
- Clarke FH: Optimization and Nonsmooth Analysis. Wiley-Interscience, New York; 1983.Google Scholar
- Di Pillo G: Exact penalty methods. In Algorithms for Continuous Optimization. Edited by: Spedicato E. Kluwer Academic, Dordrecht; 1994:209–253.View ArticleGoogle Scholar
- Fletcher R: Practical Methods of Optimization. 2nd edition. Wiley, New York; 1987.Google Scholar
- Han SP, Mangasarian OL: Exact penalty functions in nonlinear programming. Math. Program. 1979, 17: 251–269. 10.1007/BF01588250MathSciNetView ArticleGoogle Scholar
- Hoheisel T, Kanzow C, Outrata J: Exact penalty results for mathematical programs with vanishing constraints. Nonlinear Anal. 2010, 72: 2514–2526. 10.1016/j.na.2009.10.047MathSciNetView ArticleGoogle Scholar
- Huyer W, Neumaier A: A new exact penalty function. SIAM J. Optim. 2003, 13: 1141–1158. 10.1137/S1052623401390537MathSciNetView ArticleGoogle Scholar
- Li W, Peng J: Exact penalty functions for constrained minimization problems via regularized gap function for variational inequality. J. Glob. Optim. 2007, 37: 85–94.MathSciNetView ArticleGoogle Scholar
- Mangasarian OL, Fromovitz S: The Fritz John necessary optimality conditions in the presence of equality and inequality constraints. J. Math. Anal. Appl. 1967, 17: 37–47. 10.1016/0022-247X(67)90163-1MathSciNetView ArticleGoogle Scholar
- Maratos, N: Exact penalty function algorithms for finite dimensional and control optimization problems. PhD thesis, University of London (1978)Google Scholar
- Mordukhovich BS Grundlehren Series (Fundamental Principles of Mathematical Sciences) 330. In Variations Analysis and Generalized Differentiation, I: Basic Theory. Springer, Berlin; 2006.Google Scholar
- Mordukhovich BS Grundlehren Series (Fundamental Principles of Mathematical Sciences) 331. In Variations Analysis and Generalized Differentiation, II: Applications. Springer, Berlin; 2006.Google Scholar
- Nocedal J, Wright SJ: Numerical Optimization. Springer, New York; 1999.View ArticleGoogle Scholar
- Soleimani-damaneh M: The gap function for optimization problems in Banach spaces. Nonlinear Anal. 2008, 69: 716–723. 10.1016/j.na.2007.06.008MathSciNetView ArticleGoogle Scholar
- Soleimani-damaneh M: Penalization for variational inequalities. Appl. Math. Lett. 2009, 22: 347–350. 10.1016/j.aml.2008.03.029MathSciNetView ArticleGoogle Scholar
- Soleimani-damaneh M: Nonsmooth optimization using Mordukhovich’s subdifferential. SIAM J. Control Optim. 2010, 48: 3403–3432. 10.1137/070710664MathSciNetView ArticleGoogle Scholar
- Zangwill WI: Nonlinear programming via penalty function. Manag. Sci. 1967, 13: 344–358. 10.1287/mnsc.13.5.344MathSciNetView ArticleGoogle Scholar
- Zaslavski AJ: A sufficient condition for exact penalty in constrained optimization. SIAM J. Optim. 2005, 16: 250–262. 10.1137/040612294MathSciNetView ArticleGoogle Scholar
- Zaslavski AJ: Stability of exact penalty for nonconvex inequality-constrained minimization problems. Taiwan. J. Math. 2010, 14: 1–19.MathSciNetGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.