- Research
- Open access
- Published:
On the global and linear convergence of direct extension of ADMM for 3-block separable convex minimization models
Journal of Inequalities and Applications volume 2016, Article number: 227 (2016)
Abstract
In this paper, we show that when the alternating direction method of multipliers (ADMM) is extended directly to the 3-block separable convex minimization problems, it is convergent if one block in the objective possesses sub-strong monotonicity which is weaker than strong convexity. In particular, we estimate the globally linear convergence rate of the direct extension of ADMM measured by the iteration complexity under some additional conditions.
1 Introduction
Because there is still a gap between the empirical efficiency of the direct extension of ADMM for a variety of applications and the lack of theoretical conditions that can both ensure the convergence of the direct extension of ADMM and be satisfied by applications, the main attention of this paper is paid to the study of the convergence of the direct extension of ADMM for the 3-block separable convex optimization problems.
We consider the following separable convex minimization problem whose objective function is the sum of three functions without coupled variables:
where \(A_{i}\in\mathcal{R}^{l\times n_{i}}\) (\(i = 1,2,3\)), \(b\in\mathcal {R}^{l}\), and \(\theta_{i}:\mathcal{R}^{n_{i}}\rightarrow(-\infty,+\infty ]\) (\(i = 1,2,3\)) are closed proper convex (not necessarily smooth) functions. This model has a lot of applications in practice. For example, the latent variable Gaussian graphical model selection in [1], the quadratic discriminant analysis model in [2] and the robust principal component analysis model with noisy and incomplete data in [3, 4], and so on. The augmented Lagrangian function of (1) is defined as
where \(\lambda\in\mathcal{R}^{l}\) and \(\beta>0\).
The classical alternating direction method of multipliers (ADMM) for solving the 2-block separable convex minimization problems was first introduced by Gabay and Mercier [5] and Glowinski and Marrocco [6], respectively, and its iterative scheme can be described by
where \(\alpha>0\) is called step-length. The convergence of ADMM has been well established in the literature (see [5, 7, 8]). For more details of the ADMM, the reader can also refer to [5, 9–13].
Due to the classical ADMM extreme simplicity and efficiency in numerous applications such as mathematical imaging science, signal processing, and so on, it is natural to extend the classical ADMM (3a)-(3c) directly to (1). The direct extension of the ADMM for solving problem (1) consists of the following iterations:
Despite the scheme working very well for many concrete applications of (1) (see e.g. [1, 2, 4, 14]), Chen et al. [15] showed by a counter example that the convergence of (4a)-(4d) fails. The absence of the convergence of (8) has inspired some improved algorithms. These algorithms are mainly used the following two ways: One way is to correct the output of (4a)-(4d). For example, the authors of [16, 17] added an additional Gaussian back substitution correction step in each iteration after all the block variables are updated. Although, numerically, these algorithms perform slightly slower than the scheme (4a)-(4d), they possess global convergence. The other way is to employ a simple proximal term to solve inexactly the \(x_{i}\)-subproblem in (4a)-(4d), which can make the subproblems of (4a)-(4d) become much easier to carry out and the entire algorithm runs in less time. The readers can refer to [4, 18–25].
On the other hand, several researchers have also studied the convergence of the direct extension of the ADMM (4a)-(4d) by introducing some strong conditions. Han and Yuan [26] have showed that the scheme (4a)-(4d) with \(\alpha=1\) is convergent if the functions \(\theta_{i}\) (\(i=1,2,3\)) are all strongly convex and the penalty parameter β chosen in a certain interval. Subsequently, these conditions were weakened in [27, 28], and the authors showed that the condition that the two functions are strongly convex can ensure the convergence of (4a)-(4d) with \(\alpha=1\). Recently, these conditions were further weakened, Cai et al. [29] had proved that the scheme (4a)-(4d) with \(\alpha=1\) was convergent if one function in the objective is strongly convex. Very recently, Li et al. [30] showed that the directly extended 3-block ADMM with \(\alpha \in(0,(1+\sqrt{5})/2)\) is convergent, if β is smaller than a certain threshold and the first and third linear operators in the linear equation constraint are full column rank, and the second function in the objective is strongly convex. However, many applications that can be efficiently solved by the scheme (4a)-(4d) will be excluded because of the strong convexity. Thus, these conditions are of only theoretical interests and they seem to be too strict to be satisfied by many mentioned applications.
In the cyclic sense, the scheme (4a)-(4d) can be rewritten as
In this manuscript, we show that (5a)-(5d) is convergent if one function in the objective of (1) is sub-strongly monotone together with some minor restrictions on the coefficient matrices \(A_{1}\), \(A_{2}\), \(A_{3}\), and the penalty parameter β, which explains why the direct extension of ADMM (4a)-(4d) works well for some applications, even though there are not strong convex functions in such applications. Furthermore, we establish a globally linear convergence rate for the direct extension of ADMM (5a)-(5d) under some additional conditions.
After presenting in Section 2 the needed preliminary material, we devote Section 3 to a proof of the global and linear convergence of the scheme (5a)-(5d) under some assumptions. In Section 4, we construct an example which satisfies the convergence conditions given in Section 3 but do not satisfy the condition that one of the functions in the objective is strongly convex.
2 Preliminaries
In this section we summarize some of notations and the fundamental tools of variational analysis.
We use \(\langle\cdot,\cdot\rangle\) to denote the inner product of \(\mathcal{R}^{n}\), and denote by \(\|\cdot\|\) its induced norm. \(\mathbf {B}_{r}(x)\) stands for the closed ball of radius r centered at x. Throughout the paper we let all vectors be column vectors. Let A be a symmetric matrix, \(\lambda_{\min}(A)\) and \(\lambda_{\max}(A)\) to denote the smallest eigenvalue and the largest eigenvalue of A, respectively. A real symmetric matrix \(A\in\mathcal{R}^{n\times n}\) is called positive definite (or positive semi-definite) if for all \(x\neq0\), \(x^{T}Ax>0\) (or \(x^{T}Ax\geq0\)). We denote this as \(A\succ0\) (or \(A\succeq0\)). For any real symmetric matrices \(A,B\in \mathcal{R}^{n\times n}\), we use \(A\succ B\) (or \(A\succeq B\)) to mean \(A-B\succ0\) (or \(A-B\succeq0\)). We denote by \(\|x\|_{M}:=\sqrt{x^{T}Mx}\) the M-norm of the vector x if the matrix M is symmetric and positive definite. For a given matrix A, we use
to denote its norm.
Given a nonempty subset C in \(\mathcal{R}^{n}\), its indicator function is defined as
A function \(f: \mathcal{R}^{n}\rightarrow\mathcal{R}\) is convex if
and it is strongly convex with modulus \(\mu> 0\) if
A multifunction \(F: \mathcal{R}^{n}\rightrightarrows\mathcal{R}^{n}\) (see [31]) is monotone if
and strongly monotone with modulus \(\mu> 0\) if
It is well known that a function f is convex if and only if ∂f, the subdifferential of f, is monotone; and f is strongly convex if and only if ∂f is strongly monotone (see, e.g., [31]).
For a differentiable function f, the gradient ∇f is called Lipschitz continuous with constant \(L_{f}>0\) if
For any two vectors x and y with the same dimension, we have
Throughout this paper, we make the following standard assumption.
Assumption 2.1
There is a point \((\hat{x}_{1},\hat{x}_{2},\hat{x}_{3})\in\operatorname{ri} (\operatorname{dom}\theta_{1}\times\operatorname{dom}\theta_{2}\times \operatorname{dom}\theta_{3})\) such that \(A_{1}\hat{x}_{1}+A_{2}\hat{x}_{2}+A_{3}\hat{x}_{3}=b\).
Suppose that the constraint qualification (CQ) holds, then we know from Corollary 28.2.2 of [31] and Corollary 28.3.1 of [31] that \((x_{1}^{*},x_{2}^{*},x_{3}^{*})\in\operatorname{ri} (\operatorname{dom}\theta_{1}\times\operatorname{dom}\theta_{2}\times\operatorname{dom}\theta_{3})\) is an optimal solution to problem (1) if and only if there exists a Lagrange multiplier \(\lambda^{*}\in\mathcal{R}^{l}\) such that \((x_{1}^{*},x_{2}^{*},x_{3}^{*},\lambda^{*})\) is a solution to the following Karush-Kuhn-Tucher (KKT) system:
We denote by \(\mathcal{W}^{*}\) the set of the solutions of (7a)-(7d).
3 Convergence
In this section, we prove that the iterative sequence \(\{(x_{1}^{k}, x^{k}_{2}, x^{k}_{3}, \lambda^{k})\}\) generated by the direct extension of ADMM (5a)-(5d) converges to a point \((x_{1}^{*}, x^{*}_{2}, x^{*}_{3}, \lambda^{*})\) which is a solution of the KKT system (7a)-(7d) under the following assumption. In the following, the matrices \(A_{1}\), \(A_{2}\), and \(A_{3}\) are assumed to be full column rank. We define the notations
and
where \(\rho>0\) and β is the penalty parameter in the direct extension of ADMM (5a)-(5d). Then the matrices G and \(G_{1}\) are symmetric.
3.1 Global convergence
Assumption 3.1
(Sub-strong monotonicity)
There exist \((\tilde{x}_{1}^{*},\tilde{x}_{2}^{*},\tilde{x}_{3}^{*},\tilde {\lambda}^{*})\in\mathcal{W}^{*}\) and a real number \(\mu_{3}>0\) such that
Now, we start proving the convergence of the iterative scheme (5a)-(5d) under Assumption 3.1. First, we give several lemmas.
Lemma 3.1
Suppose Assumption 3.1 holds. For the iterative sequence \(\{ (x_{1}^{k}, x^{k}_{2}, x^{k}_{3}, \lambda^{k})\}\) generated by the direct extension of ADMM (5a)-(5d), then we have
where \(\tilde{v}^{*}=(\tilde{x}_{2}^{*},\tilde{x}_{3}^{*},\tilde{\lambda }^{*})\) in \((\tilde{x}_{1}^{*},\tilde{x}_{2}^{*},\tilde{x}_{3}^{*},\tilde{\lambda }^{*})\) introduced in Assumption 3.1.
Proof
Indeed, the optimality condition of subproblems in (5a)-(5d) can be written as
Using (5c), (10a)-(10c) can be rewritten as
Using the monotonicity of the subdifferential and Assumption 3.1, it follows from (7a)-(7d) and (11a)-(11c) that we have
where \(\mu_{3}>0\). Adding up these three inequalities in (12a)-(12c) and using (7d), we obtain
Using the notations G and v, we further obtain
which implies (9) and thus completes the proof. □
Lemma 3.2
There exists a real number \(\rho\in(0,1)\) such that the matrix \(G_{1}\) is symmetric and positive definite.
Proof
Let
In order to justify the matrix \(G_{1}\) is symmetric and positive definite, we only need to show the matrix \(P_{1}\) is positive definite. Since
If \(\lambda_{\max}(A_{3}A_{3}^{T})\leq\lambda_{\min}(A_{3}^{T}A_{3})\), then for any \(\rho\in(0,1)\), we have
Otherwise, for any
we have
Thus, it follows from the Schur complement [32], Section A.5.5, that there exists a real number \(\rho\in(0,1)\) such that the matrix \(P_{1}\) is symmetric and positive definite, and so is \(G_{1}\). □
Lemma 3.3
Let the iterative sequence \(\{(x_{1}^{k}, x^{k}_{2}, x^{k}_{3}, \lambda^{k})\}\) be generated by the direct extension of ADMM (5a)-(5d) with \(\beta\in(0,2\rho\mu_{3}/(5\|A_{3}^{T}A_{3}\|))\) and \(\rho\in(0,1)\) defined in Lemma 3.2. Suppose Assumption 3.1 holds. Then there is a real number \(\eta>0\) such that
Proof
Note that (11b) is also true for \(k := k - 1\), i.e.,
Using the monotonicity of the subdifferential \(\partial\theta_{2}\), we have
It follows from (6) that
and
Let
It follows from (9) that
which together with (15), (16), and (17) gives
which implies that
Using the notation \(G_{1}\) and (18), we have
To prove such \(\eta>0\) exists for (14), we only need \(2\mu_{3}-\frac{5\beta\|A_{3}^{T}A_{3}\|}{\rho}>0\), which holds if \(\beta<2\rho\mu_{3}/(5\|A_{3}^{T}A_{3}\|)\). □
Now, we are ready to prove the convergence of the sequence \(\{(x_{1}^{k}, x^{k}_{2}, x^{k}_{3}, \lambda^{k})\}\) generated by the direct extension of ADMM (5a)-(5d) under Assumption 3.1. The result is summarized in the following theorem.
Theorem 3.1
Let the iterative sequence \(\{(x_{1}^{k}, x^{k}_{2}, x^{k}_{3}, \lambda^{k})\}\) be generated by the direct extension of ADMM (5a)-(5d) with \(\beta\in(0,2\rho\mu_{3}/(5\|A_{3}^{T}A_{3}\|))\) and \(\rho\in(0,1)\) defined in Lemma 3.2. Suppose Assumption 3.1 holds. Then the sequence \(\{(x_{1}^{k}, x^{k}_{2}, x^{k}_{3}, \lambda^{k})\}\) converges to a KKT point in \(\mathcal{W}^{*}\).
Proof
It follows from (14) that
and the sequence \(\{v^{k}\}\) is bounded. Equation (5c) then further implies that \(\{x^{k}_{1}\}\) is also bounded and hence the sequence \(\{ (x_{1}^{k},x_{2}^{k},x_{3}^{k},\lambda^{k})\}\) generated by (5a)-(5d) is bounded. The boundedness of the sequence \(\{(x_{1}^{k},x_{2}^{k},x_{3}^{k},\lambda ^{k})\}\) indicates that there is at least one cluster point of \(\{ (x_{1}^{k},x_{2}^{k},x_{3}^{k},\lambda^{k})\}\). Let \(\bar{w}:=(\bar{x}_{1},\bar {x}_{2},\bar{x}_{3},\bar{\lambda})\) be an arbitrary cluster point of \(\{ (x_{1}^{k},x_{2}^{k},x_{3}^{k},\lambda^{k})\}\) and \(\{ (x_{1}^{k_{j}},x_{2}^{k_{j}},x_{3}^{k_{j}},\lambda^{k_{j}})\}\) be the subsequence converging to w̄. By the inequality (19), we have \(\bar{x}_{3}=\tilde{x}_{3}^{*}\). It follows from (5c) and (11a)-(11c) that
Taking the limit in (21a)-(21d) and using (20), we obtain
which implies that \((\bar{x}_{1},\bar{x}_{2},\bar{x}_{3},\bar{\lambda})\) is a KKT point in \(\mathcal{W}^{*}\). It follows from (14) and (5c) that the iterative sequence \(\{(x_{1}^{k},x_{2}^{k},x_{3}^{k},\lambda^{k})\}\) generated by the direct extension of ADMM (5a)-(5d) converges to a KKT point in \(\mathcal {W}^{*}\). The proof is completed. □
Remark 3.1
If the sequence \(\{x_{3}^{k}\}\) is bounded, then Assumption 3.1 can be substituted by the following.
Assumption 3.2
There exist \((\tilde{x}_{1}^{*},\tilde{x}_{2}^{*},\tilde{x}_{3}^{*},\tilde {\lambda}^{*})\in\mathcal{W}^{*}\) and a real number \(\mu_{3}>0\) such that
where \(a=\max_{k}\{\|x_{3}^{k}-\tilde{x}^{*}_{3}\|\}\).
3.2 Global linear convergence
Cai et al. [29] show that the globally linear convergence of the direct extension of ADMM (4a)-(4d) can be ensured if \(\theta_{2}\) and \(\theta_{3}\) are strongly convex. In this subsection, we will show that the globally linear convergence rate of the direct extension of ADMM (5a)-(5d) can be ensured under weaker conditions. More precisely, we establish the globally linear convergence result for the iterative scheme (5a)-(5d) by showing that there exist \(\sigma\in(0,1)\) and \(\eta _{1}>0\) such that
Assumption 3.3
\(A^{T}_{2}\) is full column rank. For any \((x_{1}^{*},x_{2}^{*},x_{3}^{*},\lambda^{*})\in \mathcal{W}^{*}\), there exists a real number \(\mu_{2}>0\) such that
Theorem 3.2
Let the iterative sequence \(\{(x_{1}^{k}, x^{k}_{2}, x^{k}_{3}, \lambda^{k})\}\) be generated by the direct extension of ADMM (5a)-(5d) with \(\beta\in(0,2\rho\mu_{3}/(5\|A_{3}^{T}A_{3}\|))\) and \(\rho\in(0,1)\) defined in Lemma 3.2. Suppose Assumption 3.1 and Assumption 3.3 hold. If the function \(\theta_{2}\) is differentiable, and its gradient \(\nabla\theta_{2}\) is Lipschitz continuous with positive constant \(L_{2}\), then there exists \(\delta>0\) such that (24) holds.
Proof
Since Assumption 3.1 and Assumption 3.3 hold, following the same discussions as of Lemma 3.1 and Lemma 3.3, we have
where \(\mu_{2}>0\) and \(\mu_{3}>0\). Thus, there is a real number \(\eta _{1}>0\) such that
Since \(\theta_{2}\) is differentiable and \(\nabla\theta_{2}\) is Lipschitz continuous with positive constant \(L_{2}\), it follows from (7b) and (11b) that
which together with (26) yields
Since the matrix \(A_{2}^{T}\) is full column rank, the inequality (27) implies that there exists \(\delta>0\) such that
Using (27) again, we obtain
Let \(\sigma=\frac{1}{\sqrt{1+\delta}}\), we see that (24) holds. □
Notice that if \(\|v^{k}-v^{k+1}\|^{2}_{G}=0\), it follows from (5c) and (11a)-(11c) that
which shows that \((x_{1}^{k+1},x_{2}^{k+1},x_{3}^{k+1},\lambda^{k+1})\) is a solution of (7a)-(7d). Thus, Theorem 3.2 establishes a globally linear convergence rate for the direct extension of ADMM (5a)-(5d), and it inspires an easily implementable stopping criterion for implementing ADMM (5a)-(5d):
4 Example
Chen et al. [15] constructed the following example of solving a 3-dimensional linear system:
to show that the directly extended alternating direction method of multipliers applied to the above 3-block (treating each variable as one block) optimization problem will diverge.
We replace the item \(0\times x_{3}\) by \(\mu\|A_{3}x_{3}\|_{1}+\delta (x_{3};\mathbf{B}_{r}(0))\) with \(\mu>0\) and arbitrary \(r>0\) in (31), and obtain
The example (32) can be rewritten as the 3-block optimization problem (1) with the following specifications:
-
\(\theta_{1}(x_{1}):=0\times x_{1}\), \(\theta _{2}(x_{2}):=0\times x_{2}\), \(\theta_{3}(x_{3}):=\mu\|A_{3}x_{3}\|_{1}+\delta (x_{3};\mathbf{B}_{r}(0))\);
-
The coefficients \(A_{i}\) (\(i = 1, 2, 3\)) and the vector b are given by
$$A_{1}:= \begin{pmatrix}1\\1\\1 \end{pmatrix} , \qquad A_{2}:= \begin{pmatrix}1\\1\\2 \end{pmatrix} , \qquad A_{3}:= \begin{pmatrix}1\\2\\2 \end{pmatrix} , \qquad b:= \begin{pmatrix}0\\0\\0 \end{pmatrix} . $$
Obviously, \(\theta_{3}(x_{3})\) is not strongly convex. For problem (32), \((x_{1},x_{2},x_{3},\lambda)=(0,0,0,0)\) is a KKT point. If the iterative sequence \(\{(x_{1}^{k}, x^{k}_{2}, x^{k}_{3}, \lambda^{k})\}\) is generated by the direct extension of ADMM (5a)-(5d), then we have \(x_{3}^{k}\in\mathbf{B}_{r}(0)\), ∀k. To justify the convergence of the direct extension of ADMM (5a)-(5d) applied to (32), one just needs to show that Assumption 3.2 holds at \((\tilde{x}_{1}^{*},\tilde{x}_{2}^{*},\tilde{x}_{3}^{*},\tilde{\lambda }^{*})=(0,0,0,0)\).
For any \(x_{3}\in\mathbf{B}_{r}(\tilde{x}_{3}^{*})\), we have
Thus,
where \(\alpha^{r}=5\mu/r\). On the other hand, since the function \(\theta_{3}\) is convex, we have
Adding up (33) and (34), we obtain
Thus Assumption 3.2 holds for (32) at \((\tilde{x}_{1}^{*},\tilde{x}_{2}^{*},\tilde{x}_{3}^{*},\tilde{\lambda }^{*})=(0,0,0,0)\) with \(\mu_{3}=5\mu/r\), it follows from Theorem 3.1 that the direct extension of ADMM (5a)-(5d) applied to (32) is convergent.
Remark 4.1
If the function \(\theta_{3}\) is strongly convex, then Assumption 3.1 or Assumption 3.2 holds trivially. The example (32) shows that the direct extension of ADMM (5a)-(5d) applied to (32) is convergent, although \(\theta_{3}\) is not strongly convex. This explains why the original scheme of the direct extension of ADMM works well for some applications even though there is not a strong convex function in the objective.
References
Chandrasekaran, V, Parrilo, PA, Willsky, AS: Latent variable graphical model selection via convex optimization. Ann. Stat. 40, 1935-1967 (2012)
McLachlan, GJ: Discriminant Analysis and Statistical Pattern Recognition. Wiley-Interscience, New York (2004)
Candès, EJ, Li, X, Ma, Y, Wright, J: Robust principal component analysis. J. ACM 58(3), Article 11 (2011)
Tao, M, Yuan, XM: Recovering low-rank and sparse components of matrices from incomplete and noisy observations. SIAM J. Optim. 21(1), 57-81 (2011)
Gabay, D, Mercier, B: A dual algorithm for the solution of nonlinear variational problems via finite-element approximations. Comput. Math. Appl. 2, 17-40 (1976)
Glowinski, R, Marrocco, A: Sur l’approximation, par éléments fins d’ordren, et la résolution, par pénalisation-dualité, d’une classe de problèmes de dirichlet nonlinéares. Rev. Fr. Autom. Inform. Rech. Opér., Anal. Numér. 9, 41-76 (1975)
Fortin, M, Glowinski, R: Augmented Lagrangian Methods. North-Holland, Amsterdam (1983)
Glowinski, R: Lectures on Numerical Methods for Nonlinear Variational Problems. Springer, Berlin (1980)
Boyd, S, Parikh, N, Chu, E, Peleato, B, Eckstein, J: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3, 1-122 (2010)
Chan, TF, Glowinski, R: Finite element approximation and iterative solution of a class of mildly non-linear elliptic equations. Technical report, Stanford University (1978)
Eckstein, J: Augmented Lagrangian and alternating direction methods for convex optimization: a tutorial and some illustrative computational results. RUTCOR research report RRR 32-2012, Rutgers University (2012)
He, BS, Liu, H, Wang, ZR, Yuan, XM: A strictly Peaceman-Rachford splitting method for convex programming. SIAM J. Optim. 24, 1011-1040 (2014)
Lin, Z, Chen, M, Wu, L, Ma, Y: The augmented Lagrange multiplier method for exact recovery of corrupted low-rank matrices. Math. Program. 9(22), 15-26 (2010)
Peng, YG, Ganesh, A, Wright, J, Xu, WL, Ma, Y: Robust alignment by sparse and low rank decomposition for linearly correlated images. IEEE Trans. Pattern Anal. Mach. Intell. 34, 2233-2246 (2012)
Chen, CH, He, BS, Ye, YY, Yuan, XM: The direct extension of ADMM for multi-block convex minimization problems is not necessarily convergent. Math. Program. 155(1-2), 57-79 (2016)
He, BS, Tao, M, Yuan, XM: Alternating direction method with Gaussian back substitution for separable convex programming. SIAM J. Optim. 22, 313-340 (2012)
He, BS, Tao, M, Yuan, XM: Convergence rate and iteration complexity on the alternating direction method of multipliers with a substitution procedure for separable convex programming. Manuscript
Chen, G, Teboulle, M: A proximal-based decomposition method for convex minimization problems. Math. Program. 64, 81-101 (1994)
Deng, W, Yin, WT: On the global and linear convergence of the generalized alternating direction method of multipliers. J. Sci. Comput. 66(3), 889-916 (2016)
Eckstein, J: Some saddle-function splitting methods for convex programming. Optim. Methods Softw. 4, 75-83 (1994)
Fazel, M, Pong, TK, Sun, DF, Tseng, P: Hankel matrix rank minimization with applications to system identification and realization. SIAM J. Matrix Anal. Appl. 34, 946-977 (2013)
He, BS, Liao, LZ, Han, D, Yang, H: A new inexact alternating direction method for monotone variational inequalities. Math. Program. 92, 103-118 (2002)
Hong, MY, Luo, ZQ: On the linear convergence of the alternating direction method of multipliers. Math. Program. (2016). doi:10.1007/s10107-016-1034-2
Xu, MH, Wu, T: A class of linearized proximal alternating direction methods. J. Optim. Theory Appl. 155, 321-337 (2011)
Yang, J, Zhang, Y: Alternating direction algorithms for \(l_{1}\) problems in compressive sensing. SIAM J. Sci. Comput. 33, 250-278 (2011)
Han, DR, Yuan, XM: A note on the alternating direction method of multipliers. J. Optim. Theory Appl. 155(1), 227-238 (2013)
Chen, CH, Shen, Y, You, YF: On the convergence analysis of the alternating direction method of multipliers with three blocks. Abstr. Appl. Anal. 2013, Article ID 183961 (2013)
Lin, TY, Ma, SQ, Zhang, SZ: On the convergence rate of multi-block ADMM (2014). arXiv:1408.4265
Cai, XJ, Han, DR, Yuan, XM: The direct extension of ADMM for three-block separable convex minimization models is convergent when one function is strongly convex. Manuscript (2014)
Li, M, Sun, DF, Toh, KC: A convergent 3-block semi-proximal ADMM for convex minimization problems with one strongly convex block. Asia-Pac. J. Oper. Res. 32(4), Article ID 1550024 (2015)
Rockafellar, RT: Convex Analysis. Princeton University Press, Princeton (1970)
Boyd, S, Vandenberghe, L: Convex Optimization. Cambridge University Press, Cambridge (2004)
Acknowledgements
This work was supported by the National Natural Sciences Grants (No. 11371116, No. 41071262, and No. 41101243).
Author information
Authors and Affiliations
Corresponding author
Additional information
Competing interests
The authors declare that they have no competing interests.
Authors’ contributions
All authors contributed equally to the writing of this paper. All authors read and approved the final manuscript.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Sun, H., Wang, J. & Deng, T. On the global and linear convergence of direct extension of ADMM for 3-block separable convex minimization models. J Inequal Appl 2016, 227 (2016). https://doi.org/10.1186/s13660-016-1173-2
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s13660-016-1173-2