Open Access

On the \(O(1/t)\) convergence rate of the alternating direction method with LQP regularization for solving structured variational inequality problems

  • Abdellah Bnouhachem1, 2Email author,
  • Abdul Latif3 and
  • Qamrul Hasan Ansari4, 5
Contributed equally
Journal of Inequalities and Applications20162016:297

https://doi.org/10.1186/s13660-016-1226-6

Received: 3 October 2016

Accepted: 8 November 2016

Published: 22 November 2016

Abstract

In this paper, we propose a parallel descent LQP alternating direction method for solving structured variational inequality with three separable operators. The \(O(1/t)\) convergence rate for this method is studied. We also present some numerical examples to illustrate the efficiency of the proposed method. The results presented in this paper extend and improve some well-known results in the literature.

Keywords

structured variational inequalitieslogarithmic-quadratic proximal methodconvergence rateprojection methodalternating direction method

MSC

90C3349J4065N30

1 Introduction

Let \(\mathbb{R}^{n}_{+} = \{ x = (x_{1},x_{2}, \ldots,x_{n}) \in \mathbb{R}^{n} : x_{i} \geq0\ \forall i=1,2, \ldots, n \}\) and \(\mathbb{R}^{n}_{++} = \{ x = (x_{1},x_{2}, \ldots,x_{n}) \in\mathbb {R}^{n} : x_{i} > 0 \ \forall i=1,2, \ldots, n \}\). The variational inequality problem is to find
$$ x\in\Omega:= \bigl\{ (u,v) : u\in\mathbb{R}_{+}^{n}, v\in \mathbb{R}_{+}^{m}, A_{1}u + A_{2}v = b \bigr\} $$
such that
$$ \bigl(x'- x\bigr)^{T} F(x) \ge0, \quad \forall x'\in\Omega, $$
(1.1)
with
$$ x=\left ( \textstyle\begin{array}{c} u \\ v \end{array}\displaystyle \right ) \quad \mbox{and} \quad F(x)=\left ( \textstyle\begin{array}{c} f_{1}(u)\\ f_{2}(v) \end{array}\displaystyle \right ), $$
(1.2)
where \(A_{1}\in\mathbb{R}^{l\times n}\), \(A_{2} \in\mathbb{R}^{l\times m}\) are given matrices, \(b\in\mathbb{R}^{l}\) is a given vector, and \(f_{1}:\mathbb{R}_{+}^{n}\to \mathbb{R}^{n}\), \(f_{2}:\mathbb{R}_{+}^{m}\to\mathbb{R}^{m}\) are given monotone operators. For further study and applications of such problems, we refer to [111] and the references therein. By attaching a Lagrange multiplier vector \(\lambda\in\mathbb{R}^{l}\) to the linear constraints \(A_{1}u + A_{2}v =b\), the problem (1.1)-(1.2) can be explained in terms of finding \(z \in\mathcal{Z}' := \mathbb{R}_{+}^{n} \times\mathbb{R}_{+}^{m} \times \mathbb{R}^{l}\) such that
$$ \bigl(z'-z\bigr)^{\top} Q(z) \geq0, \quad \forall z' \in\mathcal{Z}', $$
(1.3)
where
$$ z=\left ( \textstyle\begin{array}{c} u \\ v\\ \lambda \end{array}\displaystyle \right ), \qquad Q(z)= \left ( \textstyle\begin{array}{c} f_{1}(u)-A_{1}^{\top} \lambda\\ f_{2}(v)-A_{2}^{\top} \lambda\\ A_{1}u + A_{2}v-b \end{array}\displaystyle \right ), $$
(1.4)
and \(A_{1}^{\top}\) denotes the transpose of the matrix \(A_{1}\). The problem (1.3)-(1.4) is referred as a structured variational inequality problem (in short, SVI).
Yuan and Li [12] developed the following logarithmic-quadratic proximal (LQP)-based decomposition method by applying the LQP terms to regularize the ADM subproblems: For a given \(z^{k}=(u^{k},v^{k},\lambda^{k})\in\mathbb{R}^{n}_{++}\times\mathbb {R}^{m}_{++}\times\mathbb{R}^{l}\), and \(\mu\in(0,1)\), the new iterative \((u^{k+1}, v^{k+1}, \lambda^{k+1})\) is obtained via solving the following system:
$$\begin{aligned}& f_{1}(u) - A_{1}^{\top} \bigl[ \lambda^{k} - H \bigl(A_{1} u + A_{2} v^{k} - b\bigr) \bigr] + R \bigl[ \bigl(u-u^{k}\bigr) + \mu \bigl(u^{k} - U_{k}^{2} u^{-1}\bigr) \bigr] = 0, \end{aligned}$$
(1.5)
$$\begin{aligned}& f_{2}(v) - A_{2}^{\top} \bigl[ \lambda^{k} - H (A_{1} {u} + A_{2} v - b) \bigr] + S \bigl[ \bigl(v-v^{k}\bigr) + \mu\bigl(v^{k} - V_{k}^{2} v^{-1}\bigr) \bigr] = 0, \end{aligned}$$
(1.6)
$$\begin{aligned}& \lambda^{k+1} = \lambda^{k} - H \bigl(A_{1} u^{k} + A_{2} v^{k} - b\bigr), \end{aligned}$$
(1.7)
where \(H \in\mathbb{R}^{l \times l}\), \(R \in\mathbb{R}^{n \times n}\), and \(S \in\mathbb{R}^{m \times m}\) are symmetric positive definite.

Later, some LQP alternating direction methods have been proposed to make the LQP alternating direction method more practical, see, for example, [1317] and the references therein. Each iteration of these methods contains a prediction and a correction, the predictor is obtained via solving (1.5)-(1.7) and the new iterate is obtained by a convex combination of the previous point and the one generated by a projection-type method along a descent direction. The main disadvantage of the methods proposed in [1217] is that solving equation (1.6) requires the solution of equation (1.5). To overcome with this difficulty, Bnouhachem and Hamdi [18] proposed a parallel descent LQP alternating direction method for solving SVI.

In this paper, we propose a parallel descent LQP alternating direction method for solving the following structured variational inequality with three separable operators: Find \(y\in\Omega:= \{ (u,v,w) : u\in\mathbb {R}_{+}^{n_{1}}, v\in\mathbb{R}_{+}^{n_{2}}, w\in\mathbb{R}_{+}^{n_{2}}, A_{1}u + A_{2}v +A_{3}w= b \}\) such that
$$ \bigl(y'- y\bigr)^{\top} F(y) \ge0, \quad \forall y'\in\Omega, $$
(1.8)
with
$$ y=\left ( \textstyle\begin{array}{c} u \\ v\\ w \end{array}\displaystyle \right ),\qquad F(y)= \left ( \textstyle\begin{array}{c} f_{1}(u)\\ f_{2}(v)\\ f_{3}(w) \end{array}\displaystyle \right ), $$
(1.9)
where \(A_{1}\in\mathbb{R}^{m\times n_{1}}\), \(A_{2} \in\mathbb{R}^{m\times n_{2}}\), \(A_{3}\in\mathbb{R}^{m\times n_{3}}\) are given matrices, \(b\in\mathbb{R}^{m}\) is a given vector, and \(f_{1}:\mathbb {R}_{+}^{n_{1}}\to\mathbb{R}^{n_{1}}\), \(f_{2}:\mathbb{R}_{+}^{n_{2}}\to\mathbb{R}^{n_{2}}\), \(f_{3}:\mathbb {R}_{+}^{n_{3}}\to\mathbb{R}^{n_{3}}\) are given monotone operators. By attaching a Lagrange multiplier vector \(\lambda\in\mathbb{R}^{m}\) to the linear constraints \(A_{1}u + A_{2}v +A_{3}w= b\), the problem (1.8)-(1.9) can be explained in terms of finding \(z \in\mathcal{Z} := \mathbb{R}_{+}^{n_{1}} \times\mathbb {R}_{+}^{n_{2}} \times\mathbb{R}_{+}^{n_{3}} \times\mathbb{R}^{m}\) such that
$$ \bigl(z'-z\bigr)^{\top} Q(z) \geq0, \quad \forall z' \in \mathcal{Z}, $$
(1.10)
where
$$ z=\left ( \textstyle\begin{array}{c} u\\ v\\ w\\ \lambda \end{array}\displaystyle \right ) \quad \mbox{and} \quad Q(z)=\left ( \textstyle\begin{array}{c} f_{1}(u)-A_{1}^{\top} \lambda\\ f_{2}(v)-A_{2}^{\top} \lambda\\ f_{3}(w)-A_{3}^{\top} \lambda\\ A_{1}u + A_{2}v +A_{3}w- b \end{array}\displaystyle \right ). $$
(1.11)
The problem (1.10)-(1.11) is referred as \(\mathrm{SVI}_{3}\).

The main aim of this paper is to present the parallel descent LQP alternating direction method for solving \(\mathrm{SVI}_{3}\) and to investigate the convergence rate of this method. We show that the proposed method has the \(O(1/t)\) convergence rate. The iterative algorithm and results presented in this paper generalize, unify, and improve the previously known results in this area.

2 The proposed method

For any vector \(u\in\mathbb{R}^{n}\), \(\Vert u\Vert _{\infty}=\max\{\vert u_{1}\vert , \ldots ,\vert u_{n}\vert \}\). Let \(D \in\mathbb{R}^{n \times n}\) be a symmetry positive definite matrix, we denote the D-norm of u by \(\Vert u\Vert _{D}^{2}=u^{T}Du\).

The following lemma provides a basic property of projection operator onto a closed convex subset Ω of \(\mathbb{R}^{l}\). We denote by \(P_{\Omega,D}(\cdot)\) the projection operator under the D-norm, that is,
$$P_{\Omega,D}(v)=\operatorname{argmin} \bigl\{ \Vert v-u\Vert _{D} : u \in\Omega\bigr\} . $$

Lemma 2.1

Let D be a symmetry positive definite matrix and Ω be a nonempty closed convex subset of \(\mathbb{R}^{l}\). Then
$$ \bigl(z-P_{\Omega,D}[z]\bigr)^{\top} D \bigl(P_{\Omega,D}[z]-v\bigr) \geq0, \quad \forall z \in\mathbb{R}^{l}, v\in\Omega. $$
(2.1)

We make the following standard assumptions.

Assumption 2.1

\(f_{1}\) is monotone with respect to \(\mathbb{R}^{n_{1}}_{+}\), that is, \((f_{1}(x)-f_{1}(y))^{T}(x-y)\ge0\), \(\forall x,y\in\mathbb{R}^{n_{1}}_{+}\), \(f_{2}\) is monotone with respect to \(\mathbb{R}^{n_{2}}_{+}\), and \(f_{3}\) is monotone with respect to \(\mathbb{R}^{n_{3}}_{+}\).

Assumption 2.2

The solution set of \(\mathrm{SVI}_{3}\), denoted by \(\mathcal {Z}^{*}\), is nonempty.

We propose the following parallel LQP alternating direction method for solving \(\mathrm{SVI}_{3}\):

Algorithm 2.1

  1. Step 0.

    Given \(\varepsilon>0\), \(\mu\in(0,1)\), \(\beta\in ( \frac {\sqrt{3}}{2},1 )\), \(\gamma\in(0,2)\) and \(z^{0}=(u^{0}, v^{0},w^{0}, \lambda^{0}) \in\mathbb{R}^{n_{1}}_{++}\times \mathbb{R}^{n_{2}}_{++}\times\mathbb{R}^{n_{3}}_{++}\times\mathbb{R}^{m}\). Set \(k=0\).

     
  2. Step 1.
    Compute \(\tilde{z}^{k}=(\tilde{u}^{k},\tilde{v}^{k},\tilde{w}^{k},\tilde {\lambda}^{k})\in\mathbb{R}^{n_{1}}_{++}\times \mathbb{R}^{n_{2}}_{++}\times\mathbb{R}^{n_{3}}_{++}\times\mathbb {R}^{m}\) by solving the following system:
    $$\begin{aligned}& f_{1}(u) - A_{1}^{\top} \bigl[\lambda^{k} - H \bigl(A_{1} u + A_{2} v^{k} +A_{3} w^{k}- b\bigr) \bigr] \\& \quad {} + R_{1}\bigl[\bigl(u-u^{k}\bigr)+ \mu\bigl(u^{k} - U_{k}^{2} u^{-1}\bigr)\bigr]=0, \end{aligned}$$
    (2.2)
    $$\begin{aligned}& f_{2}(v) - A_{2}^{\top} \bigl[ \lambda^{k} - H \bigl(A_{1} u^{k} + A_{2} v +A_{3} w^{k}- b\bigr)\bigr] \\& \quad {}+ {R_{2}}\bigl[ \bigl(v-v^{k}\bigr) + \mu\bigl(v^{k} - V_{k}^{2} v^{-1}\bigr)\bigr]=0, \end{aligned}$$
    (2.3)
    $$\begin{aligned}& f_{3}(w) - A_{3}^{\top} \bigl[ \lambda^{k} - H \bigl(A_{1} u^{k} + A_{2} v^{k}+A_{3} w - b\bigr)\bigr] \\& \quad {}+ {R_{3}}\bigl[ \bigl(w-w^{k}\bigr) + \mu\bigl(w^{k} - W_{k}^{2} w^{-1}\bigr)\bigr]=0, \end{aligned}$$
    (2.4)
    $$\begin{aligned}& \tilde{\lambda}^{k} = \lambda^{k} - \beta H \bigl(A_{1} \tilde{u}^{k} + A_{2} \tilde{v}^{k}+A_{3} \tilde{w}^{k} - b\bigr) , \end{aligned}$$
    (2.5)
    where \(H \in\mathbb{R}^{m \times m}\), \({R_{1}} \in\mathbb{R}^{n_{1} \times n_{1}}\), \({R_{2}} \in\mathbb{R}^{n_{2} \times n_{2}}\) and \({R_{3}} \in\mathbb{R}^{n_{3} \times n_{3}}\) are symmetric positive definite matrices. \(U_{k}\), \(V_{k}\), and \(W_{k}\) are positive definite diagonal matrices defined by \(U_{k} =\operatorname{diag} ( u^{k}_{1}, \ldots,u^{k}_{n} )\), \(V_{k} = \operatorname{diag} ( v^{k}_{1}, \ldots,v^{k}_{n} )\), \(W_{k} = \operatorname{diag} ( w^{k}_{1}, \ldots,w^{k}_{n} )\).
     
  3. Step 2.

    If \(\max\{\Vert u^{k}-\tilde{u}^{k}\Vert _{\infty}, \Vert v^{k}-\tilde{v}^{k}\Vert _{\infty}, \Vert w^{k}-\tilde{w}^{k}\Vert _{\infty}, \Vert \lambda^{k}-\tilde{\lambda}^{k}\Vert _{\infty}\}<\epsilon\), then stop.

     
  4. Step 3.
    The new iterate \(z^{k+1}(\tau_{k})=(u^{k+1},v^{k+1},w^{k+1},\lambda ^{k+1})\) is given by
    $$ z^{k+1}(\tau_{k})= (1-\sigma) z^{k}+ \sigma P_{\mathcal{Z},G}\bigl[z^{k}-\gamma \tau_{k}G^{-1}g \bigl(z^{k},\tilde{z}^{k}\bigr)\bigr], \quad\sigma\in(0,1), $$
    (2.6)
    where
    $$\begin{aligned}& \tau_{k}=\frac{\varphi(z^{k},\tilde{z}^{k})}{\Vert z^{k}- \tilde{z}^{k}\Vert ^{2}_{G}}, \end{aligned}$$
    (2.7)
    $$\begin{aligned}& \varphi\bigl(z^{k},\tilde{z}^{k}\bigr)=\bigl\Vert z^{k}- \tilde{z}^{k}\bigr\Vert _{M}^{2} \\& \hphantom{ \varphi\bigl(z^{k},\tilde{z}^{k}\bigr)=}{}+ \frac {1}{\beta}\bigl(\lambda^{k}-\tilde{\lambda}^{k} \bigr)^{T} \bigl(A_{1}\bigl(u^{k}- \tilde{u}^{k}\bigr) +A_{2}\bigl(v^{k}- \tilde{v}^{k}\bigr)+A_{3}\bigl(w^{k}- \tilde{w}^{k}\bigr) \bigr), \end{aligned}$$
    (2.8)
    $$\begin{aligned}& g(z^{k},\tilde{z}^{k}) \\& \quad = \left ( \textstyle\begin{array}{c} f_{1}(\tilde{u}^{k}) - A_{1}^{\top} \tilde{\lambda}^{k}+A_{1}^{\top} H[A_{1}(u^{k}-\tilde{u}^{k}) +A_{2}(v^{k}-\tilde{v}^{k})+A_{3}(w^{k}-\tilde{w}^{k}) +\frac{1-\beta}{\beta }H^{-1}(\lambda^{k}-\tilde{\lambda}^{k})]\\ f_{2}(\tilde{v}^{k}) - A_{2}^{\top} \tilde{\lambda}^{k}+ A_{2}^{\top} H[A_{1}(u^{k}-\tilde{u}^{k})+A_{2}(v^{k}-\tilde{v}^{k})+A_{3}(w^{k}-\tilde{w}^{k}) +\frac{1-\beta}{\beta}H^{-1}(\lambda^{k}-\tilde{\lambda}^{k})]\\ f_{3}(\tilde{w}^{k}) - A_{3}^{\top} \tilde{\lambda}^{k}+ A_{3}^{\top} H[A_{1}(u^{k}-\tilde{u}^{k})+A_{2}(v^{k}-\tilde{v}^{k})+A_{3}(w^{k}-\tilde{w}^{k}) +\frac{1-\beta}{\beta}H^{-1}(\lambda^{k}-\tilde{\lambda}^{k})]\\ A_{1}\tilde{u}^{k}+A_{2}\tilde{v}^{k}+A_{3}\tilde{w}^{k}-b \end{array}\displaystyle \right ), \\& \\& G=\left ( \textstyle\begin{array}{c@{\quad}c@{\quad}c@{\quad}c} (1+\mu){R_{1}}+A_{1}^{\top} HA_{1} & 0& 0 &0 \\ 0 & (1+\mu){R_{2}} +A_{2}^{\top} HA_{2} & 0&0 \\ 0 &0 &(1+\mu){R_{3}} +A_{3}^{\top} HA_{3} & 0 \\ 0 & 0 &0 & \frac{1}{\beta}H^{-1} \end{array}\displaystyle \right ), \end{aligned}$$
    (2.9)
    and
    $$M=\left ( \textstyle\begin{array}{c@{\quad}c@{\quad}c@{\quad}c} {R_{1}} +A_{1}^{\top}HA_{1} & 0& 0 &0 \\ 0 & {R_{2}} +A_{2}^{\top} HA_{2} & 0&0 \\ 0 &0 & {R_{3}} +A_{3}^{\top} HA_{3} & 0 \\ 0 & 0 &0 & \frac{1}{\beta}H^{-1} \end{array}\displaystyle \right ). $$
     
Set \(k:= k + 1\) and go to Step 1.

Remark 2.1

As special cases, we can obtain some new LQP alternating methods as follows:
  1. (a)

    If \(u^{k+1}=\tilde{u}^{k}\), \(v^{k+1}=\tilde{v}^{k}\), \(w^{k+1}=\tilde{w}^{k}\) and \(\lambda^{k+1}=\tilde{\lambda}^{k}\) in (2.2), (2.3), (2.4) and (2.5), respectively, we obtain a new method which can be viewed as an extension of that proposed in [10] for solving structured variational inequality with three separable operators in a parallel way.

     
  2. (b)

    If \(u^{k+1}=\tilde{u}^{k}\), \(v^{k+1}=\tilde{v}^{k}\), \(w^{k+1}=\tilde{w}^{k}\), \(\lambda^{k+1}=\tilde{\lambda}^{k}\), and \(\beta=1\) in (2.2), (2.3), (2.4) and (2.5), respectively, we obtain a new method which can be viewed as an extension of that proposed in [12] for solving structured variational inequality with three separable operators in a parallel wise.

     
  3. (c)

    If \(\beta=1\), the proposed method can be viewed as an extension of that proposed in [18] for solving structured variational inequality with three separable operators.

     

We need the following result in the convergence analysis of the proposed method.

Lemma 2.2

[12]

Let \(q(u)\in\mathbb{R}^{n}\) be a monotone mapping of u with respect to \(\mathbb{R}^{n}_{+}\) and \(R\in\mathbb{R}^{n\times n}\) be a positive definite diagonal matrix. For a given \(u^{k} >0\), if \(U_{k}:= \operatorname {diag}(u^{k}_{1},u^{k}_{2},\ldots, u^{k}_{n})\) (the diagonal matrix with elements \(u^{k}_{1}\), \(u^{k}_{2}\), … , \(u^{k}_{n}\)) and \(u^{-1}\) be an n-vector whose jth element is \(1/u_{j}\), then the equation
$$ q(u) + R\bigl[\bigl(u -u^{k}\bigr) + \mu\bigl( u^{k} - U_{k}^{2} u^{-1}\bigr)\bigr]=0 $$
(2.10)
has a unique positive solution u. Moreover, for any \(v\geq0\), we have
$$ (v-u)^{\top} q(u) \geq{ \frac{1+\mu}{2} } \bigl(\Vert u - v \Vert _{R}^{2} - \bigl\Vert u^{k} - v \bigr\Vert _{R}^{2} \bigr) + { \frac{1-\mu}{2} }\bigl\Vert u^{k} - u \bigr\Vert _{R}^{2}. $$
(2.11)

The next theorem is useful for the convergence analysis.

Theorem 2.1

For given \(z^{k}\in \mathbb{R}^{n_{1}}_{++}\times\mathbb{R}^{n_{2}}_{++}\times \mathbb{R}^{n_{3}}_{++}\times\mathbb{R}^{m}\), let \(\tilde{z}^{k}\) be generated by (2.2)-(2.5). Then
$$\begin{aligned} \varphi\bigl(z^{k},\tilde{z}^{k}\bigr)\geq \frac{2\beta-\sqrt{3}}{2\beta} \bigl\Vert z^{k}- \tilde{z}^{k}\bigr\Vert ^{2}_{G} \end{aligned}$$
(2.12)
and
$$\begin{aligned} \tau_{k}\geq\frac{2\beta-\sqrt{3}}{2\beta}. \end{aligned}$$
(2.13)

Proof

It follows from (2.8) that
$$\begin{aligned} \varphi\bigl(z^{k},\tilde{z}^{k}\bigr) = & \bigl\Vert z^{k}- \tilde{z}^{k}\bigr\Vert _{M}^{2}+\frac{1}{\beta}\bigl(\lambda^{k}-\tilde {\lambda}^{k}\bigr)^{\top}\bigl(A_{1} \bigl(u^{k}-\tilde{u}^{k}\bigr) +A_{2} \bigl(v^{k}-\tilde{v}^{k}\bigr)+A_{3} \bigl(w^{k}-\tilde{w}^{k}\bigr)\bigr) \\ = & \bigl\Vert u^{k}-\tilde{u}^{k}\bigr\Vert _{R_{1}}^{2}+\bigl\Vert A_{1}u^{k}-A_{1} \tilde{u}^{k}\bigr\Vert _{H}^{2}+\bigl\Vert v^{k}-\tilde{v}^{k}\bigr\Vert _{R_{2}}^{2} +\bigl\Vert A_{2}v^{k} -A_{2} \tilde{v}^{k} \bigr\Vert _{H}^{2} \\ &{}+\bigl\Vert w^{k}-\tilde{w}^{k}\bigr\Vert _{R_{3}}^{2}+\bigl\Vert A_{3}w^{k} -A_{3}\tilde{w}^{k}\bigr\Vert _{H}^{2} +\frac{1}{\beta}\bigl\Vert \lambda^{k}-\tilde{\lambda}^{k} \bigr\Vert _{H^{-1}}^{2} \\ &{}+\frac{1}{\beta}\bigl(\lambda^{k}-\tilde{\lambda}^{k} \bigr)^{\top }\bigl(A_{1}\bigl(u^{k}- \tilde{u}^{k}\bigr)+A_{2}\bigl(v^{k}- \tilde{v}^{k}\bigr)+A_{3}\bigl(w^{k}- \tilde{w}^{k}\bigr)\bigr). \end{aligned}$$
(2.14)
By using the Cauchy-Schwarz inequality, we have
$$\begin{aligned}& \bigl(\lambda^{k}-\tilde{\lambda}^{k} \bigr)^{\top}\bigl(A_{1}\bigl(u^{k}- \tilde{u}^{k}\bigr)\bigr) \geq-\frac{1}{2} \biggl(\sqrt{3}\bigl\Vert A_{1}\bigl(u^{k}-\tilde{u}^{k}\bigr)\bigr\Vert _{H}^{2} +\frac{1}{\sqrt{3}}\bigl\Vert \lambda^{k}-\tilde{\lambda}^{k}\bigr\Vert _{H^{-1}}^{2} \biggr), \end{aligned}$$
(2.15)
$$\begin{aligned}& \bigl(\lambda^{k}-\tilde{\lambda}^{k} \bigr)^{\top}\bigl(A_{2}\bigl(v^{k}- \tilde{v}^{k}\bigr)\bigr) \geq-\frac{1}{2} \biggl(\sqrt{3}\bigl\Vert A_{2}\bigl(v^{k}-\tilde{v}^{k}\bigr)\bigr\Vert _{H}^{2} +\frac{1}{\sqrt{3}}\bigl\Vert \lambda^{k}-\tilde{\lambda}^{k}\bigr\Vert _{H^{-1}}^{2} \biggr) , \end{aligned}$$
(2.16)
and
$$\begin{aligned} \bigl(\lambda^{k}-\tilde{\lambda}^{k} \bigr)^{\top}\bigl(A_{3}\bigl(w^{k}-\tilde{w}^{k} \bigr)\bigr) \geq-\frac{1}{2} \biggl(\sqrt{3}\bigl\Vert A_{3} \bigl(w^{k}-\tilde{w}^{k}\bigr)\bigr\Vert _{H}^{2} +\frac{1}{\sqrt{3}}\bigl\Vert \lambda^{k}-\tilde{\lambda}^{k}\bigr\Vert _{H^{-1}}^{2} \biggr). \end{aligned}$$
(2.17)
Substituting (2.15), (2.16), and (2.17) into (2.14), we get
$$\begin{aligned} \varphi\bigl(z^{k},\tilde{z}^{k}\bigr) \geq & \frac{2\beta-\sqrt{3}}{2\beta} \bigl(\bigl\Vert A_{1}u^{k}-A_{1} \tilde{u}^{k}\bigr\Vert _{H}^{2} +\bigl\Vert A_{2}v^{k} -A_{2}\tilde{v}^{k} \bigr\Vert _{H}^{2}+\bigl\Vert A_{3}w^{k} -A_{3}\tilde{w}^{k} \bigr\Vert _{H}^{2} \bigr)\\ &{} +\frac{2-\sqrt{3}}{2\beta}\bigl\Vert \lambda^{k}-\tilde{ \lambda}^{k}\bigr\Vert _{H^{-1}}^{2} +\bigl\Vert u^{k}-\tilde{u}^{k}\bigr\Vert _{R_{1}}^{2}+\bigl\Vert v^{k}- \tilde{v}^{k}\bigr\Vert _{R_{2}}^{2} +\bigl\Vert w^{k}-\tilde{w}^{k}\bigr\Vert _{R_{3}}^{2} \\ {\geq}& \frac{2\beta-\sqrt{3}}{2\beta} \biggl(\bigl\Vert A_{1}u^{k}-A_{1} \tilde {u}^{k}\bigr\Vert _{H}^{2}+\bigl\Vert A_{2}v^{k} -A_{2}\tilde{v}^{k} \bigr\Vert _{H}^{2}\\ &{}+\bigl\Vert A_{3}w^{k}-A_{3}\tilde{w}^{k}\bigr\Vert _{H}^{2} + \frac{1}{\beta}\bigl\Vert \lambda^{k}-\tilde{\lambda}^{k} \bigr\Vert _{H^{-1}}^{2} \biggr) \\ &{}+\frac{2\beta-\sqrt{3}}{2\beta} \bigl(\bigl\Vert u^{k}-\tilde{u}^{k} \bigr\Vert _{R_{1}}^{2} +\bigl\Vert v^{k}- \tilde{v}^{k}\bigr\Vert _{R_{2}}^{2}+\bigl\Vert w^{k}-\tilde{w}^{k}\bigr\Vert _{R_{3}}^{2} \bigr) \\ =&\frac{2\beta-\sqrt{3}}{2\beta} \bigl(\bigl\Vert z^{k}-\tilde{z}^{k} \bigr\Vert ^{2}_{G}+(1-\mu)\bigl\Vert u^{k}- \tilde{u}^{k}\bigr\Vert _{R_{1}}^{2}\\ &{} +(1-\mu)\bigl\Vert v^{k}-\tilde{v}^{k}\bigr\Vert _{R_{2}}^{2}+(1- \mu)\bigl\Vert w^{k}-\tilde {w}^{k}\bigr\Vert _{R_{3}}^{2} \bigr) \\ \geq &\frac{2\beta-\sqrt{3}}{2\beta}\bigl\Vert z^{k}-\tilde {z}^{k}\bigr\Vert ^{2}_{G}. \end{aligned}$$
Therefore, it follows from (2.7) and (2.12) that
$$ \tau_{k}\geq\frac{2\beta-\sqrt{3}}{2\beta}, $$
(2.18)
and this completes the proof. □

3 Convergence rate

Recall that \(\mathcal{Z}^{*}\) can be characterized as (see (2.3.2) in p.159 of [19])
$$\mathcal{Z}^{*}=\bigcap_{z\in\mathcal{Z}} \bigl\{ \hat{z}\in \mathcal{Z} : (z-\hat{z})^{\top} Q(z)\geq0 \bigr\} . $$
This implies that is an approximate solution of \(\mathrm{SVI}_{3}\) with the accuracy \(\epsilon>0\) if it satisfies
$$ \hat{z}\in\mathcal{Z}\quad\hbox{and}\quad\sup_{z\in\mathcal {Z}} \bigl\{ (z-\hat{z})^{\top} Q(z)\bigr\} \leq\epsilon. $$
(3.1)
Now we show that after t iterations of the proposed method, we can find a \(\hat{z}\in\mathcal{Z}\) such that (3.1) is satisfied with \(\epsilon= O(1/t)\).
We introduce the following matrices,
$$ N=\left ( \textstyle\begin{array}{c@{\quad}c@{\quad}c@{\quad}c} I & 0& 0 &0 \\ 0 & I & 0& 0 \\ 0 & 0 & I& 0 \\ -\beta HA_{1} & -\beta HA_{2} &-\beta HA_{3} & \beta I \end{array}\displaystyle \right ) $$
(3.2)
and
$$ J=\left ( \textstyle\begin{array}{c@{\quad}c@{\quad}c@{\quad}c} (1+\mu){R_{1}}+A_{1}^{\top} HA_{1} & 0& 0& 0 \\ 0 &(1+\mu){R_{2}}+A_{2}^{\top} H A_{2} & 0& 0 \\ 0 &0 & (1+\mu){R_{3}}+A_{3}^{\top} H A_{3} & 0 \\ -A_{1} & -A_{2} &-A_{3} & H^{-1} \end{array}\displaystyle \right ). $$
(3.3)
By simple manipulations, we can find that \(J= GN\).
Our analysis needs a new sequence defined by
$$ \hat{z}^{k}= \left ( \textstyle\begin{array}{c} \hat{u}^{k} \\ \hat{v}^{k}\\ \hat{w}^{k}\\ \hat{\lambda}^{k} \end{array}\displaystyle \right )=\left ( \textstyle\begin{array}{c} \tilde{u}^{k}\\ \tilde{v}^{k}\\ \tilde{w}^{k}\\ {\lambda}^{k}-H(A_{1}u^{k}+A_{2}v^{k}+A_{3}w^{k}-b) \end{array}\displaystyle \right ). $$
(3.4)
Based on (3.2) and (3.4), we can easily have
$$ z^{k}-\tilde{z}^{k}=N\bigl(z^{k}- \hat{z}^{k}\bigr). $$
(3.5)
Using (1.11), (2.9), and (3.4), we obtain
$$ g\bigl(z^{k},\tilde{z}^{k}\bigr)=Q\bigl( \hat{z}^{k}\bigr). $$
(3.6)

Lemma 3.1

Let \(\hat{z}^{k}\) be defined by (3.4), \(z\in\mathcal{Z}\), and the matrix J be given by (3.3). Then
$$ \bigl(z-\hat{z}^{k}\bigr)^{\top} \bigl(Q\bigl( \hat{z}^{k}\bigr)-J\bigl(z^{k}-\hat{z}^{k}\bigr) \bigr) \geq-\mu\bigl\Vert u^{k} - \hat{u}^{k} \bigr\Vert _{R_{1}}^{2}-\mu\bigl\Vert v^{k} - \hat{v}^{k} \bigr\Vert _{R_{2}}^{2}-\mu\bigl\Vert w^{k} - \hat{w}^{k} \bigr\Vert _{R_{3}}^{2}. $$
(3.7)

Proof

Applying Lemma 2.2 to (2.2), we get
$$\begin{aligned}& \bigl(u - \tilde{u}^{k}\bigr)^{\top} \bigl\{ f_{1}\bigl(\tilde{u}^{k}\bigr) -A_{1}^{\top} \bigl[\lambda^{k} - H\bigl(A_{1} \tilde{u}^{k} + A_{2}{v}^{k}+ A_{3}{w}^{k} -b\bigr) \bigr] \bigr\} \\& \quad \geq\frac{1+\mu}{2} \bigl(\bigl\Vert \tilde{u}^{k}-u \bigr\Vert _{R_{1}}^{2} - \bigl\Vert u^{k} -u\bigr\Vert _{R_{1}}^{2} \bigr) + \frac{1-\mu}{2} \bigl\Vert u^{k} - \tilde{u}^{k} \bigr\Vert _{R_{1}}^{2}. \end{aligned}$$
(3.8)
Since
$$\begin{aligned} \bigl\Vert u^{k}-u\bigr\Vert _{R_{1}}^{2}=\bigl\Vert u^{k}-\tilde{u}^{k}\bigr\Vert _{R_{1}}^{2}+ \bigl\Vert \tilde{u}^{k}-u\bigr\Vert _{R_{1}}^{2}+2 \bigl(\tilde{u}^{k}-u\bigr)^{T}R_{1} \bigl(u^{k}-\tilde{u}^{k}\bigr), \end{aligned}$$
we have
$$ \bigl(u-\tilde{u}^{k}\bigr)^{\top} R_{1}\bigl(u^{k}-\tilde{u}^{k}\bigr) = \frac{1}{2} \bigl(\bigl\Vert \tilde{u}^{k}-u\bigr\Vert _{R_{1}}^{2}-\bigl\Vert u^{k}-u\bigr\Vert _{R_{1}}^{2} \bigr) +{\frac{1}{2}}\bigl\Vert u^{k}-\tilde{u}^{k}\bigr\Vert _{R_{1}}^{2}. $$
(3.9)
Adding (3.8) and (3.9), we obtain
$$\begin{aligned}& \bigl(u - \tilde{u}^{k}\bigr)^{\top} \bigl\{ (1+ \mu)R_{1}\bigl(u^{k}-\tilde{u}^{k}\bigr)- f_{1}\bigl(\tilde{u}^{k}\bigr) + A_{1}^{\top} \bigl[\lambda^{k} - H\bigl(A_{1}u^{k} + A_{2}{v}^{k}+ A_{3}{w}^{k} -b\bigr) \bigr] \\& \quad {}+A_{1}^{\top} HA_{1}\bigl(u^{k}- \tilde{u}^{k}\bigr) \bigr\} \leq\mu\bigl\Vert u^{k} - \tilde {u}^{k} \bigr\Vert _{R_{1}}^{2}. \end{aligned}$$
(3.10)
Similarly, applying Lemma 2.2 to (2.3), we get
$$\begin{aligned}& \bigl(v - \tilde{v}^{k}\bigr)^{\top} \bigl\{ f_{2}\bigl(\tilde{v}^{k}\bigr) - A_{2}^{\top} \bigl[\lambda^{k} - H \bigl(A_{1}u^{k} + A_{2}\tilde{v}^{k}+ A_{3}{w}^{k} -b \bigr)\bigr] \bigr\} \\& \quad \geq\frac{1+\mu}{2} \bigl(\bigl\Vert \tilde{v}^{k}-v \bigr\Vert _{R_{2}}^{2} - \bigl\Vert v^{k} -v\bigr\Vert _{R_{2}}^{2} \bigr) + \frac{1-\mu}{2} \bigl\Vert v^{k} - \tilde{v}^{k} \bigr\Vert _{R_{2}}^{2}. \end{aligned}$$
(3.11)
Similar to (3.9), we have
$$ \bigl(v-\tilde{v}^{k}\bigr)^{\top} {R_{2}}\bigl(v^{k}-\tilde{v}^{k}\bigr)={ \frac{1}{2}} \bigl(\bigl\Vert \tilde{v}^{k}-v\bigr\Vert _{R_{2}}^{2}-\bigl\Vert v^{k}-v\bigr\Vert _{R_{2}}^{2} \bigr) +{\frac{1}{2}}\bigl\Vert v^{k}-\tilde{v}^{k}\bigr\Vert _{R_{2}}^{2}. $$
(3.12)
Adding (3.11) and (3.12), we have
$$\begin{aligned}& \bigl(v - \tilde{v}^{k}\bigr)^{\top} \bigl\{ (1+ \mu){R_{2}}\bigl(v^{k}-\tilde{v}^{k}\bigr)- f_{2}\bigl(\tilde{v}^{k}\bigr) + A_{2}^{\top} \bigl[\lambda^{k} -H\bigl(A_{1}u^{k} + A_{2}{v}^{k}+ A_{3}{w}^{k} -b\bigr) \bigr] \\& \quad {}+A_{2}^{\top} HA_{2}\bigl(v^{k}- \tilde{v}^{k}\bigr) \bigr\} \leq\mu\bigl\Vert v^{k} - \tilde {v}^{k} \bigr\Vert _{R_{2}}^{2}. \end{aligned}$$
(3.13)
Similarly, we have
$$\begin{aligned}& \bigl(w - \tilde{w}^{k}\bigr)^{\top} \bigl\{ (1+ \mu){R_{3}}\bigl(w^{k}-\tilde{w}^{k}\bigr)- f_{3}\bigl(\tilde{w}^{k}\bigr) + A_{3}^{\top} \bigl[\lambda^{k} -H\bigl(A_{1}u^{k} + A_{2}{v}^{k}+ A_{3}{w}^{k} -b\bigr) \bigr] \\& \quad {}+A_{3}^{\top} HA_{3}\bigl(w^{k}- \tilde{w}^{k}\bigr) \bigr\} \le\mu\bigl\Vert w^{k} - \tilde {w}^{k} \bigr\Vert _{R_{3}}^{2}. \end{aligned}$$
(3.14)
Then, by using the notation of \(\hat{z}^{k}\) in (3.4), (3.10), (3.13), and (3.14) can be written as
$$\begin{aligned}& \bigl(u - \hat{u}^{k}\bigr)^{\top} \bigl\{ (1+\mu)R_{1}\bigl(u^{k}-\hat{u}^{k}\bigr)- f_{1}(\hat {u}^{k})+A_{1}^{\top} \hat{\lambda}^{k} +A_{1}^{\top} HA_{1}\bigl(u^{k}-\hat{u}^{k}\bigr) \bigr\} \\& \quad \leq\mu\bigl\Vert u^{k} - \hat{u}^{k} \bigr\Vert _{R_{1}}^{2}, \end{aligned}$$
(3.15)
$$\begin{aligned}& \bigl(v - \hat{v}^{k}\bigr)^{\top} \bigl\{ (1+ \mu){R_{2}}\bigl(v^{k}-\hat{v}^{k}\bigr)- f_{2}\bigl(\hat{v}^{k}\bigr)+A_{2}^{\top} \hat{\lambda}^{k} +A_{2}^{\top} HA_{2} \bigl(v^{k}-\hat{v}^{k}\bigr) \bigr\} \\& \quad \le\mu\bigl\Vert v^{k} - \hat{v}^{k} \bigr\Vert _{R_{2}}^{2}, \end{aligned}$$
(3.16)
and
$$\begin{aligned}& \bigl(w - \hat{w}^{k}\bigr)^{\top} \bigl\{ (1+ \mu){R_{3}}\bigl(w^{k}-\hat{w}^{k}\bigr)- f_{3}\bigl(\hat{w}^{k}\bigr)+A_{3}^{\top} \hat{\lambda}^{k} +A_{3}^{\top} HA_{3} \bigl(w^{k}-\hat{w}^{k}\bigr) \bigr\} \\& \quad \le\mu\bigl\Vert w^{k} - \hat{w}^{k} \bigr\Vert _{R_{3}}^{2}. \end{aligned}$$
(3.17)
In addition, it follows from (3.4) that
$$\begin{aligned}& A_{1}\hat{u}^{k}+ A_{2} \hat{v}^{k}+A_{3}\hat{w}^{k}-b +H^{-1} \bigl(\hat{\lambda }^{k}-{\lambda}^{k}\bigr) \\& \quad {}-A_{1} \bigl(\hat{u}^{k}-u^{k}\bigr)-A_{2}\bigl( \hat{v}^{k}-v^{k}\bigr)-A_{3}\bigl( \hat{w}^{k}-w^{k}\bigr)=0. \end{aligned}$$
(3.18)
Combining (3.15)-(3.18), we get
$$\begin{aligned}& \left ( \textstyle\begin{array}{c} u - \hat{u}^{k} \\ v - \hat{v}^{k}\\ w - \hat{w}^{k}\\ \lambda- \hat{\lambda}^{k} \end{array}\displaystyle \right )^{\top} \left ( \textstyle\begin{array}{c} f_{1}(\hat{u}^{k})- A_{1}^{\top} \hat{\lambda}^{k}- ((1+\mu ){R_{1}}+A^{T}HA_{1} )(u^{k}-\hat{u}^{k})\\ f_{2}(\hat{v}^{k}) - A_{2}^{\top} \hat{\lambda}^{k}- ((1+\mu ){R_{2}}+A_{2}^{\top}HA_{2} )(v^{k}-\hat{v}^{k})\\ f_{3}(\hat{w}^{k}) - A_{3}^{\top} \hat{\lambda}^{k}- ((1+\mu ){R_{3}}+A_{3}^{\top}HA_{3} )(w^{k}-\hat{w}^{k})\\ A_{1}\hat{u}^{k}+A_{2}\hat{v}^{k}+A_{3}\hat{w}^{k} -b +A_{1}(u^{k}-\hat{u}^{k})+A_{2}(v^{k}-\hat{v}^{k})+A_{3}(w^{k}-\hat {w}^{k})-H^{-1}(\lambda^{k}-\hat{\lambda}^{k}) \end{array}\displaystyle \right ) \\& \quad \geq-\mu\bigl\Vert u^{k} - \hat{u}^{k} \bigr\Vert _{R_{1}}^{2}-\mu\bigl\Vert v^{k} - \hat{v}^{k} \bigr\Vert _{R_{2}}^{2}-\mu\bigl\Vert w^{k} - \hat{w}^{k} \bigr\Vert _{R_{3}}^{2}. \end{aligned}$$
(3.19)
Recall the definition of J in (3.3), we obtain the assertion (3.7). The proof is completed. □

Lemma 3.2

For given \(z^{k}\in \mathbb{R}^{n_{1}}_{++}\times\mathbb{R}^{n_{2}}_{++}\times \mathbb{R}^{n_{3}}_{++}\times\mathbb{R}^{m}\) and \(z_{*}^{k} := P_{\mathcal{Z},G}[z^{k}-\tau_{k} G^{-1}g(z^{k},\tilde{z}^{k})]\), we have
$$ \gamma\tau_{k}\bigl(z-\hat{z}^{k} \bigr)^{\top} Q(z)+\frac{1}{2}\bigl(\bigl\Vert z-z^{k} \bigr\Vert _{G}^{2}-\bigl\Vert z-z_{*}^{k} \bigr\Vert _{G}^{2}\bigr)\geq\frac{1}{2} \gamma(2-\gamma) \tau_{k}^{2}\bigl\Vert z^{k}-\tilde{z}^{k} \bigr\Vert _{G}^{2}. $$
(3.20)

Proof

Since \(z_{*}^{k}\in\mathcal{Z}\), substituting \(z=z_{*}^{k}\) in (3.7), we get
$$\begin{aligned}& \gamma\tau_{k}\bigl(z_{*}^{k}- \hat{z}^{k}\bigr)^{\top}Q\bigl(\hat{z}^{k}\bigr) \end{aligned}$$
(3.21)
$$\begin{aligned}& \quad \geq \gamma\tau_{k}\bigl(z_{*}^{k}- \hat{z}^{k}\bigr)^{\top} J\bigl(z^{k}- \hat{z}^{k}\bigr) - \mu\gamma\tau_{k}\bigl\Vert u^{k} - \hat{u}^{k} \bigr\Vert _{R_{1}}^{2}- \mu\gamma\tau_{k}\bigl\Vert v^{k} - \hat{v}^{k} \bigr\Vert _{R_{2}}^{2} \\& \qquad {} -\mu\gamma\tau_{k}\bigl\Vert w^{k} - \hat{w}^{k}\bigr\Vert _{R_{3}}^{2} \\& \quad = \gamma\tau_{k}\bigl(z^{k}-\hat{z}^{k} \bigr)^{\top} J\bigl(z^{k}-\hat{z}^{k}\bigr)+\gamma \tau _{k}\bigl(z_{*}^{k}-z^{k} \bigr)^{\top} J\bigl(z^{k}-\hat{z}^{k}\bigr) \\& \qquad {} -\mu\gamma\tau_{k}\bigl\Vert u^{k} - \hat{u}^{k} \bigr\Vert _{R_{1}}^{2}-\mu\gamma\tau _{k}\bigl\Vert v^{k} - \hat{v}^{k} \bigr\Vert _{R_{2}}^{2} -\mu\gamma\tau_{k}\bigl\Vert w^{k} - \hat{w}^{k}\bigr\Vert _{R_{3}}^{2} \\& \quad = \gamma\tau_{k}\bigl(z^{k}-\tilde{z}^{k} \bigr)^{\top} \bigl(N^{-1}\bigr)^{\top} JN^{-1}\bigl(z^{k}-\tilde{z}^{k}\bigr) +\gamma \tau_{k}\bigl(z_{*}^{k}-z^{k} \bigr)^{\top} JN^{-1}\bigl(z^{k}-\tilde{z}^{k} \bigr) \\& \qquad {} -\gamma\tau_{k}\mu\bigl\Vert u^{k} - \hat{u}^{k} \bigr\Vert _{R_{1}}^{2} -\gamma \tau_{k}\mu\bigl\Vert v^{k} - \hat{v}^{k} \bigr\Vert _{R_{2}}^{2}-\mu\gamma\tau_{k}\bigl\Vert w^{k} - \hat{w}^{k}\bigr\Vert _{R_{3}}^{2} \\& \quad = \gamma\tau_{k}\bigl(z^{k}-\tilde{z}^{k} \bigr)^{\top} \bigl(N^{-1}\bigr)^{\top} G \bigl(z^{k}-\tilde{z}^{k}\bigr) -\gamma\tau_{k}\mu \bigl\Vert u^{k} - \hat{u}^{k} \bigr\Vert _{R_{1}}^{2} -\gamma\tau_{k}\mu\bigl\Vert v^{k} - \hat{v}^{k} \bigr\Vert _{R_{2}}^{2} \\& \qquad {} -\mu\gamma\tau_{k}\bigl\Vert w^{k} - \hat{w}^{k}\bigr\Vert _{R_{3}}^{2} +\gamma \tau_{k}\bigl(z_{*}^{k}-z^{k} \bigr)^{\top} G\bigl(z^{k}-\tilde{z}^{k}\bigr) \\& \quad = \gamma\tau_{k}\bigl(z^{k}-\tilde{z}^{k} \bigr)^{\top} \bigl(N^{-1}\bigr)^{\top} M \bigl(z^{k}-\tilde{z}^{k}\bigr) +\gamma\tau_{k} \bigl(z_{*}^{k}-z^{k}\bigr)^{\top} G \bigl(z^{k}-\tilde{z}^{k}\bigr) \\& \quad \geq \gamma\tau_{k}\varphi\bigl(z^{k},\tilde{z}^{k} \bigr)+\gamma\tau _{k}\bigl(z_{*}^{k}-z^{k} \bigr)^{\top} G\bigl(z^{k}-\tilde{z}^{k}\bigr) \\& \quad \geq \gamma\tau_{k}\varphi\bigl(z^{k},\tilde{z}^{k} \bigr)-\frac{1}{2}\bigl\Vert z^{k}-z_{*}^{k}\bigr\Vert _{G}^{2} -\frac{1}{2}\gamma^{2} \tau_{k}^{2}\bigl\Vert z^{k}-\tilde{z}^{k} \bigr\Vert _{G}^{2} \\& \quad = \frac{1}{2} \gamma(2-\gamma)\tau_{k}^{2}\bigl\Vert z^{k}-\tilde{z}^{k}\bigr\Vert _{G}^{2}- \frac{1}{2}\bigl\Vert z^{k}-z_{*}^{k}\bigr\Vert _{G}^{2}. \end{aligned}$$
(3.22)
Using (3.6), \(z_{*}^{k}\) is the projection of \(z^{k}-\gamma\tau_{k} G^{-1}Q(\hat{z}^{k})\) on \(\mathcal{Z}\), it follows from (2.1) that
$$\bigl(z^{k}-\gamma\tau_{k} G^{-1}Q\bigl( \hat{z}^{k}\bigr)-z_{*}^{k}\bigr)^{\top} G \bigl(z-z_{*}^{k}\bigr)\leq0, \quad\forall z\in\mathcal{Z}, $$
and consequently, we have
$$\gamma\tau_{k} \bigl(z-z_{*}^{k} \bigr)^{\top} Q\bigl(\hat{z}^{k}\bigr)\geq \bigl(z^{k}-z_{*}^{k} \bigr)^{\top} G\bigl(z-z_{*}^{k}\bigr). $$
Using the identity \(x^{\top} Gy =\frac{1}{2} (\Vert x\Vert _{G}^{2}-\Vert x-y\Vert _{G}^{2}+\Vert y\Vert _{G}^{2} )\) to the right hand side of the last inequality, we obtain
$$ \gamma\tau_{k} \bigl(z-z_{*}^{k} \bigr)^{\top} Q\bigl(\hat{z}^{k}\bigr)\geq\frac{1}{2} \bigl(\bigl\Vert z-z_{*}^{k}\bigr\Vert _{G}^{2}-\bigl\Vert z-z^{k}\bigr\Vert _{G}^{2} \bigr) +\frac{1}{2}\bigl\Vert z^{k}-z_{*}^{k}\bigr\Vert _{G}^{2}. $$
(3.23)
Adding (3.21) and (3.23), we get
$$\gamma\tau_{k}\bigl(z-\hat{z}^{k}\bigr)^{\top} Q \bigl(\hat{z}^{k}\bigr)+\frac{1}{2}\bigl(\bigl\Vert z-z^{k} \bigr\Vert _{G}^{2}-\bigl\Vert z-z_{*}^{k} \bigr\Vert _{G}^{2}\bigr)\geq\frac{1}{2} \gamma(2-\gamma) \tau_{k}^{2}\bigl\Vert z^{k}-\tilde{z}^{k} \bigr\Vert _{G}^{2}, $$
and by using the monotonicity of Q, we obtain (3.20) and the proof is completed. □

Lemma 3.3

Let \(z^{k}\in \mathbb{R}^{n_{1}}_{++}\times \mathbb{R}^{n_{2}}_{++}\times\mathbb{R}^{n_{3}}_{++}\times\mathbb {R}^{m}\) and \(z^{k+1}(\tau_{k})\) be generated by (2.6). Then
$$ \gamma\sigma\tau_{k}\bigl(z-\hat{z}^{k} \bigr)^{\top} Q(z)+\frac{1}{2}\bigl(\bigl\Vert z-z^{k}\bigr\Vert _{G}^{2}-\bigl\Vert z-z^{k+1}( \tau_{k})\bigr\Vert _{G}^{2}\bigr) \geq \frac{1}{2} \sigma\gamma(2-\gamma)\tau_{k}^{2}\bigl\Vert z-\tilde{z}^{k}\bigr\Vert _{G}^{2}. $$
(3.24)

Proof

We have
$$\begin{aligned}& \bigl\Vert z-z^{k}\bigr\Vert _{G}^{2}- \bigl\Vert z-z^{k+1}(\tau_{k})\bigr\Vert _{G}^{2} \end{aligned}$$
(3.25)
$$\begin{aligned}& \quad = \bigl\Vert z^{k}-z\bigr\Vert _{G}^{2}- \bigl\Vert z^{k}-\sigma\bigl(z^{k}-z_{*}^{k} \bigr)-z\bigr\Vert _{G}^{2} \\& \quad = 2\sigma\bigl(z^{k}-z\bigr)^{\top} G\bigl(z^{k}-z_{*}^{k} \bigr)-\sigma^{2}\bigl\Vert z^{k}-z_{*}^{k} \bigr\Vert _{G}^{2} \\& \quad = 2\sigma \bigl(\bigl\Vert z^{k}-z_{*}^{k} \bigr\Vert _{G}^{2}-\bigl( z-z_{*}^{k} \bigr)^{\top} G\bigl(z^{k}-z_{*}^{k}\bigr) \bigr)-\sigma^{2}\bigl\Vert z^{k}-z_{*}^{k} \bigr\Vert _{G}^{2}. \end{aligned}$$
(3.26)
Using the identity
$$\begin{aligned} \bigl(z-z_{*}^{k}\bigr)^{\top} G \bigl(z^{k}-z_{*}^{k}\bigr) = &{ \frac{1}{2}} \bigl(\bigl\Vert z_{*}^{k}-z\bigr\Vert _{G}^{2}- \bigl\Vert z^{k}-z\bigr\Vert _{G}^{2} \bigr) +{\frac{1}{2}}\bigl\Vert z^{k}-z_{*}^{k}\bigr\Vert _{G}^{2}, \end{aligned}$$
we get
$$ \bigl\Vert z^{k}-z_{*}^{k}\bigr\Vert _{G}^{2}-2\bigl(z-z_{*}^{k} \bigr)^{\top} G\bigl(z^{k}-z_{*}^{k}\bigr)= \bigl\Vert z^{k}-z\bigr\Vert _{G}^{2}-\bigl\Vert z_{*}^{k}-z\bigr\Vert _{G}^{2}. $$
(3.27)
Substituting (3.27) into (3.25), we obtain
$$\begin{aligned} \bigl\Vert z-z^{k}\bigr\Vert _{G}^{2}- \bigl\Vert z-z^{k+1}(\tau_{k})\bigr\Vert _{G}^{2} =&\sigma\bigl(\bigl\Vert z-z^{k}\bigr\Vert _{G}^{2}-\bigl\Vert z-z_{*}^{k}\bigr\Vert _{G}^{2}\bigr) +\sigma(1-\sigma)\bigl\Vert z^{k}-z_{*}^{k}\bigr\Vert _{G}^{2} \\ \geq&\sigma\bigl(\bigl\Vert z-z^{k}\bigr\Vert _{G}^{2}-\bigl\Vert z-z_{*}^{k}\bigr\Vert _{G}^{2}\bigr). \end{aligned}$$
(3.28)
Substituting (3.28) into (3.20), we obtain (3.24), the required result. □

Theorem 3.1

Let \(z^{*}\) be a solution of \(\mathrm{SVI}_{3}\) and \(z^{k+1}(\tau_{k})\) be generated by (2.6). Then \(z^{k}\) and \(\tilde{z}^{k}\) are bounded, and
$$ \bigl\Vert z^{k+1}(\tau_{k}) - z^{*}\bigr\Vert _{G}^{2}\leq\bigl\Vert z^{k} -z^{*}\bigr\Vert _{G}^{2}-c\bigl\Vert z^{k}-\tilde {z}^{k}\bigr\Vert ^{2}_{G}, $$
(3.29)
where
$$c:={ \frac{\sigma\gamma(2-\gamma)(2\beta-\sqrt {3})^{2}}{4\beta^{2}}}>0. $$

Proof

Setting \(z=z^{*}\) in (3.24), we obtain
$$\begin{aligned} \bigl\Vert z^{k+1}(\tau_{k}) - z^{*}\bigr\Vert _{G}^{2} \leq&\bigl\Vert z^{k} -z^{*}\bigr\Vert _{G}^{2}-\sigma\gamma(2-\gamma)\tau_{k}^{2} \bigl\Vert z^{k}-\tilde {z}^{k}\bigr\Vert ^{2}_{G}+ 2\gamma\sigma\tau_{k}\bigl(z^{*}-\hat{z}^{k} \bigr)^{\top} Q\bigl(z^{*}\bigr) \\ \leq&\bigl\Vert z^{k} -z^{*}\bigr\Vert _{G}^{2}- \sigma\gamma(2-\gamma)\tau_{k}^{2}\bigl\Vert z^{k}-\tilde {z}^{k}\bigr\Vert ^{2}_{G} \\ \leq&\bigl\Vert z^{k} -z^{*}\bigr\Vert _{G}^{2}- \frac{\sigma\gamma(2-\gamma)(2\beta-\sqrt {3})^{2}}{4\beta^{2}}\bigl\Vert z^{k}-\tilde{z}^{k}\bigr\Vert ^{2}_{G}. \end{aligned}$$
Then we have
$$\bigl\Vert z^{k+1}(\tau_{k})-z^{*}\bigr\Vert _{G}\leq\bigl\Vert z^{k}-z^{*}\bigr\Vert _{G} \leq\cdots\leq\bigl\Vert z^{0}-z^{*}\bigr\Vert _{G}, $$
and thus, \(\{ z^{k}\}\) is a bounded sequence.
It follows from (3.29) that
$$\sum_{k=0}^{\infty}c\bigl\Vert z^{k}-\tilde{z}^{k}\bigr\Vert ^{2}_{G}< + \infty, $$
which means that
$$ \lim_{k\to\infty}\bigl\Vert z^{k}- \tilde{z}^{k}\bigr\Vert _{G}=0. $$
(3.30)
Since \(\{ z^{k}\}\) is a bounded sequence, we conclude that \(\{\tilde {z}^{k}\}\) is also bounded. □

The global convergence of the proposed method can be proved by similar arguments as in [18]. Hence the proof is omitted.

Theorem 3.2

The sequence \(\{z^{k}\}\) generated by the proposed method converges to some \(z^{\infty}\) which is a solution of \(\mathrm{SVI}_{3}\).

Now, we are ready to present the \(O(1/t)\) convergence rate of the proposed method.

Theorem 3.3

For any integer \(t > 0\), we have a \(\hat{z}_{t}\in\mathcal{Z}\) which satisfies
$$(\hat{z}_{t}-z)^{\top} Q(z)\leq\frac{1}{2\gamma\sigma\Upsilon_{t}}\bigl\Vert z-z^{0}\bigr\Vert _{G}^{2},\quad\forall z\in \mathcal{Z}, $$
where
$$\hat{z}_{t}=\frac{1}{\Upsilon_{t}}\sum_{k=0}^{t} \tau_{k}\hat{z}^{k}\quad \textit{and} \quad\Upsilon_{t}= \sum_{k=0}^{t}\tau_{k}. $$

Proof

Summing the inequality (3.24) over \(k=0,\ldots,t\), we obtain
$$\Biggl( \Biggl(\sum_{k=0}^{t}\gamma\sigma \tau_{k} \Biggr) z-\sum_{k=0}^{t} \gamma\sigma\tau_{k}\hat{z}^{k} \Biggr)^{\top} Q(z)+ \frac{1}{2}\bigl\Vert z-z^{0}\bigr\Vert _{G}^{2} \geq0. $$
Using the notations of \(\Upsilon_{t}\) and \(\hat{z}_{t}\) in the above inequality, we derive
$$(\hat{z}_{t}-z)^{\top} Q(z)\leq\frac{1}{2\gamma\sigma\Upsilon_{t}}\bigl\Vert z-z^{0}\bigr\Vert _{G}^{2},\quad\forall z\in \mathcal{Z}. $$
Indeed, \(\hat{z}_{t}\in\mathcal{Z}\) because it is a convex combination of \(\hat{z}^{0}\), \(\hat{z}^{1}\), … , \(\hat{z}^{t}\). The proof is completed. □

Remark 3.1

It follows from (2.13) that
$$\Upsilon_{t}\geq\frac{2\beta-\sqrt{3}}{2\beta} (t+1 ). $$
Suppose that, for any compact set \(\mathcal {D}\subset\mathcal{Z}\), let \(d =\sup\{\Vert z-z^{0}\Vert _{G}\vert z\in\mathcal {D}\}\). For any given \(\epsilon>0\), after at most
$$t= \biggl[\frac{\beta d^{2}}{(2\beta-\sqrt{3})\gamma\sigma\epsilon } \biggr] $$
iterations, we have
$$(\hat{z}_{t}-z)^{T}Q(z)\leq\epsilon,\quad \forall z\in\mathcal {D}. $$
That is, the \(O(1/t)\) convergence rate is established in an ergodic sense.

4 Preliminary computational results

In this section, we present some numerical examples to illustrate the proposed method. We consider the following optimization problem with matrix variables, which is studied in [18]:
$$ \min \biggl\{ \frac{1}{2} \Vert U-C\Vert _{F}^{2}\Big\vert U\in S_{+}^{n} \biggr\} , $$
(4.1)
where \(\Vert \cdot \Vert _{F}\) is the matrix Fröbenius norm, i.e., \(\Vert C\Vert _{F} = ( \sum_{i=1}^{n}\sum_{j=1}^{n} \vert C_{ij}\vert ^{2} )^{1/2}\) and
$$S_{+}^{n}=\bigl\{ M\in\mathbb{R}^{n\times n} : M^{\top} =M, M \succeq 0 \bigr\} . $$
It has been shown in [18] that solving problem (4.1) is equivalent to the following variational inequality problem: Find \(X^{*}=(U^{*},V^{*},Z^{*})\in\Omega= S_{+}^{n} \times S_{+}^{n}\times \mathbb{R}^{n\times n}\) such that
$$\begin{aligned} \textstyle\begin{cases} \langle U-U^{*},(U^{*}-C)-Z^{*}\rangle\geq0, \\ \langle V-V^{*},(V^{*}-C)+Z^{*}\rangle\geq0,&\forall X=(U,V,Z)\in \Omega,\\ U^{*} - V^{*} =0. \end{cases}\displaystyle \end{aligned}$$
(4.2)
The problem (4.2) is a special case of (1.3)-(1.4) with matrix variables where \(A_{1}=I_{n\times n}\), \(A_{2}=-I_{n\times n}\), \(b=0\), \(f_{1}(U)=U-C\), \(f_{2}(V)=V-C\), and \(\mathcal{W} =S_{+}^{n} \times S_{+}^{n}\times\mathbb{R}^{n\times n}\). For simplification, we take \(R_{1}=r_{1}I_{n\times n}\), \(R_{2}=r_{2}I_{n\times n}\), and \(H=I_{n\times n}\) where \(r_{1}>0\) and \(r_{2}>0\) are scalars. In all tests we take \(\mu=0.5\), \(\beta=0.88\), \(C=\operatorname{rand}(n)\), and \((U^{0}, V^{0}, Z^{0})=(I_{n\times n},I_{n\times n},0_{n\times n})\) as the initial point in the test. The iteration is stopped as soon as
$$\max\bigl\{ \bigl\Vert U^{k}-\tilde{U}^{k}\bigr\Vert , \bigl\Vert V^{k}-\tilde{V}^{k}\bigr\Vert ,\bigl\Vert Z^{k}-\tilde{Z}^{k}\bigr\Vert \bigr\} \leq10^{-6}. $$
All codes were written in Matlab, we compare the proposed method with those in [18] and [17]. The iteration numbers, denoted by k, and the computational time for the problem (4.1) with different dimensions are given in Tables 1-2.
Table 1

Numerical results for problem ( 4.1 ) with \(\pmb{r_{1}=0.5}\) , \(\pmb{r_{2}=5}\)

Dimension of the problem

The proposed method

The method in [ 18 ]

The method in [ 17 ]

k

CPU (Sec.)

k

CPU (Sec.)

k

CPU (Sec.)

100

43

0.83

49

0.96

71

2.47

300

48

3.98

53

4.85

79

6.33

500

50

11.54

56

13.27

82

20.2

700

52

29.91

57

34.33

85

44.06

Table 2

Numerical results for problem ( 4.1 ) with \(\pmb{r_{1}=1}\) , \(\pmb{r_{2}=10}\)

Dimension of the problem

The proposed method

The method in [ 18 ]

The method in [ 17 ]

k

CPU (Sec.)

k

CPU (Sec.)

k

CPU (Sec.)

100

106

0.87

109

1.18

124

2.61

300

119

6.85

123

7.54

140

9.06

500

125

25.85

128

29.71

147

37.25

700

129

53.19

132

58.06

152

64.35

Tables 1-2 show that the proposed method is more flexible and efficient for the problem tested.

5 Conclusions

In this paper, we proposed a new modified logarithmic-quadratic proximal alternating direction method for solving structured variational inequalities with three separable operators. The prediction point is obtained by solving series of related systems of nonlinear equations in a parallel way. Global convergence of the proposed method is proved under mild assumptions. We have proved the \(O(1/t)\) convergence rate of the parallel LQP alternating direction method. Some preliminary numerical examples are reported to verify the effectiveness of the proposed method in practice.

Notes

Declarations

Acknowledgements

In this research, the second author was funded by the Deanship of Scientific Research (DSR), King Abdulaziz University, Jeddah. Therefore, he acknowledges with thanks DSR for the technical and financial support. The research part of the third author was done during his visit to King Fahd University of Petroleum & Minerals (KFUPM), Dhahran, Saudi Arabia. He is grateful to KFUPM for providing excellent research facilities to carry out his part of this research.

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors’ Affiliations

(1)
School of Management Science and Engineering, Nanjing University
(2)
Laboratoire d’Ingénierie des Systémes et Technologies de l’Information, ENSA, Ibn Zohr University
(3)
Department of Mathematics, King Abdulaziz University
(4)
Department of Mathematics, Aligarh Muslim University
(5)
Department of Mathematics & Statistics, King Fahd University of Petroleum & Minerals

References

  1. Fortin, M, Glowinski, R: Augmented Lagrangian Methods: Applications to the Solution of Boundary-Valued Problems. North-Holland, Amsterdam (1983) MATHGoogle Scholar
  2. Gabay, D: Applications of the method of multipliers to variational inequalities. In: Fortin, M, Glowinski, R (eds.) Augmented Lagrangian Methods: Applications to the Solution of Boundary-Valued Problems. Studies in Mathematics and Its Applications, vol. 15, pp. 299-331. North-Holland, Amsterdam, The Netherlands (1983) View ArticleGoogle Scholar
  3. Gabay, D, Mercier, B: A dual algorithm for the solution of nonlinear variational problems via finite-element approximations. Comput. Math. Appl. 2, 17-40 (1976) View ArticleMATHGoogle Scholar
  4. Glowinski, R: Numerical Methods for Nonlinear Variational Problems. Springer, New York (1984) View ArticleMATHGoogle Scholar
  5. Glowinski, R, Le Tallec, P: Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics. SIAM Studies in Applied Mathematics. SIAM, Philadelphia, PA (1989) View ArticleMATHGoogle Scholar
  6. He, BS, Yang, H: Some convergence properties of a method of multipliers for linearly constrained monotone variational inequalities. Oper. Res. Lett. 23, 151-161 (1998) MathSciNetView ArticleMATHGoogle Scholar
  7. Hou, LS: On the \(O(1/t)\) convergence rate of the parallel descent-like method and parallel splitting augmented Lagrangian method for solving a class of variational inequalities. Appl. Math. Comput. 219, 5862-5869 (2013) MathSciNetMATHGoogle Scholar
  8. Jiang, ZK, Bnouhachem, A: A projection-based prediction-correction method for structured monotone variational inequalities. Appl. Math. Comput. 202, 747-759 (2008) MathSciNetMATHGoogle Scholar
  9. Kontogiorgis, S, Meyer, RR: A variable-penalty alternating directions method for convex optimization. Math. Program. 83, 29-53 (1998) MathSciNetMATHGoogle Scholar
  10. Tao, M, Yuan, XM: On the \(O(1/t)\) convergence rate of alternating direction method with logarithmic-quadratic proximal regularization. SIAM J. Optim. 22(4), 1431-1448 (2012) MathSciNetView ArticleMATHGoogle Scholar
  11. Wang, K, Xu, LL, Han, DR: A new parallel splitting descent method for structured variational inequalities. J. Ind. Manag. Optim. 10(2), 461-476 (2014) MathSciNetView ArticleMATHGoogle Scholar
  12. Yuan, XM, Li, M: An LQP-based decomposition method for solving a class of variational inequalities. SIAM J. Optim. 21(4), 1309-1318 (2011) MathSciNetView ArticleMATHGoogle Scholar
  13. Bnouhachem, A: On LQP alternating direction method for solving variational. J. Inequal. Appl. 2014, 80 (2014) View ArticleMATHGoogle Scholar
  14. Bnouhachem, A, Ansari, QH: A descent LQP alternating direction method for solving variational inequality problems with separable structure. Appl. Math. Comput. 246, 519-532 (2014) MathSciNetMATHGoogle Scholar
  15. Bnouhachem, A, Benazza, H, Khalfaoui, M: An inexact alternating direction method for solving a class of structured variational inequalities. Appl. Math. Comput. 219, 7837-7846 (2013) MathSciNetMATHGoogle Scholar
  16. Bnouhachem, A, Xu, MH: An inexact LQP alternating direction method for solving a class of structured variational inequalities. Comput. Math. Appl. 67, 671-680 (2014) MathSciNetView ArticleMATHGoogle Scholar
  17. Li, M: A hybrid LQP-based method for structured variational inequalities. J. Comput. Math. 89(10), 1412-1425 (2012) MathSciNetMATHGoogle Scholar
  18. Bnouhachem, A, Hamdi, A: Parallel LQP alternating direction method for solving variational inequality problems with separable structure. J. Inequal. Appl. 2014, 392 (2014) MathSciNetView ArticleMATHGoogle Scholar
  19. Facchinei, F, Pang, JS: Finite-Dimensional Variational Inequalities and Complementarity Problems, Vol. I and II. Springer Series in Operations Research. Springer, New York (2003) MATHGoogle Scholar

Copyright

© Bnouhachem et al. 2016