# On the $$O(1/t)$$ convergence rate of the alternating direction method with LQP regularization for solving structured variational inequality problems

## Abstract

In this paper, we propose a parallel descent LQP alternating direction method for solving structured variational inequality with three separable operators. The $$O(1/t)$$ convergence rate for this method is studied. We also present some numerical examples to illustrate the efficiency of the proposed method. The results presented in this paper extend and improve some well-known results in the literature.

## Introduction

Let $$\mathbb{R}^{n}_{+} = \{ x = (x_{1},x_{2}, \ldots,x_{n}) \in \mathbb{R}^{n} : x_{i} \geq0\ \forall i=1,2, \ldots, n \}$$ and $$\mathbb{R}^{n}_{++} = \{ x = (x_{1},x_{2}, \ldots,x_{n}) \in\mathbb {R}^{n} : x_{i} > 0 \ \forall i=1,2, \ldots, n \}$$. The variational inequality problem is to find

$$x\in\Omega:= \bigl\{ (u,v) : u\in\mathbb{R}_{+}^{n}, v\in \mathbb{R}_{+}^{m}, A_{1}u + A_{2}v = b \bigr\}$$

such that

$$\bigl(x'- x\bigr)^{T} F(x) \ge0, \quad \forall x'\in\Omega,$$
(1.1)

with

$$x=\left ( \textstyle\begin{array}{c} u \\ v \end{array}\displaystyle \right ) \quad \mbox{and} \quad F(x)=\left ( \textstyle\begin{array}{c} f_{1}(u)\\ f_{2}(v) \end{array}\displaystyle \right ),$$
(1.2)

where $$A_{1}\in\mathbb{R}^{l\times n}$$, $$A_{2} \in\mathbb{R}^{l\times m}$$ are given matrices, $$b\in\mathbb{R}^{l}$$ is a given vector, and $$f_{1}:\mathbb{R}_{+}^{n}\to \mathbb{R}^{n}$$, $$f_{2}:\mathbb{R}_{+}^{m}\to\mathbb{R}^{m}$$ are given monotone operators. For further study and applications of such problems, we refer to  and the references therein. By attaching a Lagrange multiplier vector $$\lambda\in\mathbb{R}^{l}$$ to the linear constraints $$A_{1}u + A_{2}v =b$$, the problem (1.1)-(1.2) can be explained in terms of finding $$z \in\mathcal{Z}' := \mathbb{R}_{+}^{n} \times\mathbb{R}_{+}^{m} \times \mathbb{R}^{l}$$ such that

$$\bigl(z'-z\bigr)^{\top} Q(z) \geq0, \quad \forall z' \in\mathcal{Z}',$$
(1.3)

where

$$z=\left ( \textstyle\begin{array}{c} u \\ v\\ \lambda \end{array}\displaystyle \right ), \qquad Q(z)= \left ( \textstyle\begin{array}{c} f_{1}(u)-A_{1}^{\top} \lambda\\ f_{2}(v)-A_{2}^{\top} \lambda\\ A_{1}u + A_{2}v-b \end{array}\displaystyle \right ),$$
(1.4)

and $$A_{1}^{\top}$$ denotes the transpose of the matrix $$A_{1}$$. The problem (1.3)-(1.4) is referred as a structured variational inequality problem (in short, SVI).

Yuan and Li  developed the following logarithmic-quadratic proximal (LQP)-based decomposition method by applying the LQP terms to regularize the ADM subproblems: For a given $$z^{k}=(u^{k},v^{k},\lambda^{k})\in\mathbb{R}^{n}_{++}\times\mathbb {R}^{m}_{++}\times\mathbb{R}^{l}$$, and $$\mu\in(0,1)$$, the new iterative $$(u^{k+1}, v^{k+1}, \lambda^{k+1})$$ is obtained via solving the following system:

\begin{aligned}& f_{1}(u) - A_{1}^{\top} \bigl[ \lambda^{k} - H \bigl(A_{1} u + A_{2} v^{k} - b\bigr) \bigr] + R \bigl[ \bigl(u-u^{k}\bigr) + \mu \bigl(u^{k} - U_{k}^{2} u^{-1}\bigr) \bigr] = 0, \end{aligned}
(1.5)
\begin{aligned}& f_{2}(v) - A_{2}^{\top} \bigl[ \lambda^{k} - H (A_{1} {u} + A_{2} v - b) \bigr] + S \bigl[ \bigl(v-v^{k}\bigr) + \mu\bigl(v^{k} - V_{k}^{2} v^{-1}\bigr) \bigr] = 0, \end{aligned}
(1.6)
\begin{aligned}& \lambda^{k+1} = \lambda^{k} - H \bigl(A_{1} u^{k} + A_{2} v^{k} - b\bigr), \end{aligned}
(1.7)

where $$H \in\mathbb{R}^{l \times l}$$, $$R \in\mathbb{R}^{n \times n}$$, and $$S \in\mathbb{R}^{m \times m}$$ are symmetric positive definite.

Later, some LQP alternating direction methods have been proposed to make the LQP alternating direction method more practical, see, for example,  and the references therein. Each iteration of these methods contains a prediction and a correction, the predictor is obtained via solving (1.5)-(1.7) and the new iterate is obtained by a convex combination of the previous point and the one generated by a projection-type method along a descent direction. The main disadvantage of the methods proposed in  is that solving equation (1.6) requires the solution of equation (1.5). To overcome with this difficulty, Bnouhachem and Hamdi  proposed a parallel descent LQP alternating direction method for solving SVI.

In this paper, we propose a parallel descent LQP alternating direction method for solving the following structured variational inequality with three separable operators: Find $$y\in\Omega:= \{ (u,v,w) : u\in\mathbb {R}_{+}^{n_{1}}, v\in\mathbb{R}_{+}^{n_{2}}, w\in\mathbb{R}_{+}^{n_{2}}, A_{1}u + A_{2}v +A_{3}w= b \}$$ such that

$$\bigl(y'- y\bigr)^{\top} F(y) \ge0, \quad \forall y'\in\Omega,$$
(1.8)

with

$$y=\left ( \textstyle\begin{array}{c} u \\ v\\ w \end{array}\displaystyle \right ),\qquad F(y)= \left ( \textstyle\begin{array}{c} f_{1}(u)\\ f_{2}(v)\\ f_{3}(w) \end{array}\displaystyle \right ),$$
(1.9)

where $$A_{1}\in\mathbb{R}^{m\times n_{1}}$$, $$A_{2} \in\mathbb{R}^{m\times n_{2}}$$, $$A_{3}\in\mathbb{R}^{m\times n_{3}}$$ are given matrices, $$b\in\mathbb{R}^{m}$$ is a given vector, and $$f_{1}:\mathbb {R}_{+}^{n_{1}}\to\mathbb{R}^{n_{1}}$$, $$f_{2}:\mathbb{R}_{+}^{n_{2}}\to\mathbb{R}^{n_{2}}$$, $$f_{3}:\mathbb {R}_{+}^{n_{3}}\to\mathbb{R}^{n_{3}}$$ are given monotone operators. By attaching a Lagrange multiplier vector $$\lambda\in\mathbb{R}^{m}$$ to the linear constraints $$A_{1}u + A_{2}v +A_{3}w= b$$, the problem (1.8)-(1.9) can be explained in terms of finding $$z \in\mathcal{Z} := \mathbb{R}_{+}^{n_{1}} \times\mathbb {R}_{+}^{n_{2}} \times\mathbb{R}_{+}^{n_{3}} \times\mathbb{R}^{m}$$ such that

$$\bigl(z'-z\bigr)^{\top} Q(z) \geq0, \quad \forall z' \in \mathcal{Z},$$
(1.10)

where

$$z=\left ( \textstyle\begin{array}{c} u\\ v\\ w\\ \lambda \end{array}\displaystyle \right ) \quad \mbox{and} \quad Q(z)=\left ( \textstyle\begin{array}{c} f_{1}(u)-A_{1}^{\top} \lambda\\ f_{2}(v)-A_{2}^{\top} \lambda\\ f_{3}(w)-A_{3}^{\top} \lambda\\ A_{1}u + A_{2}v +A_{3}w- b \end{array}\displaystyle \right ).$$
(1.11)

The problem (1.10)-(1.11) is referred as $$\mathrm{SVI}_{3}$$.

The main aim of this paper is to present the parallel descent LQP alternating direction method for solving $$\mathrm{SVI}_{3}$$ and to investigate the convergence rate of this method. We show that the proposed method has the $$O(1/t)$$ convergence rate. The iterative algorithm and results presented in this paper generalize, unify, and improve the previously known results in this area.

## The proposed method

For any vector $$u\in\mathbb{R}^{n}$$, $$\Vert u\Vert _{\infty}=\max\{\vert u_{1}\vert , \ldots ,\vert u_{n}\vert \}$$. Let $$D \in\mathbb{R}^{n \times n}$$ be a symmetry positive definite matrix, we denote the D-norm of u by $$\Vert u\Vert _{D}^{2}=u^{T}Du$$.

The following lemma provides a basic property of projection operator onto a closed convex subset Ω of $$\mathbb{R}^{l}$$. We denote by $$P_{\Omega,D}(\cdot)$$ the projection operator under the D-norm, that is,

$$P_{\Omega,D}(v)=\operatorname{argmin} \bigl\{ \Vert v-u\Vert _{D} : u \in\Omega\bigr\} .$$

### Lemma 2.1

Let D be a symmetry positive definite matrix and Ω be a nonempty closed convex subset of $$\mathbb{R}^{l}$$. Then

$$\bigl(z-P_{\Omega,D}[z]\bigr)^{\top} D \bigl(P_{\Omega,D}[z]-v\bigr) \geq0, \quad \forall z \in\mathbb{R}^{l}, v\in\Omega.$$
(2.1)

We make the following standard assumptions.

### Assumption 2.1

$$f_{1}$$ is monotone with respect to $$\mathbb{R}^{n_{1}}_{+}$$, that is, $$(f_{1}(x)-f_{1}(y))^{T}(x-y)\ge0$$, $$\forall x,y\in\mathbb{R}^{n_{1}}_{+}$$, $$f_{2}$$ is monotone with respect to $$\mathbb{R}^{n_{2}}_{+}$$, and $$f_{3}$$ is monotone with respect to $$\mathbb{R}^{n_{3}}_{+}$$.

### Assumption 2.2

The solution set of $$\mathrm{SVI}_{3}$$, denoted by $$\mathcal {Z}^{*}$$, is nonempty.

We propose the following parallel LQP alternating direction method for solving $$\mathrm{SVI}_{3}$$:

### Algorithm 2.1

1. Step 0.

Given $$\varepsilon>0$$, $$\mu\in(0,1)$$, $$\beta\in ( \frac {\sqrt{3}}{2},1 )$$, $$\gamma\in(0,2)$$ and $$z^{0}=(u^{0}, v^{0},w^{0}, \lambda^{0}) \in\mathbb{R}^{n_{1}}_{++}\times \mathbb{R}^{n_{2}}_{++}\times\mathbb{R}^{n_{3}}_{++}\times\mathbb{R}^{m}$$. Set $$k=0$$.

2. Step 1.

Compute $$\tilde{z}^{k}=(\tilde{u}^{k},\tilde{v}^{k},\tilde{w}^{k},\tilde {\lambda}^{k})\in\mathbb{R}^{n_{1}}_{++}\times \mathbb{R}^{n_{2}}_{++}\times\mathbb{R}^{n_{3}}_{++}\times\mathbb {R}^{m}$$ by solving the following system:

\begin{aligned}& f_{1}(u) - A_{1}^{\top} \bigl[\lambda^{k} - H \bigl(A_{1} u + A_{2} v^{k} +A_{3} w^{k}- b\bigr) \bigr] \\& \quad {} + R_{1}\bigl[\bigl(u-u^{k}\bigr)+ \mu\bigl(u^{k} - U_{k}^{2} u^{-1}\bigr)\bigr]=0, \end{aligned}
(2.2)
\begin{aligned}& f_{2}(v) - A_{2}^{\top} \bigl[ \lambda^{k} - H \bigl(A_{1} u^{k} + A_{2} v +A_{3} w^{k}- b\bigr)\bigr] \\& \quad {}+ {R_{2}}\bigl[ \bigl(v-v^{k}\bigr) + \mu\bigl(v^{k} - V_{k}^{2} v^{-1}\bigr)\bigr]=0, \end{aligned}
(2.3)
\begin{aligned}& f_{3}(w) - A_{3}^{\top} \bigl[ \lambda^{k} - H \bigl(A_{1} u^{k} + A_{2} v^{k}+A_{3} w - b\bigr)\bigr] \\& \quad {}+ {R_{3}}\bigl[ \bigl(w-w^{k}\bigr) + \mu\bigl(w^{k} - W_{k}^{2} w^{-1}\bigr)\bigr]=0, \end{aligned}
(2.4)
\begin{aligned}& \tilde{\lambda}^{k} = \lambda^{k} - \beta H \bigl(A_{1} \tilde{u}^{k} + A_{2} \tilde{v}^{k}+A_{3} \tilde{w}^{k} - b\bigr) , \end{aligned}
(2.5)

where $$H \in\mathbb{R}^{m \times m}$$, $${R_{1}} \in\mathbb{R}^{n_{1} \times n_{1}}$$, $${R_{2}} \in\mathbb{R}^{n_{2} \times n_{2}}$$ and $${R_{3}} \in\mathbb{R}^{n_{3} \times n_{3}}$$ are symmetric positive definite matrices. $$U_{k}$$, $$V_{k}$$, and $$W_{k}$$ are positive definite diagonal matrices defined by $$U_{k} =\operatorname{diag} ( u^{k}_{1}, \ldots,u^{k}_{n} )$$, $$V_{k} = \operatorname{diag} ( v^{k}_{1}, \ldots,v^{k}_{n} )$$, $$W_{k} = \operatorname{diag} ( w^{k}_{1}, \ldots,w^{k}_{n} )$$.

3. Step 2.

If $$\max\{\Vert u^{k}-\tilde{u}^{k}\Vert _{\infty}, \Vert v^{k}-\tilde{v}^{k}\Vert _{\infty}, \Vert w^{k}-\tilde{w}^{k}\Vert _{\infty}, \Vert \lambda^{k}-\tilde{\lambda}^{k}\Vert _{\infty}\}<\epsilon$$, then stop.

4. Step 3.

The new iterate $$z^{k+1}(\tau_{k})=(u^{k+1},v^{k+1},w^{k+1},\lambda ^{k+1})$$ is given by

$$z^{k+1}(\tau_{k})= (1-\sigma) z^{k}+ \sigma P_{\mathcal{Z},G}\bigl[z^{k}-\gamma \tau_{k}G^{-1}g \bigl(z^{k},\tilde{z}^{k}\bigr)\bigr], \quad\sigma\in(0,1),$$
(2.6)

where

\begin{aligned}& \tau_{k}=\frac{\varphi(z^{k},\tilde{z}^{k})}{\Vert z^{k}- \tilde{z}^{k}\Vert ^{2}_{G}}, \end{aligned}
(2.7)
\begin{aligned}& \varphi\bigl(z^{k},\tilde{z}^{k}\bigr)=\bigl\Vert z^{k}- \tilde{z}^{k}\bigr\Vert _{M}^{2} \\& \hphantom{ \varphi\bigl(z^{k},\tilde{z}^{k}\bigr)=}{}+ \frac {1}{\beta}\bigl(\lambda^{k}-\tilde{\lambda}^{k} \bigr)^{T} \bigl(A_{1}\bigl(u^{k}- \tilde{u}^{k}\bigr) +A_{2}\bigl(v^{k}- \tilde{v}^{k}\bigr)+A_{3}\bigl(w^{k}- \tilde{w}^{k}\bigr) \bigr), \end{aligned}
(2.8)
\begin{aligned}& g(z^{k},\tilde{z}^{k}) \\& \quad = \left ( \textstyle\begin{array}{c} f_{1}(\tilde{u}^{k}) - A_{1}^{\top} \tilde{\lambda}^{k}+A_{1}^{\top} H[A_{1}(u^{k}-\tilde{u}^{k}) +A_{2}(v^{k}-\tilde{v}^{k})+A_{3}(w^{k}-\tilde{w}^{k}) +\frac{1-\beta}{\beta }H^{-1}(\lambda^{k}-\tilde{\lambda}^{k})]\\ f_{2}(\tilde{v}^{k}) - A_{2}^{\top} \tilde{\lambda}^{k}+ A_{2}^{\top} H[A_{1}(u^{k}-\tilde{u}^{k})+A_{2}(v^{k}-\tilde{v}^{k})+A_{3}(w^{k}-\tilde{w}^{k}) +\frac{1-\beta}{\beta}H^{-1}(\lambda^{k}-\tilde{\lambda}^{k})]\\ f_{3}(\tilde{w}^{k}) - A_{3}^{\top} \tilde{\lambda}^{k}+ A_{3}^{\top} H[A_{1}(u^{k}-\tilde{u}^{k})+A_{2}(v^{k}-\tilde{v}^{k})+A_{3}(w^{k}-\tilde{w}^{k}) +\frac{1-\beta}{\beta}H^{-1}(\lambda^{k}-\tilde{\lambda}^{k})]\\ A_{1}\tilde{u}^{k}+A_{2}\tilde{v}^{k}+A_{3}\tilde{w}^{k}-b \end{array}\displaystyle \right ), \\& \\& G=\left ( \textstyle\begin{array}{c@{\quad}c@{\quad}c@{\quad}c} (1+\mu){R_{1}}+A_{1}^{\top} HA_{1} & 0& 0 &0 \\ 0 & (1+\mu){R_{2}} +A_{2}^{\top} HA_{2} & 0&0 \\ 0 &0 &(1+\mu){R_{3}} +A_{3}^{\top} HA_{3} & 0 \\ 0 & 0 &0 & \frac{1}{\beta}H^{-1} \end{array}\displaystyle \right ), \end{aligned}
(2.9)

and

$$M=\left ( \textstyle\begin{array}{c@{\quad}c@{\quad}c@{\quad}c} {R_{1}} +A_{1}^{\top}HA_{1} & 0& 0 &0 \\ 0 & {R_{2}} +A_{2}^{\top} HA_{2} & 0&0 \\ 0 &0 & {R_{3}} +A_{3}^{\top} HA_{3} & 0 \\ 0 & 0 &0 & \frac{1}{\beta}H^{-1} \end{array}\displaystyle \right ).$$

Set $$k:= k + 1$$ and go to Step 1.

### Remark 2.1

As special cases, we can obtain some new LQP alternating methods as follows:

1. (a)

If $$u^{k+1}=\tilde{u}^{k}$$, $$v^{k+1}=\tilde{v}^{k}$$, $$w^{k+1}=\tilde{w}^{k}$$ and $$\lambda^{k+1}=\tilde{\lambda}^{k}$$ in (2.2), (2.3), (2.4) and (2.5), respectively, we obtain a new method which can be viewed as an extension of that proposed in  for solving structured variational inequality with three separable operators in a parallel way.

2. (b)

If $$u^{k+1}=\tilde{u}^{k}$$, $$v^{k+1}=\tilde{v}^{k}$$, $$w^{k+1}=\tilde{w}^{k}$$, $$\lambda^{k+1}=\tilde{\lambda}^{k}$$, and $$\beta=1$$ in (2.2), (2.3), (2.4) and (2.5), respectively, we obtain a new method which can be viewed as an extension of that proposed in  for solving structured variational inequality with three separable operators in a parallel wise.

3. (c)

If $$\beta=1$$, the proposed method can be viewed as an extension of that proposed in  for solving structured variational inequality with three separable operators.

We need the following result in the convergence analysis of the proposed method.

### Lemma 2.2



Let $$q(u)\in\mathbb{R}^{n}$$ be a monotone mapping of u with respect to $$\mathbb{R}^{n}_{+}$$ and $$R\in\mathbb{R}^{n\times n}$$ be a positive definite diagonal matrix. For a given $$u^{k} >0$$, if $$U_{k}:= \operatorname {diag}(u^{k}_{1},u^{k}_{2},\ldots, u^{k}_{n})$$ (the diagonal matrix with elements $$u^{k}_{1}$$, $$u^{k}_{2}$$, … , $$u^{k}_{n}$$) and $$u^{-1}$$ be an n-vector whose jth element is $$1/u_{j}$$, then the equation

$$q(u) + R\bigl[\bigl(u -u^{k}\bigr) + \mu\bigl( u^{k} - U_{k}^{2} u^{-1}\bigr)\bigr]=0$$
(2.10)

has a unique positive solution u. Moreover, for any $$v\geq0$$, we have

$$(v-u)^{\top} q(u) \geq{ \frac{1+\mu}{2} } \bigl(\Vert u - v \Vert _{R}^{2} - \bigl\Vert u^{k} - v \bigr\Vert _{R}^{2} \bigr) + { \frac{1-\mu}{2} }\bigl\Vert u^{k} - u \bigr\Vert _{R}^{2}.$$
(2.11)

The next theorem is useful for the convergence analysis.

### Theorem 2.1

For given $$z^{k}\in \mathbb{R}^{n_{1}}_{++}\times\mathbb{R}^{n_{2}}_{++}\times \mathbb{R}^{n_{3}}_{++}\times\mathbb{R}^{m}$$, let $$\tilde{z}^{k}$$ be generated by (2.2)-(2.5). Then

\begin{aligned} \varphi\bigl(z^{k},\tilde{z}^{k}\bigr)\geq \frac{2\beta-\sqrt{3}}{2\beta} \bigl\Vert z^{k}- \tilde{z}^{k}\bigr\Vert ^{2}_{G} \end{aligned}
(2.12)

and

\begin{aligned} \tau_{k}\geq\frac{2\beta-\sqrt{3}}{2\beta}. \end{aligned}
(2.13)

### Proof

It follows from (2.8) that

\begin{aligned} \varphi\bigl(z^{k},\tilde{z}^{k}\bigr) = & \bigl\Vert z^{k}- \tilde{z}^{k}\bigr\Vert _{M}^{2}+\frac{1}{\beta}\bigl(\lambda^{k}-\tilde {\lambda}^{k}\bigr)^{\top}\bigl(A_{1} \bigl(u^{k}-\tilde{u}^{k}\bigr) +A_{2} \bigl(v^{k}-\tilde{v}^{k}\bigr)+A_{3} \bigl(w^{k}-\tilde{w}^{k}\bigr)\bigr) \\ = & \bigl\Vert u^{k}-\tilde{u}^{k}\bigr\Vert _{R_{1}}^{2}+\bigl\Vert A_{1}u^{k}-A_{1} \tilde{u}^{k}\bigr\Vert _{H}^{2}+\bigl\Vert v^{k}-\tilde{v}^{k}\bigr\Vert _{R_{2}}^{2} +\bigl\Vert A_{2}v^{k} -A_{2} \tilde{v}^{k} \bigr\Vert _{H}^{2} \\ &{}+\bigl\Vert w^{k}-\tilde{w}^{k}\bigr\Vert _{R_{3}}^{2}+\bigl\Vert A_{3}w^{k} -A_{3}\tilde{w}^{k}\bigr\Vert _{H}^{2} +\frac{1}{\beta}\bigl\Vert \lambda^{k}-\tilde{\lambda}^{k} \bigr\Vert _{H^{-1}}^{2} \\ &{}+\frac{1}{\beta}\bigl(\lambda^{k}-\tilde{\lambda}^{k} \bigr)^{\top }\bigl(A_{1}\bigl(u^{k}- \tilde{u}^{k}\bigr)+A_{2}\bigl(v^{k}- \tilde{v}^{k}\bigr)+A_{3}\bigl(w^{k}- \tilde{w}^{k}\bigr)\bigr). \end{aligned}
(2.14)

By using the Cauchy-Schwarz inequality, we have

\begin{aligned}& \bigl(\lambda^{k}-\tilde{\lambda}^{k} \bigr)^{\top}\bigl(A_{1}\bigl(u^{k}- \tilde{u}^{k}\bigr)\bigr) \geq-\frac{1}{2} \biggl(\sqrt{3}\bigl\Vert A_{1}\bigl(u^{k}-\tilde{u}^{k}\bigr)\bigr\Vert _{H}^{2} +\frac{1}{\sqrt{3}}\bigl\Vert \lambda^{k}-\tilde{\lambda}^{k}\bigr\Vert _{H^{-1}}^{2} \biggr), \end{aligned}
(2.15)
\begin{aligned}& \bigl(\lambda^{k}-\tilde{\lambda}^{k} \bigr)^{\top}\bigl(A_{2}\bigl(v^{k}- \tilde{v}^{k}\bigr)\bigr) \geq-\frac{1}{2} \biggl(\sqrt{3}\bigl\Vert A_{2}\bigl(v^{k}-\tilde{v}^{k}\bigr)\bigr\Vert _{H}^{2} +\frac{1}{\sqrt{3}}\bigl\Vert \lambda^{k}-\tilde{\lambda}^{k}\bigr\Vert _{H^{-1}}^{2} \biggr) , \end{aligned}
(2.16)

and

\begin{aligned} \bigl(\lambda^{k}-\tilde{\lambda}^{k} \bigr)^{\top}\bigl(A_{3}\bigl(w^{k}-\tilde{w}^{k} \bigr)\bigr) \geq-\frac{1}{2} \biggl(\sqrt{3}\bigl\Vert A_{3} \bigl(w^{k}-\tilde{w}^{k}\bigr)\bigr\Vert _{H}^{2} +\frac{1}{\sqrt{3}}\bigl\Vert \lambda^{k}-\tilde{\lambda}^{k}\bigr\Vert _{H^{-1}}^{2} \biggr). \end{aligned}
(2.17)

Substituting (2.15), (2.16), and (2.17) into (2.14), we get

\begin{aligned} \varphi\bigl(z^{k},\tilde{z}^{k}\bigr) \geq & \frac{2\beta-\sqrt{3}}{2\beta} \bigl(\bigl\Vert A_{1}u^{k}-A_{1} \tilde{u}^{k}\bigr\Vert _{H}^{2} +\bigl\Vert A_{2}v^{k} -A_{2}\tilde{v}^{k} \bigr\Vert _{H}^{2}+\bigl\Vert A_{3}w^{k} -A_{3}\tilde{w}^{k} \bigr\Vert _{H}^{2} \bigr)\\ &{} +\frac{2-\sqrt{3}}{2\beta}\bigl\Vert \lambda^{k}-\tilde{ \lambda}^{k}\bigr\Vert _{H^{-1}}^{2} +\bigl\Vert u^{k}-\tilde{u}^{k}\bigr\Vert _{R_{1}}^{2}+\bigl\Vert v^{k}- \tilde{v}^{k}\bigr\Vert _{R_{2}}^{2} +\bigl\Vert w^{k}-\tilde{w}^{k}\bigr\Vert _{R_{3}}^{2} \\ {\geq}& \frac{2\beta-\sqrt{3}}{2\beta} \biggl(\bigl\Vert A_{1}u^{k}-A_{1} \tilde {u}^{k}\bigr\Vert _{H}^{2}+\bigl\Vert A_{2}v^{k} -A_{2}\tilde{v}^{k} \bigr\Vert _{H}^{2}\\ &{}+\bigl\Vert A_{3}w^{k}-A_{3}\tilde{w}^{k}\bigr\Vert _{H}^{2} + \frac{1}{\beta}\bigl\Vert \lambda^{k}-\tilde{\lambda}^{k} \bigr\Vert _{H^{-1}}^{2} \biggr) \\ &{}+\frac{2\beta-\sqrt{3}}{2\beta} \bigl(\bigl\Vert u^{k}-\tilde{u}^{k} \bigr\Vert _{R_{1}}^{2} +\bigl\Vert v^{k}- \tilde{v}^{k}\bigr\Vert _{R_{2}}^{2}+\bigl\Vert w^{k}-\tilde{w}^{k}\bigr\Vert _{R_{3}}^{2} \bigr) \\ =&\frac{2\beta-\sqrt{3}}{2\beta} \bigl(\bigl\Vert z^{k}-\tilde{z}^{k} \bigr\Vert ^{2}_{G}+(1-\mu)\bigl\Vert u^{k}- \tilde{u}^{k}\bigr\Vert _{R_{1}}^{2}\\ &{} +(1-\mu)\bigl\Vert v^{k}-\tilde{v}^{k}\bigr\Vert _{R_{2}}^{2}+(1- \mu)\bigl\Vert w^{k}-\tilde {w}^{k}\bigr\Vert _{R_{3}}^{2} \bigr) \\ \geq &\frac{2\beta-\sqrt{3}}{2\beta}\bigl\Vert z^{k}-\tilde {z}^{k}\bigr\Vert ^{2}_{G}. \end{aligned}

Therefore, it follows from (2.7) and (2.12) that

$$\tau_{k}\geq\frac{2\beta-\sqrt{3}}{2\beta},$$
(2.18)

and this completes the proof. □

## Convergence rate

Recall that $$\mathcal{Z}^{*}$$ can be characterized as (see (2.3.2) in p.159 of )

$$\mathcal{Z}^{*}=\bigcap_{z\in\mathcal{Z}} \bigl\{ \hat{z}\in \mathcal{Z} : (z-\hat{z})^{\top} Q(z)\geq0 \bigr\} .$$

This implies that is an approximate solution of $$\mathrm{SVI}_{3}$$ with the accuracy $$\epsilon>0$$ if it satisfies

$$\hat{z}\in\mathcal{Z}\quad\hbox{and}\quad\sup_{z\in\mathcal {Z}} \bigl\{ (z-\hat{z})^{\top} Q(z)\bigr\} \leq\epsilon.$$
(3.1)

Now we show that after t iterations of the proposed method, we can find a $$\hat{z}\in\mathcal{Z}$$ such that (3.1) is satisfied with $$\epsilon= O(1/t)$$.

We introduce the following matrices,

$$N=\left ( \textstyle\begin{array}{c@{\quad}c@{\quad}c@{\quad}c} I & 0& 0 &0 \\ 0 & I & 0& 0 \\ 0 & 0 & I& 0 \\ -\beta HA_{1} & -\beta HA_{2} &-\beta HA_{3} & \beta I \end{array}\displaystyle \right )$$
(3.2)

and

$$J=\left ( \textstyle\begin{array}{c@{\quad}c@{\quad}c@{\quad}c} (1+\mu){R_{1}}+A_{1}^{\top} HA_{1} & 0& 0& 0 \\ 0 &(1+\mu){R_{2}}+A_{2}^{\top} H A_{2} & 0& 0 \\ 0 &0 & (1+\mu){R_{3}}+A_{3}^{\top} H A_{3} & 0 \\ -A_{1} & -A_{2} &-A_{3} & H^{-1} \end{array}\displaystyle \right ).$$
(3.3)

By simple manipulations, we can find that $$J= GN$$.

Our analysis needs a new sequence defined by

$$\hat{z}^{k}= \left ( \textstyle\begin{array}{c} \hat{u}^{k} \\ \hat{v}^{k}\\ \hat{w}^{k}\\ \hat{\lambda}^{k} \end{array}\displaystyle \right )=\left ( \textstyle\begin{array}{c} \tilde{u}^{k}\\ \tilde{v}^{k}\\ \tilde{w}^{k}\\ {\lambda}^{k}-H(A_{1}u^{k}+A_{2}v^{k}+A_{3}w^{k}-b) \end{array}\displaystyle \right ).$$
(3.4)

Based on (3.2) and (3.4), we can easily have

$$z^{k}-\tilde{z}^{k}=N\bigl(z^{k}- \hat{z}^{k}\bigr).$$
(3.5)

Using (1.11), (2.9), and (3.4), we obtain

$$g\bigl(z^{k},\tilde{z}^{k}\bigr)=Q\bigl( \hat{z}^{k}\bigr).$$
(3.6)

### Lemma 3.1

Let $$\hat{z}^{k}$$ be defined by (3.4), $$z\in\mathcal{Z}$$, and the matrix J be given by (3.3). Then

$$\bigl(z-\hat{z}^{k}\bigr)^{\top} \bigl(Q\bigl( \hat{z}^{k}\bigr)-J\bigl(z^{k}-\hat{z}^{k}\bigr) \bigr) \geq-\mu\bigl\Vert u^{k} - \hat{u}^{k} \bigr\Vert _{R_{1}}^{2}-\mu\bigl\Vert v^{k} - \hat{v}^{k} \bigr\Vert _{R_{2}}^{2}-\mu\bigl\Vert w^{k} - \hat{w}^{k} \bigr\Vert _{R_{3}}^{2}.$$
(3.7)

### Proof

Applying Lemma 2.2 to (2.2), we get

\begin{aligned}& \bigl(u - \tilde{u}^{k}\bigr)^{\top} \bigl\{ f_{1}\bigl(\tilde{u}^{k}\bigr) -A_{1}^{\top} \bigl[\lambda^{k} - H\bigl(A_{1} \tilde{u}^{k} + A_{2}{v}^{k}+ A_{3}{w}^{k} -b\bigr) \bigr] \bigr\} \\& \quad \geq\frac{1+\mu}{2} \bigl(\bigl\Vert \tilde{u}^{k}-u \bigr\Vert _{R_{1}}^{2} - \bigl\Vert u^{k} -u\bigr\Vert _{R_{1}}^{2} \bigr) + \frac{1-\mu}{2} \bigl\Vert u^{k} - \tilde{u}^{k} \bigr\Vert _{R_{1}}^{2}. \end{aligned}
(3.8)

Since

\begin{aligned} \bigl\Vert u^{k}-u\bigr\Vert _{R_{1}}^{2}=\bigl\Vert u^{k}-\tilde{u}^{k}\bigr\Vert _{R_{1}}^{2}+ \bigl\Vert \tilde{u}^{k}-u\bigr\Vert _{R_{1}}^{2}+2 \bigl(\tilde{u}^{k}-u\bigr)^{T}R_{1} \bigl(u^{k}-\tilde{u}^{k}\bigr), \end{aligned}

we have

$$\bigl(u-\tilde{u}^{k}\bigr)^{\top} R_{1}\bigl(u^{k}-\tilde{u}^{k}\bigr) = \frac{1}{2} \bigl(\bigl\Vert \tilde{u}^{k}-u\bigr\Vert _{R_{1}}^{2}-\bigl\Vert u^{k}-u\bigr\Vert _{R_{1}}^{2} \bigr) +{\frac{1}{2}}\bigl\Vert u^{k}-\tilde{u}^{k}\bigr\Vert _{R_{1}}^{2}.$$
(3.9)

Adding (3.8) and (3.9), we obtain

\begin{aligned}& \bigl(u - \tilde{u}^{k}\bigr)^{\top} \bigl\{ (1+ \mu)R_{1}\bigl(u^{k}-\tilde{u}^{k}\bigr)- f_{1}\bigl(\tilde{u}^{k}\bigr) + A_{1}^{\top} \bigl[\lambda^{k} - H\bigl(A_{1}u^{k} + A_{2}{v}^{k}+ A_{3}{w}^{k} -b\bigr) \bigr] \\& \quad {}+A_{1}^{\top} HA_{1}\bigl(u^{k}- \tilde{u}^{k}\bigr) \bigr\} \leq\mu\bigl\Vert u^{k} - \tilde {u}^{k} \bigr\Vert _{R_{1}}^{2}. \end{aligned}
(3.10)

Similarly, applying Lemma 2.2 to (2.3), we get

\begin{aligned}& \bigl(v - \tilde{v}^{k}\bigr)^{\top} \bigl\{ f_{2}\bigl(\tilde{v}^{k}\bigr) - A_{2}^{\top} \bigl[\lambda^{k} - H \bigl(A_{1}u^{k} + A_{2}\tilde{v}^{k}+ A_{3}{w}^{k} -b \bigr)\bigr] \bigr\} \\& \quad \geq\frac{1+\mu}{2} \bigl(\bigl\Vert \tilde{v}^{k}-v \bigr\Vert _{R_{2}}^{2} - \bigl\Vert v^{k} -v\bigr\Vert _{R_{2}}^{2} \bigr) + \frac{1-\mu}{2} \bigl\Vert v^{k} - \tilde{v}^{k} \bigr\Vert _{R_{2}}^{2}. \end{aligned}
(3.11)

Similar to (3.9), we have

$$\bigl(v-\tilde{v}^{k}\bigr)^{\top} {R_{2}}\bigl(v^{k}-\tilde{v}^{k}\bigr)={ \frac{1}{2}} \bigl(\bigl\Vert \tilde{v}^{k}-v\bigr\Vert _{R_{2}}^{2}-\bigl\Vert v^{k}-v\bigr\Vert _{R_{2}}^{2} \bigr) +{\frac{1}{2}}\bigl\Vert v^{k}-\tilde{v}^{k}\bigr\Vert _{R_{2}}^{2}.$$
(3.12)

Adding (3.11) and (3.12), we have

\begin{aligned}& \bigl(v - \tilde{v}^{k}\bigr)^{\top} \bigl\{ (1+ \mu){R_{2}}\bigl(v^{k}-\tilde{v}^{k}\bigr)- f_{2}\bigl(\tilde{v}^{k}\bigr) + A_{2}^{\top} \bigl[\lambda^{k} -H\bigl(A_{1}u^{k} + A_{2}{v}^{k}+ A_{3}{w}^{k} -b\bigr) \bigr] \\& \quad {}+A_{2}^{\top} HA_{2}\bigl(v^{k}- \tilde{v}^{k}\bigr) \bigr\} \leq\mu\bigl\Vert v^{k} - \tilde {v}^{k} \bigr\Vert _{R_{2}}^{2}. \end{aligned}
(3.13)

Similarly, we have

\begin{aligned}& \bigl(w - \tilde{w}^{k}\bigr)^{\top} \bigl\{ (1+ \mu){R_{3}}\bigl(w^{k}-\tilde{w}^{k}\bigr)- f_{3}\bigl(\tilde{w}^{k}\bigr) + A_{3}^{\top} \bigl[\lambda^{k} -H\bigl(A_{1}u^{k} + A_{2}{v}^{k}+ A_{3}{w}^{k} -b\bigr) \bigr] \\& \quad {}+A_{3}^{\top} HA_{3}\bigl(w^{k}- \tilde{w}^{k}\bigr) \bigr\} \le\mu\bigl\Vert w^{k} - \tilde {w}^{k} \bigr\Vert _{R_{3}}^{2}. \end{aligned}
(3.14)

Then, by using the notation of $$\hat{z}^{k}$$ in (3.4), (3.10), (3.13), and (3.14) can be written as

\begin{aligned}& \bigl(u - \hat{u}^{k}\bigr)^{\top} \bigl\{ (1+\mu)R_{1}\bigl(u^{k}-\hat{u}^{k}\bigr)- f_{1}(\hat {u}^{k})+A_{1}^{\top} \hat{\lambda}^{k} +A_{1}^{\top} HA_{1}\bigl(u^{k}-\hat{u}^{k}\bigr) \bigr\} \\& \quad \leq\mu\bigl\Vert u^{k} - \hat{u}^{k} \bigr\Vert _{R_{1}}^{2}, \end{aligned}
(3.15)
\begin{aligned}& \bigl(v - \hat{v}^{k}\bigr)^{\top} \bigl\{ (1+ \mu){R_{2}}\bigl(v^{k}-\hat{v}^{k}\bigr)- f_{2}\bigl(\hat{v}^{k}\bigr)+A_{2}^{\top} \hat{\lambda}^{k} +A_{2}^{\top} HA_{2} \bigl(v^{k}-\hat{v}^{k}\bigr) \bigr\} \\& \quad \le\mu\bigl\Vert v^{k} - \hat{v}^{k} \bigr\Vert _{R_{2}}^{2}, \end{aligned}
(3.16)

and

\begin{aligned}& \bigl(w - \hat{w}^{k}\bigr)^{\top} \bigl\{ (1+ \mu){R_{3}}\bigl(w^{k}-\hat{w}^{k}\bigr)- f_{3}\bigl(\hat{w}^{k}\bigr)+A_{3}^{\top} \hat{\lambda}^{k} +A_{3}^{\top} HA_{3} \bigl(w^{k}-\hat{w}^{k}\bigr) \bigr\} \\& \quad \le\mu\bigl\Vert w^{k} - \hat{w}^{k} \bigr\Vert _{R_{3}}^{2}. \end{aligned}
(3.17)

In addition, it follows from (3.4) that

\begin{aligned}& A_{1}\hat{u}^{k}+ A_{2} \hat{v}^{k}+A_{3}\hat{w}^{k}-b +H^{-1} \bigl(\hat{\lambda }^{k}-{\lambda}^{k}\bigr) \\& \quad {}-A_{1} \bigl(\hat{u}^{k}-u^{k}\bigr)-A_{2}\bigl( \hat{v}^{k}-v^{k}\bigr)-A_{3}\bigl( \hat{w}^{k}-w^{k}\bigr)=0. \end{aligned}
(3.18)

Combining (3.15)-(3.18), we get

\begin{aligned}& \left ( \textstyle\begin{array}{c} u - \hat{u}^{k} \\ v - \hat{v}^{k}\\ w - \hat{w}^{k}\\ \lambda- \hat{\lambda}^{k} \end{array}\displaystyle \right )^{\top} \left ( \textstyle\begin{array}{c} f_{1}(\hat{u}^{k})- A_{1}^{\top} \hat{\lambda}^{k}- ((1+\mu ){R_{1}}+A^{T}HA_{1} )(u^{k}-\hat{u}^{k})\\ f_{2}(\hat{v}^{k}) - A_{2}^{\top} \hat{\lambda}^{k}- ((1+\mu ){R_{2}}+A_{2}^{\top}HA_{2} )(v^{k}-\hat{v}^{k})\\ f_{3}(\hat{w}^{k}) - A_{3}^{\top} \hat{\lambda}^{k}- ((1+\mu ){R_{3}}+A_{3}^{\top}HA_{3} )(w^{k}-\hat{w}^{k})\\ A_{1}\hat{u}^{k}+A_{2}\hat{v}^{k}+A_{3}\hat{w}^{k} -b +A_{1}(u^{k}-\hat{u}^{k})+A_{2}(v^{k}-\hat{v}^{k})+A_{3}(w^{k}-\hat {w}^{k})-H^{-1}(\lambda^{k}-\hat{\lambda}^{k}) \end{array}\displaystyle \right ) \\& \quad \geq-\mu\bigl\Vert u^{k} - \hat{u}^{k} \bigr\Vert _{R_{1}}^{2}-\mu\bigl\Vert v^{k} - \hat{v}^{k} \bigr\Vert _{R_{2}}^{2}-\mu\bigl\Vert w^{k} - \hat{w}^{k} \bigr\Vert _{R_{3}}^{2}. \end{aligned}
(3.19)

Recall the definition of J in (3.3), we obtain the assertion (3.7). The proof is completed. □

### Lemma 3.2

For given $$z^{k}\in \mathbb{R}^{n_{1}}_{++}\times\mathbb{R}^{n_{2}}_{++}\times \mathbb{R}^{n_{3}}_{++}\times\mathbb{R}^{m}$$ and $$z_{*}^{k} := P_{\mathcal{Z},G}[z^{k}-\tau_{k} G^{-1}g(z^{k},\tilde{z}^{k})]$$, we have

$$\gamma\tau_{k}\bigl(z-\hat{z}^{k} \bigr)^{\top} Q(z)+\frac{1}{2}\bigl(\bigl\Vert z-z^{k} \bigr\Vert _{G}^{2}-\bigl\Vert z-z_{*}^{k} \bigr\Vert _{G}^{2}\bigr)\geq\frac{1}{2} \gamma(2-\gamma) \tau_{k}^{2}\bigl\Vert z^{k}-\tilde{z}^{k} \bigr\Vert _{G}^{2}.$$
(3.20)

### Proof

Since $$z_{*}^{k}\in\mathcal{Z}$$, substituting $$z=z_{*}^{k}$$ in (3.7), we get

\begin{aligned}& \gamma\tau_{k}\bigl(z_{*}^{k}- \hat{z}^{k}\bigr)^{\top}Q\bigl(\hat{z}^{k}\bigr) \end{aligned}
(3.21)
\begin{aligned}& \quad \geq \gamma\tau_{k}\bigl(z_{*}^{k}- \hat{z}^{k}\bigr)^{\top} J\bigl(z^{k}- \hat{z}^{k}\bigr) - \mu\gamma\tau_{k}\bigl\Vert u^{k} - \hat{u}^{k} \bigr\Vert _{R_{1}}^{2}- \mu\gamma\tau_{k}\bigl\Vert v^{k} - \hat{v}^{k} \bigr\Vert _{R_{2}}^{2} \\& \qquad {} -\mu\gamma\tau_{k}\bigl\Vert w^{k} - \hat{w}^{k}\bigr\Vert _{R_{3}}^{2} \\& \quad = \gamma\tau_{k}\bigl(z^{k}-\hat{z}^{k} \bigr)^{\top} J\bigl(z^{k}-\hat{z}^{k}\bigr)+\gamma \tau _{k}\bigl(z_{*}^{k}-z^{k} \bigr)^{\top} J\bigl(z^{k}-\hat{z}^{k}\bigr) \\& \qquad {} -\mu\gamma\tau_{k}\bigl\Vert u^{k} - \hat{u}^{k} \bigr\Vert _{R_{1}}^{2}-\mu\gamma\tau _{k}\bigl\Vert v^{k} - \hat{v}^{k} \bigr\Vert _{R_{2}}^{2} -\mu\gamma\tau_{k}\bigl\Vert w^{k} - \hat{w}^{k}\bigr\Vert _{R_{3}}^{2} \\& \quad = \gamma\tau_{k}\bigl(z^{k}-\tilde{z}^{k} \bigr)^{\top} \bigl(N^{-1}\bigr)^{\top} JN^{-1}\bigl(z^{k}-\tilde{z}^{k}\bigr) +\gamma \tau_{k}\bigl(z_{*}^{k}-z^{k} \bigr)^{\top} JN^{-1}\bigl(z^{k}-\tilde{z}^{k} \bigr) \\& \qquad {} -\gamma\tau_{k}\mu\bigl\Vert u^{k} - \hat{u}^{k} \bigr\Vert _{R_{1}}^{2} -\gamma \tau_{k}\mu\bigl\Vert v^{k} - \hat{v}^{k} \bigr\Vert _{R_{2}}^{2}-\mu\gamma\tau_{k}\bigl\Vert w^{k} - \hat{w}^{k}\bigr\Vert _{R_{3}}^{2} \\& \quad = \gamma\tau_{k}\bigl(z^{k}-\tilde{z}^{k} \bigr)^{\top} \bigl(N^{-1}\bigr)^{\top} G \bigl(z^{k}-\tilde{z}^{k}\bigr) -\gamma\tau_{k}\mu \bigl\Vert u^{k} - \hat{u}^{k} \bigr\Vert _{R_{1}}^{2} -\gamma\tau_{k}\mu\bigl\Vert v^{k} - \hat{v}^{k} \bigr\Vert _{R_{2}}^{2} \\& \qquad {} -\mu\gamma\tau_{k}\bigl\Vert w^{k} - \hat{w}^{k}\bigr\Vert _{R_{3}}^{2} +\gamma \tau_{k}\bigl(z_{*}^{k}-z^{k} \bigr)^{\top} G\bigl(z^{k}-\tilde{z}^{k}\bigr) \\& \quad = \gamma\tau_{k}\bigl(z^{k}-\tilde{z}^{k} \bigr)^{\top} \bigl(N^{-1}\bigr)^{\top} M \bigl(z^{k}-\tilde{z}^{k}\bigr) +\gamma\tau_{k} \bigl(z_{*}^{k}-z^{k}\bigr)^{\top} G \bigl(z^{k}-\tilde{z}^{k}\bigr) \\& \quad \geq \gamma\tau_{k}\varphi\bigl(z^{k},\tilde{z}^{k} \bigr)+\gamma\tau _{k}\bigl(z_{*}^{k}-z^{k} \bigr)^{\top} G\bigl(z^{k}-\tilde{z}^{k}\bigr) \\& \quad \geq \gamma\tau_{k}\varphi\bigl(z^{k},\tilde{z}^{k} \bigr)-\frac{1}{2}\bigl\Vert z^{k}-z_{*}^{k}\bigr\Vert _{G}^{2} -\frac{1}{2}\gamma^{2} \tau_{k}^{2}\bigl\Vert z^{k}-\tilde{z}^{k} \bigr\Vert _{G}^{2} \\& \quad = \frac{1}{2} \gamma(2-\gamma)\tau_{k}^{2}\bigl\Vert z^{k}-\tilde{z}^{k}\bigr\Vert _{G}^{2}- \frac{1}{2}\bigl\Vert z^{k}-z_{*}^{k}\bigr\Vert _{G}^{2}. \end{aligned}
(3.22)

Using (3.6), $$z_{*}^{k}$$ is the projection of $$z^{k}-\gamma\tau_{k} G^{-1}Q(\hat{z}^{k})$$ on $$\mathcal{Z}$$, it follows from (2.1) that

$$\bigl(z^{k}-\gamma\tau_{k} G^{-1}Q\bigl( \hat{z}^{k}\bigr)-z_{*}^{k}\bigr)^{\top} G \bigl(z-z_{*}^{k}\bigr)\leq0, \quad\forall z\in\mathcal{Z},$$

and consequently, we have

$$\gamma\tau_{k} \bigl(z-z_{*}^{k} \bigr)^{\top} Q\bigl(\hat{z}^{k}\bigr)\geq \bigl(z^{k}-z_{*}^{k} \bigr)^{\top} G\bigl(z-z_{*}^{k}\bigr).$$

Using the identity $$x^{\top} Gy =\frac{1}{2} (\Vert x\Vert _{G}^{2}-\Vert x-y\Vert _{G}^{2}+\Vert y\Vert _{G}^{2} )$$ to the right hand side of the last inequality, we obtain

$$\gamma\tau_{k} \bigl(z-z_{*}^{k} \bigr)^{\top} Q\bigl(\hat{z}^{k}\bigr)\geq\frac{1}{2} \bigl(\bigl\Vert z-z_{*}^{k}\bigr\Vert _{G}^{2}-\bigl\Vert z-z^{k}\bigr\Vert _{G}^{2} \bigr) +\frac{1}{2}\bigl\Vert z^{k}-z_{*}^{k}\bigr\Vert _{G}^{2}.$$
(3.23)

Adding (3.21) and (3.23), we get

$$\gamma\tau_{k}\bigl(z-\hat{z}^{k}\bigr)^{\top} Q \bigl(\hat{z}^{k}\bigr)+\frac{1}{2}\bigl(\bigl\Vert z-z^{k} \bigr\Vert _{G}^{2}-\bigl\Vert z-z_{*}^{k} \bigr\Vert _{G}^{2}\bigr)\geq\frac{1}{2} \gamma(2-\gamma) \tau_{k}^{2}\bigl\Vert z^{k}-\tilde{z}^{k} \bigr\Vert _{G}^{2},$$

and by using the monotonicity of Q, we obtain (3.20) and the proof is completed. □

### Lemma 3.3

Let $$z^{k}\in \mathbb{R}^{n_{1}}_{++}\times \mathbb{R}^{n_{2}}_{++}\times\mathbb{R}^{n_{3}}_{++}\times\mathbb {R}^{m}$$ and $$z^{k+1}(\tau_{k})$$ be generated by (2.6). Then

$$\gamma\sigma\tau_{k}\bigl(z-\hat{z}^{k} \bigr)^{\top} Q(z)+\frac{1}{2}\bigl(\bigl\Vert z-z^{k}\bigr\Vert _{G}^{2}-\bigl\Vert z-z^{k+1}( \tau_{k})\bigr\Vert _{G}^{2}\bigr) \geq \frac{1}{2} \sigma\gamma(2-\gamma)\tau_{k}^{2}\bigl\Vert z-\tilde{z}^{k}\bigr\Vert _{G}^{2}.$$
(3.24)

### Proof

We have

\begin{aligned}& \bigl\Vert z-z^{k}\bigr\Vert _{G}^{2}- \bigl\Vert z-z^{k+1}(\tau_{k})\bigr\Vert _{G}^{2} \end{aligned}
(3.25)
\begin{aligned}& \quad = \bigl\Vert z^{k}-z\bigr\Vert _{G}^{2}- \bigl\Vert z^{k}-\sigma\bigl(z^{k}-z_{*}^{k} \bigr)-z\bigr\Vert _{G}^{2} \\& \quad = 2\sigma\bigl(z^{k}-z\bigr)^{\top} G\bigl(z^{k}-z_{*}^{k} \bigr)-\sigma^{2}\bigl\Vert z^{k}-z_{*}^{k} \bigr\Vert _{G}^{2} \\& \quad = 2\sigma \bigl(\bigl\Vert z^{k}-z_{*}^{k} \bigr\Vert _{G}^{2}-\bigl( z-z_{*}^{k} \bigr)^{\top} G\bigl(z^{k}-z_{*}^{k}\bigr) \bigr)-\sigma^{2}\bigl\Vert z^{k}-z_{*}^{k} \bigr\Vert _{G}^{2}. \end{aligned}
(3.26)

Using the identity

\begin{aligned} \bigl(z-z_{*}^{k}\bigr)^{\top} G \bigl(z^{k}-z_{*}^{k}\bigr) = &{ \frac{1}{2}} \bigl(\bigl\Vert z_{*}^{k}-z\bigr\Vert _{G}^{2}- \bigl\Vert z^{k}-z\bigr\Vert _{G}^{2} \bigr) +{\frac{1}{2}}\bigl\Vert z^{k}-z_{*}^{k}\bigr\Vert _{G}^{2}, \end{aligned}

we get

$$\bigl\Vert z^{k}-z_{*}^{k}\bigr\Vert _{G}^{2}-2\bigl(z-z_{*}^{k} \bigr)^{\top} G\bigl(z^{k}-z_{*}^{k}\bigr)= \bigl\Vert z^{k}-z\bigr\Vert _{G}^{2}-\bigl\Vert z_{*}^{k}-z\bigr\Vert _{G}^{2}.$$
(3.27)

Substituting (3.27) into (3.25), we obtain

\begin{aligned} \bigl\Vert z-z^{k}\bigr\Vert _{G}^{2}- \bigl\Vert z-z^{k+1}(\tau_{k})\bigr\Vert _{G}^{2} =&\sigma\bigl(\bigl\Vert z-z^{k}\bigr\Vert _{G}^{2}-\bigl\Vert z-z_{*}^{k}\bigr\Vert _{G}^{2}\bigr) +\sigma(1-\sigma)\bigl\Vert z^{k}-z_{*}^{k}\bigr\Vert _{G}^{2} \\ \geq&\sigma\bigl(\bigl\Vert z-z^{k}\bigr\Vert _{G}^{2}-\bigl\Vert z-z_{*}^{k}\bigr\Vert _{G}^{2}\bigr). \end{aligned}
(3.28)

Substituting (3.28) into (3.20), we obtain (3.24), the required result. □

### Theorem 3.1

Let $$z^{*}$$ be a solution of $$\mathrm{SVI}_{3}$$ and $$z^{k+1}(\tau_{k})$$ be generated by (2.6). Then $$z^{k}$$ and $$\tilde{z}^{k}$$ are bounded, and

$$\bigl\Vert z^{k+1}(\tau_{k}) - z^{*}\bigr\Vert _{G}^{2}\leq\bigl\Vert z^{k} -z^{*}\bigr\Vert _{G}^{2}-c\bigl\Vert z^{k}-\tilde {z}^{k}\bigr\Vert ^{2}_{G},$$
(3.29)

where

$$c:={ \frac{\sigma\gamma(2-\gamma)(2\beta-\sqrt {3})^{2}}{4\beta^{2}}}>0.$$

### Proof

Setting $$z=z^{*}$$ in (3.24), we obtain

\begin{aligned} \bigl\Vert z^{k+1}(\tau_{k}) - z^{*}\bigr\Vert _{G}^{2} \leq&\bigl\Vert z^{k} -z^{*}\bigr\Vert _{G}^{2}-\sigma\gamma(2-\gamma)\tau_{k}^{2} \bigl\Vert z^{k}-\tilde {z}^{k}\bigr\Vert ^{2}_{G}+ 2\gamma\sigma\tau_{k}\bigl(z^{*}-\hat{z}^{k} \bigr)^{\top} Q\bigl(z^{*}\bigr) \\ \leq&\bigl\Vert z^{k} -z^{*}\bigr\Vert _{G}^{2}- \sigma\gamma(2-\gamma)\tau_{k}^{2}\bigl\Vert z^{k}-\tilde {z}^{k}\bigr\Vert ^{2}_{G} \\ \leq&\bigl\Vert z^{k} -z^{*}\bigr\Vert _{G}^{2}- \frac{\sigma\gamma(2-\gamma)(2\beta-\sqrt {3})^{2}}{4\beta^{2}}\bigl\Vert z^{k}-\tilde{z}^{k}\bigr\Vert ^{2}_{G}. \end{aligned}

Then we have

$$\bigl\Vert z^{k+1}(\tau_{k})-z^{*}\bigr\Vert _{G}\leq\bigl\Vert z^{k}-z^{*}\bigr\Vert _{G} \leq\cdots\leq\bigl\Vert z^{0}-z^{*}\bigr\Vert _{G},$$

and thus, $$\{ z^{k}\}$$ is a bounded sequence.

It follows from (3.29) that

$$\sum_{k=0}^{\infty}c\bigl\Vert z^{k}-\tilde{z}^{k}\bigr\Vert ^{2}_{G}< + \infty,$$

which means that

$$\lim_{k\to\infty}\bigl\Vert z^{k}- \tilde{z}^{k}\bigr\Vert _{G}=0.$$
(3.30)

Since $$\{ z^{k}\}$$ is a bounded sequence, we conclude that $$\{\tilde {z}^{k}\}$$ is also bounded. □

The global convergence of the proposed method can be proved by similar arguments as in . Hence the proof is omitted.

### Theorem 3.2

The sequence $$\{z^{k}\}$$ generated by the proposed method converges to some $$z^{\infty}$$ which is a solution of $$\mathrm{SVI}_{3}$$.

Now, we are ready to present the $$O(1/t)$$ convergence rate of the proposed method.

### Theorem 3.3

For any integer $$t > 0$$, we have a $$\hat{z}_{t}\in\mathcal{Z}$$ which satisfies

$$(\hat{z}_{t}-z)^{\top} Q(z)\leq\frac{1}{2\gamma\sigma\Upsilon_{t}}\bigl\Vert z-z^{0}\bigr\Vert _{G}^{2},\quad\forall z\in \mathcal{Z},$$

where

$$\hat{z}_{t}=\frac{1}{\Upsilon_{t}}\sum_{k=0}^{t} \tau_{k}\hat{z}^{k}\quad \textit{and} \quad\Upsilon_{t}= \sum_{k=0}^{t}\tau_{k}.$$

### Proof

Summing the inequality (3.24) over $$k=0,\ldots,t$$, we obtain

$$\Biggl( \Biggl(\sum_{k=0}^{t}\gamma\sigma \tau_{k} \Biggr) z-\sum_{k=0}^{t} \gamma\sigma\tau_{k}\hat{z}^{k} \Biggr)^{\top} Q(z)+ \frac{1}{2}\bigl\Vert z-z^{0}\bigr\Vert _{G}^{2} \geq0.$$

Using the notations of $$\Upsilon_{t}$$ and $$\hat{z}_{t}$$ in the above inequality, we derive

$$(\hat{z}_{t}-z)^{\top} Q(z)\leq\frac{1}{2\gamma\sigma\Upsilon_{t}}\bigl\Vert z-z^{0}\bigr\Vert _{G}^{2},\quad\forall z\in \mathcal{Z}.$$

Indeed, $$\hat{z}_{t}\in\mathcal{Z}$$ because it is a convex combination of $$\hat{z}^{0}$$, $$\hat{z}^{1}$$, … , $$\hat{z}^{t}$$. The proof is completed. □

### Remark 3.1

It follows from (2.13) that

$$\Upsilon_{t}\geq\frac{2\beta-\sqrt{3}}{2\beta} (t+1 ).$$

Suppose that, for any compact set $$\mathcal {D}\subset\mathcal{Z}$$, let $$d =\sup\{\Vert z-z^{0}\Vert _{G}\vert z\in\mathcal {D}\}$$. For any given $$\epsilon>0$$, after at most

$$t= \biggl[\frac{\beta d^{2}}{(2\beta-\sqrt{3})\gamma\sigma\epsilon } \biggr]$$

iterations, we have

$$(\hat{z}_{t}-z)^{T}Q(z)\leq\epsilon,\quad \forall z\in\mathcal {D}.$$

That is, the $$O(1/t)$$ convergence rate is established in an ergodic sense.

## Preliminary computational results

In this section, we present some numerical examples to illustrate the proposed method. We consider the following optimization problem with matrix variables, which is studied in :

$$\min \biggl\{ \frac{1}{2} \Vert U-C\Vert _{F}^{2}\Big\vert U\in S_{+}^{n} \biggr\} ,$$
(4.1)

where $$\Vert \cdot \Vert _{F}$$ is the matrix Fröbenius norm, i.e., $$\Vert C\Vert _{F} = ( \sum_{i=1}^{n}\sum_{j=1}^{n} \vert C_{ij}\vert ^{2} )^{1/2}$$ and

$$S_{+}^{n}=\bigl\{ M\in\mathbb{R}^{n\times n} : M^{\top} =M, M \succeq 0 \bigr\} .$$

It has been shown in  that solving problem (4.1) is equivalent to the following variational inequality problem: Find $$X^{*}=(U^{*},V^{*},Z^{*})\in\Omega= S_{+}^{n} \times S_{+}^{n}\times \mathbb{R}^{n\times n}$$ such that

\begin{aligned} \textstyle\begin{cases} \langle U-U^{*},(U^{*}-C)-Z^{*}\rangle\geq0, \\ \langle V-V^{*},(V^{*}-C)+Z^{*}\rangle\geq0,&\forall X=(U,V,Z)\in \Omega,\\ U^{*} - V^{*} =0. \end{cases}\displaystyle \end{aligned}
(4.2)

The problem (4.2) is a special case of (1.3)-(1.4) with matrix variables where $$A_{1}=I_{n\times n}$$, $$A_{2}=-I_{n\times n}$$, $$b=0$$, $$f_{1}(U)=U-C$$, $$f_{2}(V)=V-C$$, and $$\mathcal{W} =S_{+}^{n} \times S_{+}^{n}\times\mathbb{R}^{n\times n}$$. For simplification, we take $$R_{1}=r_{1}I_{n\times n}$$, $$R_{2}=r_{2}I_{n\times n}$$, and $$H=I_{n\times n}$$ where $$r_{1}>0$$ and $$r_{2}>0$$ are scalars. In all tests we take $$\mu=0.5$$, $$\beta=0.88$$, $$C=\operatorname{rand}(n)$$, and $$(U^{0}, V^{0}, Z^{0})=(I_{n\times n},I_{n\times n},0_{n\times n})$$ as the initial point in the test. The iteration is stopped as soon as

$$\max\bigl\{ \bigl\Vert U^{k}-\tilde{U}^{k}\bigr\Vert , \bigl\Vert V^{k}-\tilde{V}^{k}\bigr\Vert ,\bigl\Vert Z^{k}-\tilde{Z}^{k}\bigr\Vert \bigr\} \leq10^{-6}.$$

All codes were written in Matlab, we compare the proposed method with those in  and . The iteration numbers, denoted by k, and the computational time for the problem (4.1) with different dimensions are given in Tables 1-2.

Tables 1-2 show that the proposed method is more flexible and efficient for the problem tested.

## Conclusions

In this paper, we proposed a new modified logarithmic-quadratic proximal alternating direction method for solving structured variational inequalities with three separable operators. The prediction point is obtained by solving series of related systems of nonlinear equations in a parallel way. Global convergence of the proposed method is proved under mild assumptions. We have proved the $$O(1/t)$$ convergence rate of the parallel LQP alternating direction method. Some preliminary numerical examples are reported to verify the effectiveness of the proposed method in practice.

## References

1. Fortin, M, Glowinski, R: Augmented Lagrangian Methods: Applications to the Solution of Boundary-Valued Problems. North-Holland, Amsterdam (1983)

2. Gabay, D: Applications of the method of multipliers to variational inequalities. In: Fortin, M, Glowinski, R (eds.) Augmented Lagrangian Methods: Applications to the Solution of Boundary-Valued Problems. Studies in Mathematics and Its Applications, vol. 15, pp. 299-331. North-Holland, Amsterdam, The Netherlands (1983)

3. Gabay, D, Mercier, B: A dual algorithm for the solution of nonlinear variational problems via finite-element approximations. Comput. Math. Appl. 2, 17-40 (1976)

4. Glowinski, R: Numerical Methods for Nonlinear Variational Problems. Springer, New York (1984)

5. Glowinski, R, Le Tallec, P: Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics. SIAM Studies in Applied Mathematics. SIAM, Philadelphia, PA (1989)

6. He, BS, Yang, H: Some convergence properties of a method of multipliers for linearly constrained monotone variational inequalities. Oper. Res. Lett. 23, 151-161 (1998)

7. Hou, LS: On the $$O(1/t)$$ convergence rate of the parallel descent-like method and parallel splitting augmented Lagrangian method for solving a class of variational inequalities. Appl. Math. Comput. 219, 5862-5869 (2013)

8. Jiang, ZK, Bnouhachem, A: A projection-based prediction-correction method for structured monotone variational inequalities. Appl. Math. Comput. 202, 747-759 (2008)

9. Kontogiorgis, S, Meyer, RR: A variable-penalty alternating directions method for convex optimization. Math. Program. 83, 29-53 (1998)

10. Tao, M, Yuan, XM: On the $$O(1/t)$$ convergence rate of alternating direction method with logarithmic-quadratic proximal regularization. SIAM J. Optim. 22(4), 1431-1448 (2012)

11. Wang, K, Xu, LL, Han, DR: A new parallel splitting descent method for structured variational inequalities. J. Ind. Manag. Optim. 10(2), 461-476 (2014)

12. Yuan, XM, Li, M: An LQP-based decomposition method for solving a class of variational inequalities. SIAM J. Optim. 21(4), 1309-1318 (2011)

13. Bnouhachem, A: On LQP alternating direction method for solving variational. J. Inequal. Appl. 2014, 80 (2014)

14. Bnouhachem, A, Ansari, QH: A descent LQP alternating direction method for solving variational inequality problems with separable structure. Appl. Math. Comput. 246, 519-532 (2014)

15. Bnouhachem, A, Benazza, H, Khalfaoui, M: An inexact alternating direction method for solving a class of structured variational inequalities. Appl. Math. Comput. 219, 7837-7846 (2013)

16. Bnouhachem, A, Xu, MH: An inexact LQP alternating direction method for solving a class of structured variational inequalities. Comput. Math. Appl. 67, 671-680 (2014)

17. Li, M: A hybrid LQP-based method for structured variational inequalities. J. Comput. Math. 89(10), 1412-1425 (2012)

18. Bnouhachem, A, Hamdi, A: Parallel LQP alternating direction method for solving variational inequality problems with separable structure. J. Inequal. Appl. 2014, 392 (2014)

19. Facchinei, F, Pang, JS: Finite-Dimensional Variational Inequalities and Complementarity Problems, Vol. I and II. Springer Series in Operations Research. Springer, New York (2003)

## Acknowledgements

In this research, the second author was funded by the Deanship of Scientific Research (DSR), King Abdulaziz University, Jeddah. Therefore, he acknowledges with thanks DSR for the technical and financial support. The research part of the third author was done during his visit to King Fahd University of Petroleum & Minerals (KFUPM), Dhahran, Saudi Arabia. He is grateful to KFUPM for providing excellent research facilities to carry out his part of this research.

## Author information

Authors

### Corresponding author

Correspondence to Abdellah Bnouhachem.

### Competing interests

The authors declare that they have no competing interests.

### Authors’ contributions

All authors contributed equally and significantly in writing this article. All authors read and approved the final manuscript.

Abdellah Bnouhachem, Abdul Latif and Qamrul Hasan Ansari contributed equally to this work.

## Rights and permissions 