On the $O(1/t)$ convergence rate of the alternating direction method with LQP regularization for solving structured variational inequality problems

Abstract

In this paper, we propose a parallel descent LQP alternating direction method for solving structured variational inequality with three separable operators. The $O(1/t)$ convergence rate for this method is studied. We also present some numerical examples to illustrate the efficiency of the proposed method. The results presented in this paper extend and improve some well-known results in the literature.

1 Introduction

Let $\mathbb{R}^{n}_{+} = \{ x = (x_{1},x_{2}, \ldots,x_{n}) \in \mathbb{R}^{n} : x_{i} \geq0\ \forall i=1,2, \ldots, n \}$ and $\mathbb{R}^{n}_{++} = \{ x = (x_{1},x_{2}, \ldots,x_{n}) \in\mathbb {R}^{n} : x_{i} > 0 \ \forall i=1,2, \ldots, n \}$. The variational inequality problem is to find

$$ x\in\Omega:= \bigl\{ (u,v) : u\in\mathbb{R}_{+}^{n}, v\in \mathbb{R}_{+}^{m}, A_{1}u + A_{2}v = b \bigr\} $$

such that

$$ \bigl(x'- x\bigr)^{T} F(x) \ge0, \quad \forall x'\in\Omega, $$

(1.1)

with

$$ x=\left ( \textstyle\begin{array}{c} u \\ v \end{array}\displaystyle \right ) \quad \mbox{and} \quad F(x)=\left ( \textstyle\begin{array}{c} f_{1}(u)\\ f_{2}(v) \end{array}\displaystyle \right ), $$

(1.2)

where $A_{1}\in\mathbb{R}^{l\times n}$, $A_{2} \in\mathbb{R}^{l\times m}$ are given matrices, $b\in\mathbb{R}^{l}$ is a given vector, and $f_{1}:\mathbb{R}_{+}^{n}\to \mathbb{R}^{n}$, $f_{2}:\mathbb{R}_{+}^{m}\to\mathbb{R}^{m}$ are given monotone operators. For further study and applications of such problems, we refer to [1–11] and the references therein. By attaching a Lagrange multiplier vector $\lambda\in\mathbb{R}^{l}$ to the linear constraints $A_{1}u + A_{2}v =b$, the problem (1.1)-(1.2) can be explained in terms of finding $z \in\mathcal{Z}' := \mathbb{R}_{+}^{n} \times\mathbb{R}_{+}^{m} \times \mathbb{R}^{l}$ such that

$$ \bigl(z'-z\bigr)^{\top} Q(z) \geq0, \quad \forall z' \in\mathcal{Z}', $$

(1.3)

where

$$ z=\left ( \textstyle\begin{array}{c} u \\ v\\ \lambda \end{array}\displaystyle \right ), \qquad Q(z)= \left ( \textstyle\begin{array}{c} f_{1}(u)-A_{1}^{\top} \lambda\\ f_{2}(v)-A_{2}^{\top} \lambda\\ A_{1}u + A_{2}v-b \end{array}\displaystyle \right ), $$

(1.4)

and $A_{1}^{\top}$ denotes the transpose of the matrix $A_{1}$. The problem (1.3)-(1.4) is referred as a structured variational inequality problem (in short, SVI).

Yuan and Li [12] developed the following logarithmic-quadratic proximal (LQP)-based decomposition method by applying the LQP terms to regularize the ADM subproblems: For a given $z^{k}=(u^{k},v^{k},\lambda^{k})\in\mathbb{R}^{n}_{++}\times\mathbb {R}^{m}_{++}\times\mathbb{R}^{l}$, and $\mu\in(0,1)$, the new iterative $(u^{k+1}, v^{k+1}, \lambda^{k+1})$ is obtained via solving the following system:

$$\begin{aligned}& f_{1}(u) - A_{1}^{\top} \bigl[ \lambda^{k} - H \bigl(A_{1} u + A_{2} v^{k} - b\bigr) \bigr] + R \bigl[ \bigl(u-u^{k}\bigr) + \mu \bigl(u^{k} - U_{k}^{2} u^{-1}\bigr) \bigr] = 0, \end{aligned}$$

(1.5)

$$\begin{aligned}& f_{2}(v) - A_{2}^{\top} \bigl[ \lambda^{k} - H (A_{1} {u} + A_{2} v - b) \bigr] + S \bigl[ \bigl(v-v^{k}\bigr) + \mu\bigl(v^{k} - V_{k}^{2} v^{-1}\bigr) \bigr] = 0, \end{aligned}$$

(1.6)

$$\begin{aligned}& \lambda^{k+1} = \lambda^{k} - H \bigl(A_{1} u^{k} + A_{2} v^{k} - b\bigr), \end{aligned}$$

(1.7)

where $H \in\mathbb{R}^{l \times l}$, $R \in\mathbb{R}^{n \times n}$, and $S \in\mathbb{R}^{m \times m}$ are symmetric positive definite.

Later, some LQP alternating direction methods have been proposed to make the LQP alternating direction method more practical, see, for example, [13–17] and the references therein. Each iteration of these methods contains a prediction and a correction, the predictor is obtained via solving (1.5)-(1.7) and the new iterate is obtained by a convex combination of the previous point and the one generated by a projection-type method along a descent direction. The main disadvantage of the methods proposed in [12–17] is that solving equation (1.6) requires the solution of equation (1.5). To overcome with this difficulty, Bnouhachem and Hamdi [18] proposed a parallel descent LQP alternating direction method for solving SVI.

In this paper, we propose a parallel descent LQP alternating direction method for solving the following structured variational inequality with three separable operators: Find $y\in\Omega:= \{ (u,v,w) : u\in\mathbb {R}_{+}^{n_{1}}, v\in\mathbb{R}_{+}^{n_{2}}, w\in\mathbb{R}_{+}^{n_{2}}, A_{1}u + A_{2}v +A_{3}w= b \}$ such that

$$ \bigl(y'- y\bigr)^{\top} F(y) \ge0, \quad \forall y'\in\Omega, $$

(1.8)

with

$$ y=\left ( \textstyle\begin{array}{c} u \\ v\\ w \end{array}\displaystyle \right ),\qquad F(y)= \left ( \textstyle\begin{array}{c} f_{1}(u)\\ f_{2}(v)\\ f_{3}(w) \end{array}\displaystyle \right ), $$

(1.9)

where $A_{1}\in\mathbb{R}^{m\times n_{1}}$, $A_{2} \in\mathbb{R}^{m\times n_{2}}$, $A_{3}\in\mathbb{R}^{m\times n_{3}}$ are given matrices, $b\in\mathbb{R}^{m}$ is a given vector, and $f_{1}:\mathbb {R}_{+}^{n_{1}}\to\mathbb{R}^{n_{1}}$, $f_{2}:\mathbb{R}_{+}^{n_{2}}\to\mathbb{R}^{n_{2}}$, $f_{3}:\mathbb {R}_{+}^{n_{3}}\to\mathbb{R}^{n_{3}}$ are given monotone operators. By attaching a Lagrange multiplier vector $\lambda\in\mathbb{R}^{m}$ to the linear constraints $A_{1}u + A_{2}v +A_{3}w= b$, the problem (1.8)-(1.9) can be explained in terms of finding $z \in\mathcal{Z} := \mathbb{R}_{+}^{n_{1}} \times\mathbb {R}_{+}^{n_{2}} \times\mathbb{R}_{+}^{n_{3}} \times\mathbb{R}^{m}$ such that

$$ \bigl(z'-z\bigr)^{\top} Q(z) \geq0, \quad \forall z' \in \mathcal{Z}, $$

(1.10)

where

$$ z=\left ( \textstyle\begin{array}{c} u\\ v\\ w\\ \lambda \end{array}\displaystyle \right ) \quad \mbox{and} \quad Q(z)=\left ( \textstyle\begin{array}{c} f_{1}(u)-A_{1}^{\top} \lambda\\ f_{2}(v)-A_{2}^{\top} \lambda\\ f_{3}(w)-A_{3}^{\top} \lambda\\ A_{1}u + A_{2}v +A_{3}w- b \end{array}\displaystyle \right ). $$

(1.11)

The problem (1.10)-(1.11) is referred as $\mathrm{SVI}_{3}$.

The main aim of this paper is to present the parallel descent LQP alternating direction method for solving $\mathrm{SVI}_{3}$ and to investigate the convergence rate of this method. We show that the proposed method has the $O(1/t)$ convergence rate. The iterative algorithm and results presented in this paper generalize, unify, and improve the previously known results in this area.

2 The proposed method

For any vector $u\in\mathbb{R}^{n}$, $\Vert u\Vert _{\infty}=\max\{\vert u_{1}\vert , \ldots ,\vert u_{n}\vert \}$. Let $D \in\mathbb{R}^{n \times n}$ be a symmetry positive definite matrix, we denote the D-norm of u by $\Vert u\Vert _{D}^{2}=u^{T}Du$.

The following lemma provides a basic property of projection operator onto a closed convex subset Ω of $\mathbb{R}^{l}$. We denote by $P_{\Omega,D}(\cdot)$ the projection operator under the D-norm, that is,

$$P_{\Omega,D}(v)=\operatorname{argmin} \bigl\{ \Vert v-u\Vert _{D} : u \in\Omega\bigr\} . $$

Lemma 2.1

Let D be a symmetry positive definite matrix and Ω be a nonempty closed convex subset of $\mathbb{R}^{l}$. Then

$$ \bigl(z-P_{\Omega,D}[z]\bigr)^{\top} D \bigl(P_{\Omega,D}[z]-v\bigr) \geq0, \quad \forall z \in\mathbb{R}^{l}, v\in\Omega. $$

(2.1)

We make the following standard assumptions.

Assumption 2.1

$f_{1}$ is monotone with respect to $\mathbb{R}^{n_{1}}_{+}$, that is, $(f_{1}(x)-f_{1}(y))^{T}(x-y)\ge0$, $\forall x,y\in\mathbb{R}^{n_{1}}_{+}$, $f_{2}$ is monotone with respect to $\mathbb{R}^{n_{2}}_{+}$, and $f_{3}$ is monotone with respect to $\mathbb{R}^{n_{3}}_{+}$.

Assumption 2.2

The solution set of $\mathrm{SVI}_{3}$, denoted by $\mathcal {Z}^{*}$, is nonempty.

We propose the following parallel LQP alternating direction method for solving $\mathrm{SVI}_{3}$:

Algorithm 2.1

• Step 0. Given $\varepsilon>0$, $\mu\in(0,1)$, $\beta\in ( \frac {\sqrt{3}}{2},1 )$, $\gamma\in(0,2)$ and $z^{0}=(u^{0}, v^{0},w^{0}, \lambda^{0}) \in\mathbb{R}^{n_{1}}_{++}\times \mathbb{R}^{n_{2}}_{++}\times\mathbb{R}^{n_{3}}_{++}\times\mathbb{R}^{m}$. Set $k=0$.

• Step 1. Compute $\tilde{z}^{k}=(\tilde{u}^{k},\tilde{v}^{k},\tilde{w}^{k},\tilde {\lambda}^{k})\in\mathbb{R}^{n_{1}}_{++}\times \mathbb{R}^{n_{2}}_{++}\times\mathbb{R}^{n_{3}}_{++}\times\mathbb {R}^{m}$ by solving the following system:

  $$\begin{aligned}& f_{1}(u) - A_{1}^{\top} \bigl[\lambda^{k} - H \bigl(A_{1} u + A_{2} v^{k} +A_{3} w^{k}- b\bigr) \bigr] \\& \quad {} + R_{1}\bigl[\bigl(u-u^{k}\bigr)+ \mu\bigl(u^{k} - U_{k}^{2} u^{-1}\bigr)\bigr]=0, \end{aligned}$$

  (2.2)
  $$\begin{aligned}& f_{2}(v) - A_{2}^{\top} \bigl[ \lambda^{k} - H \bigl(A_{1} u^{k} + A_{2} v +A_{3} w^{k}- b\bigr)\bigr] \\& \quad {}+ {R_{2}}\bigl[ \bigl(v-v^{k}\bigr) + \mu\bigl(v^{k} - V_{k}^{2} v^{-1}\bigr)\bigr]=0, \end{aligned}$$

  (2.3)
  $$\begin{aligned}& f_{3}(w) - A_{3}^{\top} \bigl[ \lambda^{k} - H \bigl(A_{1} u^{k} + A_{2} v^{k}+A_{3} w - b\bigr)\bigr] \\& \quad {}+ {R_{3}}\bigl[ \bigl(w-w^{k}\bigr) + \mu\bigl(w^{k} - W_{k}^{2} w^{-1}\bigr)\bigr]=0, \end{aligned}$$

  (2.4)
  $$\begin{aligned}& \tilde{\lambda}^{k} = \lambda^{k} - \beta H \bigl(A_{1} \tilde{u}^{k} + A_{2} \tilde{v}^{k}+A_{3} \tilde{w}^{k} - b\bigr) , \end{aligned}$$

  (2.5)

  where $H \in\mathbb{R}^{m \times m}$, ${R_{1}} \in\mathbb{R}^{n_{1} \times n_{1}}$, ${R_{2}} \in\mathbb{R}^{n_{2} \times n_{2}}$ and ${R_{3}} \in\mathbb{R}^{n_{3} \times n_{3}}$ are symmetric positive definite matrices. $U_{k}$, $V_{k}$, and $W_{k}$ are positive definite diagonal matrices defined by $U_{k} =\operatorname{diag} ( u^{k}_{1}, \ldots,u^{k}_{n} )$, $V_{k} = \operatorname{diag} ( v^{k}_{1}, \ldots,v^{k}_{n} )$, $W_{k} = \operatorname{diag} ( w^{k}_{1}, \ldots,w^{k}_{n} )$.

• Step 2. If $\max\{\Vert u^{k}-\tilde{u}^{k}\Vert _{\infty}, \Vert v^{k}-\tilde{v}^{k}\Vert _{\infty}, \Vert w^{k}-\tilde{w}^{k}\Vert _{\infty}, \Vert \lambda^{k}-\tilde{\lambda}^{k}\Vert _{\infty}\}<\epsilon$, then stop.

• Step 3. The new iterate $z^{k+1}(\tau_{k})=(u^{k+1},v^{k+1},w^{k+1},\lambda ^{k+1})$ is given by

  $$ z^{k+1}(\tau_{k})= (1-\sigma) z^{k}+ \sigma P_{\mathcal{Z},G}\bigl[z^{k}-\gamma \tau_{k}G^{-1}g \bigl(z^{k},\tilde{z}^{k}\bigr)\bigr], \quad\sigma\in(0,1), $$

  (2.6)

  where

  $$\begin{aligned}& \tau_{k}=\frac{\varphi(z^{k},\tilde{z}^{k})}{\Vert z^{k}- \tilde{z}^{k}\Vert ^{2}_{G}}, \end{aligned}$$

  (2.7)
  $$\begin{aligned}& \varphi\bigl(z^{k},\tilde{z}^{k}\bigr)=\bigl\Vert z^{k}- \tilde{z}^{k}\bigr\Vert _{M}^{2} \\& \hphantom{ \varphi\bigl(z^{k},\tilde{z}^{k}\bigr)=}{}+ \frac {1}{\beta}\bigl(\lambda^{k}-\tilde{\lambda}^{k} \bigr)^{T} \bigl(A_{1}\bigl(u^{k}- \tilde{u}^{k}\bigr) +A_{2}\bigl(v^{k}- \tilde{v}^{k}\bigr)+A_{3}\bigl(w^{k}- \tilde{w}^{k}\bigr) \bigr), \end{aligned}$$

  (2.8)
  $$\begin{aligned}& g(z^{k},\tilde{z}^{k}) \\& \quad = \left ( \textstyle\begin{array}{c} f_{1}(\tilde{u}^{k}) - A_{1}^{\top} \tilde{\lambda}^{k}+A_{1}^{\top} H[A_{1}(u^{k}-\tilde{u}^{k}) +A_{2}(v^{k}-\tilde{v}^{k})+A_{3}(w^{k}-\tilde{w}^{k}) +\frac{1-\beta}{\beta }H^{-1}(\lambda^{k}-\tilde{\lambda}^{k})]\\ f_{2}(\tilde{v}^{k}) - A_{2}^{\top} \tilde{\lambda}^{k}+ A_{2}^{\top} H[A_{1}(u^{k}-\tilde{u}^{k})+A_{2}(v^{k}-\tilde{v}^{k})+A_{3}(w^{k}-\tilde{w}^{k}) +\frac{1-\beta}{\beta}H^{-1}(\lambda^{k}-\tilde{\lambda}^{k})]\\ f_{3}(\tilde{w}^{k}) - A_{3}^{\top} \tilde{\lambda}^{k}+ A_{3}^{\top} H[A_{1}(u^{k}-\tilde{u}^{k})+A_{2}(v^{k}-\tilde{v}^{k})+A_{3}(w^{k}-\tilde{w}^{k}) +\frac{1-\beta}{\beta}H^{-1}(\lambda^{k}-\tilde{\lambda}^{k})]\\ A_{1}\tilde{u}^{k}+A_{2}\tilde{v}^{k}+A_{3}\tilde{w}^{k}-b \end{array}\displaystyle \right ), \\& \\& G=\left ( \textstyle\begin{array}{c@{\quad}c@{\quad}c@{\quad}c} (1+\mu){R_{1}}+A_{1}^{\top} HA_{1} & 0& 0 &0 \\ 0 & (1+\mu){R_{2}} +A_{2}^{\top} HA_{2} & 0&0 \\ 0 &0 &(1+\mu){R_{3}} +A_{3}^{\top} HA_{3} & 0 \\ 0 & 0 &0 & \frac{1}{\beta}H^{-1} \end{array}\displaystyle \right ), \end{aligned}$$

  (2.9)

  and

  $$M=\left ( \textstyle\begin{array}{c@{\quad}c@{\quad}c@{\quad}c} {R_{1}} +A_{1}^{\top}HA_{1} & 0& 0 &0 \\ 0 & {R_{2}} +A_{2}^{\top} HA_{2} & 0&0 \\ 0 &0 & {R_{3}} +A_{3}^{\top} HA_{3} & 0 \\ 0 & 0 &0 & \frac{1}{\beta}H^{-1} \end{array}\displaystyle \right ). $$

Set $k:= k + 1$ and go to Step 1.

Remark 2.1

As special cases, we can obtain some new LQP alternating methods as follows:

• (a) If $u^{k+1}=\tilde{u}^{k}$, $v^{k+1}=\tilde{v}^{k}$, $w^{k+1}=\tilde{w}^{k}$ and $\lambda^{k+1}=\tilde{\lambda}^{k}$ in (2.2), (2.3), (2.4) and (2.5), respectively, we obtain a new method which can be viewed as an extension of that proposed in [10] for solving structured variational inequality with three separable operators in a parallel way.

• (b) If $u^{k+1}=\tilde{u}^{k}$, $v^{k+1}=\tilde{v}^{k}$, $w^{k+1}=\tilde{w}^{k}$, $\lambda^{k+1}=\tilde{\lambda}^{k}$, and $\beta=1$ in (2.2), (2.3), (2.4) and (2.5), respectively, we obtain a new method which can be viewed as an extension of that proposed in [12] for solving structured variational inequality with three separable operators in a parallel wise.

• (c) If $\beta=1$, the proposed method can be viewed as an extension of that proposed in [18] for solving structured variational inequality with three separable operators.

We need the following result in the convergence analysis of the proposed method.

Lemma 2.2

[12]

Let $q(u)\in\mathbb{R}^{n}$ be a monotone mapping of u with respect to $\mathbb{R}^{n}_{+}$ and $R\in\mathbb{R}^{n\times n}$ be a positive definite diagonal matrix. For a given $u^{k} >0$, if $U_{k}:= \operatorname {diag}(u^{k}_{1},u^{k}_{2},\ldots, u^{k}_{n})$ (the diagonal matrix with elements $u^{k}_{1}$, $u^{k}_{2}$, … , $u^{k}_{n}$) and $u^{-1}$ be an n-vector whose jth element is $1/u_{j}$, then the equation

$$ q(u) + R\bigl[\bigl(u -u^{k}\bigr) + \mu\bigl( u^{k} - U_{k}^{2} u^{-1}\bigr)\bigr]=0 $$

(2.10)

has a unique positive solution u. Moreover, for any $v\geq0$, we have

$$ (v-u)^{\top} q(u) \geq{ \frac{1+\mu}{2} } \bigl(\Vert u - v \Vert _{R}^{2} - \bigl\Vert u^{k} - v \bigr\Vert _{R}^{2} \bigr) + { \frac{1-\mu}{2} }\bigl\Vert u^{k} - u \bigr\Vert _{R}^{2}. $$

(2.11)

The next theorem is useful for the convergence analysis.

Theorem 2.1

For given $z^{k}\in \mathbb{R}^{n_{1}}_{++}\times\mathbb{R}^{n_{2}}_{++}\times \mathbb{R}^{n_{3}}_{++}\times\mathbb{R}^{m}$, let $\tilde{z}^{k}$ be generated by (2.2)-(2.5). Then

$$\begin{aligned} \varphi\bigl(z^{k},\tilde{z}^{k}\bigr)\geq \frac{2\beta-\sqrt{3}}{2\beta} \bigl\Vert z^{k}- \tilde{z}^{k}\bigr\Vert ^{2}_{G} \end{aligned}$$

(2.12)

and

$$\begin{aligned} \tau_{k}\geq\frac{2\beta-\sqrt{3}}{2\beta}. \end{aligned}$$

(2.13)

Proof

It follows from (2.8) that

$$\begin{aligned} \varphi\bigl(z^{k},\tilde{z}^{k}\bigr) = & \bigl\Vert z^{k}- \tilde{z}^{k}\bigr\Vert _{M}^{2}+\frac{1}{\beta}\bigl(\lambda^{k}-\tilde {\lambda}^{k}\bigr)^{\top}\bigl(A_{1} \bigl(u^{k}-\tilde{u}^{k}\bigr) +A_{2} \bigl(v^{k}-\tilde{v}^{k}\bigr)+A_{3} \bigl(w^{k}-\tilde{w}^{k}\bigr)\bigr) \\ = & \bigl\Vert u^{k}-\tilde{u}^{k}\bigr\Vert _{R_{1}}^{2}+\bigl\Vert A_{1}u^{k}-A_{1} \tilde{u}^{k}\bigr\Vert _{H}^{2}+\bigl\Vert v^{k}-\tilde{v}^{k}\bigr\Vert _{R_{2}}^{2} +\bigl\Vert A_{2}v^{k} -A_{2} \tilde{v}^{k} \bigr\Vert _{H}^{2} \\ &{}+\bigl\Vert w^{k}-\tilde{w}^{k}\bigr\Vert _{R_{3}}^{2}+\bigl\Vert A_{3}w^{k} -A_{3}\tilde{w}^{k}\bigr\Vert _{H}^{2} +\frac{1}{\beta}\bigl\Vert \lambda^{k}-\tilde{\lambda}^{k} \bigr\Vert _{H^{-1}}^{2} \\ &{}+\frac{1}{\beta}\bigl(\lambda^{k}-\tilde{\lambda}^{k} \bigr)^{\top }\bigl(A_{1}\bigl(u^{k}- \tilde{u}^{k}\bigr)+A_{2}\bigl(v^{k}- \tilde{v}^{k}\bigr)+A_{3}\bigl(w^{k}- \tilde{w}^{k}\bigr)\bigr). \end{aligned}$$

(2.14)

By using the Cauchy-Schwarz inequality, we have

$$\begin{aligned}& \bigl(\lambda^{k}-\tilde{\lambda}^{k} \bigr)^{\top}\bigl(A_{1}\bigl(u^{k}- \tilde{u}^{k}\bigr)\bigr) \geq-\frac{1}{2} \biggl(\sqrt{3}\bigl\Vert A_{1}\bigl(u^{k}-\tilde{u}^{k}\bigr)\bigr\Vert _{H}^{2} +\frac{1}{\sqrt{3}}\bigl\Vert \lambda^{k}-\tilde{\lambda}^{k}\bigr\Vert _{H^{-1}}^{2} \biggr), \end{aligned}$$

(2.15)

$$\begin{aligned}& \bigl(\lambda^{k}-\tilde{\lambda}^{k} \bigr)^{\top}\bigl(A_{2}\bigl(v^{k}- \tilde{v}^{k}\bigr)\bigr) \geq-\frac{1}{2} \biggl(\sqrt{3}\bigl\Vert A_{2}\bigl(v^{k}-\tilde{v}^{k}\bigr)\bigr\Vert _{H}^{2} +\frac{1}{\sqrt{3}}\bigl\Vert \lambda^{k}-\tilde{\lambda}^{k}\bigr\Vert _{H^{-1}}^{2} \biggr) , \end{aligned}$$

(2.16)

and

$$\begin{aligned} \bigl(\lambda^{k}-\tilde{\lambda}^{k} \bigr)^{\top}\bigl(A_{3}\bigl(w^{k}-\tilde{w}^{k} \bigr)\bigr) \geq-\frac{1}{2} \biggl(\sqrt{3}\bigl\Vert A_{3} \bigl(w^{k}-\tilde{w}^{k}\bigr)\bigr\Vert _{H}^{2} +\frac{1}{\sqrt{3}}\bigl\Vert \lambda^{k}-\tilde{\lambda}^{k}\bigr\Vert _{H^{-1}}^{2} \biggr). \end{aligned}$$

(2.17)

Substituting (2.15), (2.16), and (2.17) into (2.14), we get

$$\begin{aligned} \varphi\bigl(z^{k},\tilde{z}^{k}\bigr) \geq & \frac{2\beta-\sqrt{3}}{2\beta} \bigl(\bigl\Vert A_{1}u^{k}-A_{1} \tilde{u}^{k}\bigr\Vert _{H}^{2} +\bigl\Vert A_{2}v^{k} -A_{2}\tilde{v}^{k} \bigr\Vert _{H}^{2}+\bigl\Vert A_{3}w^{k} -A_{3}\tilde{w}^{k} \bigr\Vert _{H}^{2} \bigr)\\ &{} +\frac{2-\sqrt{3}}{2\beta}\bigl\Vert \lambda^{k}-\tilde{ \lambda}^{k}\bigr\Vert _{H^{-1}}^{2} +\bigl\Vert u^{k}-\tilde{u}^{k}\bigr\Vert _{R_{1}}^{2}+\bigl\Vert v^{k}- \tilde{v}^{k}\bigr\Vert _{R_{2}}^{2} +\bigl\Vert w^{k}-\tilde{w}^{k}\bigr\Vert _{R_{3}}^{2} \\ {\geq}& \frac{2\beta-\sqrt{3}}{2\beta} \biggl(\bigl\Vert A_{1}u^{k}-A_{1} \tilde {u}^{k}\bigr\Vert _{H}^{2}+\bigl\Vert A_{2}v^{k} -A_{2}\tilde{v}^{k} \bigr\Vert _{H}^{2}\\ &{}+\bigl\Vert A_{3}w^{k}-A_{3}\tilde{w}^{k}\bigr\Vert _{H}^{2} + \frac{1}{\beta}\bigl\Vert \lambda^{k}-\tilde{\lambda}^{k} \bigr\Vert _{H^{-1}}^{2} \biggr) \\ &{}+\frac{2\beta-\sqrt{3}}{2\beta} \bigl(\bigl\Vert u^{k}-\tilde{u}^{k} \bigr\Vert _{R_{1}}^{2} +\bigl\Vert v^{k}- \tilde{v}^{k}\bigr\Vert _{R_{2}}^{2}+\bigl\Vert w^{k}-\tilde{w}^{k}\bigr\Vert _{R_{3}}^{2} \bigr) \\ =&\frac{2\beta-\sqrt{3}}{2\beta} \bigl(\bigl\Vert z^{k}-\tilde{z}^{k} \bigr\Vert ^{2}_{G}+(1-\mu)\bigl\Vert u^{k}- \tilde{u}^{k}\bigr\Vert _{R_{1}}^{2}\\ &{} +(1-\mu)\bigl\Vert v^{k}-\tilde{v}^{k}\bigr\Vert _{R_{2}}^{2}+(1- \mu)\bigl\Vert w^{k}-\tilde {w}^{k}\bigr\Vert _{R_{3}}^{2} \bigr) \\ \geq &\frac{2\beta-\sqrt{3}}{2\beta}\bigl\Vert z^{k}-\tilde {z}^{k}\bigr\Vert ^{2}_{G}. \end{aligned}$$

Therefore, it follows from (2.7) and (2.12) that

$$ \tau_{k}\geq\frac{2\beta-\sqrt{3}}{2\beta}, $$

(2.18)

and this completes the proof. □

3 Convergence rate

Recall that $\mathcal{Z}^{*}$ can be characterized as (see (2.3.2) in p.159 of [19])

$$\mathcal{Z}^{*}=\bigcap_{z\in\mathcal{Z}} \bigl\{ \hat{z}\in \mathcal{Z} : (z-\hat{z})^{\top} Q(z)\geq0 \bigr\} . $$

This implies that ẑ is an approximate solution of $\mathrm{SVI}_{3}$ with the accuracy $\epsilon>0$ if it satisfies

$$ \hat{z}\in\mathcal{Z}\quad\hbox{and}\quad\sup_{z\in\mathcal {Z}} \bigl\{ (z-\hat{z})^{\top} Q(z)\bigr\} \leq\epsilon. $$

(3.1)

Now we show that after t iterations of the proposed method, we can find a $\hat{z}\in\mathcal{Z}$ such that (3.1) is satisfied with $\epsilon= O(1/t)$.

We introduce the following matrices,

$$ N=\left ( \textstyle\begin{array}{c@{\quad}c@{\quad}c@{\quad}c} I & 0& 0 &0 \\ 0 & I & 0& 0 \\ 0 & 0 & I& 0 \\ -\beta HA_{1} & -\beta HA_{2} &-\beta HA_{3} & \beta I \end{array}\displaystyle \right ) $$

(3.2)

and

$$ J=\left ( \textstyle\begin{array}{c@{\quad}c@{\quad}c@{\quad}c} (1+\mu){R_{1}}+A_{1}^{\top} HA_{1} & 0& 0& 0 \\ 0 &(1+\mu){R_{2}}+A_{2}^{\top} H A_{2} & 0& 0 \\ 0 &0 & (1+\mu){R_{3}}+A_{3}^{\top} H A_{3} & 0 \\ -A_{1} & -A_{2} &-A_{3} & H^{-1} \end{array}\displaystyle \right ). $$

(3.3)

By simple manipulations, we can find that $J= GN$.

Our analysis needs a new sequence defined by

$$ \hat{z}^{k}= \left ( \textstyle\begin{array}{c} \hat{u}^{k} \\ \hat{v}^{k}\\ \hat{w}^{k}\\ \hat{\lambda}^{k} \end{array}\displaystyle \right )=\left ( \textstyle\begin{array}{c} \tilde{u}^{k}\\ \tilde{v}^{k}\\ \tilde{w}^{k}\\ {\lambda}^{k}-H(A_{1}u^{k}+A_{2}v^{k}+A_{3}w^{k}-b) \end{array}\displaystyle \right ). $$

(3.4)

Based on (3.2) and (3.4), we can easily have

$$ z^{k}-\tilde{z}^{k}=N\bigl(z^{k}- \hat{z}^{k}\bigr). $$

(3.5)

Using (1.11), (2.9), and (3.4), we obtain

$$ g\bigl(z^{k},\tilde{z}^{k}\bigr)=Q\bigl( \hat{z}^{k}\bigr). $$

(3.6)

Lemma 3.1

Let $\hat{z}^{k}$ be defined by (3.4), $z\in\mathcal{Z}$, and the matrix J be given by (3.3). Then

$$ \bigl(z-\hat{z}^{k}\bigr)^{\top} \bigl(Q\bigl( \hat{z}^{k}\bigr)-J\bigl(z^{k}-\hat{z}^{k}\bigr) \bigr) \geq-\mu\bigl\Vert u^{k} - \hat{u}^{k} \bigr\Vert _{R_{1}}^{2}-\mu\bigl\Vert v^{k} - \hat{v}^{k} \bigr\Vert _{R_{2}}^{2}-\mu\bigl\Vert w^{k} - \hat{w}^{k} \bigr\Vert _{R_{3}}^{2}. $$

(3.7)

Proof

Applying Lemma 2.2 to (2.2), we get

$$\begin{aligned}& \bigl(u - \tilde{u}^{k}\bigr)^{\top} \bigl\{ f_{1}\bigl(\tilde{u}^{k}\bigr) -A_{1}^{\top} \bigl[\lambda^{k} - H\bigl(A_{1} \tilde{u}^{k} + A_{2}{v}^{k}+ A_{3}{w}^{k} -b\bigr) \bigr] \bigr\} \\& \quad \geq\frac{1+\mu}{2} \bigl(\bigl\Vert \tilde{u}^{k}-u \bigr\Vert _{R_{1}}^{2} - \bigl\Vert u^{k} -u\bigr\Vert _{R_{1}}^{2} \bigr) + \frac{1-\mu}{2} \bigl\Vert u^{k} - \tilde{u}^{k} \bigr\Vert _{R_{1}}^{2}. \end{aligned}$$

(3.8)

Since

$$\begin{aligned} \bigl\Vert u^{k}-u\bigr\Vert _{R_{1}}^{2}=\bigl\Vert u^{k}-\tilde{u}^{k}\bigr\Vert _{R_{1}}^{2}+ \bigl\Vert \tilde{u}^{k}-u\bigr\Vert _{R_{1}}^{2}+2 \bigl(\tilde{u}^{k}-u\bigr)^{T}R_{1} \bigl(u^{k}-\tilde{u}^{k}\bigr), \end{aligned}$$

we have

$$ \bigl(u-\tilde{u}^{k}\bigr)^{\top} R_{1}\bigl(u^{k}-\tilde{u}^{k}\bigr) = \frac{1}{2} \bigl(\bigl\Vert \tilde{u}^{k}-u\bigr\Vert _{R_{1}}^{2}-\bigl\Vert u^{k}-u\bigr\Vert _{R_{1}}^{2} \bigr) +{\frac{1}{2}}\bigl\Vert u^{k}-\tilde{u}^{k}\bigr\Vert _{R_{1}}^{2}. $$

(3.9)

Adding (3.8) and (3.9), we obtain

$$\begin{aligned}& \bigl(u - \tilde{u}^{k}\bigr)^{\top} \bigl\{ (1+ \mu)R_{1}\bigl(u^{k}-\tilde{u}^{k}\bigr)- f_{1}\bigl(\tilde{u}^{k}\bigr) + A_{1}^{\top} \bigl[\lambda^{k} - H\bigl(A_{1}u^{k} + A_{2}{v}^{k}+ A_{3}{w}^{k} -b\bigr) \bigr] \\& \quad {}+A_{1}^{\top} HA_{1}\bigl(u^{k}- \tilde{u}^{k}\bigr) \bigr\} \leq\mu\bigl\Vert u^{k} - \tilde {u}^{k} \bigr\Vert _{R_{1}}^{2}. \end{aligned}$$

(3.10)

Similarly, applying Lemma 2.2 to (2.3), we get

$$\begin{aligned}& \bigl(v - \tilde{v}^{k}\bigr)^{\top} \bigl\{ f_{2}\bigl(\tilde{v}^{k}\bigr) - A_{2}^{\top} \bigl[\lambda^{k} - H \bigl(A_{1}u^{k} + A_{2}\tilde{v}^{k}+ A_{3}{w}^{k} -b \bigr)\bigr] \bigr\} \\& \quad \geq\frac{1+\mu}{2} \bigl(\bigl\Vert \tilde{v}^{k}-v \bigr\Vert _{R_{2}}^{2} - \bigl\Vert v^{k} -v\bigr\Vert _{R_{2}}^{2} \bigr) + \frac{1-\mu}{2} \bigl\Vert v^{k} - \tilde{v}^{k} \bigr\Vert _{R_{2}}^{2}. \end{aligned}$$

(3.11)

Similar to (3.9), we have

$$ \bigl(v-\tilde{v}^{k}\bigr)^{\top} {R_{2}}\bigl(v^{k}-\tilde{v}^{k}\bigr)={ \frac{1}{2}} \bigl(\bigl\Vert \tilde{v}^{k}-v\bigr\Vert _{R_{2}}^{2}-\bigl\Vert v^{k}-v\bigr\Vert _{R_{2}}^{2} \bigr) +{\frac{1}{2}}\bigl\Vert v^{k}-\tilde{v}^{k}\bigr\Vert _{R_{2}}^{2}. $$

(3.12)

Adding (3.11) and (3.12), we have

$$\begin{aligned}& \bigl(v - \tilde{v}^{k}\bigr)^{\top} \bigl\{ (1+ \mu){R_{2}}\bigl(v^{k}-\tilde{v}^{k}\bigr)- f_{2}\bigl(\tilde{v}^{k}\bigr) + A_{2}^{\top} \bigl[\lambda^{k} -H\bigl(A_{1}u^{k} + A_{2}{v}^{k}+ A_{3}{w}^{k} -b\bigr) \bigr] \\& \quad {}+A_{2}^{\top} HA_{2}\bigl(v^{k}- \tilde{v}^{k}\bigr) \bigr\} \leq\mu\bigl\Vert v^{k} - \tilde {v}^{k} \bigr\Vert _{R_{2}}^{2}. \end{aligned}$$

(3.13)

Similarly, we have

$$\begin{aligned}& \bigl(w - \tilde{w}^{k}\bigr)^{\top} \bigl\{ (1+ \mu){R_{3}}\bigl(w^{k}-\tilde{w}^{k}\bigr)- f_{3}\bigl(\tilde{w}^{k}\bigr) + A_{3}^{\top} \bigl[\lambda^{k} -H\bigl(A_{1}u^{k} + A_{2}{v}^{k}+ A_{3}{w}^{k} -b\bigr) \bigr] \\& \quad {}+A_{3}^{\top} HA_{3}\bigl(w^{k}- \tilde{w}^{k}\bigr) \bigr\} \le\mu\bigl\Vert w^{k} - \tilde {w}^{k} \bigr\Vert _{R_{3}}^{2}. \end{aligned}$$

(3.14)

Then, by using the notation of $\hat{z}^{k}$ in (3.4), (3.10), (3.13), and (3.14) can be written as

$$\begin{aligned}& \bigl(u - \hat{u}^{k}\bigr)^{\top} \bigl\{ (1+\mu)R_{1}\bigl(u^{k}-\hat{u}^{k}\bigr)- f_{1}(\hat {u}^{k})+A_{1}^{\top} \hat{\lambda}^{k} +A_{1}^{\top} HA_{1}\bigl(u^{k}-\hat{u}^{k}\bigr) \bigr\} \\& \quad \leq\mu\bigl\Vert u^{k} - \hat{u}^{k} \bigr\Vert _{R_{1}}^{2}, \end{aligned}$$

(3.15)

$$\begin{aligned}& \bigl(v - \hat{v}^{k}\bigr)^{\top} \bigl\{ (1+ \mu){R_{2}}\bigl(v^{k}-\hat{v}^{k}\bigr)- f_{2}\bigl(\hat{v}^{k}\bigr)+A_{2}^{\top} \hat{\lambda}^{k} +A_{2}^{\top} HA_{2} \bigl(v^{k}-\hat{v}^{k}\bigr) \bigr\} \\& \quad \le\mu\bigl\Vert v^{k} - \hat{v}^{k} \bigr\Vert _{R_{2}}^{2}, \end{aligned}$$

(3.16)

and

$$\begin{aligned}& \bigl(w - \hat{w}^{k}\bigr)^{\top} \bigl\{ (1+ \mu){R_{3}}\bigl(w^{k}-\hat{w}^{k}\bigr)- f_{3}\bigl(\hat{w}^{k}\bigr)+A_{3}^{\top} \hat{\lambda}^{k} +A_{3}^{\top} HA_{3} \bigl(w^{k}-\hat{w}^{k}\bigr) \bigr\} \\& \quad \le\mu\bigl\Vert w^{k} - \hat{w}^{k} \bigr\Vert _{R_{3}}^{2}. \end{aligned}$$

(3.17)

In addition, it follows from (3.4) that

$$\begin{aligned}& A_{1}\hat{u}^{k}+ A_{2} \hat{v}^{k}+A_{3}\hat{w}^{k}-b +H^{-1} \bigl(\hat{\lambda }^{k}-{\lambda}^{k}\bigr) \\& \quad {}-A_{1} \bigl(\hat{u}^{k}-u^{k}\bigr)-A_{2}\bigl( \hat{v}^{k}-v^{k}\bigr)-A_{3}\bigl( \hat{w}^{k}-w^{k}\bigr)=0. \end{aligned}$$

(3.18)

Combining (3.15)-(3.18), we get

$$\begin{aligned}& \left ( \textstyle\begin{array}{c} u - \hat{u}^{k} \\ v - \hat{v}^{k}\\ w - \hat{w}^{k}\\ \lambda- \hat{\lambda}^{k} \end{array}\displaystyle \right )^{\top} \left ( \textstyle\begin{array}{c} f_{1}(\hat{u}^{k})- A_{1}^{\top} \hat{\lambda}^{k}- ((1+\mu ){R_{1}}+A^{T}HA_{1} )(u^{k}-\hat{u}^{k})\\ f_{2}(\hat{v}^{k}) - A_{2}^{\top} \hat{\lambda}^{k}- ((1+\mu ){R_{2}}+A_{2}^{\top}HA_{2} )(v^{k}-\hat{v}^{k})\\ f_{3}(\hat{w}^{k}) - A_{3}^{\top} \hat{\lambda}^{k}- ((1+\mu ){R_{3}}+A_{3}^{\top}HA_{3} )(w^{k}-\hat{w}^{k})\\ A_{1}\hat{u}^{k}+A_{2}\hat{v}^{k}+A_{3}\hat{w}^{k} -b +A_{1}(u^{k}-\hat{u}^{k})+A_{2}(v^{k}-\hat{v}^{k})+A_{3}(w^{k}-\hat {w}^{k})-H^{-1}(\lambda^{k}-\hat{\lambda}^{k}) \end{array}\displaystyle \right ) \\& \quad \geq-\mu\bigl\Vert u^{k} - \hat{u}^{k} \bigr\Vert _{R_{1}}^{2}-\mu\bigl\Vert v^{k} - \hat{v}^{k} \bigr\Vert _{R_{2}}^{2}-\mu\bigl\Vert w^{k} - \hat{w}^{k} \bigr\Vert _{R_{3}}^{2}. \end{aligned}$$

(3.19)

Recall the definition of J in (3.3), we obtain the assertion (3.7). The proof is completed. □

Lemma 3.2

For given $z^{k}\in \mathbb{R}^{n_{1}}_{++}\times\mathbb{R}^{n_{2}}_{++}\times \mathbb{R}^{n_{3}}_{++}\times\mathbb{R}^{m}$ and $z_{*}^{k} := P_{\mathcal{Z},G}[z^{k}-\tau_{k} G^{-1}g(z^{k},\tilde{z}^{k})]$, we have

$$ \gamma\tau_{k}\bigl(z-\hat{z}^{k} \bigr)^{\top} Q(z)+\frac{1}{2}\bigl(\bigl\Vert z-z^{k} \bigr\Vert _{G}^{2}-\bigl\Vert z-z_{*}^{k} \bigr\Vert _{G}^{2}\bigr)\geq\frac{1}{2} \gamma(2-\gamma) \tau_{k}^{2}\bigl\Vert z^{k}-\tilde{z}^{k} \bigr\Vert _{G}^{2}. $$

(3.20)

Proof

Since $z_{*}^{k}\in\mathcal{Z}$, substituting $z=z_{*}^{k}$ in (3.7), we get

$$\begin{aligned}& \gamma\tau_{k}\bigl(z_{*}^{k}- \hat{z}^{k}\bigr)^{\top}Q\bigl(\hat{z}^{k}\bigr) \end{aligned}$$

(3.21)

$$\begin{aligned}& \quad \geq \gamma\tau_{k}\bigl(z_{*}^{k}- \hat{z}^{k}\bigr)^{\top} J\bigl(z^{k}- \hat{z}^{k}\bigr) - \mu\gamma\tau_{k}\bigl\Vert u^{k} - \hat{u}^{k} \bigr\Vert _{R_{1}}^{2}- \mu\gamma\tau_{k}\bigl\Vert v^{k} - \hat{v}^{k} \bigr\Vert _{R_{2}}^{2} \\& \qquad {} -\mu\gamma\tau_{k}\bigl\Vert w^{k} - \hat{w}^{k}\bigr\Vert _{R_{3}}^{2} \\& \quad = \gamma\tau_{k}\bigl(z^{k}-\hat{z}^{k} \bigr)^{\top} J\bigl(z^{k}-\hat{z}^{k}\bigr)+\gamma \tau _{k}\bigl(z_{*}^{k}-z^{k} \bigr)^{\top} J\bigl(z^{k}-\hat{z}^{k}\bigr) \\& \qquad {} -\mu\gamma\tau_{k}\bigl\Vert u^{k} - \hat{u}^{k} \bigr\Vert _{R_{1}}^{2}-\mu\gamma\tau _{k}\bigl\Vert v^{k} - \hat{v}^{k} \bigr\Vert _{R_{2}}^{2} -\mu\gamma\tau_{k}\bigl\Vert w^{k} - \hat{w}^{k}\bigr\Vert _{R_{3}}^{2} \\& \quad = \gamma\tau_{k}\bigl(z^{k}-\tilde{z}^{k} \bigr)^{\top} \bigl(N^{-1}\bigr)^{\top} JN^{-1}\bigl(z^{k}-\tilde{z}^{k}\bigr) +\gamma \tau_{k}\bigl(z_{*}^{k}-z^{k} \bigr)^{\top} JN^{-1}\bigl(z^{k}-\tilde{z}^{k} \bigr) \\& \qquad {} -\gamma\tau_{k}\mu\bigl\Vert u^{k} - \hat{u}^{k} \bigr\Vert _{R_{1}}^{2} -\gamma \tau_{k}\mu\bigl\Vert v^{k} - \hat{v}^{k} \bigr\Vert _{R_{2}}^{2}-\mu\gamma\tau_{k}\bigl\Vert w^{k} - \hat{w}^{k}\bigr\Vert _{R_{3}}^{2} \\& \quad = \gamma\tau_{k}\bigl(z^{k}-\tilde{z}^{k} \bigr)^{\top} \bigl(N^{-1}\bigr)^{\top} G \bigl(z^{k}-\tilde{z}^{k}\bigr) -\gamma\tau_{k}\mu \bigl\Vert u^{k} - \hat{u}^{k} \bigr\Vert _{R_{1}}^{2} -\gamma\tau_{k}\mu\bigl\Vert v^{k} - \hat{v}^{k} \bigr\Vert _{R_{2}}^{2} \\& \qquad {} -\mu\gamma\tau_{k}\bigl\Vert w^{k} - \hat{w}^{k}\bigr\Vert _{R_{3}}^{2} +\gamma \tau_{k}\bigl(z_{*}^{k}-z^{k} \bigr)^{\top} G\bigl(z^{k}-\tilde{z}^{k}\bigr) \\& \quad = \gamma\tau_{k}\bigl(z^{k}-\tilde{z}^{k} \bigr)^{\top} \bigl(N^{-1}\bigr)^{\top} M \bigl(z^{k}-\tilde{z}^{k}\bigr) +\gamma\tau_{k} \bigl(z_{*}^{k}-z^{k}\bigr)^{\top} G \bigl(z^{k}-\tilde{z}^{k}\bigr) \\& \quad \geq \gamma\tau_{k}\varphi\bigl(z^{k},\tilde{z}^{k} \bigr)+\gamma\tau _{k}\bigl(z_{*}^{k}-z^{k} \bigr)^{\top} G\bigl(z^{k}-\tilde{z}^{k}\bigr) \\& \quad \geq \gamma\tau_{k}\varphi\bigl(z^{k},\tilde{z}^{k} \bigr)-\frac{1}{2}\bigl\Vert z^{k}-z_{*}^{k}\bigr\Vert _{G}^{2} -\frac{1}{2}\gamma^{2} \tau_{k}^{2}\bigl\Vert z^{k}-\tilde{z}^{k} \bigr\Vert _{G}^{2} \\& \quad = \frac{1}{2} \gamma(2-\gamma)\tau_{k}^{2}\bigl\Vert z^{k}-\tilde{z}^{k}\bigr\Vert _{G}^{2}- \frac{1}{2}\bigl\Vert z^{k}-z_{*}^{k}\bigr\Vert _{G}^{2}. \end{aligned}$$

(3.22)

Using (3.6), $z_{*}^{k}$ is the projection of $z^{k}-\gamma\tau_{k} G^{-1}Q(\hat{z}^{k})$ on $\mathcal{Z}$, it follows from (2.1) that

$$\bigl(z^{k}-\gamma\tau_{k} G^{-1}Q\bigl( \hat{z}^{k}\bigr)-z_{*}^{k}\bigr)^{\top} G \bigl(z-z_{*}^{k}\bigr)\leq0, \quad\forall z\in\mathcal{Z}, $$

and consequently, we have

$$\gamma\tau_{k} \bigl(z-z_{*}^{k} \bigr)^{\top} Q\bigl(\hat{z}^{k}\bigr)\geq \bigl(z^{k}-z_{*}^{k} \bigr)^{\top} G\bigl(z-z_{*}^{k}\bigr). $$

Using the identity $x^{\top} Gy =\frac{1}{2} (\Vert x\Vert _{G}^{2}-\Vert x-y\Vert _{G}^{2}+\Vert y\Vert _{G}^{2} )$ to the right hand side of the last inequality, we obtain

$$ \gamma\tau_{k} \bigl(z-z_{*}^{k} \bigr)^{\top} Q\bigl(\hat{z}^{k}\bigr)\geq\frac{1}{2} \bigl(\bigl\Vert z-z_{*}^{k}\bigr\Vert _{G}^{2}-\bigl\Vert z-z^{k}\bigr\Vert _{G}^{2} \bigr) +\frac{1}{2}\bigl\Vert z^{k}-z_{*}^{k}\bigr\Vert _{G}^{2}. $$

(3.23)

Adding (3.21) and (3.23), we get

$$\gamma\tau_{k}\bigl(z-\hat{z}^{k}\bigr)^{\top} Q \bigl(\hat{z}^{k}\bigr)+\frac{1}{2}\bigl(\bigl\Vert z-z^{k} \bigr\Vert _{G}^{2}-\bigl\Vert z-z_{*}^{k} \bigr\Vert _{G}^{2}\bigr)\geq\frac{1}{2} \gamma(2-\gamma) \tau_{k}^{2}\bigl\Vert z^{k}-\tilde{z}^{k} \bigr\Vert _{G}^{2}, $$

and by using the monotonicity of Q, we obtain (3.20) and the proof is completed. □

Lemma 3.3

Let $z^{k}\in \mathbb{R}^{n_{1}}_{++}\times \mathbb{R}^{n_{2}}_{++}\times\mathbb{R}^{n_{3}}_{++}\times\mathbb {R}^{m}$ and $z^{k+1}(\tau_{k})$ be generated by (2.6). Then

$$ \gamma\sigma\tau_{k}\bigl(z-\hat{z}^{k} \bigr)^{\top} Q(z)+\frac{1}{2}\bigl(\bigl\Vert z-z^{k}\bigr\Vert _{G}^{2}-\bigl\Vert z-z^{k+1}( \tau_{k})\bigr\Vert _{G}^{2}\bigr) \geq \frac{1}{2} \sigma\gamma(2-\gamma)\tau_{k}^{2}\bigl\Vert z-\tilde{z}^{k}\bigr\Vert _{G}^{2}. $$

(3.24)

Proof

We have

$$\begin{aligned}& \bigl\Vert z-z^{k}\bigr\Vert _{G}^{2}- \bigl\Vert z-z^{k+1}(\tau_{k})\bigr\Vert _{G}^{2} \end{aligned}$$

(3.25)

$$\begin{aligned}& \quad = \bigl\Vert z^{k}-z\bigr\Vert _{G}^{2}- \bigl\Vert z^{k}-\sigma\bigl(z^{k}-z_{*}^{k} \bigr)-z\bigr\Vert _{G}^{2} \\& \quad = 2\sigma\bigl(z^{k}-z\bigr)^{\top} G\bigl(z^{k}-z_{*}^{k} \bigr)-\sigma^{2}\bigl\Vert z^{k}-z_{*}^{k} \bigr\Vert _{G}^{2} \\& \quad = 2\sigma \bigl(\bigl\Vert z^{k}-z_{*}^{k} \bigr\Vert _{G}^{2}-\bigl( z-z_{*}^{k} \bigr)^{\top} G\bigl(z^{k}-z_{*}^{k}\bigr) \bigr)-\sigma^{2}\bigl\Vert z^{k}-z_{*}^{k} \bigr\Vert _{G}^{2}. \end{aligned}$$

(3.26)

Using the identity

$$\begin{aligned} \bigl(z-z_{*}^{k}\bigr)^{\top} G \bigl(z^{k}-z_{*}^{k}\bigr) = &{ \frac{1}{2}} \bigl(\bigl\Vert z_{*}^{k}-z\bigr\Vert _{G}^{2}- \bigl\Vert z^{k}-z\bigr\Vert _{G}^{2} \bigr) +{\frac{1}{2}}\bigl\Vert z^{k}-z_{*}^{k}\bigr\Vert _{G}^{2}, \end{aligned}$$

we get

$$ \bigl\Vert z^{k}-z_{*}^{k}\bigr\Vert _{G}^{2}-2\bigl(z-z_{*}^{k} \bigr)^{\top} G\bigl(z^{k}-z_{*}^{k}\bigr)= \bigl\Vert z^{k}-z\bigr\Vert _{G}^{2}-\bigl\Vert z_{*}^{k}-z\bigr\Vert _{G}^{2}. $$

(3.27)

Substituting (3.27) into (3.25), we obtain

$$\begin{aligned} \bigl\Vert z-z^{k}\bigr\Vert _{G}^{2}- \bigl\Vert z-z^{k+1}(\tau_{k})\bigr\Vert _{G}^{2} =&\sigma\bigl(\bigl\Vert z-z^{k}\bigr\Vert _{G}^{2}-\bigl\Vert z-z_{*}^{k}\bigr\Vert _{G}^{2}\bigr) +\sigma(1-\sigma)\bigl\Vert z^{k}-z_{*}^{k}\bigr\Vert _{G}^{2} \\ \geq&\sigma\bigl(\bigl\Vert z-z^{k}\bigr\Vert _{G}^{2}-\bigl\Vert z-z_{*}^{k}\bigr\Vert _{G}^{2}\bigr). \end{aligned}$$

(3.28)

Substituting (3.28) into (3.20), we obtain (3.24), the required result. □

Theorem 3.1

Let $z^{*}$ be a solution of $\mathrm{SVI}_{3}$ and $z^{k+1}(\tau_{k})$ be generated by (2.6). Then $z^{k}$ and $\tilde{z}^{k}$ are bounded, and

$$ \bigl\Vert z^{k+1}(\tau_{k}) - z^{*}\bigr\Vert _{G}^{2}\leq\bigl\Vert z^{k} -z^{*}\bigr\Vert _{G}^{2}-c\bigl\Vert z^{k}-\tilde {z}^{k}\bigr\Vert ^{2}_{G}, $$

(3.29)

where

$$c:={ \frac{\sigma\gamma(2-\gamma)(2\beta-\sqrt {3})^{2}}{4\beta^{2}}}>0. $$

Proof

Setting $z=z^{*}$ in (3.24), we obtain

$$\begin{aligned} \bigl\Vert z^{k+1}(\tau_{k}) - z^{*}\bigr\Vert _{G}^{2} \leq&\bigl\Vert z^{k} -z^{*}\bigr\Vert _{G}^{2}-\sigma\gamma(2-\gamma)\tau_{k}^{2} \bigl\Vert z^{k}-\tilde {z}^{k}\bigr\Vert ^{2}_{G}+ 2\gamma\sigma\tau_{k}\bigl(z^{*}-\hat{z}^{k} \bigr)^{\top} Q\bigl(z^{*}\bigr) \\ \leq&\bigl\Vert z^{k} -z^{*}\bigr\Vert _{G}^{2}- \sigma\gamma(2-\gamma)\tau_{k}^{2}\bigl\Vert z^{k}-\tilde {z}^{k}\bigr\Vert ^{2}_{G} \\ \leq&\bigl\Vert z^{k} -z^{*}\bigr\Vert _{G}^{2}- \frac{\sigma\gamma(2-\gamma)(2\beta-\sqrt {3})^{2}}{4\beta^{2}}\bigl\Vert z^{k}-\tilde{z}^{k}\bigr\Vert ^{2}_{G}. \end{aligned}$$

Then we have

$$\bigl\Vert z^{k+1}(\tau_{k})-z^{*}\bigr\Vert _{G}\leq\bigl\Vert z^{k}-z^{*}\bigr\Vert _{G} \leq\cdots\leq\bigl\Vert z^{0}-z^{*}\bigr\Vert _{G}, $$

and thus, $\{ z^{k}\}$ is a bounded sequence.

It follows from (3.29) that

$$\sum_{k=0}^{\infty}c\bigl\Vert z^{k}-\tilde{z}^{k}\bigr\Vert ^{2}_{G}< + \infty, $$

which means that

$$ \lim_{k\to\infty}\bigl\Vert z^{k}- \tilde{z}^{k}\bigr\Vert _{G}=0. $$

(3.30)

Since $\{ z^{k}\}$ is a bounded sequence, we conclude that $\{\tilde {z}^{k}\}$ is also bounded. □

The global convergence of the proposed method can be proved by similar arguments as in [18]. Hence the proof is omitted.

Theorem 3.2

The sequence $\{z^{k}\}$ generated by the proposed method converges to some $z^{\infty}$ which is a solution of $\mathrm{SVI}_{3}$.

Now, we are ready to present the $O(1/t)$ convergence rate of the proposed method.

Theorem 3.3

For any integer $t > 0$, we have a $\hat{z}_{t}\in\mathcal{Z}$ which satisfies

$$(\hat{z}_{t}-z)^{\top} Q(z)\leq\frac{1}{2\gamma\sigma\Upsilon_{t}}\bigl\Vert z-z^{0}\bigr\Vert _{G}^{2},\quad\forall z\in \mathcal{Z}, $$

where

$$\hat{z}_{t}=\frac{1}{\Upsilon_{t}}\sum_{k=0}^{t} \tau_{k}\hat{z}^{k}\quad \textit{and} \quad\Upsilon_{t}= \sum_{k=0}^{t}\tau_{k}. $$

Proof

Summing the inequality (3.24) over $k=0,\ldots,t$, we obtain

$$\Biggl( \Biggl(\sum_{k=0}^{t}\gamma\sigma \tau_{k} \Biggr) z-\sum_{k=0}^{t} \gamma\sigma\tau_{k}\hat{z}^{k} \Biggr)^{\top} Q(z)+ \frac{1}{2}\bigl\Vert z-z^{0}\bigr\Vert _{G}^{2} \geq0. $$

Using the notations of $\Upsilon_{t}$ and $\hat{z}_{t}$ in the above inequality, we derive

$$(\hat{z}_{t}-z)^{\top} Q(z)\leq\frac{1}{2\gamma\sigma\Upsilon_{t}}\bigl\Vert z-z^{0}\bigr\Vert _{G}^{2},\quad\forall z\in \mathcal{Z}. $$

Indeed, $\hat{z}_{t}\in\mathcal{Z}$ because it is a convex combination of $\hat{z}^{0}$, $\hat{z}^{1}$, … , $\hat{z}^{t}$. The proof is completed. □

Remark 3.1

It follows from (2.13) that

$$\Upsilon_{t}\geq\frac{2\beta-\sqrt{3}}{2\beta} (t+1 ). $$

Suppose that, for any compact set $\mathcal {D}\subset\mathcal{Z}$, let $d =\sup\{\Vert z-z^{0}\Vert _{G}\vert z\in\mathcal {D}\}$. For any given $\epsilon>0$, after at most

$$t= \biggl[\frac{\beta d^{2}}{(2\beta-\sqrt{3})\gamma\sigma\epsilon } \biggr] $$

iterations, we have

$$(\hat{z}_{t}-z)^{T}Q(z)\leq\epsilon,\quad \forall z\in\mathcal {D}. $$

That is, the $O(1/t)$ convergence rate is established in an ergodic sense.

4 Preliminary computational results

In this section, we present some numerical examples to illustrate the proposed method. We consider the following optimization problem with matrix variables, which is studied in [18]:

$$ \min \biggl\{ \frac{1}{2} \Vert U-C\Vert _{F}^{2}\Big\vert U\in S_{+}^{n} \biggr\} , $$

(4.1)

where $\Vert \cdot \Vert _{F}$ is the matrix Fröbenius norm, i.e., $\Vert C\Vert _{F} = ( \sum_{i=1}^{n}\sum_{j=1}^{n} \vert C_{ij}\vert ^{2} )^{1/2}$ and

$$S_{+}^{n}=\bigl\{ M\in\mathbb{R}^{n\times n} : M^{\top} =M, M \succeq 0 \bigr\} . $$

It has been shown in [18] that solving problem (4.1) is equivalent to the following variational inequality problem: Find $X^{*}=(U^{*},V^{*},Z^{*})\in\Omega= S_{+}^{n} \times S_{+}^{n}\times \mathbb{R}^{n\times n}$ such that

$$\begin{aligned} \textstyle\begin{cases} \langle U-U^{*},(U^{*}-C)-Z^{*}\rangle\geq0, \\ \langle V-V^{*},(V^{*}-C)+Z^{*}\rangle\geq0,&\forall X=(U,V,Z)\in \Omega,\\ U^{*} - V^{*} =0. \end{cases}\displaystyle \end{aligned}$$

(4.2)

The problem (4.2) is a special case of (1.3)-(1.4) with matrix variables where $A_{1}=I_{n\times n}$, $A_{2}=-I_{n\times n}$, $b=0$, $f_{1}(U)=U-C$, $f_{2}(V)=V-C$, and $\mathcal{W} =S_{+}^{n} \times S_{+}^{n}\times\mathbb{R}^{n\times n}$. For simplification, we take $R_{1}=r_{1}I_{n\times n}$, $R_{2}=r_{2}I_{n\times n}$, and $H=I_{n\times n}$ where $r_{1}>0$ and $r_{2}>0$ are scalars. In all tests we take $\mu=0.5$, $\beta=0.88$, $C=\operatorname{rand}(n)$, and $(U^{0}, V^{0}, Z^{0})=(I_{n\times n},I_{n\times n},0_{n\times n})$ as the initial point in the test. The iteration is stopped as soon as

$$\max\bigl\{ \bigl\Vert U^{k}-\tilde{U}^{k}\bigr\Vert , \bigl\Vert V^{k}-\tilde{V}^{k}\bigr\Vert ,\bigl\Vert Z^{k}-\tilde{Z}^{k}\bigr\Vert \bigr\} \leq10^{-6}. $$

All codes were written in Matlab, we compare the proposed method with those in [18] and [17]. The iteration numbers, denoted by k, and the computational time for the problem (4.1) with different dimensions are given in Tables 1-2.

Table 1 Numerical results for problem ( 4.1 ) with $\pmb{r_{1}=0.5}$ , $\pmb{r_{2}=5}$

Table 2 Numerical results for problem ( 4.1 ) with $\pmb{r_{1}=1}$ , $\pmb{r_{2}=10}$

Tables 1-2 show that the proposed method is more flexible and efficient for the problem tested.

5 Conclusions

In this paper, we proposed a new modified logarithmic-quadratic proximal alternating direction method for solving structured variational inequalities with three separable operators. The prediction point is obtained by solving series of related systems of nonlinear equations in a parallel way. Global convergence of the proposed method is proved under mild assumptions. We have proved the $O(1/t)$ convergence rate of the parallel LQP alternating direction method. Some preliminary numerical examples are reported to verify the effectiveness of the proposed method in practice.