To prove the convergence of the HSM, we will show first in this section that the sequence \(\{w^{k}\}\) generated by the HSM is Fejér monotone with respect to the solution set \(\mathcal{W}^{*}\) of the problem (2.12).
Due to (1.6), multiplying both sides of the third inequality by 2, and multiplying both sides of the last inequality by 3, respectively, we get
$$ \left \{ \textstyle\begin{array}{l} (x'-x^{*})^{T}[A^{T}(Ax^{*}-b)-\lambda^{*}]\geq 0, \\ (y'-y^{*})^{T}[\delta\Delta^{T}\Delta{y^{*}}+\lambda^{*}-\mu^{*}]\geq0, \\ 2\times(\lambda'-\lambda^{*})^{T}(x^{*}-y^{*})\geq0, \\ \eta\|z'\|_{1}-\eta\|z^{*}\|_{1}+(z'-z^{*})^{T}[\mu^{*}]\geq0, \\ 3\times(\mu'-\mu^{*})^{T}(y^{*}-z^{*})\geq0, \end{array}\displaystyle \right .\quad \forall w'=\bigl(x', y', \lambda',z', \mu'\bigr)\in{\mathcal{W}}, $$
(3.1)
and (3.1) can be written as
$$ \bigl(w'-w^{*}\bigr)^{T} H F\bigl(w^{*}\bigr) \ge0,\quad \forall w'\in\mathcal{W}. $$
(3.2)
Lemma 3.1
For a given
\(w^{k}=(x^{k},y^{k},\lambda^{k},z^{k},\mu^{k})\), if
\(\tilde{w}^{k}=(\tilde{x}^{k},\tilde{y}^{k}, \tilde{\lambda}^{k},\tilde {z}^{k},\tilde{\mu}^{k})\)
is generated by (2.1)-(2.5), then we have
$$ \bigl(\tilde{w}^{k}-w^{*}\bigr)^{T} F\bigl( \tilde {w}^{k}\bigr)\ge0 $$
(3.3)
and
$$ \bigl(\tilde{w}^{k}-w^{*}\bigr)^{T} H F\bigl( \tilde {w}^{k}\bigr)\ge0, $$
(3.4)
where
\(w^{*}=(x^{*}, y^{*}, \lambda^{*},z^{*},\mu^{*})\in{\mathcal{W}}^{*}\)
is a solution.
Proof
It is easy to show that \(F(w)\) is linear and consequently monotone. Indeed, let
$$Q=\left ( \textstyle\begin{array}{@{}c@{\quad}c@{\quad}c@{\quad}c@{\quad}c@{}} A^{T}A & 0 & -I & 0 & 0 \\ 0 & \delta\Delta^{T} \Delta& I & 0 & -I \\ I & -I & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & I \\ 0 & I & 0 & -I & 0 \end{array}\displaystyle \right ) \quad \mbox{and} \quad e= \left ( \textstyle\begin{array}{@{}c@{}} -A^{T} b \\ 0 \\ 0 \\ 0 \\ 0 \end{array}\displaystyle \right ), $$
then \(F(w)=Qw+e\). By the monotonicity of F and HF we have
$$\bigl(\tilde{w}^{k}-w^{*}\bigr)^{T} \bigl(F\bigl( \tilde{w}^{k}\bigr)-F\bigl(w^{*}\bigr)\bigr)\ge0 $$
and
$$\bigl(\tilde{w}^{k}-w^{*}\bigr)^{T} \bigl(HF\bigl( \tilde{w}^{k}\bigr)-HF\bigl(w^{*}\bigr)\bigr)\ge0, $$
respectively, which results in (by (2.11))
$$\bigl(\tilde{w}^{k}-w^{*}\bigr)^{T} F\bigl( \tilde{w}^{k}\bigr)\ge\bigl(\tilde{w}^{k}-w^{*} \bigr)^{T} F\bigl(w^{*}\bigr)\ge 0, $$
and by (3.2)
$$\bigl(\tilde{w}^{k}-w^{*}\bigr)^{T} HF\bigl( \tilde{w}^{k}\bigr)\ge\bigl(\tilde{w}^{k}-w^{*} \bigr)^{T} H F\bigl(w^{*}\bigr)\ge0. $$
□
Lemma 3.2
For a given
\(w^{k}=(x^{k},y^{k},\lambda^{k},z^{k},\mu^{k})\), if
\(\tilde{w}^{k}=(\tilde {x}^{k},\tilde{y}^{k}, \tilde{\lambda}^{k},\tilde{z}^{k},\tilde{\mu}^{k})\)
is generated by (2.1)-(2.5), then we have
$$\begin{aligned}& \bigl(\tilde{w}^{k}-w^{*}\bigr)^{T}\bigl[HF \bigl(\tilde{w}^{k}\bigr)+g\bigl(w^{k},\tilde{w}^{k} \bigr)\bigr] \\& \quad \ge \bigl(\lambda^{k}-\tilde{\lambda}^{k} \bigr)^{T}\bigl(x^{k}-\tilde{x}^{k}\bigr)-\bigl( \lambda ^{k}-\tilde{\lambda}^{k}\bigr)^{T} \bigl(y^{k}-\tilde{y}^{k}\bigr) \\& \qquad {} +\bigl(\mu^{k}-\tilde{\mu}^{k}\bigr)^{T} \bigl(y^{k}-\tilde{y}^{k}\bigr)-\bigl(\mu^{k}- \tilde{\mu }^{k}\bigr)^{T}\bigl(z^{k}- \tilde{z}^{k}\bigr). \end{aligned}$$
(3.5)
Proof
By a direct computation, we get
$$\begin{aligned}& \bigl(\tilde{w}^{k}-w^{*}\bigr)^{T}\bigl[HF\bigl( \tilde{w}^{k}\bigr)+g\bigl(w^{k},\tilde{w}^{k}\bigr) \bigr] \\& \quad = \bigl(\tilde{w}^{k}-w^{*}\bigr)^{T}HF\bigl( \tilde{w}^{k}\bigr)+\bigl(\tilde{w}^{k}-w^{*} \bigr)^{T} g\bigl(w^{k},\tilde{w}^{k}\bigr) \\& \quad \geq \bigl(\tilde{w}^{k}-w^{*}\bigr)^{T}g \bigl(w^{k},\tilde{w}^{k}\bigr) \\& \quad = \bigl(\tilde{x}^{k}-x^{*}\bigr)^{T}\beta_{k} \bigl(x^{k}-\tilde{x}^{k}\bigr)-\bigl(\tilde {x}^{k}-x^{*}\bigr)^{T}\beta_{k}\bigl(y^{k}- \tilde{y}^{k}\bigr)-\bigl(\tilde{y}^{k}-y^{*}\bigr)\beta _{k}\bigl(x^{k}-\tilde{x}^{k}\bigr) \\& \qquad {} +\bigl(\tilde{y}^{k}-y^{*}\bigr)^{T} \beta_{k}\bigl(y^{k}-\tilde{y}^{k}\bigr)+\bigl( \tilde {y}^{k}-y^{*}\bigr)\rho_{k}\bigl(y^{k}- \tilde{y}^{k}\bigr)-\bigl(\tilde{y}^{k}-y^{*}\bigr)\rho _{k}\bigl(z^{k}-\tilde{z}^{k}\bigr) \\& \qquad {} +\bigl(\tilde{z}^{k}-z^{*}\bigr)\rho_{k} \bigl(z^{k}-\tilde{z}^{k}\bigr)-\bigl(\tilde{z}^{k}-z^{*} \bigr)\rho _{k}\bigl(y^{k}-\tilde{y}^{k}\bigr) \\& \quad = \bigl[\bigl(\tilde{x}^{k}-\tilde{y}^{k}\bigr)- \bigl(x^{*}-y^{*}\bigr)\bigr]^{T}\beta_{k}\bigl(x^{k}- \tilde {x}^{k}\bigr)-\bigl[\bigl(\tilde{x}^{k}- \tilde{y}^{k}\bigr)-\bigl(x^{*}-y^{*}\bigr)\bigr]^{T} \beta_{k}\bigl(y^{k}-\tilde {y}^{k}\bigr) \\& \qquad {} +\bigl[\bigl(\tilde{y}^{k}-\tilde{z}^{k}\bigr)- \bigl(y^{*}-z^{*}\bigr)\bigr]^{T}\rho_{k}\bigl(y^{k}- \tilde {y}^{k}\bigr)-\bigl[\bigl(\tilde{y}^{k}- \tilde{z}^{k}\bigr)-\bigl(y^{*}-z^{*}\bigr)\bigr]^{T} \rho_{k}\bigl(z^{k}-\tilde {z}^{k}\bigr). \end{aligned}$$
Substituting \(x^{*}-y^{*}=0\), \(y^{*}-z^{*}=0\) into the last equation, we get
$$\begin{aligned}& \bigl(\tilde{w}^{k}-w^{*}\bigr)^{T}\bigl[HF \bigl(\tilde{w}^{k}\bigr)+g\bigl(w^{k},\tilde{w}^{k} \bigr)\bigr] \\& \quad \geq \bigl(\tilde{x}^{k}-\tilde{y}^{k}\bigr) \beta_{k}\bigl(x^{k}-\tilde{x}^{k}\bigr)-\bigl( \tilde {x}^{k}-\tilde{y}^{k}\bigr)\beta_{k} \bigl(y^{k}-\tilde{y}^{k}\bigr) \\& \qquad {}+\bigl(\tilde{y}^{k}-\tilde{z}^{k}\bigr) \rho_{k}\bigl(y^{k}-\tilde{y}^{k}\bigr)-\bigl(\tilde {y}^{k}-\tilde{z}^{k}\bigr)\rho_{k} \bigl(z^{k}-\tilde{z}^{k}\bigr). \end{aligned}$$
(3.6)
By (2.3) and (2.5), we get
$$\begin{aligned}& \bigl(\tilde{x}^{k}-\tilde{y}^{k}\bigr)= \frac {1}{\beta_{k}}\bigl(\lambda^{k}-\tilde{\lambda}^{k}\bigr), \end{aligned}$$
(3.7)
$$\begin{aligned}& \bigl(\tilde{y}^{k}-\tilde{z}^{k}\bigr)= \frac {1}{\rho_{k}}\bigl(\mu^{k}-\tilde{\mu}^{k}\bigr). \end{aligned}$$
(3.8)
Substituting (3.7) and (3.8) into the right-hand side of (3.6), we obtain
$$\begin{aligned}& \bigl(\tilde{w}^{k}-w^{*}\bigr)^{T}\bigl[HF\bigl( \tilde{w}^{k}\bigr)+g\bigl(w^{k},\tilde{w}^{k}\bigr) \bigr] \\& \quad \geq \bigl(\lambda^{k}-\tilde{\lambda}^{k}\bigr) \bigl(x^{k}-\tilde{x}^{k}\bigr)-\bigl(\lambda^{k}- \tilde {\lambda}^{k}\bigr) \bigl(y^{k}-\tilde{y}^{k} \bigr) \\& \qquad {}+\bigl(\mu^{k}-\tilde{\mu}^{k}\bigr) \bigl(y^{k}-\tilde{y}^{k}\bigr)-\bigl(\mu^{k}- \tilde{\mu }^{k}\bigr) \bigl(z^{k}-\tilde{z}^{k} \bigr). \end{aligned}$$
□
Lemma 3.3
For a given
\(w^{k}=(x^{k},y^{k},\lambda^{k},z^{k},\mu^{k})\), if
\(\tilde{w}^{k}=(\tilde {x}^{k},\tilde{y}^{k}, \tilde{\lambda}^{k},\tilde{z}^{k},\tilde{\mu}^{k})\)
is generated by (2.1)-(2.5), then we have
$$\begin{aligned}& \eta\bigl\Vert z'\bigr\Vert _{1}-\eta \bigl\Vert \tilde{z}^{k}\bigr\Vert _{1}+ \bigl(w'-\tilde{w}^{k}\bigr)^{T}\bigl[HF\bigl( \tilde {w}^{k}\bigr)+g\bigl(w^{k},\tilde{w}^{k} \bigr)-d\bigl(w^{k},\tilde{w}^{k}\bigr) \bigr] \\& \quad \ge0,\quad \forall z'\in\mathcal{X}, w'\in{ \mathcal{W}} . \end{aligned}$$
(3.9)
Proof
Combining (2.1)-(2.5) together, we have
$$\begin{aligned}& \eta\bigl\Vert z'\bigr\Vert _{1}-\eta\bigl\Vert \tilde{z}^{k}\bigr\Vert _{1}+\bigl(w'- \tilde{w}^{k}\bigr)^{T}\left ( \textstyle\begin{array}{@{}c@{}} A^{T}(A\tilde{x}^{k}-b)-\lambda^{k}+\beta_{k}(\tilde{x}^{k}-y^{k}) \\ \delta\Delta^{T}\Delta\tilde{y}^{k}+\lambda^{k}-\mu^{k} -\beta_{k}(x^{k}-\tilde{y}^{k})+\rho_{k}(\tilde{y}^{k}-z^{k}) \\ 2(\tilde{x}^{k}-\tilde{y}^{k})-\frac{1}{\beta_{k}} (\lambda^{k}-\tilde{\lambda}^{k}) \\ \mu^{k}-\rho_{k}(\tilde{y}^{k}-\tilde{z}^{k})\\ 3( \tilde{y}^{k}-\tilde{z}^{k})-\frac{1}{\rho_{k}}(\mu^{k}-\tilde{\mu }^{k}) \end{array}\displaystyle \right ) \\& \quad \geq0. \end{aligned}$$
By a manipulation, we get
$$\begin{aligned}& \eta\bigl\Vert z'\bigr\Vert _{1}-\eta \bigl\Vert \tilde{z}^{k}\bigr\Vert _{1}+ \bigl(w'-\tilde{w}^{k}\bigr)^{T}\left \{ \left ( \textstyle\begin{array}{@{}c@{}} A^{T}(A\tilde{x}^{k}-b)-\tilde{\lambda}^{k} \\ \delta\Delta^{T}\Delta\tilde{y}^{k}+\tilde{\lambda}^{k}-\tilde{\mu}^{k} \\ 2(\tilde{x}^{k}-\tilde{y}^{k})\\ \tilde{\mu}^{k}\\ 3(\tilde{y}^{k}-\tilde{z}^{k}) \end{array}\displaystyle \right )+g\bigl(w^{k}, \tilde{w}^{k}\bigr)-M\bigl(w^{k}-\tilde{w}^{k}\bigr) \right \} \\& \quad \geq0. \end{aligned}$$
(3.10)
The assertion (3.9) is just a compact form of (3.10). □
Theorem 3.1
For a given
\(w^{k}=(x^{k},y^{k},\lambda^{k},z^{k},\mu^{k})\), if
\(\tilde{w}^{k}=(\tilde {x}^{k},\tilde{y}^{k}, \tilde{\lambda}^{k},\tilde{z}^{k},\tilde{\mu}^{k})\)
is generated by (2.1)-(2.5), then for any
\(w^{*}=(x^{*}, y^{*}, \lambda^{*},z^{*},\mu^{*}) \in{\mathcal{W}}^{*}\)
we have
$$ \bigl(\tilde{w}^{k}-w^{*}\bigr)^{T}\bigl[ H F \bigl(\tilde{w}^{k}\bigr)+g\bigl(w^{k},\tilde{w}^{k} \bigr)\bigr]\ge\varphi \bigl(w^{k},\tilde{w}^{k}\bigr)- \bigl(w^{k}-\tilde{w}^{k}\bigr)^{T}d \bigl(w^{k},\tilde{w}^{k}\bigr). $$
(3.11)
Proof
Note \(d(w^{k},\tilde{w}^{k})=M(w^{k}-\tilde{w}^{k})\), by (3.5) we get
$$\begin{aligned}& \bigl(\tilde{w}^{k}-w^{*}\bigr)^{T}\bigl[HF\bigl( \tilde{w}^{k}\bigr)+g\bigl(w^{k},\tilde{w}^{k}\bigr) \bigr] +\bigl(w^{k}-\tilde{w}^{k}\bigr)^{T}d \bigl(w^{k},\tilde{w}^{k}\bigr) \\& \quad \geq \bigl(\lambda^{k}-\tilde{\lambda}^{k} \bigr)^{T}\bigl(x^{k}-\tilde{x}^{k}\bigr) -\bigl( \lambda^{k}-\tilde{\lambda}^{k}\bigr)^{T} \bigl(y^{k}-\tilde{y}^{k}\bigr) \\& \qquad {}+\bigl(\mu^{k}- \tilde{\mu}^{k}\bigr)^{T}\bigl(y^{k}- \tilde{y}^{k}\bigr) -\bigl(\mu^{k}-\tilde{\mu}^{k} \bigr)^{T}\bigl(z^{k}-\tilde{z}^{k}\bigr) \\& \qquad {} +\beta_{k}\bigl\Vert x^{k}-\tilde{x}^{k} \bigr\Vert +\beta_{k}\bigl\Vert y^{k} - \tilde{y}^{k}\bigr\Vert +\rho_{k}\bigl\Vert y^{k}-\tilde{y}^{k}\bigr\Vert \\& \qquad {}-\frac{1}{2} \rho_{k}\bigl(y^{k}-\tilde{y}^{k} \bigr)^{T}\bigl(z^{k}-\tilde{z}^{k}\bigr) + \frac{2}{\beta_{k}}\bigl\Vert \lambda^{k}-\tilde{\lambda}^{k} \bigr\Vert ^{2} \\& \qquad {}-\frac{\rho_{k}}{2}\bigl(z^{k}-\tilde{z}^{k} \bigr)^{T}\bigl(y^{k}-\tilde{y}^{k}\bigr) + \rho_{k}\bigl\Vert z^{k}-\tilde{z}^{k}\bigr\Vert ^{2}+\frac{3}{\rho_{k}}\bigl\Vert \mu^{k} -\tilde{ \mu}^{k}\bigr\Vert ^{2} \\& \quad \geq \bigl(\lambda^{k}-\tilde{\lambda}^{k} \bigr)^{T}\bigl(x^{k}-\tilde{x}^{k}\bigr) -\bigl( \lambda^{k}-\tilde{\lambda}^{k}\bigr)^{T} \bigl(y^{k}-\tilde{y}^{k}\bigr) \\& \qquad {}+\bigl(\mu^{k}- \tilde{\mu}^{k}\bigr)^{T}\bigl(y^{k}- \tilde{y}^{k}\bigr) -\bigl(\mu^{k}-\tilde{\mu}^{k} \bigr)^{T}\bigl(z^{k}-\tilde{z}^{k}\bigr) \\ & \qquad {} -\rho_{k}\bigl(y^{k}-\tilde{y}^{k} \bigr)^{T}\bigl(z^{k}-\tilde{z}^{k}\bigr) + \beta_{k}\bigl\Vert x^{k}-\tilde{x}^{k}\bigr\Vert ^{2} \\ & \qquad {}+\beta_{k}\bigl\Vert y^{k} - \tilde{y}^{k}\bigr\Vert ^{2}+\rho_{k}\bigl\Vert y^{k}-\tilde{y}^{k}\bigr\Vert ^{2} + \rho_{k}\bigl\Vert z^{k}-\tilde{z}^{k}\bigr\Vert ^{2} \\ & \qquad {} +\frac{2}{\beta_{k}}\bigl\Vert \lambda^{k}-\tilde{ \lambda}^{k}\bigr\Vert ^{2} +\frac{3}{\rho_{k}}\bigl\Vert \mu^{k}-\tilde{\mu}^{k}\bigr\Vert ^{2} \\ & \quad = \biggl[\frac{1}{2}\beta_{k}\bigl\Vert x^{k}- \tilde{x}^{k}\bigr\Vert ^{2} +\bigl(\lambda^{k}- \tilde{\lambda}^{k}\bigr)^{T}\bigl(x^{k}- \tilde{x}^{k}\bigr) +\frac{1}{2\beta_{k}}\bigl\Vert \lambda^{k}-\tilde{\lambda}^{k}\bigr\Vert ^{2} \biggr] \\ & \qquad {} + \biggl[\frac{1}{2}\beta_{k}\bigl\Vert y^{k}-\tilde{y}^{k}\bigr\Vert ^{2} +\bigl( \lambda^{k}-\tilde{\lambda}^{k}\bigr)^{T} \bigl(y^{k}-\tilde{y}^{k}\bigr) +\frac{1}{2\beta_{k}}\bigl\Vert \lambda^{k}-\tilde{\lambda}^{k}\bigr\Vert ^{2} \biggr] \\ & \qquad {} + \biggl[\frac{1}{4}\rho_{k}\bigl\Vert y^{k}-\tilde{y}^{k}\bigr\Vert ^{2} +\bigl( \mu^{k}-\tilde{\mu}^{k}\bigr)^{T} \bigl(y^{k}-\tilde{y}^{k}\bigr)+\frac{1}{\rho_{k}}\bigl\Vert \mu^{k} -\tilde{\mu}^{k}\bigr\Vert ^{2} \biggr] \\ & \qquad {} + \biggl[\frac{1}{4}\rho_{k}\bigl\Vert z^{k}-\tilde{z}^{k}\bigr\Vert ^{2} +\bigl( \mu^{k}-\tilde{\mu}^{k}\bigr)^{T} \bigl(z^{k}-\tilde{z}^{k}\bigr)+\frac{1}{\rho_{k}}\bigl\Vert \mu^{k} -\tilde{\mu}^{k}\bigr\Vert ^{2} \biggr] \\ & \qquad {} + \biggl[\frac{1}{2}\rho_{k}\bigl\Vert y^{k}-\tilde{y}^{k}\bigr\Vert ^{2} - \rho_{k}\bigl(y^{k}-\tilde{y}\bigr)^{T} \bigl(z^{k}-\tilde{z}^{k}\bigr) +\frac{\rho_{k}}{2}\bigl\Vert z^{k}-\tilde{z}^{k}\bigr\Vert ^{2} \biggr] \\ & \qquad {} +\frac{1}{2}\beta_{k}\bigl\Vert x^{k}- \tilde{x}^{k}\bigr\Vert ^{2} +\frac{1}{2} \beta_{k}\bigl\Vert y^{k}-\tilde{y}^{k}\bigr\Vert ^{2} +\frac{1}{4}\rho_{k}\bigl\Vert y^{k}-\tilde{y}^{k}\bigr\Vert ^{2} \\ & \qquad {} +\frac{1}{\beta_{k}}\bigl\Vert \lambda^{k}-\tilde{ \lambda}^{k}\bigr\Vert ^{2} +\frac {1}{4} \rho_{k}\bigl\Vert z^{k}-\tilde{z}^{k}\bigr\Vert ^{2} +\frac{1}{\rho_{k}}\bigl\Vert \mu^{k}-\tilde{ \mu}^{k}\bigr\Vert ^{2} \\ & \quad = \frac{1}{2} \biggl[\sqrt{\beta_{k}}\bigl\Vert x^{k}-\tilde{x}^{k}\bigr\Vert +\frac{1}{\sqrt{\beta_{k}}}\bigl\Vert \lambda^{k}-\tilde{\lambda}^{k}\bigr\Vert \biggr]^{2} +\frac{1}{2} \biggl[\sqrt{\beta_{k}}\bigl\Vert y^{k}-\tilde{y}^{k}\bigr\Vert -\frac{1}{\sqrt{\beta_{k}}} \bigl\Vert \lambda^{k}-\tilde{\lambda}^{k}\bigr\Vert \biggr]^{2} \\ & \qquad {} + \biggl[\frac{\sqrt{\rho_{k}}}{2}\bigl\Vert y^{k}- \tilde{y}^{k}\bigr\Vert + \frac {1}{\sqrt{\rho_{k}}}\bigl\Vert \mu^{k}-\tilde{\mu}^{k}\bigr\Vert \biggr]^{2} + \biggl[\frac{\sqrt{\rho_{k}}}{2}\bigl\Vert z^{k}-\tilde{z}^{k}\bigr\Vert -\frac{1}{\sqrt{\rho_{k}}}\bigl\Vert \mu^{k}-\tilde{ \mu}^{k}\bigr\Vert \biggr]^{2} \\ & \qquad {} + \frac{1}{2} \bigl[\sqrt{\rho_{k}}\bigl\Vert y^{k}-\tilde{y}^{k}\bigr\Vert -\sqrt{\rho_{k}} \bigl\Vert z^{k}-\tilde{z}^{k}\bigr\Vert \bigr]^{2} \\ & \qquad {}+\frac{1}{2}\beta_{k}\bigl\Vert x^{k} -\tilde{x}^{k}\bigr\Vert ^{2}+ \frac{1}{2}\beta_{k}\bigl\Vert y^{k}- \tilde{y}^{k}\bigr\Vert ^{2} +\frac{1}{4} \rho_{k}\bigl\Vert y^{k}-\tilde{y}^{k}\bigr\Vert ^{2} \\ & \qquad {} + \frac{1}{\beta_{k}}\bigl\Vert \lambda^{k} -\tilde{ \lambda}^{k}\bigr\Vert ^{2} +\frac{1}{4} \rho_{k}\bigl\Vert z^{k}-\tilde{z}^{k}\bigr\Vert ^{2} +\frac{1}{\rho_{k}}\bigl\Vert \mu^{k}-\tilde{ \mu}^{k}\bigr\Vert ^{2} \\ & \quad \geq \frac{1}{2}\beta_{k}\bigl\Vert x^{k}- \tilde{x}^{k}\bigr\Vert ^{2} +\frac{1}{2} \beta_{k}\bigl\Vert y^{k}-\tilde{y}^{k}\bigr\Vert ^{2}+\frac{1}{4}\rho_{k}\bigl\Vert y^{k} -\tilde{y}^{k}\bigr\Vert ^{2}+ \frac{1}{\beta_{k}}\bigl\Vert \lambda^{k}-\tilde{\lambda}^{k} \bigr\Vert ^{2} \\ & \qquad {} +\frac{1}{4}\rho_{k}\bigl\Vert z^{k}- \tilde{z}^{k}\bigr\Vert ^{2}+\frac{1}{\rho_{k}}\bigl\Vert \mu^{k} -\tilde{\mu}^{k}\bigr\Vert ^{2} \\ & \quad = \bigl(w^{k}-\tilde{w}^{k}\bigr)^{T}G \bigl(w^{k}-\tilde{w}^{k}\bigr) , \end{aligned}$$
where
$$G= \left ( \textstyle\begin{array}{@{}c@{\quad}c@{\quad}c@{\quad}c@{\quad}c@{}} \frac{1}{2}\beta_{k} & & & & \\ & \frac{2\beta_{k}+\rho_{k}}{4} & & & \\ & & \frac{1}{\beta_{k}} & & \\ & & & \frac{1}{4}\rho_{k}& \\ & & & & \frac{1}{\rho_{k}} \end{array}\displaystyle \right ). $$
Thus, we have
$$\begin{aligned}& \bigl(\tilde{w}^{k}-w^{*}\bigr)^{T}\bigl[HF \bigl(\tilde{w}^{k}\bigr)+g\bigl(w^{k},\tilde{w}^{k} \bigr)\bigr] +\bigl(w^{k}-\tilde{w}^{k}\bigr)^{T}d \bigl(w^{k},\tilde{w}^{k}\bigr) \\& \quad \geq \bigl(w^{k}-\tilde{w}^{k}\bigr)^{T}G \bigl(w^{k}-\tilde{w}^{k}\bigr) = \bigl\Vert w^{k}-\tilde{w}^{k}\bigr\Vert ^{2}_{G }:= \varphi\bigl(w^{k},\tilde{w}^{k}\bigr). \end{aligned}$$
(3.12)
This theorem follows from (3.12) directly. □
Lemma 3.4
If
\(\tilde{w}^{k}=(\tilde{x}^{k},\tilde {y}^{k}, \tilde{\lambda}^{k},\tilde{z}^{k},\tilde{\mu}^{k})\)
is generated by (2.1)-(2.5) from a given
\(w^{k}=(x^{k},y^{k}, \lambda^{k},z^{k},\mu^{k})\), then for any
\(w^{*}=(x^{*}, y^{*}, \lambda^{*},z^{*},\mu^{*}) \in{\mathcal{W}}^{*}\), we have
$$ \bigl(w^{k}-w^{*}\bigr)^{T} G d \bigl(w^{k},\tilde{w}^{k}\bigr) \ge\varphi \bigl(w^{k},\tilde{w}^{k}\bigr). $$
(3.13)
Proof
We omit the proof of Lemma 3.4 here. A similar proof can be found in [15]. □
Theorem 3.2
For a given
\(w^{k}=(x^{k},y^{k},\lambda^{k},z^{k},\mu^{k})\), if
\(\tilde{w}^{k}=(\tilde {x}^{k},\tilde{y}^{k}, \tilde{\lambda}^{k},\tilde{z}^{k},\tilde{\mu}^{k})\)
is generated by (2.1)-(2.5), then for any
\(w^{*}=(x^{*}, y^{*}, \lambda^{*},z^{*},\mu^{*}) \in{\mathcal{W}}^{*}\)
we have
$$ \bigl\Vert w^{k+1}-w^{*}\bigr\Vert ^{2}_{G} \leq\bigl\Vert w^{k}-w^{*}\bigr\Vert ^{2}_{G} - \gamma(2-\gamma)\frac{1}{\kappa}\bigl\Vert w^{k}- \tilde{w}^{k}\bigr\Vert ^{2}_{G}, $$
(3.14)
where
\(0<\gamma<2\), \(\kappa>0 \).
Proof
By the iterative formula (2.6), and Lemma 3.4, we have
$$\begin{aligned} \bigl\Vert w^{k+1}-w^{*}\bigr\Vert ^{2}_{G} =& \bigl\Vert w^{k}-w^{*}-\alpha_{k} d\bigl(w^{k}, \tilde{w}^{k}\bigr)\bigr\Vert ^{2}_{G} \\ =&\bigl\Vert w^{k}-w^{*}\bigr\Vert ^{2}_{G}-2 \alpha_{k} \bigl\langle w^{k}-w^{*},Gd\bigl(w^{k}, \tilde{w}^{k}\bigr)\bigr\rangle +\alpha_{k} ^{2} \bigl\Vert d\bigl(w^{k},\tilde {w}^{k}\bigr)\bigr\Vert ^{2}_{G} \\ \leq&\bigl\Vert w^{k}-w^{*}\bigr\Vert ^{2}_{G}-2 \alpha_{k} \varphi\bigl(w^{k},\tilde{w}^{k}\bigr)+ \alpha ^{2}_{k}\bigl\Vert d\bigl(w^{k}, \tilde{w}^{k}\bigr)\bigr\Vert ^{2}_{G}. \end{aligned}$$
Following from (2.7) and (2.8), we have
$$\begin{aligned} \bigl\Vert w^{k+1}-w^{*}\bigr\Vert ^{2}_{G} \leq&\bigl\Vert w^{k}-w^{*}\bigr\Vert ^{2}_{G} -2\gamma\alpha^{*^{2}}_{k}\bigl\Vert d\bigl(w^{k}, \tilde{w}^{k}\bigr)\bigr\Vert ^{2}_{G} + \gamma^{2}\alpha^{*^{2}}_{k}\bigl\Vert d \bigl(w^{k},\tilde{w}^{k}\bigr)\bigr\Vert ^{2}_{G} \\ =&\bigl\Vert w^{k}-w^{*}\bigr\Vert ^{2}_{G} - \gamma(2-\gamma)\alpha^{*^{2}}_{k}\bigl\Vert d \bigl(w^{k},\tilde{w}^{k}\bigr)\bigr\Vert ^{2}_{G} \\ =& \bigl\Vert w^{k}-w^{*}\bigr\Vert ^{2}_{G} -\gamma(2-\gamma)\alpha^{*}_{k} \bigl\Vert w^{k}-\tilde {w}^{k}\bigr\Vert _{G}^{2} \\ \le& \bigl\Vert w^{k}-w^{*}\bigr\Vert ^{2}_{G} -\gamma(2-\gamma)\frac{1}{\kappa} \bigl\Vert w^{k}- \tilde{w}^{k}\bigr\Vert _{G}^{2}. \end{aligned}$$
The last inequality follows from (2.10). □
Theorem 3.2 claims the Fejér monotonicity of the sequence \(\{ w^{k}\}\) generated by the HSM. Adding (3.14) from 0 to ∞ with respect to k yields
$$ \gamma(2-\gamma)\frac{1}{\kappa}\sum_{k=0}^{\infty}\bigl\Vert w^{k}-\tilde{w}^{k}\bigr\Vert ^{2}_{G}\le\bigl\Vert w^{0}-w^{*}\bigr\Vert ^{2}_{G}-\lim_{k\rightarrow\infty}\bigl\Vert w^{k+1}-w^{*}\bigr\Vert ^{2}_{G}\le\bigl\Vert w^{0}-w^{*}\bigr\Vert ^{2}_{G}< \infty . $$
Thus we have
$$ \lim_{k \rightarrow\infty} \bigl\Vert w^{k}- \tilde{w}^{k}\bigr\Vert ^{2}_{G} =0, $$
(3.15)
which results in both \(\{w^{k}\}\) and \(\{\tilde{w}^{k}\}\) being bounded sequences and having cluster points. Let \(w^{\infty}\) be a cluster point of \(\{\tilde{w}^{k}\}\) and \(\{\tilde{w}^{k_{j}}\}\) be a subsequence converging to \(w^{\infty}\).
By the HSM, \(\tilde{w}^{k_{j}}\) is a solution of (2.12), thus
$$\begin{aligned}& \eta\bigl\Vert z'\bigr\Vert _{1}-\eta \bigl\Vert \tilde{z}^{k_{j}}\bigr\Vert _{1}+ \bigl(w'-\tilde{w}^{k_{j}}\bigr)^{T} \bigl[F\bigl( \tilde{w}^{k_{j}}\bigr)+g\bigl(w^{k_{j}},\tilde {w}^{k_{j}} \bigr)-M\bigl(w^{k_{j}}-\tilde{w}^{k_{j}}\bigr)\bigr] \\& \quad \ge0, \quad \forall w'\in \mathcal{W}. \end{aligned}$$
(3.16)
The limit (3.15) also results in \(\lim_{k_{j}\rightarrow \infty} g (w^{k_{j}}, \tilde{w}^{k_{j}})=0\) and \(\lim_{k_{j}\rightarrow\infty} d (w^{k_{j}}, \tilde{w}^{k_{j}})=M (w^{k_{j}}-\tilde{w}^{k_{j}})=0\) by positive-definiteness of G and M. Taking the limit on both sides of (3.16) we have
$$ \eta\bigl\Vert z'\bigr\Vert _{1}-\eta \bigl\Vert {z}^{\infty}\bigr\Vert _{1}+ \bigl(w'- {w}^{\infty}\bigr)^{T} F\bigl( {w}^{\infty}\bigr)\ge0, \quad \forall w'\in \mathcal{W}, $$
(3.17)
which implies \(w^{\infty}\) is a solution of the problem (1.4) by the optimality condition (2.11).
Note that (3.14) holds for all solutions of (1.4), we get
$$ \bigl\Vert w^{k+1}-w^{\infty}\bigr\Vert ^{2}_{G}\le\bigl\Vert w^{k}-w^{\infty}\bigr\Vert ^{2}_{G}, \quad \forall k. $$
(3.18)
Since \(\tilde{w}^{k_{j}} \rightarrow w^{\infty}\) and \(\|w^{k_{j}}-\tilde {w}^{k_{j}}\|^{2}_{G}\rightarrow0\) as \(k_{j} \rightarrow\infty\), for \(\forall\varepsilon>0\), there exists an integer \(k_{l}>0\) such that for all k and \(k_{j}\) larger than \(k_{l}\), we have
$$ \bigl\Vert \tilde{w}^{k_{j}}-w^{\infty}\bigr\Vert _{G}< \frac {\varepsilon}{2}, \qquad \bigl\Vert w^{k}- \tilde{w}^{k_{j}} \bigr\Vert _{G}< \frac{\varepsilon }{2}. $$
(3.19)
It follows from (3.18) and (3.19) that
$$\bigl\Vert w^{k}-w^{\infty}\bigr\Vert _{G}=\bigl\Vert w^{k}-\tilde{w}^{k_{j}}+\tilde{w}^{k_{j}}-w^{\infty}\bigr\Vert _{G}\le\bigl\Vert w^{k}-\tilde{w}^{k_{j}} \bigr\Vert _{G}+ \bigl\Vert \tilde{w}^{k_{j}}-w^{\infty}\bigr\Vert _{G}< \varepsilon. $$
Thus, the sequence \(\{w^{k}\}\) converges to \(w^{\infty}\), which is a solution of (1.4), or equivalently, of the problem (1.1).