Kernel function based interior-point methods for horizontal linear complementarity problems

Abstract

It is well known that each kernel function defines an interior-point algorithm. In this paper we propose new classes of kernel functions whose form is different from known kernel functions and define interior-point methods (IPMs) based on these functions whose barrier term is exponential power of exponential functions for $P_{\ast }(\kappa )$-horizontal linear complementarity problems (HLCPs). New search directions and proximity measures are defined by these kernel functions. We obtain so far the best known complexity results for large- and small-update methods.

1 Introduction

In this paper we consider $P_{\ast }(\kappa )$-horizontal linear complementarity problem (HLCP) as follows.

Given $\{M,N\}$, a $P_{\ast }(\kappa )$-pair, $M,N\in {\mathbf{R}}^{n\times n}$, $q\in {\mathbf{R}}^n$, and $\kappa \ge 0$, find a pair $(x;s)\in {\mathbf{R}}^{2n}$ such that

$$-Mx+Ns=q,xs=0,(x;s)\ge 0.$$

(1)

Note that $\{M,N\}$ is called a $P_{\ast }(\kappa )$-pair if $-Mx+Ns=0$ implies that

$$(1+4\kappa )\sum _{i\in I_{+}(x)}{x}_i{s}_i+\sum _{i\in I_{-}(x)}{x}_i{s}_i\ge 0,$$

where $I_{+}(x):=\{i\in I:x_i{s}_i\ge 0\}$, $I_{-}(x):=\{i\in I:x_i{s}_i<0\}$, and $I:=\{1,2,\dots ,n\}$.

$P_{\ast }(\kappa )$-HLCPs have many applications in economic equilibrium problems, noncooperative games, traffic assignment problems, and optimization problems [1, 2]. $P_{\ast }(\kappa )$-HLCP (1) includes the standard linear complementarity problem (LCP), linear, and quadratic optimization problems. Indeed, when N is nonsingular, then $P_{\ast }(\kappa )$-HLCP reduces to $P_{\ast }(\kappa )$-LCP. Furthermore, when $\kappa =0$, $P_{\ast }(0)$-HLCP is monotone LCP.

Recently, Bai et al. [3] defined the concept of eligible kernel functions which require four conditions and proposed primal-dual IPMs for linear optimization (LO) problems based on these functions, and some of these methods achieved the best known complexity results for both large- and small-update methods. Cho [4] and Cho et al. [5] extended these algorithms for LO to $P_{\ast }(\kappa )$-linear complementarity problems (LCPs) and obtained the similar complexity results as LO problems for large-update methods. Amini et al. [6, 7] introduced new IPMs based on parametric versions of kernel functions in [3] and obtained the better iteration bounds than the bound of the algorithm in [3] with numerical tests. Wang et al. [2] generalized polynomial IPMs for LO problem to $P_{\ast }(\kappa )$-HLCP based on a finite kernel function, which was first defined in [8], and obtained the same iteration bounds for large- and small-update methods as an LO problem. Ghami et al. [9] extended IPMs for LO problems to the $P_{\ast }(\kappa )$-LCPs based on eligible kernel functions, which were defined in [3], and proposed large- as well as small-update methods. Lesaja et al. [10] also proposed IPMs for $P_{\ast }(\kappa )$-LCPs based on ten kernel functions which were defined for LO problems. Ghami et al. [11] proposed IPM for an LO problem based on a kernel function whose barrier term is a trigonometric function. However, this method does not have the best known iteration bound for a large-update method. Cho et al. [12] defined a new kernel function, whose barrier term is the exponential power of the exponential function for LO problems, and obtained the best known iteration bounds for large- and small-update methods.

Motivated by these works, we introduce new classes of eligible kernel functions, which are different from known kernel functions in [3, 6, 7] and have the exponential power of exponential barrier term, and propose a complexity analysis of the IPMs for $P_{\ast }(\kappa )$-HLCP based on these kernel functions. We show that these algorithms have $\mathcal{O}((1+2\kappa )\sqrt{n}lognlog\frac{n{\mu }^0}{ϵ})$ and $\mathcal{O}((1+2\kappa )\sqrt{n}log\frac{n{\mu }^0}{ϵ})$ iteration bounds for large- and small-update methods, respectively, which are currently the best known iteration bounds for such methods.

The paper is organized as follows. In Section 2 we propose some basic concepts and a generic interior point algorithm for $P_{\ast }(\kappa )$-HLCP. In Section 3 we introduce new classes of eligible kernel functions and their technical properties. Finally, we derive the framework for analyzing the iteration bounds and the complexity results of the algorithms based on these kernel functions in Section 4.

Notational conventions: ${\mathbf{R}}_{+}^n$ and ${\mathbf{R}}_{++}^n$ denote the sets of n-dimensional nonnegative vectors and positive vectors, respectively. For $x,s\in {\mathbf{R}}^n$, $x_{min}$, xs, and $(x;s)$ denote the smallest component of the vector x, the componentwise product of the vectors x and s, and the column vector ${(x^T,s^T)}^T$, respectively. We denote by D the diagonal matrix from a vector d, i.e., $D=diag(d)$. e denotes the n-dimensional vector of ones. For $f(x)$, $g(x):{\mathbf{R}}_{++}\to {\mathbf{R}}_{++}$, $f(x)=\mathcal{O}(g(x))$ if $f(x)\le c_{1}g(x)$ for some positive constant $c_1$ and $f(x)=\mathrm{\Theta }(g(x))$ if $c_{2}g(x)\le f(x)\le c_{3}g(x)$ for some positive constants $c_2$ and $c_3$.

2 Preliminaries

In this section we recall some basic definitions and introduce a generic interior point algorithm for $P_{\ast }(\kappa )$-HLCP.

Definition 2.1 [13]

Let $M\in {\mathbf{R}}^{n\times n}$, $x\in {\mathbf{R}}^n$, and $\kappa \ge 0$.

• (i) M is called a positive semidefinite matrix if $x^T(Mx)\ge 0$.

• (ii) M is called a $P_0$-matrix if there exists an index $i\in I$ such that $x_i\ne 0$ and $x_i{[Mx]}_i\ge 0$.

• (iii) M is called a $P_{\ast }(\kappa )$-matrix if

  $$(1+4\kappa )\sum _{i\in I_{+}(x)}{x}_i{[Mx]}_i+\sum _{i\in I_{-}(x)}{x}_i{[Mx]}_i\ge 0,$$

where ${[Mx]}_i$ denotes the i th component of the vector Mx, $I_{+}(x)=\{i\in I:x_i{[Mx]}_i\ge 0\}$, and $I_{-}(x)=\{i\in I:x_i{[Mx]}_i<0\}$.

Definition 2.2 [14]

Let $M,N\in {\mathbf{R}}^{n\times n}$, $x,s\in {\mathbf{R}}^n$, and $\kappa \ge 0$.

• (i) $\{M,N\}$ is called a monotone pair if $-Mx+Ns=0$ implies $x^{T}s\ge 0$.

• (ii) $\{M,N\}$ is called a $P_0$-pair if $-Mx+Ns=0$ and $(x;s)\ne 0$ implies that there exists an index $i\in I$ such that $x_i\ne 0$ or $s_i\ne 0$, and $x_i{s}_i\ge 0$.

• (iii) $\{M,N\}$ is called a $P_{\ast }(\kappa )$-pair if $-Mx+Ns=0$ implies that $x^{T}s\ge -4\kappa {\sum }_{i\in I_{+}}{x}_i{s}_i$, where $I_{+}(x)=\{i\in I:x_i{s}_i\ge 0\}$.

Lemma 2.3 If $\{M,N\}$ is a $P_0$-pair, then

$$M^{\prime }=\left(\begin{array}{cc}-M& N\\ S& X\end{array}\right)$$

is a nonsingular matrix for any positive diagonal matrices $X,S\in {\mathbf{R}}^{n\times n}$.

Proof Assume that the matrix $M^{\prime }$ is singular. Then $M^{\prime }\zeta =0$ for some nonzero $\zeta =(\xi ;\eta )\in R^{2n}$, i.e., $-M\xi +N\eta =0$ and $s_i{\xi }_i+x_i{\eta }_i=0$, $i\in I$. Hence $(\xi ;\eta )\ne 0$, and we have an index $i\in I$ such that ${\xi }_i\ne 0$ or ${\eta }_i\ne 0$, and ${\xi }_i{\eta }_i\ge 0$, since $\{M,N\}$ is a $P_0$-pair. On the other hand, ${\xi }_i{\eta }_i=-x_i{({\eta }_i)}^2/s_i<0$. This is a contradiction. This completes the proof. □

Since the class of $P_0$-pairs includes the class of $P_{\ast }(\kappa )$-pairs, we obtain the following corollary.

Corollary 2.4 Let $\{M,N\}$ be a $P_{\ast }(\kappa )$-pair and $x,s\in {\mathbf{R}}_{++}^n$. Then all $c\in {\mathbf{R}}^n$ the system

$$-M\mathrm{\Delta }x+N\mathrm{\Delta }s=0,S\mathrm{\Delta }x+X\mathrm{\Delta }s=c$$

has a unique solution $(\mathrm{\Delta }x;\mathrm{\Delta }s)$.

The basic idea of generic IPMs is to replace the second equation of (1) by the parameterized equation $xs=\mu \mathbf{e}$ with $\mu >0$, i.e., we consider the following system:

$$-Mx+Ns=q,xs=\mu \mathbf{e},(x;s)>0.$$

(2)

Without loss of generality, we assume that (1) satisfies the interior-point condition (IPC), i.e., there exists $(x^0;s^0)>0$ such that $-M{x}^0+N{s}^0=q$ [15]. Since $\{M,N\}$ is a $P_{\ast }(\kappa )$-pair and (1) satisfies IPC, the system (2) has a unique solution $(x(\mu );s(\mu ))$ for each $\mu >0$, which is called the μ-center. The set of μ-centers is called the central path of (1). The limit of the central path exists, and since the limit point satisfies (1), it naturally yields the solution for (1) [16]. IPMs follow this central path approximately and approach the solution of (1) as $\mu \to 0$.

For given $(x;s):=(x^0;s^0)$, by applying Newton’s method to the system (2), we have the Newton-system as follows:

$$-M\mathrm{\Delta }x+N\mathrm{\Delta }s=0,S\mathrm{\Delta }x+X\mathrm{\Delta }s=\mu \mathbf{e}-xs.$$

(3)

By taking a step along the search direction $(\mathrm{\Delta }x;\mathrm{\Delta }s)$, we define a new iteration $(x_{+};s_{+})$, where for some $\alpha \ge 0$,

$$x_{+}:=x+\alpha \mathrm{\Delta }x,s_{+}:=s+\alpha \mathrm{\Delta }s.$$

(4)

To have the motivation of a new algorithm, we define the following scaled vectors:

$$v:=\sqrt{\frac{xs}{\mu }},d:=\sqrt{\frac{x}{s}},d_x:=\frac{v\mathrm{\Delta }x}{x},d_s:=\frac{v\mathrm{\Delta }s}{s}.$$

(5)

Using (5), we can rewrite the Newton-system (3) as follows:

$$-\overline{M}{d}_x+\overline{N}{d}_s=0,d_x+d_s=v^{-1}-v,$$

(6)

where $\overline{M}:=DMD$, $\overline{N}:=DND$, and $D:=diag(d)$. Note that the right-hand side of the second equation of (6) equals the negative gradient of the logarithmic barrier function ${\mathrm{\Psi }}_l(v):={\sum }_{i=1}^n{\psi }_l(v_i)$ and ${\psi }_l(t)=\frac{t^2-1}{2}-logt$, i.e.,

$$d_x+d_s=-\mathrm{\nabla }{\mathrm{\Psi }}_l(v).$$

(7)

The interior-point algorithm works as follows. Assume that we are given a strictly feasible point $(x;s)$ which is in a τ-neighborhood of the given μ-center. Then we update μ to ${\mu }_{+}=(1-\theta )\mu$ for some fixed $\theta \in (0,1)$ and solve the system (3) to obtain the search direction. The positivity condition of a new iteration is ensured with the right choice of the step size α. This procedure is repeated until we find a new iteration $(x_{+};s_{+})$ that is in a τ-neighborhood of the ${\mu }_{+}$-center and then we let $\mu :={\mu }_{+}$ and $(x;s):=(x_{+};s_{+})$. We repeat the process until $n\mu <\epsilon$ (see Algorithm 1).

Algorithm 1

Generic interior-point algorithm for $P_{\ast }(\kappa )$-HLCP

If $\tau =\mathcal{O}(n)$ and $\theta =\mathrm{\Theta }(1)$, then the algorithm is called a large-update method. When $\tau =\mathcal{O}(1)$ and $\theta =\mathrm{\Theta }(\frac{1}{\sqrt{n}})$, we call the algorithm a small-update method.

3 New kernel function

In this section we define new classes of kernel functions and give their essential properties.

$\psi :{\mathbf{R}}_{++}\to {\mathbf{R}}_{+}$ is called a kernel function if ψ is twice differentiable and satisfies the following conditions:

$${\psi }^{\prime }(1)=\psi (1)=0,{\psi }^{″}(t)>0,\mathrm{\forall }t>0,\underset{t\to 0^{+}}{lim}\psi (t)=\underset{t\to \mathrm{\infty }}{lim}\psi (t)=\mathrm{\infty }.$$

(8)

We define new classes of kernel functions ${\psi }_j(t)$, $j\in \{1,2\}$, in Table 1 and give the first three derivatives of ${\psi }_j(t)$, $j\in \{1,2\}$, in Table 2 and Table 3.

Table 1 Kernel functions

Table 2 The first two derivatives of the kernel functions

Table 3 The third derivative of the kernel functions

In the following lemma, we show that $\psi (t):={\psi }_j(t)$, $j\in \{1,2\}$, are eligible [3].

Lemma 3.1 Let $\psi (t):={\psi }_j(t)$, $j\in \{1,2\}$, be defined as in Table 1. Then ${\psi }_j$, $j\in \{1,2\}$, satisfy the following eligible conditions:

• (a) $t{\psi }^{″}(t)+{\psi }^{\prime }(t)>0$, $t>0$, i.e., ψ is exponential convex,

• (b) $t{\psi }^{″}(t)-{\psi }^{\prime }(t)>0$, $t>0$,

• (c) ${\psi }_j^{(3)}(t)<0$, $t>0$,

• (d) $2{({\psi }^{″}(t))}^2-{\psi }^{\prime }(t){\psi }_j^{(3)}(t)>0$, $t>0$.

Proof From Table 4, Table 3, and Table 5, we show that ${\psi }_j(t)$, $j\in \{1,2\}$, satisfy eligible conditions (a)-(d).  □

Table 4 Conditions (a) and (b)

Table 5 Condition (d)

Remark 3.2 For ${\psi }_j(t)$, $j\in \{1,2\}$, let ${\psi }_{b1}(t)={\psi }_1(t)-\frac{e(t^2-1)}{2}$, ${\psi }_{b2}(t)={\psi }_2(t)-\frac{t^2-1}{2}$.

From Table 2,

$${\psi }_1^{″}(t)\ge e,{\psi }_2^{″}(t)\ge 1,t>0.$$

(9)

Since ${\psi }_{bj}^{\prime }(t)<0$, $j\in \{1,2\}$, from Table 2, ${\psi }_{bj}(t)$, $j\in \{1,2\}$, are monotonically decreasing with respect to $t>0$.

Let ${\rho }_j:[0,\mathrm{\infty })\to (0,1]$ and ${\varrho }_j:[0,\mathrm{\infty })\to [1,\mathrm{\infty })$ denote the inverse functions of the restriction of $-\frac{1}{2}{\psi }_j^{\prime }(t)$ for $0<t\le 1$ and ${\psi }_j(t)$ for $t\ge 1$, respectively, $j\in \{1,2\}$. Then

$$z=-\frac{1}{2}{\psi }_j^{\prime }(t)\iff t={\rho }_j(z),0<t\le 1,$$

(10)

and

$$u={\psi }_j(t)\iff t={\varrho }_j(u),t\ge 1.$$

(11)

Lemma 3.3 Let ${\rho }_j(z)$, $j\in \{1,2\}$, be defined as in (10). Then we have, for $p\ge 1$, $r\ge 1$,

• (i) ${\rho }_1(z)\ge {(log(e+p^{-1}log(e+2z)))}^{-\frac{1}{r}}$, $z\ge 0$,

• (ii) ${\rho }_2(z)\ge {(1+log(1+p^{-1}log(1+2z)))}^{-\frac{1}{r}}$, $z\ge 0$.

Proof For (i), using (10) and Table 2, we have the equation

$$-et+e^{p(g_1(t)-e)}{g}_1(t)t^{-r-1}=2z,g_1(t)=e^{t^{-r}},0<t\le 1.$$

Since $0<t\le 1$,

$$e^{p(g_1(t)-e)}{g}_1(t)t^{-r-1}=et+2z\le e+2z,g_1(t)=e^{t^{-r}}.$$

(12)

By taking the natural logarithm on both sides of (12), we have $e^{t^{-r}}\le e+p^{-1}log(e+2z)$. Hence we have

$${\rho }_1(z)\ge {(log(e+p^{-1}log(e+2z)))}^{-\frac{1}{r}}.$$

By the same way as (i), we obtain the result (ii). This completes the proof. □

Lemma 3.4 Let ${\psi }_j(t)$, $j\in \{1,2\}$, be defined as in Table 1. Then we have

• (i) $\frac{e}{2}{(t-1)}^2\le {\psi }_1(t)\le \frac{1}{2e}{({\psi }_1^{\prime }(t))}^2$, $t>0$,

• (ii) $\frac{1}{2}{(t-1)}^2\le {\psi }_2(t)\le \frac{1}{2}{({\psi }_2^{\prime }(t))}^2$, $t>0$.

Proof For (i), using the first condition of (8) and (9), we have

$${\psi }_1(t)={\int }_1^t{\int }_1^{\xi }{\psi }_1^{″}(\zeta )d\zeta d\xi \ge e{\int }_1^t{\int }_1^{\xi }d\zeta d\xi =\frac{e}{2}{(t-1)}^2,$$

which proves the first inequality. The second inequality is obtained as follows:

$$\begin{array}{rcl}{\psi }_1(t)& =& {\int }_1^t{\int }_1^{\xi }{\psi }_1^{″}(\zeta )d\zeta d\xi \le \frac{1}{e}{\int }_1^t{\int }_1^{\xi }{\psi }_1^{″}(\xi ){\psi }_1^{″}(\zeta )d\zeta d\xi \\ =& \frac{1}{e}{\int }_1^t{\psi }_1^{″}(\xi ){\psi }_1^{\prime }(\xi )d\xi =\frac{1}{e}{\int }_1^t{\psi }_1^{\prime }(\xi )d{\psi }_1^{\prime }(\xi )=\frac{1}{2e}{({\psi }_1^{\prime }(t))}^2.\end{array}$$

For (ii), by the same way as above, we obtain the result. This completes the proof. □

Lemma 3.5 Let ${\varrho }_j(u)$, $j\in \{1,2\}$, be defined as in (11). Then we have

• (i) ${\varrho }_1(u)\le 1+\sqrt{\frac{2u}{e}}$, $u\ge 0$,

• (ii) ${\varrho }_2(u)\le 1+\sqrt{2u}$, $u\ge 0$.

Proof For (i), using the first inequality in Lemma 3.4, we have $u={\psi }_1(t)\ge \frac{e}{2}{(t-1)}^2$. Then we have

$$t={\varrho }_1(u)\le 1+\sqrt{\frac{2u}{e}},u\ge 0.$$

Similarly, we obtain the result (ii). This completes the proof. □

In this paper we replace the logarithmic barrier function ${\mathrm{\Psi }}_l(v)$ in (7) by a strictly convex function $\mathrm{\Psi }(v)$ as follows:

$$d_x+d_s=-\mathrm{\nabla }\mathrm{\Psi }(v),$$

(13)

where

$$\mathrm{\Psi }(v):={\mathrm{\Psi }}_j(v)=\sum _{i=1}^n{\psi }_j(v_i),j\in \{1,2\},$$

(14)

and ${\psi }_j(t)$, $j\in \{1,2\}$, are defined in Table 1. Since $\mathrm{\Psi }(v)$ is strictly convex and minimal at $v=\mathbf{e}$, we have

$$\mathrm{\Psi }(v)=0\iff v=\mathbf{e}\iff x=x(\mu ),s=s(\mu ).$$

Using (5) and (13), we modify the Newton-system (3) as follows:

$$-M\mathrm{\Delta }x+N\mathrm{\Delta }s=0,S\mathrm{\Delta }x+X\mathrm{\Delta }s=-\mu v\mathrm{\nabla }\mathrm{\Psi }(v).$$

(15)

By Corollary 2.4, the system (15) has a unique solution $(\mathrm{\Delta }x;\mathrm{\Delta }s)$ which is the modified Newton search direction. Consequently, we use $\mathrm{\Psi }(v)$ as the proximity function to find a search direction and to measure the proximity between the current iteration and the μ-center. We also define the norm-based proximity measure ${\delta }_j(v)$, $j\in \{1,2\}$, as follows:

$${\delta }_j(v):=\frac{1}{2}\parallel \mathrm{\nabla }{\mathrm{\Psi }}_j(v)\parallel =\frac{1}{2}\parallel d_x+d_s\parallel .$$

(16)

The following lemma gives a relation between two proximity measures.

Lemma 3.6 Let ${\delta }_j(v)$ and ${\mathrm{\Psi }}_j(v)$, $j\in \{1,2\}$, be defined as in (16) and (14), respectively. Then we have

• (i) ${\delta }_1(v)\ge \sqrt{\frac{e{\mathrm{\Psi }}_1(v)}{2}}$,

• (ii) ${\delta }_2(v)\ge \sqrt{\frac{{\mathrm{\Psi }}_2(v)}{2}}$.

Proof For (i), using (16) and the second inequality in Lemma 3.4, we have

$${\delta }_1^2(v)=\frac{1}{4}{\parallel \mathrm{\nabla }{\mathrm{\Psi }}_1(v)\parallel }^2=\frac{1}{4}\sum _{i=1}^n{({\psi }_1^{\prime }(v_i))}^2\ge \frac{e{\mathrm{\Psi }}_1(v)}{2}.$$

Hence we have ${\delta }_1(v)\ge \sqrt{\frac{e{\mathrm{\Psi }}_1(v)}{2}}$.

For (ii), by the same way as above, we obtain the result. This completes the proof. □

Using the eligible conditions (b) and (c) in Lemma 3.1, we obtain the following lemma.

Lemma 3.7 (Theorem 3.2 in [3])

Let ${\varrho }_j$, $j\in \{1,2\}$, be defined as in (11). Then we have

$${\mathrm{\Psi }}_j(\beta v)\le n\psi \left(\beta {\varrho }_j\left(\frac{\mathrm{\Psi }(v)}{n}\right)\right),v\in {\mathbf{R}}_{++},\beta \ge 1.$$

In the following lemma, we give upper bounds of ${\mathrm{\Psi }}_j(v)$, $j\in \{1,2\}$, after a μ-update.

Lemma 3.8 Let ${\mathrm{\Psi }}_j(v)$, $j\in \{1,2\}$, be defined as in (14), $0<\theta <1$, and $v_{+}=\frac{v}{\sqrt{1-\theta }}$. If ${\mathrm{\Psi }}_j(v)\le \tau$, $j\in \{1,2\}$, then we have

• (i) ${\mathrm{\Psi }}_1(v_{+})\le \frac{en\theta +2\tau +2\sqrt{2en\tau }}{2(1-\theta )}$ or ${\mathrm{\Psi }}_1(v_{+})\le \frac{{\psi }_1^{″}(1){(\sqrt{\frac{2\tau }{e}}+\theta \sqrt{n})}^2}{2(1-\theta )}$,

• (ii) ${\mathrm{\Psi }}_2(v_{+})\le \frac{n\theta +2\tau +2\sqrt{2n\tau }}{2(1-\theta )}$ or ${\mathrm{\Psi }}_2(v_{+})\le \frac{{\psi }_2^{″}(1){(\sqrt{2\tau }+\theta \sqrt{n})}^2}{2(1-\theta )}$.

Proof For the first inequality of (i), using Remark 3.2 with ${\psi }_{b1}(1)=0$ and ${\psi }_{b1}^{\prime }(t)<0$, we get

$${\psi }_1(t)\le \frac{e(t^2-1)}{2},t\ge 1.$$

(17)

Using Lemma 3.7, (17), and Lemma 3.5(i), we have

$$\begin{array}{rcl}{\mathrm{\Psi }}_1(v_{+})& \le & \frac{en}{2}(\frac{{\varrho }_1^2(\frac{\tau }{n})}{1-\theta }-1)\le \frac{en}{2}(\frac{{(1+\sqrt{\frac{2\tau }{en}})}^2}{1-\theta }-1)\\ =& \frac{en\theta +2\tau +2\sqrt{2en\tau }}{2(1-\theta )}.\end{array}$$

For the second inequality of (i), using Taylor’s theorem, ${\psi }_1(1)={\psi }_1^{\mathrm{\prime }}(1)=0$ and ${\psi }_1^{(3)}(t)<0$, we have

$$\begin{array}{rcl}{\psi }_1(t)& =& {\psi }_1(1)+{\psi }_1^{\mathrm{\prime }}(1)(t-1)+\frac{1}{2}{\psi }_1^{\mathrm{\prime }\mathrm{\prime }}(1){(t-1)}^2+\frac{1}{3!}{\psi }_1^{(3)}(\xi ){(t-1)}^3\\ =& \frac{1}{2}{\psi }_1^{\mathrm{\prime }\mathrm{\prime }}(1){(t-1)}^2+\frac{1}{3!}{\psi }_1^{(3)}(\xi ){(t-1)}^3\\ <& \frac{{\psi }_1^{\mathrm{\prime }\mathrm{\prime }}(1)}{2}{(t-1)}^2\end{array}$$

(18)

for some ξ, $1\le \xi \le t$. Since $\frac{1}{\sqrt{1-\theta }}\ge 1$ and ${\varrho }_1(\frac{\tau }{n})\ge 1$, we have $\frac{{\varrho }_1(\frac{\tau }{n})}{\sqrt{1-\theta }}\ge 1$. Using Lemma 3.7, (18), and Lemma 3.5(i), we have

$$\begin{array}{rcl}{\mathrm{\Psi }}_1(v_{+})& \le & \frac{n{\psi }_1^{\mathrm{\prime }\mathrm{\prime }}(1)}{2}{(\frac{{\varrho }_1(\frac{\tau }{n})}{\sqrt{1-\theta }}-1)}^2\\ \le & \frac{n{\psi }_1^{\mathrm{\prime }\mathrm{\prime }}(1)}{2}{\left(\frac{1+\sqrt{\frac{2\tau }{en}}-\sqrt{1-\theta }}{\sqrt{1-\theta }}\right)}^2\\ \le & \frac{n{\psi }_1^{\mathrm{\prime }\mathrm{\prime }}(1)}{2}{\left(\frac{\sqrt{\frac{2\tau }{en}}+\theta }{\sqrt{1-\theta }}\right)}^2=\frac{{\psi }_1^{\mathrm{\prime }\mathrm{\prime }}(1)}{2(1-\theta )}{(\sqrt{\frac{2\tau }{e}}+\theta \sqrt{n})}^2,\end{array}$$

where the last inequality holds from $1-\sqrt{1-\theta }=\frac{\theta }{1+\sqrt{1-\theta }}\le \theta$, $0<\theta <1$.

By the same way as the proof of (i), we obtain the result (ii). This completes the proof. □

Define

$${\overline{\mathrm{\Psi }}}_{1,0}:=\frac{en\theta +2\tau +2\sqrt{2en\tau }}{2(1-\theta )},{\tilde{\mathrm{\Psi }}}_{1,0}:=\frac{{\psi }_1^{\mathrm{\prime }\mathrm{\prime }}(1)}{2(1-\theta )}{(\sqrt{\frac{2\tau }{e}}+\theta \sqrt{n})}^2$$

(19)

and

$${\overline{\mathrm{\Psi }}}_{2,0}:=\frac{n\theta +2\tau +2\sqrt{2n\tau }}{2(1-\theta )},{\tilde{\mathrm{\Psi }}}_{2,0}:=\frac{{\psi }_2^{\mathrm{\prime }\mathrm{\prime }}(1)}{2(1-\theta )}{(\sqrt{2\tau }+\theta \sqrt{n})}^2.$$

(20)

We will use ${\overline{\mathrm{\Psi }}}_{j,0}$ and ${\tilde{\mathrm{\Psi }}}_{j,0}$ for the upper bounds of ${\mathrm{\Psi }}_j(v)$ from (14) for large- and small-update methods, respectively, $j\in \{1,2\}$.

Remark 3.9 For the large-update method with $\tau =\mathcal{O}(n)$ and $\theta =\mathrm{\Theta }(1)$, ${\overline{\mathrm{\Psi }}}_{j,0}=\mathcal{O}(n)$, $j\in \{1,2\}$, and for the small-update method with $\tau =\mathcal{O}(1)$ and $\theta =\mathrm{\Theta }(\frac{1}{\sqrt{n}})$, ${\tilde{\mathrm{\Psi }}}_{j,0}=\mathcal{O}({\psi }_j^{″}(1))$, $j\in \{1,2\}$.

For fixed μ, if we take a step size α, using (4) and (5), we have new iterations

$$x_{+}=x(\mathbf{e}+\alpha \frac{\mathrm{\Delta }x}{x})=x(\mathbf{e}+\alpha \frac{d_x}{v})=\frac{x}{v}(v+\alpha d_x)$$

and

$$s_{+}=s(\mathbf{e}+\alpha \frac{\mathrm{\Delta }s}{s})=s(\mathbf{e}+\alpha \frac{d_s}{v})=\frac{s}{v}(v+\alpha d_s).$$

For fixed $\mu >0$,

$$v_{+}:=\sqrt{\frac{x_{+}{s}_{+}}{\mu }}=\sqrt{(v+\alpha d_x)(v+\alpha d_s)}.$$

For notational convenience, let $\mathrm{\Psi }(v):={\mathrm{\Psi }}_j(v)$ and $\psi (t):={\psi }_j(t)$, $j\in \{1,2\}$.

For $\alpha >0$, we define

$$f(\alpha ):=\mathrm{\Psi }(v_{+})-\mathrm{\Psi }(v),$$

where $f(\alpha )$ is the difference of proximities between a new iteration and a current iteration for fixed μ. By the condition (a) in Lemma 3.1, we have

$$\mathrm{\Psi }(v_{+})=\mathrm{\Psi }\left(\sqrt{(v+\alpha d_x)(v+\alpha d_s)}\right)\le \frac{1}{2}(\mathrm{\Psi }(v+\alpha d_x)+\mathrm{\Psi }(v+\alpha d_s)).$$

Hence we have $f(\alpha )\le f_1(\alpha )$, where

$$f_1(\alpha ):=\frac{1}{2}(\mathrm{\Psi }(v+\alpha d_x)+\mathrm{\Psi }(v+\alpha d_s))-\mathrm{\Psi }(v).$$

Then, we have $f(0)=f_1(0)=0$. Differentiating $f_1(\alpha )$ with respect to α, we have

$$f_1^{\prime }(\alpha )=\frac{1}{2}\sum _{i=1}^n({\psi }^{\prime }(v_i+\alpha {[d_x]}_i){[d_x]}_i+{\psi }^{\prime }(v_i+\alpha {[d_s]}_i){[d_s]}_i),$$

where ${[d_x]}_i$ and ${[d_s]}_i$ denote the i th components of the vectors $d_x$ and $d_s$, respectively. Using (13) and (16), we have

$$f_1^{\prime }(0)=\frac{1}{2}\mathrm{\nabla }\mathrm{\Psi }{(v)}^T(d_x+d_s)=-\frac{1}{2}\mathrm{\nabla }\mathrm{\Psi }{(v)}^T\mathrm{\nabla }\mathrm{\Psi }(v)=-2{\delta }^2(v).$$

By taking the derivative of $f_1^{\prime }(\alpha )$ with respect to α, we have

$$f_1^{″}(\alpha )=\frac{1}{2}\sum _{i=1}^n({\psi }^{″}(v_i+\alpha {[d_x]}_i){[d_x]}_i^2+{\psi }^{″}(v_i+\alpha {[d_s]}_i){[d_s]}_i^2).$$

Since $f_1^{″}(\alpha )>0$, $f_1(\alpha )$ is strictly convex in α unless $d_x=d_s=0$. Since $\{M,N\}$ is a $P_{\ast }(\kappa )$-pair and $-M\mathrm{\Delta }x+N\mathrm{\Delta }s=0$ from (15), for $(\mathrm{\Delta }x;\mathrm{\Delta }s)\in {\mathbf{R}}^{2n}$,

$$(1+4\kappa )\sum _{i\in I_{+}}{[\mathrm{\Delta }x]}_i{[\mathrm{\Delta }s]}_i+\sum _{i\in I_{-}}{[\mathrm{\Delta }x]}_i{[\mathrm{\Delta }s]}_i\ge 0,$$

where $I_{+}=\{i\in I:{[\mathrm{\Delta }x]}_i{[\mathrm{\Delta }s]}_i\ge 0\}$, $I_{-}=I-I_{+}$. Since $d_x{d}_s=\frac{v^2\mathrm{\Delta }x\mathrm{\Delta }s}{xs}=\frac{\mathrm{\Delta }x\mathrm{\Delta }s}{\mu }$ and $\mu >0$, we have

$$(1+4\kappa )\sum _{i\in I_{+}}{[d_x]}_i{[d_s]}_i+\sum _{i\in I_{-}}{[d_x]}_i{[d_s]}_i\ge 0.$$

For notational convenience, we denote $\mathrm{\Psi }:={\mathrm{\Psi }}_j(v)$ and $\delta :={\delta }_j(v)$, $j\in \{1,2\}$.

In the following lemmas, we state same technical properties in [5].

Lemma 3.10 (Lemma 4.4 in [5])

$f_1^{\mathrm{\prime }}(\alpha )\le 0$ if α satisfies

$$-{\psi }^{\prime }(v_{min}-2\alpha \delta \sqrt{1+2\kappa })+{\psi }^{\mathrm{\prime }}(v_{min})\le \frac{2\delta }{\sqrt{1+2\kappa }}.$$

(21)

Lemma 3.11 (Lemma 4.5 in [5])

Let $\rho :={\rho }_j(\delta )$, $j\in \{1,2\}$, be defined as in (10). Then, in the worst case, the largest step size α satisfying (21) is given by

$$\overline{\alpha }:=\frac{1}{2\delta \sqrt{1+2\kappa }}(\rho (\delta )-\rho \left((1+\frac{1}{\sqrt{1+2\kappa }})\delta \right)).$$

Lemma 3.12 (Lemma 4.6 in [5])

Let ρ and $\overline{\alpha }$ be defined as in Lemma  3.11. Then

$$\overline{\alpha }\ge \frac{1}{(1+2\kappa ){\psi }^{″}(\rho ((1+\frac{1}{\sqrt{1+2\kappa }})\delta ))}.$$

Define

$$\tilde{\alpha }:=\frac{1}{(1+2\kappa ){\psi }^{″}(\rho ((1+\frac{1}{\sqrt{1+2\kappa }})\delta ))}.$$

(22)

Then we have $\tilde{\alpha }\le \overline{\alpha }$.

Lemma 3.13 Let $\tilde{\alpha }$ be defined as in (22). Then for $\kappa \ge 0$, we have

$$f(\tilde{\alpha })\le -\frac{{\delta }^2}{(1+2\kappa ){\psi }^{″}(\rho ((1+\frac{1}{\sqrt{1+2\kappa }})\delta ))}.$$

Lemma 3.14 (Lemma 4.10 in [5])

The right-hand sides in Lemma  3.13 are monotonically decreasing with respect to δ.

Lemma 3.15 (Proposition 1.3.2 in [17])

Let $t_0,t_1,\dots ,t_K$ be a sequence of positive numbers such that

$$t_{k+1}\le t_k-\lambda t_k^{1-\gamma },k=0,1,\dots ,K,$$

where $\lambda >0$ and $0<\gamma \le 1$. Then $K\le \lceil \frac{t_0^{\gamma }}{\lambda \gamma }\rceil$.

We define the value of $\mathrm{\Psi }(v)$ after the μ-update as ${\mathrm{\Psi }}_0$, and the subsequent values in the same outer iteration are denoted as ${\mathrm{\Psi }}_k$, $k=1,2,\dots$ . Then we have

$${\mathrm{\Psi }}_{K-1}>\tau ,0\le {\mathrm{\Psi }}_K\le \tau .$$

Theorem 3.16 Let a $P_{\ast }(\kappa )$-HLCP be given. If $\tau \ge 1$, then the upper bound of a total number of iterations is given by

$$\lceil \frac{{\mathrm{\Psi }}_0^{\gamma }}{\theta \lambda \gamma }log\frac{n{\mu }^0}{ϵ}\rceil .$$

Proof From Lemma 3.15 and Lemma II. 17 in [18], the number of inner and outer iterations is given by $\lceil \frac{{\mathrm{\Psi }}_0^{\gamma }}{\lambda \gamma }\rceil$ and $\lceil \frac{1}{\theta }log\frac{n{\mu }^0}{ϵ}\rceil$, respectively. For the total number of iterations, we multiply the number of inner iterations by that of outer iterations. Hence we have the desired results. This completes the proof. □

4 Application to new kernel functions

For the complexity analysis, we follow a similar framework in [3] for LO problems.

We apply the framework in Table 6 to the specific kernel function

$${\psi }_1(t)=\frac{e(t^2-1)}{2}+\frac{e^{p(g_1(t)-e)}-1}{pr},g_1(t)=e^{t^{-r}},p\ge 1,r\ge 1.$$

Table 6 Framework for analyzing the iteration bounds

Step 1: Using Lemma 3.3, ${\rho }_1(z)\ge {(log(e+p^{-1}log(e+2z)))}^{-\frac{1}{r}}$, $z\ge 0$.

Step 2: By Lemma 3.5, the inverse function of ${\psi }_1(t)$ for $t\ge 1$ satisfies

$${\varrho }_1(u)\le 1+\sqrt{\frac{2u}{e}},u\ge 0.$$

Step 3: Using Lemma 3.6, we obtain

$${\delta }_1(v)\ge \sqrt{\frac{e{\mathrm{\Psi }}_1(v)}{2}},v>0.$$

Step 4: Using (19) and ${\psi }_1^{″}(1)=e(pre+2r+2)$ from Table 2, we have the following:

• (i) For the large-update method, ${\mathrm{\Psi }}_0\le \frac{en\theta +2\tau +2\sqrt{2en\tau }}{2(1-\theta )}:={\overline{\mathrm{\Psi }}}_{1,0}$.

• (ii) For the small-update method, ${\mathrm{\Psi }}_0\le \frac{e(pre+2r+2){(\sqrt{\frac{2\tau }{e}}+\theta \sqrt{n})}^2}{2(1-\theta )}:={\tilde{\mathrm{\Psi }}}_{1,0}$.

Step 5: Define $L_1({\mathrm{\Psi }}_1,p):=e+p^{-1}log(e+2\sqrt{2e{\mathrm{\Psi }}_1})$. Using ${\psi }_1^{(3)}(t)<0$, Step 1, $1+\frac{1}{\sqrt{1+2\kappa }}\le 2$, and Table 2, we have

(23)

where the last inequality follows from the assumption ${\mathrm{\Psi }}_1\ge \tau \ge 1$. Using Lemma 3.13, Lemma 3.14, Lemma 3.6, and (23), we have

$$\begin{array}{rcl}f(\tilde{\alpha })& \le & -\frac{{\delta }^2}{(1+2\kappa ){\psi }_1^{″}({\rho }_1((1+\frac{1}{\sqrt{1+2\kappa }})\delta ))}\\ \le & -\frac{\frac{e{\mathrm{\Psi }}_1}{2}}{(1+2\kappa ){\psi }_1^{″}({\rho }_1((1+\frac{1}{\sqrt{1+2\kappa }})\sqrt{\frac{e{\mathrm{\Psi }}_1}{2}}))}\\ \le & -\frac{\frac{e{\mathrm{\Psi }}_1}{2}}{4(1+2\kappa )e\sqrt{{\mathrm{\Psi }}_1}{L}_1({\mathrm{\Psi }}_1,p){(log{L}_1({\mathrm{\Psi }}_1,p))}^{\frac{2(r+1)}{r}}(pr{L}_1({\mathrm{\Psi }}_1,p)+2r+1)}\\ =& -\frac{\sqrt{{\mathrm{\Psi }}_1}}{8(1+2\kappa )L_1({\mathrm{\Psi }}_1,p){(log{L}_1({\mathrm{\Psi }}_1,p))}^{\frac{2(r+1)}{r}}(pr{L}_1({\mathrm{\Psi }}_1,p)+2r+1)}\\ \le & -\frac{\sqrt{{\mathrm{\Psi }}_1}}{8(1+2\kappa )L_1({\mathrm{\Psi }}_{1,0},p){(log{L}_1({\mathrm{\Psi }}_{1,0},p))}^{\frac{2(r+1)}{r}}(pr{L}_1({\mathrm{\Psi }}_{1,0},p)+2r+1)},\end{array}$$

where the last inequality follows from $L_1({\mathrm{\Psi }}_{1,0},p):=e+p^{-1}log(e+2\sqrt{2e{\mathrm{\Psi }}_{1,0}})$ and the assumption ${\mathrm{\Psi }}_{1,0}\ge {\mathrm{\Psi }}_1$.

Step 6: Using Theorem 3.16, Step 4 with ${\mathrm{\Psi }}_{1,0}\le {\overline{\mathrm{\Psi }}}_{1,0}$, and ${\mathrm{\Psi }}_{1,0}\le {\tilde{\mathrm{\Psi }}}_{1,0}$, and Step 5 with $\gamma =\frac{1}{2}$ and $\frac{1}{\lambda }=8(1+2\kappa )L_1({\mathrm{\Psi }}_{1,0},p){(log{L}_1({\mathrm{\Psi }}_{1,0},p))}^{\frac{2(r+1)}{r}}(pr{L}_1({\mathrm{\Psi }}_{1,0},p)+2r+1)$, we have the upper bounds of the total number of iterations for large- and small-update methods as follows.

• (i) For large-update methods,

  $$\lceil 8(1+2\kappa )L_1({\overline{\mathrm{\Psi }}}_{1,0},p){(log{L}_1({\overline{\mathrm{\Psi }}}_{1,0},p))}^{\frac{2(r+1)}{r}}(pr{L}_1({\overline{\mathrm{\Psi }}}_{1,0},p)+2r+1){\overline{\mathrm{\Psi }}}_{1,0}^{\frac{1}{2}}\frac{1}{\theta }log\frac{n{\mu }^0}{ϵ}\rceil ,$$

  where $L_1({\overline{\mathrm{\Psi }}}_{1,0},p):=e+p^{-1}log(e+2\sqrt{2e{\overline{\mathrm{\Psi }}}_{1,0}})$.

• (ii) For small-update methods,

  $$\lceil 8(1+2\kappa )L_1({\tilde{\mathrm{\Psi }}}_{1,0},p){(log{L}_1({\tilde{\mathrm{\Psi }}}_{1,0},p))}^{\frac{2(r+1)}{r}}(pr{L}_1({\tilde{\mathrm{\Psi }}}_{1,0},p)+2r+1){\tilde{\mathrm{\Psi }}}_{1,0}^{\frac{1}{2}}\frac{1}{\theta }log\frac{n{\mu }^0}{ϵ}\rceil ,$$

where $L_1({\tilde{\mathrm{\Psi }}}_{1,0},p):=e+p^{-1}log(e+2\sqrt{2e{\tilde{\mathrm{\Psi }}}_{1,0}})$.

Step 7: Using Step 6 and Remark 3.9, for the large-update method with $p=log(e+2\sqrt{2e{\overline{\mathrm{\Psi }}}_{1,0}})=\mathcal{O}(logn)$ and $r=1$, the algorithm has $\mathcal{O}((1+2\kappa )\sqrt{n}lognlog\frac{n{\mu }^0}{ϵ})$ complexity. For the small-update method with $p=1$ and $r=1$, the algorithm has $\mathcal{O}((1+2\kappa )\sqrt{n}log\frac{n{\mu }^0}{ϵ})$ complexity. These are currently the best known complexity results.

Remark 4.1 For the kernel function ${\psi }_2(t)$ in Table 1, by applying the framework, the algorithms have $\lceil 8(1+2\kappa )L_2({\overline{\mathrm{\Psi }}}_{2,0},p){(log{L}_2({\overline{\mathrm{\Psi }}}_{2,0},p))}^{\frac{2(r+1)}{r}}(pr{L}_2({\overline{\mathrm{\Psi }}}_{2,0},p)+2r+1){\overline{\mathrm{\Psi }}}_{2,0}^{\frac{1}{2}}\frac{1}{\theta }log\frac{n{\mu }^0}{ϵ}\rceil$ and $\lceil 8(1+2\kappa )L_2({\tilde{\mathrm{\Psi }}}_{2,0},p){(log{L}_2({\tilde{\mathrm{\Psi }}}_{2,0},p))}^{\frac{2(r+1)}{r}}(pr{L}_2({\tilde{\mathrm{\Psi }}}_{2,0},p)+2r+1){\tilde{\mathrm{\Psi }}}_{2,0}^{\frac{1}{2}}\frac{1}{\theta }log\frac{n{\mu }^0}{ϵ}\rceil$ iteration bounds for large- and small-update methods, respectively, where $L_2({\overline{\mathrm{\Psi }}}_{2,0},p):=1+p^{-1}log(1+2\sqrt{2{\overline{\mathrm{\Psi }}}_{2,0}})$ and $L_2({\tilde{\mathrm{\Psi }}}_{2,0},p):=1+p^{-1}log(1+2\sqrt{2{\tilde{\mathrm{\Psi }}}_{1,0}})$. By taking $p=log(1+2\sqrt{2{\overline{\mathrm{\Psi }}}_{2,0}})=\mathcal{O}(logn)$ and $r=1$, the algorithm has $\mathcal{O}((1+2\kappa )\sqrt{n}lognlog\frac{n{\mu }^0}{ϵ})$ complexity for large-update methods. Choosing $p=1$ and $r=1$, the algorithm has $\mathcal{O}((1+2\kappa )\sqrt{n}log\frac{n{\mu }^0}{ϵ})$ for small-update methods. In conclusion, we obtain so far the best known iteration bounds of large- and small-update methods for kernel functions ${\psi }_j$, $j\in \{1,2\}$, in Table 1.