Skip to main content

A semi-smoothing augmented Lagrange multiplier algorithm for low-rank Toeplitz matrix completion


The smoothing augmented Lagrange multiplier (SALM) algorithm is a generalization of the augmented Lagrange multiplier algorithm for completing a Toeplitz matrix, which saves computational cost of the singular value decomposition (SVD) and approximates well the solution. However, the communication of numerous data is computationally demanding at each iteration step. In this paper, we propose an accelerated scheme to the SALM algorithm for the Toeplitz matrix completion (TMC), which will reduce the extra load coming from data communication under reasonable smoothing. It has resulted in a semi-smoothing augmented Lagrange multiplier (SSALM) algorithm. Meanwhile, we demonstrate the convergence theory of the new algorithm. Finally, numerical experiments show that the new algorithm is more effective/economic than the original algorithm.


Completing a low-rank matrix from a subset of its entries has been a hot problem recently, first introduced by [8], that has arisen in a wide variety of practical contexts across all disciplines of engineering and computational science such as model reduction [19], machine learning [1, 2], control [22], pattern recognition [12], imaging inpainting [3], video denoising [16], computer vision [28], and so on. Despite matrix completion (MC) requiring the global solution of a non-convex objective, there are many computational efficient algorithms which are effective for a broad class of matrices. The problem has received intensive research from both theoretical and algorithmic aspects, see, e.g., [4,5,6,7,8,9,10,11, 13,14,15,16,17,18, 21, 23, 26, 27, 29,30,31,32, 34, 35], and the references therein for partial review. It is well known that the mathematical model of the MC problem is of the following form:

$$ \begin{aligned} &\min_{A\in \mathbb{R}^{m\times n}} \Vert A \Vert _{\ast }, \\ &\text{subject to } \mathcal{P}_{\varOmega }(A)=\mathcal{P}_{\varOmega }(M), \end{aligned} $$

where the matrix \(M\in \mathbb{R}^{m\times n}\) is an underlying matrix to be completed, Ω is a random subset of indices for the available entries, and \(\mathcal{P}_{\varOmega }\) is the associated sampling orthogonal projection operator which acquires only the entries indexed by \(\varOmega \subset \{1,2,\ldots ,m\}\times \{1,2,\ldots ,n\}\).

In the current MC problems, the matrix M is of special structure in general. Therefore, much attention has been paid to the completion of Toeplitz and Hankel matrices in recent years [20, 24, 25, 30, 32]. Many scholars have conducted in-depth research on the special structure, property, and application of the Toeplitz and Hankel matrices; for example, nuclear norm minimization for the low-rank Hankel matrix reconstruction problem under the random Gaussian sampling model is investigated in [7]. In addition, Hankel matrix reconstitution in the sense of minimizing the nuclear norm with non-uniform sampling of entries is researched in [11]. To make full use of the special structure of a Toeplitz matrix, a mean value algorithm is presented in [30]; the modified Lagrange multiplier (MALM) algorithm [31] and the smoothing augmented Lagrange multiplier (SALM) algorithm [34] are also proposed. Therefore, the Toeplitz matrix completion (TMC) is one of the most important MC problems and has attracted a large amount of attention recently. As is well known, an \(n\times n\) Toeplitz matrix is of the following form:

T= ( t j i ) i , j = 1 n = ( t 0 t 1 t n 2 t n 1 t 1 t 0 t n 3 t n 2 t n + 2 t n + 3 t 0 t 1 t n + 1 t n + 2 t 1 t 0 ) T n × n R n × n ,

which is determined by \(2n-1\) entries, say, the first row and the first column. Explicitly seeking the lowest rank Toeplitz matrix consistent with the known entries is mathematically considered as

$$ \begin{aligned}& \min_{A\in \mathbb{T}^{n\times n}} \Vert A \Vert _{\ast }, \\ &\text{subject to } H\circ A=M, \end{aligned}$$

where “” is the Hadamard product, \(H=(H_{ij})\in \mathbb{R} ^{n\times n}\) is the weighted matrix with entries \(H_{ij}=1\) for \(j-i\in \varOmega \subset \{-n+1,\ldots ,n-1\}\) and \(H_{ij}=0\) for any other \((i,j)\), \(M=(M_{ij})\in \mathbb{T}^{n\times n}\) is the underlying Toeplitz matrix to be completed, namely \(M_{ij}=0\) for \(j-i\notin \varOmega \).

The SALM algorithm switches the iteration matrix into the Toeplitz structure at each iteration step by the smoothing operator, which saves computational cost of the singular value decomposition and approximates well the solution. Unfortunately, numerous data have to be shifted at each iteration step in the process of implementing this algorithm. However, there is a cost, sometimes relatively great, associated with the moving of data. The control of memory traffic is crucial to performance in many computers.

These factors motivated us to reduce the traffic jam of data, resulting in a semi-smoothing augmented Lagrange multiplier (SSALM) algorithm based on the selecting technique of the optimal parameter \(\omega ^{(k)}\) at each of the five iteration steps in [33]. Compared with the SALM algorithm, the new algorithm either saves computation cost or reduces data communication. Two aspects are taken into account, which results in more practical or economic implementation. The new algorithm not only overcomes the slowness of the SVD for the original ALM algorithm, but also reduces the greatness of the data communication for the ALM algorithm. We can see that the CPU of SSALM algorithm is reduced to 30.44% from the numerical experiments.

The rest of this paper is organized as follows. Some preliminaries are provided in Sect. 2. Section 3 presents the semi-smoothing augmented Lagrange multiplier (SSALM) algorithm after giving an outline of the ALM algorithm, the dual approach, and the SALM algorithm. The convergence property of the SSALM algorithm is constructed in Sect. 4. We report the numerical results to indicate the effectiveness of the SSALM algorithm in Sect. 5. Finally, we end the paper with the concluding remarks in Sect. 6.


This section is devoted to some of the necessary notations and preliminaries. \(\mathbb{R}^{m\times n}\) denotes the set of \(m\times n\) real matrices, \(\mathbb{T}^{n\times n}\) is the set of \(n\times n\) real Toeplitz matrices. The nuclear norm of a matrix A is denoted by \(\|A\|_{\ast }\), and the Frobenius norm \(\|A\|_{F}\) is the maximum absolute value of the matrix entries of a matrix A. \(A^{T}\) is used to express the transpose of a matrix \(A\in \mathbb{R}^{n\times n}\), \(\operatorname{rank}(A)\) is equal to the rank of a matrix A, and \(\operatorname{tr}(A)\) represents the trace of A. The standard inner product between two matrices is denoted by \(\langle X,Y\rangle = \operatorname{tr}(X^{T}Y)\). For \(A=(a_{ij}) \in \mathbb{R}^{m\times n}\), \(B=(b_{ij})\in \mathbb{R}^{m\times n}\), their Hadamard product \(A\circ B\) is an \(m\times n\) matrix whose \((i,j)\) entry is the \(a_{ij}b_{ij}\), i.e., \(A\circ B=(a_{ij}b_{ij})\in \mathbb{R} ^{m\times n}\).

The singular value decomposition (SVD) of a matrix \(A\in \mathbb{R} ^{m\times n}\) of r-rank is defined by

$$ A=U\varSigma _{r} V^{T}, \qquad \varSigma _{r}= \operatorname{diag}(\sigma _{1}, \sigma _{2}, \ldots , \sigma _{r}), $$

where \(U\in \mathbb{R}^{m\times r}\) and \(V \in \mathbb{R}^{n\times r}\) are column orthonormal matrices, and \(\sigma _{1} \geq \sigma _{2} \geq \cdots \geq \sigma _{r}>0\).

Definition 2.1

(Singular value thresholding operator [6])

For each \(\tau \geq 0\), the singular value thresholding operator \(\mathcal{D}_{\tau }\) is defined as follows:

$$ \mathcal{D}_{\tau }(A):=U\mathcal{D}_{\tau }(\varSigma )V^{T},\qquad \mathcal{D}_{\tau }(\varSigma )={\operatorname{diag}}\bigl(\{ \sigma _{i}-\tau \}_{+}\bigr), $$


$$\begin{aligned}& A=U\varSigma _{r} V^{T}\in \mathbb{R}^{m\times n}, \qquad \{\sigma _{i}-\tau \}_{+}=\textstyle\begin{cases} \sigma _{i}-\tau , & \text{if } \sigma _{i}>\tau, \\ 0, & \text{if } \sigma _{i}\leq \tau. \end{cases}\displaystyle \end{aligned}$$

\(I_{n}=(e_{1},e_{2},\ldots ,e_{n})\in \mathbb{R}^{n\times n}\) denotes the \(n\times n\) identity matrix and \(Z_{n}=(e_{2},e_{3},\ldots ,e_{n},0) \in \mathbb{R}^{n\times n}\) is called the shift matrix. It is clear that

Z n r = { ( O O I n r O ) , 1 < r < n , O , r n ,

where “O” stands for a zero-matrix. Thus, a Toeplitz matrix \(T \in \mathbb{T} ^{n\times n}\), shown in (1.2), can be written as a linear combination of these shift matrices, that is,

$$ T=\sum_{l=1}^{n-1}t_{-l}Z_{n}^{l}+ \sum_{l=0}^{n-1}t_{l}\bigl(Z _{n}^{T}\bigr)^{l}. $$

\(\varOmega \subset \{-n+1,\ldots ,n-1\}\) is an indices set of observed diagonals of a Toeplitz matrix \(M\in \mathbb{T}^{n\times n}\), Ω̄ is the complementary set of Ω. For any Toeplitz matrix \(A\in \mathbb{T}^{n\times n}\), the vector vec\((A,\alpha )\) denotes the αth diagonal of A, \(\alpha =-n+1, -n+2,\ldots ,n-1\), that is to say,

$$\operatorname{vec}(H\circ A,\alpha )=\textstyle\begin{cases} \operatorname{vec}(A,\alpha ) ,& \alpha \in \varOmega, \\ \mathbf{0} ,& \alpha \notin \varOmega, \end{cases}\displaystyle \quad (\mathbf{0}\mbox{ is a zero-vector}). $$

Definition 2.2

(Toeplitz structure smoothing operator [34])

For any matrix \(A=(a_{ij})\in \mathbb{R}^{n\times n}\), the Toeplitz structure smoothing operator \(\mathcal{T}\) is defined as follows:

$$ \mathcal{T}(A):=\sum_{l=1}^{n-1} \tilde{a}_{-l}Z_{n}^{l}+ \sum _{l=0}^{n-1}\tilde{a}_{l} \bigl(Z_{n}^{T}\bigr)^{l}, $$

where \(\tilde{a}_{\alpha }=\frac{_{\alpha }A^{\min }+_{\alpha }A^{ \max }}{2}\), \(\alpha =-n+1,-n+2,\ldots , n-1\) with

$$ _{\alpha }A^{\min }=\min_{i,j\in \{1,2,\ldots ,n\}} \{a_{ij},i-j= \alpha \}, \quad \text{and}\quad {} _{\alpha }A^{\max }= \max_{i,j\in \{1,2,\ldots ,n\}}\{a_{ij},i-j=\alpha \}. $$

It is clear that \(\mathcal{T}(A)\) is a Toeplitz matrix derived from the matrix A. Namely, any \(A\in \mathbb{R}^{n\times n}\) can be changed into a Toeplitz structure via the smoothing operator \(\mathcal{T}( \cdot )\).


First of all in this section, for completeness as well as for the purpose of comparison, we briefly review and summarize some relative algorithms for approximately minimizing the nuclear norm of a matrix under convex constraints.

Since the matrix completion problem is closely connected to the robust principal component analysis (RPCA) problem, then it can be formulated in the same way as RPCA, an equivalent problem of (1.1) can be considered as follows.

As E will compensate for the unknown entries of M, the unknown entries of M are simply set as zeros. Suppose that the given data are arranged as the columns of a large matrix \(M\in \mathbb{R}^{m\times n}\). The mathematical model for estimating the low-dimensional subspace is to find a low-rank matrix \(A\in \mathbb{R}^{m\times n}\) such that the discrepancy between A and M is minimized, leading to the following constrained optimization:

$$ \begin{aligned} &\min_{A,E\in \mathbb{R}^{m\times n}} \Vert A \Vert _{\ast }, \\ &\text{subject to } A+E=M, \mathcal{P}_{\varOmega }(E)=0, \end{aligned} $$

where E will compensate for the unknown entries of M, the unknown entries of \(M\in \mathbb{R}^{m\times n}\) are simply set as zeros. And \(\mathcal{P}_{\varOmega }: \mathbb{R}^{m\times n}\rightarrow \mathbb{R} ^{m\times n}\) is a linear operator that keeps the entries in Ω unchanged and sets those outside Ω (say, in Ω̅) zeros.

The dual algorithm

The dual algorithm proposed in [10] tackles problem (3.1) via its dual. That is, one first solves the dual problem

$$ \begin{aligned} &\max_{Y} \langle M,Y \rangle , \\ &\text{subject to } J(Y)\leq 1, \end{aligned} $$

for the optimal Lagrange multiplier Y, where

$$ J(Y)=\max \bigl( \Vert Y \Vert _{2},\lambda ^{-1} \Vert Y \Vert _{\infty }\bigr). $$

A steepest ascend algorithm constrained on the surface \(\{Y|J(Y)=1\}\) can be adopted to solve (3.2), where the constrained steepest ascend direction is obtained by projecting M onto the tangent cone of the convex body \(\{Y|J(Y)\leq 1\}\). It turns out that the optimal solution to the primal problem (3.1) can be obtained during the process of finding the constrained steepest ascend direction.

The augmented Lagrange multiplier algorithm

The augmented Lagrange multiplier (ALM) algorithm was proposed in [18] for solving a convex optimization (1.1). It should be described subsequently.

It is famous that the partial augmented Lagrangian function of problem (3.1) is

$$ \mathcal{L}(A,E,Y,\mu )= \Vert A \Vert _{\ast }+\langle Y,M-A-E \rangle +\frac{ \mu }{2} \Vert M-A-E \Vert _{F}^{2}. $$

Hence, the augmented Lagrange multiplier algorithm is designed as follows.

Algorithm 3.1


Given a sampled set Ω, a sampled matrix \(D=\mathcal{P}_{\varOmega }(M)\), \(\mu _{0}>0\), \(\rho >1\). Given also two initial matrices \(Y_{0}=0\), \(E_{0}=0\). \(k:=0\).

  1. 1.

    Compute the SVD of the matrix \((D-E_{k}+\mu _{k}^{-1}Y_{k})\):

    $$ [U_{k},\varSigma _{k},V_{k}]=\operatorname{svd}\bigl(D-E_{k}+\mu _{k}^{-1}Y_{k}\bigr); $$
  2. 2.


    $$ A_{k+1}=U_{k}\mathcal{D}_{\mu _{k}^{-1}}(\varSigma _{k})V_{k}^{T}, $$

    solve \(E_{k+1}=\arg \min_{\mathcal{P}_{\varOmega }(E)=0} \mathcal{L}(A_{k+1},E,Y_{k},\mu _{k})\),

    $$ E_{k+1}=\mathcal{P}_{\bar{\varOmega }}\bigl(D-A_{k+1}+\mu _{k}^{-1}Y_{k}\bigr); $$
  3. 3.

    If \(\|D-A_{k+1}-E_{k+1}\|_{F}/\|D\|_{F}<\epsilon _{1}\) and \(\mu _{k}\|E_{k+1}-E_{k}\|_{F}/\|D\|_{F}<\epsilon _{2}\), stop; otherwise, go to the next step;

  4. 4.

    Set \(Y_{k+1}=Y_{k}+\mu _{k}(D-A_{k+1}-E_{k+1})\).

If \(\mu _{k}\|E_{k+1}-E_{k}\|_{F}/\|D\|_{F}<\epsilon _{2}\), set \(\mu _{k+1}=\rho \mu _{k}\); otherwise, go to Step 1.

Note that the ALM algorithm performs better both in theory and practice than other algorithms with a Q-linear convergence speed globally. It is of much better numerical behavior or higher accuracy. It is reported that the ALM algorithm has been applied to MC problem (1.1). The algorithm, however, has the disadvantage of the penalty function, namely the matrix sequences \(\{A_{k}\}\) generated by the ALM algorithm are the accepted solutions but feasible ones.

The smoothing augmented Lagrange multiplier algorithm

In this subsection, we make mention of a stepped-up scheme for the TMC problem. The smoothing augmented Lagrange multiplier (SALM) approach employs the smoothing operator \(\mathcal{T}\) (see (2.1)) to approximate a matrix generated in the kth iteration so that the current approximation is of a Toeplitz structure.

Then our problem can be expressed as the following convex programming:

$$ \begin{aligned} &\min_{A,E\in \mathbb{T}^{n\times n}} \Vert A \Vert _{\ast }, \\ & \text{subject to}\quad A+E=\mathcal{P}_{\varOmega }(M), \qquad \mathcal{P}_{\varOmega }(E)=0, \end{aligned} $$

where \(M\in \mathbb{T}^{n\times n}\) is a real Toeplitz matrix, and \(\varOmega \subset \{-n+1,\ldots ,n-1\}\).

Let \(D=\mathcal{P}_{\varOmega }(M)\). Then the partial augmented Lagrangian function is

$$ \mathcal{L}(A,E,Y,\mu )= \Vert A \Vert _{\ast }+ \langle Y,D-A-E\rangle +\frac{ \mu }{2} \Vert D-A-E \Vert _{F}^{2}, $$

where \(Y\in \mathbb{R}^{n\times n}\).

Algorithm 3.2


Given a sampled set Ω, a sampled matrix D, \(\mu _{0}>0\), \(\rho >1\). Given also two initial matrices \(Y_{0}=0\), \(E_{0}=0\). \(k:=0\).

  1. 1.

    Compute the SVD of the matrix \((D-E_{k}+\mu _{k}^{-1}Y_{k})\) using the Lanczos method

    $$ [U_{k},\varSigma _{k},V_{k}]=\operatorname{lansvd}\bigl(D-E_{k}+\mu _{k}^{-1}Y_{k}\bigr); $$
  2. 2.


    $$ X_{k+1}=U_{k}\mathcal{D}_{\mu _{k}^{-1}}(\varSigma _{k})V_{k}^{T}, $$

    compute for smoothing \(\tilde{a}_{\alpha }=\frac{_{\alpha }X_{k+1} ^{\min }+_{\alpha }X_{k+1}^{\max }}{2}\), \(\alpha \in \{-n+1,-n+2, \ldots ,n-1\}\), and

    $$\begin{aligned}& A_{k+1}= \mathcal{T}(X_{k+1})=\sum _{l=1}^{n-1}\tilde{a}_{-l}Z _{n}^{l}+\sum_{l=0}^{n-1} \tilde{a}_{l}\bigl(Z_{n}^{T}\bigr)^{l}, \\& E_{k+1}=\mathcal{P}_{\bar{\varOmega }}\bigl(D-A_{k+1}+\mu _{k}^{-1}Y_{k}\bigr); \end{aligned}$$
  3. 3.

    If \(\|D-A_{k+1}-E_{k+1}\|_{F}/\|D\|_{F}<\epsilon _{1}\) and \(\mu _{k}\|E_{k+1}-E_{k}\|_{F}/\|D\|_{F}<\epsilon _{2}\), stop; otherwise, go to the next step;

  4. 4.

    Set \(Y_{k+1}=Y_{k}+\mu _{k}(D-A_{k+1}-E_{k+1})\).

If \(\mu _{k}\|E_{k+1}-E_{k}\|_{F}/\|D\|_{F}<\epsilon _{2}\), set \(\mu _{k+1}=\rho \mu _{k}\); otherwise, go to Step 1.

It is reported that the convergence speed of the SALM algorithm is greater than that of the ALM and APG algorithms. A merit of smoothing is that the fast SVD procedure can be utilized to reduce the computation.

As we know, the SVD time is saved at the expense of data communication. Sometimes, this is not worth the candle. This motivated us to come up with the following algorithm.

The semi-smoothing augmented Lagrange multiplier algorithm

In this subsection, we propose a semi-smoothing augmented Lagrange multiplier algorithm based on the ALM and SALM algorithms for the TMC problem. The new algorithm consists of two stages: one is \(\ell -1\) iterations by the ALM scheme, which is free moving of data; another is the th smoothing by the SALM procedure, which is keeping the iteration matrix as a Toeplitz structure.

Now, the semi-smoothing augmented Lagrange multiplier (SSALM) algorithm will be presented in the following.

Algorithm 3.3

(SSALM algorithm)

Input: A sampled set Ω, a sampled matrix D, \(Y_{0,0}=0\), \(E_{0,0}=0\); parameters \(\mu _{0}>0\), \(\rho >1\), \(\ell >1\), \(\epsilon _{1}\), \(\epsilon _{2}\).

Let \(k:=0\), \(q:=1\), \(q=1,2,\ldots ,\ell -1\).


  1. 1.

    \(\ell -1\) iterations.

    (1) :

    Compute the SVD of the matrix \((D-E_{k,q}+\mu _{k,q} ^{-1}Y_{k,q})\):

    $$ [U_{k,q},\varSigma _{k,q},V_{k,q}]=\operatorname{lansvd}\bigl(D-E_{k,q}+\mu _{k,q} ^{-1}Y_{k,q} \bigr); $$
    (2) :


    $$\begin{aligned}& X_{k+1,q+1}=U_{k,q}\mathcal{D}_{\mu _{k,q}^{-1}}(\varSigma _{k,q})V_{k,q} ^{T}, \\& E_{k+1,q+1}=\mathcal{P}_{\bar{\varOmega }}\bigl(D-X_{k+1,q+1}+\mu _{k,q}^{-1}Y _{k,q}\bigr); \end{aligned}$$
    (3) :

    If \(\|D-X_{k+1,q+1}-E_{k+1,q+1}\|_{F}/\|D\|_{F}<\epsilon _{1}\) and \(\mu _{k,q}\|E_{k+1,q+1}-E_{k,q}\|_{F}/\|D\|_{F}<\epsilon _{2}\), stop; otherwise, go to the next step;

    (4) :

    Set \(Y_{k+1,q+1}=Y_{k,q}+\mu _{k,q}(D-X_{k+1,q+1}-E_{k+1,q+1})\), \(\mu _{k+1,q+1}=\rho \mu _{k,q}\); otherwise, go to step (1);

  2. 2.

    th smoothing.

    (1) :

    Compute the SVD of the matrix \((D-E_{k,\ell }+\mu _{k, \ell }^{-1}Y_{k,\ell })\):

    $$ [U_{k,\ell },\varSigma _{k,\ell },V_{k,\ell }]=\operatorname{svd}\bigl(D-E_{k, \ell }+\mu _{k,\ell }^{-1}Y_{k,\ell }\bigr); $$
    (2) :


    $$ X_{k+1,\ell }=U_{k,\ell }\mathcal{D}_{\mu _{k,\ell }^{-1}}(\varSigma _{k, \ell })V_{k,\ell }^{T}, $$

    compute the factors of smoothing \(\tilde{a}_{\alpha }=\frac{_{\alpha }X_{k+1,\ell }^{\min }+_{\alpha }X_{k+1,\ell }^{\max }}{2}\), \(\alpha =-n+1,-n+2,\ldots , n-1\),

    and set

    $$ A_{k+1,\ell }= \mathcal{T}(X_{k+1,\ell })=\sum _{l=1}^{n-1} \tilde{a}_{-l}Z_{n}^{l}+ \sum_{l=0}^{n-1}\tilde{a}_{l} \bigl(Z_{n} ^{T}\bigr)^{l}. $$


    $$ E_{k+1,\ell }=\mathcal{P}_{\bar{\varOmega }}\bigl(D-A_{k+1,\ell }+\mu _{k, \ell }^{-1}Y_{k+1,\ell }\bigr); $$
  3. 3.

    If \(\|D-A_{k+1,\ell }-E_{k+1,\ell }\|_{F}/\|D\|_{F}<\epsilon _{1}\) and \(\mu _{k,\ell }\|E_{k+1,\ell }-E_{k,\ell }\|_{F}/\|D\|_{F}< \epsilon _{2}\), stop; otherwise, go to the next step;

  4. 4.

    Set \(Y_{k+1,q+1}=Y_{k,q}+\mu _{k,q}(D-A_{k+1,q+1}-E_{k+1,q+1})\).

If \(\mu _{k,q}\|E_{k+1,q+1}-E_{k,q}\|_{F}/\|D\|_{F}< \epsilon _{2}\), set \(\mu _{k+1,q+1}=\rho \mu _{k,q}\); otherwise, go to Step 1.

In fact, Algorithm 3.3 includes the above Algorithm 3.2 as a special case if \(\ell =1\). Obviously, the algorithm is an acceleration of the SALM algorithm in [34].

Convergence analysis

This section will analyze the convergence of Algorithm 3.3.

Let \((\ddot{A},\ddot{E})\) be the solution of model (3.5) and Ÿ be that of the optimal problem (3.2). We provide some lemmas in the following firstly.

Lemma 4.1


Let \(A\in \mathbb{R}^{m\times n}\) be an arbitrary matrix and \(U\varSigma V^{T}\) be its SVD. Then the set of subgradients of the nuclear norm of A is given by

$$ \partial \Vert A \Vert _{*}=\bigl\{ UV^{T}+W:W\in \mathbb{R}^{m\times n}, U^{T}W=0,WV=0, \Vert W \Vert _{2}\leq 1\bigr\} . $$

Lemma 4.2


If \(\mu _{k}\) is nondecreasing, then each entry of the following series is nonnegative and their sum is finite, i.e.,

$$\begin{aligned}& \sum_{k=1}^{+\infty }{\mu _{k}}^{-1} \bigl(\langle Y_{k+1}-Y_{k},E _{k+1}-E_{k} \rangle +\langle A_{k+1}-\ddot{A},\hat{Y}_{k+1}-\ddot{Y} \rangle +\langle E_{k+1}-\ddot{E},Y_{k+1}-\ddot{Y}\rangle \bigr) \\& \quad < +\infty . \end{aligned}$$

Lemma 4.3


The sequences \(\{\ddot{Y}_{k}\}\), \(\{Y_{k}\}\), and \(\{\hat{Y}_{k}\}\) are all bounded, where \(\hat{Y} _{k}=Y_{k+1}+\mu _{k-1}(D-A_{k}-E_{k-1})\).

Lemma 4.4

The sequence \(\{Y_{k,q}\}\) generated by Algorithm 3.3 is bounded.


Let \(B=\mu _{k,q}(D-E_{k,q}+\mu _{k,q}^{-1}Y_{k,q}-X _{k+1,q+1})\), \(\mathcal{T}(B)=\sum_{l=1}^{n-1}\tilde{b}_{-l}Z _{n}^{l}+\sum_{l=0}^{n-1}\tilde{b}_{l}(Z_{n}^{T})^{l}\), defined as (2.1).

First of all, we indicate that \(Y_{k,q}\), \(E_{k,q}\), \(k=1,2,\ldots \) , \(q=1,2, \ldots ,\ell -1\), are all the Toeplitz matrices. Clearly, \(Y_{0,0}=0\), \(E _{0,0}=0\) are both smoothed into a Toeplitz structure. Suppose that \(Y_{k,q}\), \(E_{k,q}\) are both the Toeplitz matrices, so is \(E_{k+1,q+1}= \mathcal{P}_{\bar{\varOmega }}(D-X_{k+1,q+1}+\mu _{k,q}^{-1}Y_{k,q})\). Thus, \(Y_{k+1,q+1}\) is a Toeplitz matrix also because of Step 4 in Algorithm 3.3.

$$\begin{aligned} Y_{k+1,q+1} =& Y_{k,q}+\mu _{k,q}(D-A_{k+1,q+1}-E_{k+1,q+1}) \\ =& Y_{k,q}+ \mu _{k,q}(D-A_{k+1,q+1}-E_{k,q})+ \mu _{k,q}(E_{k,q}-E_{k+1,q+1}). \end{aligned}$$


$$\begin{aligned} \mu _{k,q}(E_{k,q}-E_{k+1,q+1}) =& \mu _{k,q} \mathcal{P}_{\bar{\varOmega }}\bigl(E_{k,q}-\bigl(D-A_{k+1,q+1}+\mu _{k,q}^{-1}Y_{k,q}\bigr)\bigr) \\ =& \mu _{k,q}\mathcal{P}_{\bar{\varOmega }}\bigl(E_{k,q}-\bigl(D- \mathcal{T}(X_{k+1,q+1})+ \mu _{k,q}^{-1}Y_{k,q} \bigr)\bigr) \\ =& \mu _{k,q}\mathcal{P}_{\bar{\varOmega }}\bigl(E_{k,q}-\bigl(D- \mathcal{T}\bigl(D-E _{k,q}+\mu _{k,q}^{-1}Y_{k,q}- \mu _{k,q}^{-1}B\bigr)+\mu _{k,q}^{-1}Y_{k,q} \bigr)\bigr) \\ =& \mu _{k,q}\mathcal{P}_{\bar{\varOmega }}\bigl(E_{k,q}-\bigl(D- \bigl(D-E_{k,q}+\mu _{k,q}^{-1}Y_{k,q}-\mu _{k,q}^{-1}\mathcal{T}(B)\bigr)+\mu _{k,q}^{-1}Y_{k,q} \bigr)\bigr) \\ =& \mathcal{P}_{\bar{\varOmega }}\mathcal{T}(B). \end{aligned}$$

It is clear that by Steps 1–2 in Algorithm 3.3 we have

$$ D-E_{k,q}+\mu _{k,q}^{-1}Y_{k,q}= \check{U}_{k,q}\check{\varSigma }_{k,q} \check{V}_{k,q}^{T}+ \tilde{U}_{k,q}\tilde{\varSigma }_{k,q}\tilde{V}_{k,q} ^{T}, $$

where \(\check{U}_{k,q}\), \(\check{V}_{k,q}\) are the singular vectors associated with singular values that are more than \(\frac{1}{\mu _{k,q}}\) and \(\tilde{U}_{k,q}\), \(\tilde{V}_{k,q}\) are those associated with singular values that are less than or equal to \(\frac{1}{ \mu _{k,q}}\), the elements of the diagonal matrix \(\check{\varSigma } _{k,q}\) are more than \(\frac{1}{\mu _{k,q}}\), and those of the diagonal matrix \(\tilde{\varSigma }_{k,q}\) are less than or equal to \(\frac{1}{ \mu _{k,q}}\). Hence, it is drawn that \(A_{k+1,q+1}=\check{U}_{k,q}( \check{\varSigma }_{k,q}-\mu _{k,q}^{-1}I)\check{V}_{k,q}^{T}\), and

$$\begin{aligned} \Vert B \Vert _{F} =& \bigl\Vert \mu _{k,q} \bigl(D-E_{k,q}+\mu _{k,q}^{-1}Y_{k,q}-A_{k+1,q+1} \bigr) \bigr\Vert _{F} \\ =& \bigl\Vert \mu _{k,q}\bigl(\mu _{k,q}^{-1} \check{U}_{k,q}\check{V}_{k,q}^{T}+ \tilde{U}_{k,q}\tilde{\varSigma }_{k,q}\tilde{V}_{k,q}^{T} \bigr) \bigr\Vert _{F} \\ =& \bigl\Vert \check{U}_{k,q}\check{V}_{k,q}^{T}+ \mu _{k,q}\tilde{U}_{k,q} \tilde{\varSigma }_{k,q} \tilde{V}_{k,q}^{T} \bigr\Vert _{F} \\ \leq & \sqrt{n}. \end{aligned}$$

We can obtain hence that \(Y_{k,q}+\mu _{k,q}(D-A_{k+1,q+1}-E_{k,q}) \in \partial \|A_{k+1,q+1}\|_{*}\) from Lemmas 4.2 and 4.3.

It is known that, for \(A_{k+1,q+1}=U\varSigma V^{T}\), by Lemma 4.1,

$$ \partial \Vert A_{k+1,q+1} \Vert _{*}=\bigl\{ UV^{T}+W:W\in \mathbb{R}^{n\times n},U ^{T}W=0,WV=0, \Vert W \Vert _{2}\leq 1\bigr\} . $$

We also have

$$ \bigl\Vert UV^{T}+W \bigr\Vert _{F}^{2}=\operatorname{tr}\bigl(\bigl(UV^{T}+W\bigr)^{T}\bigl(UV^{T}+W\bigr) \bigr)= \operatorname{tr}\bigl(VV^{T}+W^{T}W\bigr)\leq n. $$

Therefore, the following inequalities can be obtained:

$$ \bigl\Vert Y_{k,q}+\mu _{k,q}(D-A_{k+1,q+1}-E_{k,q}) \bigr\Vert _{F}\leq \sqrt{n} $$


$$ \bigl\Vert \mathcal{P}_{\bar{\varOmega }}\bigl(\mathcal{T}(B)\bigr) \bigr\Vert _{F}\leq \big\| \mathcal{T}(B))\big\| _{F} \leq \Vert B \Vert _{F}\leq \sqrt{n}. $$

It is evident that the sequence \(\{Y_{k,q}\}\) is bounded. □

Theorem 4.1

Suppose that \(\langle A_{k+1,q+1}-A_{k,q},D-A _{k+1,q+1}-E_{k,q}\rangle \geq 0\), then the sequence \(\{A_{k,q}\}\) converges to the solution of (3.5) when \(\mu _{k,q}\rightarrow \infty \) and \(\sum_{k=1}^{+\infty }\mu _{k,q}^{-1}=+\infty \).


It is true that

$$ \lim_{k\rightarrow \infty }(D-A_{k+1}-E_{k+1})=0 $$

since \(\mu _{k,q}^{-1}(Y_{k+1,q+1}-Y_{k,q})=D-A_{k+1,q+1}-E_{k+1,q+1}\) and Lemma 4.4.

Let \((\ddot{A},\ddot{E})\) be the solution of (3.5). Then \(A _{k+1,q+1},Y_{k+1,q+1},E_{k+1,q+1}\), \(k=1,2,\ldots \) , are all Toeplitz matrices since \(\ddot{A}+\ddot{E}=D\). We prove first that

$$\begin{aligned}& \Vert E_{k+1,q+1}-\ddot{E} \Vert _{F}^{2}+ \mu _{k,q}^{-2} \Vert Y_{k+1,q+1}-\ddot{Y} \Vert _{F}^{2} \\& \quad = \Vert E_{k,q}-\ddot{E} \Vert _{F}^{2}- \Vert E_{k+1,q+1}-E_{k,q} \Vert _{F}^{2}+ \mu _{k,q}^{-2} \Vert Y_{k,q}-\ddot{Y} \Vert _{F}^{2} \\& \qquad {}- \mu _{k,q}^{-2} \Vert Y_{k+1,q+1}-Y_{k,q} \Vert _{F}^{2}-2\mu _{k,q}^{-1}\langle A_{k+1,q+1}-\ddot{A},\hat{Y}_{k+1,q+1}-\ddot{Y}\rangle , \end{aligned}$$

where \(\hat{Y}_{k+1,q+1}=Y_{k,q}+\mu _{k,q}(D-A_{k+1,q+1}-E_{k,q})\), Ÿ is the optimal solution to the dual problem (3.2).

$$\begin{aligned} \Vert E_{k,q}-\ddot{E} \Vert _{F}^{2} =& \bigl\Vert \mathcal{P}_{\bar{\varOmega }}(E_{k,q}- \ddot{E}) \bigr\Vert _{F}^{2} \\ =& \bigl\Vert \mathcal{P}_{\bar{\varOmega }}(E_{k+1,q+1}- \ddot{E}-E_{k+1,q+1}+E _{k,q}) \bigr\Vert _{F}^{2} \\ =& \bigl\Vert \mathcal{P}_{\bar{\varOmega }}(E_{k+1,q+1}-\ddot{E}) \bigr\Vert _{F}^{2}+ \bigl\Vert \mathcal{P}_{\bar{\varOmega }}(E_{k+1,q+1}-E_{k,q}) \bigr\Vert _{F}^{2} \\ & {}-2\bigl\langle \mathcal{P}_{\bar{\varOmega }}(E_{k+1,q+1}-\ddot{E}), \mathcal{P}_{\bar{\varOmega }}(E_{k+1,q+1}-E_{k,q})\bigr\rangle \\ =& \Vert E_{k+1,q+1}-\ddot{E} \Vert _{F}^{2}+ \Vert E_{k+1,q+1}-E_{k,q} \Vert _{F}^{2} \\ &{} +2\mu _{k,q}^{-1}\bigl\langle \mathcal{P}_{\bar{\varOmega }}(A_{k+1,q+1}- \ddot{A}),\hat{Y}_{k+1,q+1}-\ddot{Y}\bigr\rangle . \end{aligned}$$

We obtain the following result with the same analysis:

$$\begin{aligned} \mu _{k,q}^{-2} \Vert Y_{k,q}-\ddot{Y} \Vert _{F}^{2} =& \mu _{k,q}^{-2} \bigl\Vert \mathcal{P}_{\varOmega }(Y_{k,q}-\ddot{Y}) \bigr\Vert _{F}^{2} \\ =&\mu _{k,q}^{-2} \bigl\Vert \mathcal{P}_{\varOmega }(Y_{k+1,q+1}- \ddot{Y}) \bigr\Vert _{F} ^{2}+\mu _{k,q}^{-2} \bigl\Vert \mathcal{P}_{\varOmega }(Y_{k+1,q+1}-Y_{k,q}) \bigr\Vert _{F}^{2} \\ &{} -2\mu _{k,q}^{-1}\bigl\langle \mathcal{P}_{\varOmega }(Y_{k+1,q+1}- \ddot{Y}), \mathcal{P}_{\varOmega }(Y_{k+1,q+1}-Y_{k,q})\bigr\rangle \\ =&\mu _{k,q}^{-2} \Vert Y_{k+1,q+1}-\ddot{Y} \Vert _{F}^{2}+\mu _{k,q}^{-2} \Vert Y _{k+1,q+1}-Y_{k,q} \Vert _{F}^{2} \\ & {}+2\mu _{k,q}^{-1}\bigl\langle \mathcal{P}_{\varOmega }(A_{k+1,q+1}- \ddot{A}), \hat{Y}_{k+1,q+1}-\ddot{Y}\bigr\rangle . \end{aligned}$$


$$ \min_{A\in \mathbb{T}^{m\times n}} \mu \Vert A \Vert _{*}+\frac{1}{2} \big\Vert \mathcal{P}_{\varOmega }(A-M) \big\Vert _{F}^{2} $$

holds true.

The sum \(\sum_{k=1}^{\infty }\mu _{k,q}^{-1}\langle A_{k,q}- \ddot{A},\hat{Y}_{k,q}-\ddot{Y}\rangle <+\infty \) and \(\|E_{k,q}- \ddot{E}\|^{2}+\mu _{k,q}^{-2}\|Y_{k,q}-\ddot{Y}\|^{2}\) is nonincreasing since \(\|A\|_{\ast }\) is a convex function and \(\hat{Y}_{k+1,q+1} \in \partial \|A\|_{\ast }\), \(\langle A_{k+1,q+1}-\ddot{A}, \hat{Y} _{k+1,q+1}-\ddot{Y}\rangle \geq 0\). On the other hand, the following are true by Algorithm 3.3:

$$ \langle Y_{k,q},E_{k,q}-\ddot{E}\rangle =0 \quad \text{and} \quad \langle \ddot{Y},E_{k,q}-\ddot{E}\rangle =0. $$

Therefore, by the same idea of Theorem 2 in [18], it is obtained that Ä is the solution of (3.5). □

Theorem 4.2

Let \(X=(x_{ij})\in \mathbb{R}^{n\times n}\), \(\mathcal{T}(X)=(\tilde{x}_{ij})\in \mathbb{T}^{n\times n}\) be the Toeplitz matrix derived from X, introduced in (2.1). Then, for any Toeplitz matrix \(Y=(y_{ij})\in \mathbb{T}^{n\times n}\), \(\langle X-\mathcal{T}(X),Y\rangle =0\).


By the definition of \(\mathcal{T}(X)\), we have \(\sum_{i,j}(x_{ij}-\tilde{x}_{ij})=0\), \(i,j=1,2,\ldots ,n\). Since Y is a Toeplitz matrix and \(y_{\alpha }=y_{ij}\), \(\alpha =i-j\), \(i,j=1,2,\ldots , n\), then

$$\begin{aligned} \bigl\langle X-\mathcal{T}(X),Y\bigr\rangle =& \sum _{i=1}^{n}\sum_{j=1}^{n}(x_{ij}- \tilde{x}_{ij})y_{ji} \\ =& \sum_{l=1}^{n-1}{y}_{-l}(x_{ij}- \tilde{x}_{ij})+\sum_{l=0}^{n-1}{y}_{l}(x_{ij}- \tilde{x}_{ij}) \\ =& 0. \end{aligned}$$


Theorem 4.3

In Algorithm 3.3, \(A_{k,q}\) is a Toeplitz matrix generated by \(X_{k,q}\), then

$$ \Vert A_{k,q}-\ddot{A} \Vert _{F}< \Vert X_{k,q}-\ddot{A} \Vert _{F} $$

is true with Ä being the solution of (3.5).


$$\begin{aligned} \Vert X_{k,q}-\ddot{A} \Vert _{F}^{2} =& \Vert X_{k,q}-A_{k,q}+A_{k,q}-\ddot{A} \Vert _{F}^{2} \\ =& \langle X_{k,q}-A_{k,q},X_{k,q}-A_{k,q} \rangle +2\langle X_{k,q}-A _{k,q},A_{k,q}-\ddot{A} \rangle \\ & {}+\langle A_{k,q}-\ddot{A},A_{k,q}-\ddot{A}\rangle \\ =& \Vert X_{k,q}-A_{k,q} \Vert _{F}^{2}+ \Vert A_{k,q}-\ddot{A} \Vert _{F}^{2}. \end{aligned}$$

Thus, \(\|A_{k,q}-\ddot{A}\|_{F}<\|X_{k,q}-\ddot{A}\|_{F}\) holds true. □

Numerical experiments

In this section we report some original numerical results of two algorithms (SALM, SSALM) for some \(n\times n\) matrices with different ranks. All the experiments are conducted on the same workstation with an Intel(R) Core(TM) i7-6700 CPU @ 3.40 GHz that has 16 GB memory and 16-bit operating system, running Windows 7 and Matlab (vision 2016a). We analyze and compare iteration numbers (IT), computing time in seconds (time(s)), deviation (error 1, error 2), and ratio which are defined in the following. It can be seen that the SSALM algorithm proposed in this study is highly effective compared with the SALM algorithm.

$$\begin{aligned}& {\text{\textsc{error }}} 1 =\frac{ \Vert A+E-D \Vert _{F}}{ \Vert D \Vert _{F}}, \qquad {\text{\textsc{error }}} 2 =\frac{ \Vert A-M \Vert _{F}}{ \Vert M \Vert _{F}}, \\& {\text{\textsc{ratio }}}=\frac{{\text{the CPU of the SSALM algorithm}}}{ {\text{the CPU of the SALM algorithm}}}\times 100\%. \end{aligned}$$

In the experiments, \(M\in \mathbb{T}^{n\times n} \) represents an uncompleted Toeplitz matrix, the sampling density \(p= m/(2n-1)\), where m is the number of the known diagonal entries. By the way, \(p\in \{0.3,0.4,0.5,0.6\}\) here. Due to the special structure of a Toeplitz matrix, we have \(0\leq m \leq 2n-1\). The SALM and SSALM algorithms share the same values of all parameters, say, \(\tau _{0}=1/ \|D\|_{2}\), \(\delta =1.2172 + {1.8588m}/{n^{2}}\), \(\epsilon _{1}=10^{-9}\), \(\epsilon _{2}=5\times 10^{-6}\). In the test of the SSALM algorithm, \(\ell =2\) or \(\ell =3\) as a rule of semi-smoothing.

The experimental results of two algorithms are presented in Tables 16. From the tables, two algorithms can successfully compute an approximate solution of the prescriptive stop condition for all the test matrix M. And computing time of our SSALM algorithm in far less than that of the SALM algorithm. In particular, compared with the cost of the SALM algorithm, we can find that the cost of the SSALM algorithm is decreased up to 30.44%. The “ratio” in Tables 56 can show this effectiveness.

Table 1 Comparison between SALM and SSALM for \(p=0.6\)
Table 2 Comparison between SALM and SSALM for \(p=0.5\)
Table 3 Comparison between SALM and SSALM for \(p=0.4\)
Table 4 Comparison between SALM and SSALM for \(p=0.3\)
Table 5 The values of ratio for \(\ell =2\)
Table 6 Comparison between SALM and SSALM for \(\ell =3\)

Concluding remarks

As is known to all, matrix completion usually means to reconstruct a matrix from a subset of the items of a matrix by taking advantage of low-rank structure matrix interdependencies between the entries. It is well known but NP-hard in general. In recent years, the Toeplitz matrix completion has attracted widespread attention and has become one of the most important completion problems. In order to solve such problems, we proposed an accelerated technique of the SALM algorithm in this study, namely the SSALM algorithm, and the theory of the SSALM algorithm convergence is established. Theoretical analysis and numerical results have shown that the SSALM scheme is an effective algorithm for solving the TMC problem. In particular, the CPU of the SSALM algorithm is consistently reduced up to 30.44% in all cases. The SSALM algorithm overcomes the original ALM algorithm both complexity of the singular value decomposition and surmounts the property of the extra load of the SALM algorithm. The reason is that data communication congestion is far more expensive than computing in many computers. Therefore, the SSALM algorithm has better numerical behavior for solving the TMC problem than the SALM algorithm (Tables 16).



matrix completion


Toeplitz matrix completion


singular value decomposition


singular value thresholding


accelerated proximal gradient


augmented Lagrange multiplier


modified augmented Lagrange multiplier


soothing augmented Lagrange multiplier


semi-soothing augmented Lagrange multiplier


robust principal component analysis


iteration number


computing time


  1. 1.

    Amit, Y., Fink, M., Srebro, N., Ullman, S.: Uncovering shared structures in multiclass classification. In: Proceeding of the 24th International Conference on Machine Learning, pp. 17–24. ACM, New York (2007)

    Google Scholar 

  2. 2.

    Argyriou, A., Evgeniou, T., Pontil, M.: Multi-task feature learning. Adv. Neural Inf. Process. Syst. 19, 41–48 (2007)

    Google Scholar 

  3. 3.

    Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C.: Multi-task feature learning, image inpainting. Comput. Graph. 34, 417–424 (2000)

    Google Scholar 

  4. 4.

    Blanchard, J., Tanner, J., Wei, K.: CGIHT: conjugate gradient iterative hard thresholding for compressed sensing and matrix completion. In: Numerical Analysis Group (2014) Preprint 14/08

    Google Scholar 

  5. 5.

    Boumal, N., Absil, P.A.: RTRMC: a Riemannian trust-region method for low-rank matrix completion. In: Shawe-Taylor, J., Zemel, R.S., Bartlett, P., Pereira, F.C.N., Weinberger, K.Q. (eds.) Advances in Neural Inf. Processing Systems, NIPS, vol. 24, pp. 406–414 (2011)

    Google Scholar 

  6. 6.

    Cai, J.F., Candès, E.J., Shen, Z.: A singular value thresholding method for matrix completion. SIAM J. Optim. 20(4), 1956–1982 (2010)

    MathSciNet  Article  Google Scholar 

  7. 7.

    Cai, J.F., Qu, X., Xu, W., Ye, G.B.: Robust recovery of complex exponential signals from random Gaussian projections via low rank Hankel matrix reconstruction. Appl. Comput. Harmon. Anal. 41(2), 470–490 (2016)

    MathSciNet  Article  Google Scholar 

  8. 8.

    Candès, E.J., Recht, B.: Exact matrix completion via convex optimization. Found. Comput. Math. 9(6), 717–772 (2009)

    MathSciNet  Article  Google Scholar 

  9. 9.

    Chen, C., He, B., Yuan, X.: Matrix completion via an alternating direction method. IMA J. Numer. Anal. 32, 227–245 (2012)

    MathSciNet  Article  Google Scholar 

  10. 10.

    Chen, M., Ganesh, A., Lin, Z., Ma, Y., Wright, J., Wu, L.: Fast convex optimization algorithms for exact recovery of a corrupted low-rank matrix. J. Mar. Biol. Assoc. UK 56(3), 707–722 (2009)

    Google Scholar 

  11. 11.

    Chen, Y., Chi, Y.: Robust spectral compressed sensing via structured matrix completion. IEEE Trans. Inf. Theory 60(10), 6576–6601 (2014)

    MathSciNet  Article  Google Scholar 

  12. 12.

    Eldén, L.: Matrix Methods in Data Mining and Pattern Recognition. Society for Industrial and Applied Mathematics, Philadelphia (2007)

    Google Scholar 

  13. 13.

    Hu, Y., Zhang, D.B., Liu, J., Ye, J.P., He, X.F.: Accelerated singular value thresholding for matrix completion. In: KDD’12, Beijing, August 12-16, 2012 (2012)

    Google Scholar 

  14. 14.

    Jain, P., Meka, R., Dhillon, I.: Guaranteed rank minimization via singular value projection. In: Proceeding of the Neural Information Processing Systems Conf., NIPS, pp. 937–945 (2010)

    Google Scholar 

  15. 15.

    Jain, P., Netrapalli, P., Sanghavi, S.: Low-rank matrix completion using alternating minimization. In: Proceedings of the 45th Annual ACM Symposium on Theory of Computing (STOC), pp. 665–674 (2013)

    Google Scholar 

  16. 16.

    Ji, H., Liu, C., Shen, Z., Xu, Y.: Robust video denoising using low rank matrix completion. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition CVPR (2010)

  17. 17.

    Kyrillidis, A., Cevher, V.: Matrix recipes for hard thresholding methods. J. Math. Imaging Vis. 48(2), 235–265 (2013)

    MathSciNet  Article  Google Scholar 

  18. 18.

    Lin, Z., Chen, M., Wu, L., Ma, Y.: The augmented Lagrange multiplier method for exact recovery of corrupted low-rank matrices. UIUC Technicial Report UIUL-ENG-09-2214, (2010)

  19. 19.

    Liu, Z., Vandenberghe, L.: Interior-point method for nuclear norm approximation with application to system identification. SIAM J. Matrix Anal. Appl. 31(3), 1235–1256 (2009)

    MathSciNet  Article  Google Scholar 

  20. 20.

    Luk, F.T., Qiao, S.: A fast singular value algorithm for Hankel matrices. In: Olshevsky, V. (ed.) Fast Algorithms for Structured Matrices: Theory and Applications. Contemporary Mathematics, vol. 323. Am. Math. Soc., Providence (2003)

    Google Scholar 

  21. 21.

    Ma, S., Goldfarb, D., Chen, L.: Fixed point and Bregman iterative methods for matrix rank minimization. Math. Program. 128, 321–353 (2011)

    MathSciNet  Article  Google Scholar 

  22. 22.

    Mesbahi, M., Papavassilopoulos, G.P.: On the rank minimization problem over a positive semidefinite linear matrix inequality. IEEE Trans. Autom. Control 42(2), 239–243 (1997)

    MathSciNet  Article  Google Scholar 

  23. 23.

    Mishra, B., Apuroop, K.A., Sepulchre, R.: A Riemannian geometry for low-rank matrix completion (2013) arXiv:1306.2672

  24. 24.

    Sebert, F., Zou, Y.M., Ying, L.: Toeplitz block matrices in compressed sensing and their applications in imaging. IEEE, Inf. Tech. Appl. in Biomedicine, 47–50 (2008)

  25. 25.

    Shaw, A.K., Pokala, S., Kumaresan, R.: Toeplitz and Hankel approximation using structured approach. Acoustics. Speech and signal processing. Proc. IEEE Int. Conf. 2349, 12–15 (1998)

    Google Scholar 

  26. 26.

    Tanner, J., Wei, K.: Low rank matrix completion by alternating steepest descent methods. Appl. Comput. Harmon. Anal. 40, 417–429 (2016)

    MathSciNet  Article  Google Scholar 

  27. 27.

    Toh, K.C., Yun, S.: An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems. Pac. J. Optim. 6, 615–640 (2010)

    MathSciNet  MATH  Google Scholar 

  28. 28.

    Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: a factorization method. Int. J. Comput. Vis. 9(2), 137–154 (1992)

    Article  Google Scholar 

  29. 29.

    Vandereycken, B.: Low rank matrix completion by Riemannian optimization. SIAM J. Optim. 23(2), 1214–1236 (2013)

    MathSciNet  Article  Google Scholar 

  30. 30.

    Wang, C.L., Li, C.: A mean value algorithm for Toeplitz matrix completion. Appl. Math. Lett. 41, 35–40 (2015)

    MathSciNet  Article  Google Scholar 

  31. 31.

    Wang, C.L., Li, C., Wang, J.: A modified augmented Lagrange multiplier algorithm for Toeplitz matrix completion. Adv. Comput. Math. 42, 1209–1224 (2016)

    MathSciNet  Article  Google Scholar 

  32. 32.

    Wang, C.L., Li, C., Wang, J.: Comparisons of several algorithms for Toeplitz matrix recovery. Comput. Math. Appl. 71(1), 133–146 (2016)

    MathSciNet  Article  Google Scholar 

  33. 33.

    Wen, R.P., Li, S.D., Meng, G.Y.: SOR-like methods with optimization model for augmented linear systems. East Asian J. Appl. Math. 7(1), 101–115 (2017)

    MathSciNet  Article  Google Scholar 

  34. 34.

    Wen, R.P., Li, S.Z., Zhou, F.: Toeplitz matrix completion via smoothing augmented Lagrange multiplier algorithm. Appl. Math. Comput. 355, 299–310 (2019)

    MathSciNet  Google Scholar 

  35. 35.

    Wen, R.P., Yan, X.H.: A new gradient projection method for matrix completion. Appl. Math. Comput. 258, 537–544 (2015)

    MathSciNet  MATH  Google Scholar 

Download references


The authors are very much indebted to the editor and anonymous referees for their helpful comments and suggestions which greatly improved the original manuscript of this paper. The authors also are thankful for the support from the NSF of Shanxi Province (201601D011004) and the SYYJSKC-1803.

Availability of data and materials

Please contact author for data requests.


It is not applicable.

Author information




All authors contributed equally to the writing of this paper. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Ruiping Wen.

Ethics declarations

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Wen, R., Li, S. & Duan, Y. A semi-smoothing augmented Lagrange multiplier algorithm for low-rank Toeplitz matrix completion. J Inequal Appl 2019, 83 (2019).

Download citation


  • Toeplitz matrix
  • Completion
  • Augmented Lagrange multiplier
  • Data communication