A new reweighted minimization algorithm for image deblurring
© Qiao et al.; licensee Springer 2014
Received: 22 February 2013
Accepted: 16 March 2014
Published: 16 June 2014
In this paper, a new reweighted minimization algorithm for image deblurring is proposed. The algorithm is based on a generalized inverse iteration and linearized Bregman iteration, which is used for the weighted minimization problem . In the computing process, the effective using of signal information can make up the detailed features of image, which may be lost in the deblurring process. Numerical experiments confirm that the new reweighted algorithm for image restoration is effective and competitive to the recent state-of-the-art algorithms.
Image deblurring is a fundamental problem in image processing, since many real-life problems can be modeled as deblurring problems . In this paper, a new reweighted minimization algorithm for image deblurring is proposed. The algorithm is obtained based on a generalized inverse iteration and a linearized Bregman iteration.
where is an additive noise and is a linear blurring operator. This problem is ill-posed due to the large condition number of the matrix A. Any small perturbation on the observed blurred image f may cause the direct solution , which is very difficult to obtain from the original image u . This is a widely studied subject and many corresponding approaches have been developed, and one of them is to minimize some cost functionals . The simplest method is a Tikhonov regularization, which minimizes an energy consisting of a data fidelity term and an norm regularization term. A is a convolution, which can solve the problem in the Fourier domain. In this case, the method is called a Wiener filter , this is a linear method, and the edges of restored image are usually smeared. To overcome this, a total variation (TV)-based regularization was proposed by Rudin et al. in , which is known as the ROF model. Due to its virtue of preserving edges, it is widely used in image processing, such as blind deconvolution, inpainting, and superresolution; see . However, as we know, for the TV yields staircasing [5, 6], these TV-based methods do not preserve the fine structures, details, and textures. To avoid these drawbacks, nonlocal methods were proposed for denoising [7, 8], and then extended to deblurring . Also, the Bregman iteration, introduced to image science , was shown to improve TV-based blind deconvolution [11–13]. Recently, a nonlocal TV regularization was invented based on graph theory  and applied to image deblurring . Another approach for deblurring is the wavelet-based method, etc. .
where is a continuous convex function, and when is strictly or strongly convex, the solution of (1.2) is unique.
This constrained optimization problem (1.2) arise in many applications, like in image compression, reconstruction, inpainting, segmentation, compressed sensing, etc. The problem (1.2) can be transformed into a linear programming problem, and then solved by a conventional linear programming solver in many cases. Recently, fixed-point continuation method  and Bregman iteration  are very popular. Specially, Bregman iterative regularization was proposed by Osher et al. . In the past few years, a series of new methods have been developed, and among them, the linearized Bregman method [19–22] and the split Bregman method [23–26] got most attention.
Obviously, the problem (1.3) is an -norm minimization problem. Since many practical problems related to the sparsity of the solution make the problem (1.3) stay on focus for years, like in signal processing, compressive sensing etc. [18, 19]. Similar to the problem (1.2), the problem (1.3) also can be transformed into a linear program and then solved by conventional linear programming solvers. However, such solvers are not tailored for the matrix A that is large-scale and completely dense. Fortunately, the problem (1.3) can be solved very effectively by the linearized Bregman method [19–22, 27]. The computing speed of its simplified form with soft threshold operator is faster [19, 21, 22]. The corresponding convergence analysis was discussed in .
On this basis, the authors propose a new reweighted minimization method to solve the problem (1.5) and illustrate by numerical experiments.
The rest of the paper is organized as follows. In Section 2, we summarize the existing methods for solving the constrained problem (1.3). In Section 3, the generalized shrinkage operator is proposed. The new algorithm is proposed in Section 4. Numerical results are shown in Section 5. Finally, we draw some conclusions in Section 6.
2.1 Generalized inverse
We are interested in the iterative formula of the generalized inverse, because it is used by our new algorithm. Therefore, before we give a detailed discussion, we first give some definitions and lemmas.
Definition 2.1 
Remark 2.1 The inner inverse is not unique. In general, the set of the inner inverses of the matrix A is denoted .
Definition 2.2 
is called the range of .
Lemma 2.1 
is convergent to .
2.2 Linearized Bregman iteration
where is an element in the subgradient set of J at the point v. In general and the triangle inequality is not satisfied, so is not a distance in the usual sense. For details, see .
where δ is a constant and . Hereafter, we use to denote the norm.
Namely, the algorithm (2.8) is called an linearized Bregman iteration.
where is generalized inverse of matrix A.
3 The generalized shrinkage operator
Theorem 3.1 .
- (1)If , and notice that then , for this case gets its minimum at point along the direction and the minimum is(3.2)
- (2)If , and notice that , again we find that decreases along the direction :(3.3)
Since , along the direction we find that the minimizer of is .
- (1)If , since , increases along the direction :(3.4)
- (2)If , since we have , the minimizer of along the direction is and the corresponding minimum is(3.5)
Since , we can get the minimizer of at along the direction .
- (1)If , since , increases along the direction :(3.6)
- (2)If , since , decreases along the direction :(3.7)
when , the minimum of along the direction is .
The generalized shrinkage operator leads to the sparse solution and removes noises. Hence, the algorithm with the generalized shrinkage operator converges to a sparse solution and is robust to noises.
4 The new reweighted minimization algorithm
where , and . The corresponding sequence also converges to an optimal solution of the problem (1.3).
where is a penalty function such as the norm. This minimization can be accomplished by solving a sequence of weighted least-squares problems where the weights depend on the previous residuals . The typical choice of ρ is inversely proportional to the residual, so that the large residuals will be penalized less in the subsequent iterations. Then an IRLS involving an iteratively reweighted -norm can be better approximated by an -like criterion. Inspired by the above idea, in order to better approximate an -like criterion , our algorithm involves the iteratively reweighted -norm.
where , , and .
5 Numerical experiments
where , , and are the restored image, original image, and average operator, respectively.
Our code is written in MATLAB and run on a Windows PC with a Intel(R) Core(TM) 2 Duo CPU T8100 @ 2.10 GHz 2.10 GHz and 1.5 GB memory. The MATLAB version is 7.1.
Reweighted minimization algorithm: Step 1. Set , , , , , , .
Step 2. The sequence generated by (4.4).
Step 3. Until .
We demonstrate the performance of the reweighted minimization algorithm, the chaotic iterative algorithm, the Bregman iteration, and the Bregman iteration with in MATLAB.
In fact, the complexity analysis also shows comparative results of several methods. Set the same loop number is K. So, the workload of the algorithm (2.11) is two parts. They are the workload of the and the loop of the (2.11). The workload is during the computation of , , when , because of the singular value decomposition involving multiplication of the matrix and matrix and eigenvalue calculation. The workload of the loop of the (2.11) is , because the loop only contains multiplication of matrix and vector. Therefore, the total workload of the algorithm (2.11) is . The workload of the chaotic iteration (4.1), the reweighted minimization algorithm (4.4) and the Bregman iteration (2.8) are , respectively. Obviously, , the workload of the algorithm (2.11) is bigger than the other three algorithms.
The comparison of different algorithms
256 × 256
15 × 15 ‘disk’
256 × 256
7 × 15 ’Gaussian’
64 × 80
3 × 5 ’motion’
In this paper, we propose the reweighted minimization algorithm for image deblurring. Above all, we can see that the recovery of the image effect is obvious. Especially in the case of a large degree of blurring and difficult to recover details, it is stable and effective. In addition, we can improve the efficiency of this reweighted minimization algorithm combining with the ‘kicking’ technology. Because of the scale factor and efficiency of the algorithm , the new method proposed in this paper can be used in a parallel operation to get a better algorithm.
This research was partly supported by Fund of Oceanic Telemetry Engineering and Technology Research Center, State Oceanic Administration (grant no. 2012003), the NSFC (grant nos. 60971132,61101208) and Fundamental Research Funds for the Central Universities (grant no. 13CX02086A).
- Chan TF, Shen J: Image Processing and Analysis. SIAM, Philadelphia; 2005.View ArticleGoogle Scholar
- Aubert G, Kornprobst P Appl. Math. Sci. 147. In Mathematical Problems in Image Processing. 2nd edition. Springer, New York; 2006.Google Scholar
- Andrews HC, Hunt BR: Digital Image Restoration. Prentice Hall, Englewood Cliffs; 1977.Google Scholar
- Rudin L, Osher S, Fatemi E: Nonlinear total variation based noise removal algorithms. Physica D 1992, 60: 259–268. 10.1016/0167-2789(92)90242-FView ArticleGoogle Scholar
- Dobson DC, Santosa F: Recovery of blocky images from noise and blurred data. SIAM J. Appl. Math. 1996, 56: 1181–1198. 10.1137/S003613999427560XMathSciNetView ArticleGoogle Scholar
- Nikolova M: Local strong homogeneity of a regularized estimator. SIAM J. Appl. Math. 2000, 61: 633–658. 10.1137/S0036139997327794MathSciNetView ArticleGoogle Scholar
- Buades A, Coll B, Morel JM: A review of image denoising algorithms, with a new one. Multiscale Model. Simul. 2005, 4: 490–530. 10.1137/040616024MathSciNetView ArticleGoogle Scholar
- Tomasi C, Manduchi R: Bilateral filtering for gray and color images. Proceedings of the 1998 IEEE International Conference on Computer Vision 1998. Bombay, IndiaGoogle Scholar
- Buades, A, Coll, B, Morel, JM: Image enhancement by non-local reverse heat equation. CMLA Tech. rep. 22 (2006)Google Scholar
- Osher S, Burger M, Goldfarb D, Xu J, Yin W: An iterative regularization method for total variation-based image restoration. Multiscale Model. Simul. 2005, 4: 460–489. 10.1137/040605412MathSciNetView ArticleGoogle Scholar
- He L, Marquina A, Osher S: Blind deconvolution using TV regularization and Bregman iteration. Int. J. Imaging Syst. Technol. 2005, 15: 74–83. 10.1002/ima.20040View ArticleGoogle Scholar
- Marquina, A: Inverse scale space methods for blind deconvolution. UCLA-CAM-Report 06–36 (2006)Google Scholar
- Marquina A, Osher S: Image super-resolution by TV-regularization and Bregman iteration. J. Sci. Comput. 2008, 37: 367–382. 10.1007/s10915-008-9214-8MathSciNetView ArticleGoogle Scholar
- Gilboa G, Osher S: Nonlocal linear image regularization and supervised segmentation. Multiscale Model. Simul. 2007, 6: 595–630. 10.1137/060669358MathSciNetView ArticleGoogle Scholar
- Lou, Y, Zhang, X, Osher, S, Bertozzi, A: Image recovery via nonlocal operators. UCLA-CAM-Report 08–35 (2008)Google Scholar
- Cofiman RR, Donoho DL: Translation-invariant de-noising. Lecture Notes in Statistics 103. In Wavelets and Statistics. Edited by: Antoniadis A, Oppenheim G. Springer, New York; 1995.Google Scholar
- Hale, E, Yin, W, Zhang, Y: A fixed-point continuation method for l 1 -regularization with application to compressed sensing. CAAM-TR07–07 (2007)Google Scholar
- Yin W, Osher S, Goldfarb D, Darbon J: Bregman iterative algorithms for -minimization with applications to compressed sensing. SIAM J. Imaging Sci. 2008, 1: 143–168. 10.1137/070703983MathSciNetView ArticleGoogle Scholar
- Cai J, Osher S, Shen Z: Linearized Bregman iterations for compressed sensing. Math. Comput. 2009,78(267):1515–1536. 10.1090/S0025-5718-08-02189-3MathSciNetView ArticleGoogle Scholar
- Cai J, Osher S, Shen Z: Convergence of the linearized Bregman iteration for -norm minimization. Math. Comput. 2009,78(268):2127–2136. 10.1090/S0025-5718-09-02242-XMathSciNetView ArticleGoogle Scholar
- Osher, S, Mao, Y, Dong, B, Yin, W: Fast linearized Bregman iteration for compressive sensing and sparse denoising. UCLA-CAM-Report 08–37 (2008)Google Scholar
- Cai J, Osher S, Shen Z: Linearized Bregman iterations for frame-based image deblurring. SIAM J. Imaging Sci. 2009,2(1):226–252. 10.1137/080733371MathSciNetView ArticleGoogle Scholar
- Goldstein T, Osher S: The split Bregman method for -regularized problems. SIAM J. Imaging Sci. 2009,2(2):323–343. 10.1137/080725891MathSciNetView ArticleGoogle Scholar
- Cai J, Osher S, Shen Z: Split Bregman method and frame based image restoration. Multiscale Model. Simul. 2009,8(2):337–369.MathSciNetView ArticleGoogle Scholar
- Wu C, Tai X: Augmented Lagrangian method, dual methods, and split Bregman iteration for ROF, vectorial TV, and high order models. SIAM J. Imaging Sci. 2010,3(3):300–339. 10.1137/090767558MathSciNetView ArticleGoogle Scholar
- Yang, Y, Möller, M, Osher, S: A dual split Bregman method for fast minimization. UCLA-CAM-Report 11–57 (2011)Google Scholar
- Zhang H, Cheng L: Linearized Bregman iteration algorithm. Math. Numer. Sin. 2010, 32: 97–104. (in Chinese)MathSciNetGoogle Scholar
- Candés EJ, Wakin MB, Boyd SP: Enhancing sparsity by reweighted minimization. J. Fourier Anal. Appl. 2008,14(5):877–905.MathSciNetView ArticleGoogle Scholar
- Wang G, Wei Y, Qiao S: Generalized Inverses: Theory and Computations. Science Press, Beijing; 2004.Google Scholar
- Wang S, Yang Z: Generalized Inverse Matrix and Its Applications. Beijing University of Technology Press, Beijing; 1996.Google Scholar
- Bregman L: The relaxation method of finding the common points of convex sets and its application to the solution of problems in convex programming. USSR Comput. Math. Math. Phys. 1967,7(3):200–217. 10.1016/0041-5553(67)90040-7View ArticleGoogle Scholar
- Donoho D: De-noising by soft-thresholding. IEEE Trans. Inf. Theory 1995, 41: 613–627. 10.1109/18.382009MathSciNetView ArticleGoogle Scholar
- Schlossmacher EJ: An iterative technique for absolute deviations curve fitting. J. Am. Stat. Assoc. 1973, 68: 857–859. 10.1080/01621459.1973.10481436View ArticleGoogle Scholar
- Zhao YB, Li D: Reweighted -minimization for sparse solutions to underdetermined linear systems. SIAM J. Optim. 2012,22(3):1065–1088. 10.1137/110847445MathSciNetView ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.