Skip to content


  • Research Article
  • Open Access

The Convergence Rate for a -Functional in Learning Theory

Journal of Inequalities and Applications20102010:249507

  • Received: 11 November 2009
  • Accepted: 21 February 2010
  • Published:


It is known that in the field of learning theory based on reproducing kernel Hilbert spaces the upper bounds estimate for a -functional is needed. In the present paper, the upper bounds for the -functional on the unit sphere are estimated with spherical harmonics approximation. The results show that convergence rate of the -functional depends upon the smoothness of both the approximated function and the reproducing kernels.


  • Spherical Harmonic
  • Tikhonov Regularization
  • Cauchy Inequality
  • Jacobi Weight
  • Mercer Kernel

1. Introduction

It is known that the goal of learning theory is to approximate a function (or some function features) from data samples.

Let be a compact subset of -dimensional Euclidean spaces , . Then, learning theory is to find a function related the input to the output (see [13]). The function is determined by a probability distribution on where is the marginal distribution on and is the condition probability of for a given

Generally, the distribution is known only through a set of sample independently drawn according to . Given a sample , the regression problem based on Support Vector Machine (SVM) learning is to find a function such that is a good estimate of when a new input is provided. The binary classification problem based on SVM learning is to find a function which divides into two parts. Here is often induced by a real-valued function with the form of where if , otherwise, . The functions are often generated from the following Tikhonov regularization scheme (see, e.g., [49]) associated with a reproducing kernel Hilbert space (RKHS) (defined below) and a sample :


where is a positive constant called the regularization parameter and ( ) called -norm SVM loss.

In addition, the Tikhonov regularization scheme involving offset (see, e.g., [4, 10, 11]) can be presented below with a similar way to (1.1)


We are in a position to define reproducing kernel Hilbert space. A function is called a Mercer kernel if it is continuous, symmetric, and positive semidefinite, that is, for any finite set of distinct points , the matrix is positive semidefinite.

The reproducing kernel Hilbert space (RKHS) (see [12]) associated with the Mercer kernel is defined to be the closure of the linear span of the set of functions with the inner product satisfying and the reproducing property


If , then . Denote as the space of continuous function on with the norm . Let Then the reproducing property tells that


It is easy to see that is a subset of We say that is a universal kernel if for any compact subset is dense in (see [13, Page 2652]).

Let be a given discrete set of finite points. Then, we may define an RKHS by the linear span of the set of functions . Then, it is easy to see that and for any there holds

Define and where the minimum is taken over all measurable functions. Then, to estimate the explicit learning rate, one needs to estimate the regularization errors (see, e.g., [4, 7, 9, 14])


The convergence rate of (1.5) is controlled by the -functional (see, e.g., [9])


and (1.6) is controlled by another -functional (see, e.g., [4])


where with


We notice that, on one hand, the -functionals (1.7) and (1.8) are the modifications of the -functional of interpolation theory (see [15]) since the interpolation relation (1.4). On the other hand, they are different from the usual -functionals (see e.g., [1630]) since the term However, they have some similar point. For example, if is a universal kernel, is dense in (see e.g., [31]). Moreover, some classical function spaces such as the polynomial spaces (see [2, 32]) and even some Sobolev spaces may be regarded as RKHS (see e.g., [33]).

In learning theory we often require and for some (see e.g., [1, 7, 14]). Many results on this topic have been achieved. With the weighted Durrmeyer operators [8, 9] showed the decay by taking to be the algebraic polynomials kernels on or on the simplex in .

However, in general case, the convergence of -functional (1.8) should also be considered since the offset often has influences on the solution of the learning algorithms (see e.g., [6, 11]). Hence, the purpose of this paper is twofold. One is to provide the convergence rates of (1.7) and (1.8) when is a general Mercer kernel on the unit sphere and The other is how to construct functions of the type of


to obtain the convergence rate of (1.8). The translation networks constructed in [3437] have the form of (1.10) and the zonal networks constructed in [38, 39] have the form of (1.10) with . So the methods used by these references may be used here to estimate the convergence rates of (1.7) and (1.8) if one can bound the term

In the present paper, we shall give the convergence rate of (1.7) and (1.8) for a general kernel defined on the unit sphere and with being the usual Lebesgue measure on . If there is a distortion between and the convergence rate of (1.7)-(1.8) in the general case may be obtained according to the way used by [1, 8].

The rest of this paper is organized as follows. In Section 2, we shall restate some notations on spherical harmonics and present the main results. Some useful lemmas dealing with the approximation order for the de la Vallée means of the spherical harmonics, the Gauss integral formula, the Marcinkiewicz-Zygmund with respect to the scattered data obtained by G. Brown and F. Dai and a result on the zonal networks approximation provided by H. N. Mhaskar will be given in Section 3. A kind of weighted norm estimate for the Mercer kernel matrices on the unit sphere will be given in Lemma 3.8. Our main results are proved in the last section.

Throughout the paper, we shall write if there exists a constant such that . We write if and .

2. Notations and Results

To state the results of this paper, we need some notations and results on spherical harmonics.

2.1. Notations

For integers , , the class of all one variable algebraic polynomials of degree defined on is denoted by , the class of all spherical harmonics of degree will be denoted by , and the class of all spherical harmonics of degree will be denoted by . The dimension of is given by (see [40, Page 65])


and that of is One has the following well-known addition formula (see [41, Page 10, Theorem ]):


where is the degree- generalized Legendre polynomial. The Legendre polynomials are normalized so that and satisfy the orthogonality relations


Define and by taking to be the usual volume element of and the Jacobi weights functions , , , respectively. For any we have the following relation (see [42, Page 312]):


The orthogonal projections of a function on are defined by (see e.g., [43])


where denotes the inner product of and .

2.2. Main Results

Let satisfy and . Define


Then, by [44, Chapter 17] we know that is positive semidefinite on and the right of (2.6) is convergence absolutely and uniformly since . Therefore, is a Mercer kernel on By [13, Theorem ] we know that is a universal kernel on . We suppose that there is a constant depending only on such for any


Given a finite set , we denote by the cardinality of . For and we say that a finite subset is an -covering of if


where with being the geodesic distance between and .

Let be an integer, a sequence of real numbers. Define forward difference operators by , ,


We say a finite subset is a subset of interpolatory type if for any real numbers there is a such that , This kind of subsets may be found from [45, 46].

Let be the set of all sequence for which and the set of all sequence for which

Let be a real number, Then, we say if there is a function such that


We now give the results of this paper.

Theorem 2.1.

If there is a constant depending only on such that is a subset of interpolatory type and a -covering of satisfying with and being a given positive integer. is an integer. is a real number such that there is and , satisfies and . is the reproducing kernel space reproduced by and the kernel (2.6). . Then there is a constant depending only on and and a function with and a constant such that

The functions satisfying the conditions of Theorem 2.1 may be found in [39, Page 357].

Corollary 2.2.

Under the conditions of Theorem 2.1. If , then

Corollary 2.2 shows that the convergence rate of the -functional (1.8) is controlled by the smoothness of both the reproducing kernels and the approximated function .

Theorem 2.3.

If there is a constant depending only on such that is a subset of interpolatory type and a -covering of satisfying with and being a given positive integer. is the reproducing kernel space reproducing by and the kernel (2.6) with satisfying and Then, for and there holds


3. Some Lemmas

To prove Theorems 2.1 and 2.3, we need some lemmas. The first one is about the Gauss integral formula and Marcinkiewicz inequalities.

Lemma 3.1 (see [4750]).

There exist constants depending only on such that for any positive integer and any -covering of satisfying , there exists a set of real numbers , such that
for any and for

where the constants of equivalence depending only on , , , and when is small. Here one employs the slight abuse of notation that

The second lemma we shall use is the Nikolskii inequality for the spherical harmonics.

Lemma 3.2 (see [38, 45, 49, 51, 52]).

If , , then one has the following Nikolskii inequality:

where the constant depends only on .

We now restate the general approximation frame of the Cesàro means and de la Vallée Poussin means provided by Dai and Ditzian (see [53]).

Lemma 3.3.

Let be a positive measure on . is a sequence of finite-dimensional spaces satisfying the following:

(I) .

(II) is orthogonal to (in ) when

(III) is dense in for all .

(IV) is the collection of the constants.

The Cesàro means of is given by

for , where

and is an orthogonal base of in One sets,for a given , and if there exists such that

Let be defined as for and for and is a nonegative and nonincrease function. are the de la Vallée Poussin means defined as

Then, If for some , , then, and

Lemma 3.3 makes the following Lemma 3.4.

Lemma 3.4.

Let be the function defined as in Lemma 3.3. Define two kinds of operators, respectively, by

Then, for any and for any . Moreover,

where for one defines


By [54, Lemma ] we know for some . Hence, (3.9) holds by (3.7). By [19, Theorem ] we know for Hence, (3.10) holds by (3.7).

Let be a finite set. Then we call an M-Z quadrature measure of order if (3.1) and (3.2) hold for By this definition one knows the finite set in Lemma 3.1 is an M-Z quadrature measure of order .

Define an operator as


Then, we have the following results.

Lemma 3.5 (see [39]).

For a given integer let be an M-Z quadrature measure of order , , an integer, , , where satisfies which satisfies if and if . defined in Lemma 3.3 is a nonnegative and non-increasing function. Let satisfy . Then, for , , where consists of for which the derivative of order ; that is, , belongs to . Then, there is an operator such that

(i)(see [39, Proposition , (b)]). for



(ii)(see [39, Theorem ]). Moreover, if one adds an assumption that then, there are constants and such that

and for

Lemma 3.6 (see e.g., [29, Page 230]).

Let . Then,

Following Lemma 3.7 deals with the orthogonality of the Legendre polynomials

Lemma 3.7.

For the generalized Legendre polynomials one has


It may be obtained by (2.2).

Lemma 3.8.

Let satisfy (2.7) for and . is a finite set satisfying the conditions of Theorem 2.1. Then, there is a constant depending only on such that


Define a matrix by , where with and Then,

By the Parseval equality we have


Let satisfy , . Then, by (3.1)

Hence, . On the other hand, since , , we have for any that
It follows for that
Define . Then, (3.24), (3.10), the Cauchy inequality, and the fact make
It follows that

Equation (3.2) thus holds.

4. Proof of the Main Results

We now show Theorems 2.1 and 2.3, respectively.

Proof of Theorem 2.1.

Lemma in [39] gave the following results.

Let , , be an integer, and a sequence of real numbers such . Then, there exists such that ,

Since and we have a such that Hence, and

and for there holds for that
It follows for that
On the other hand, since
where for , we have by (4.3)
Hence, above equation and (3.1)-(3.2) makes
where , Define
Then, we know and by (3.9)

It follows by (3.9) that


On the other hand, by the definition of and (3.14) we have for that

where denotes the operator of Lemma 3.5 for Hence,
Equation (3.2) and the definition of make
The Hölder inequality, the of Lemma 3.5, and the fact that make . Therefore,

Take then

Equations (3.2), (3.17), (3.16), and the Cauchy inequality make

Let be the Gamma function. Then, it is well known that Therefore,

Equations (4.14) and (4.4) make
and hence

Since , we have (2.11) by (4.20). Equation (2.12) follows by (4.3), (4.4), and (3.19).

Proof of Corollary 2.2.

By (2.11)-(2.12) one has

Proof of Theorem 2.3.

Take the place of in Lemma 3.5 with denote still by the operator in Lemma 3.5 with and
then, and by (3.15) In this case,
Since is a spherical harmonics of order , we know by of Lemma 3.5 that are also spherical harmonics of order Then, (3.2), of Lemma 3.5, (3.3), and (3.16) make

Hence, (3.19) and above equation make . Equation (2.14) follows by (3.15).



This work is supported by the National NSF (10871226) of China. The authors thank the reviewers for giving very valuable suggestions.

Authors’ Affiliations

Department of Mathematics, Shaoxing University, Shaoxing, Zhejiang, 312000, China
Department of Mathematics, Zhejiang Normal University, Jinhua, Zhejiang, 321004, China


  1. Cucker F, Smale S: On the mathematical foundations of learning. Bulletin of the American Mathematical Society 2002, 39(1):1–49.MathSciNetView ArticleMATHGoogle Scholar
  2. Cucker F, Zhou D-X: Learning Theory: An Approximation Theory Viewpoint, Cambridge Monographs on Applied and Computational Mathematics. Cambridge University Press, Cambridge, Mass, USA; 2007:xii+224.View ArticleMATHGoogle Scholar
  3. Vapnik VN: Statistical Learning Theory, Adaptive and Learning Systems for Signal Processing, Communications, and Control. John Wiley & Sons, New York, NY, USA; 1998:xxvi+736.MATHGoogle Scholar
  4. Chen DR, Wu Q, Ying YM, Zhou DX: Support vector machine soft margin classifier: error analysis. Journal of Machine Learning and Research 2004, 5: 1143–1175.MathSciNetMATHGoogle Scholar
  5. Evgeniou T, Pontil M, Poggio T: Regularization networks and support vector machines. Advances in Computational Mathematics 2000, 13(1):1–50. 10.1023/A:1018946025316MathSciNetView ArticleMATHGoogle Scholar
  6. Li Y, Liu Y, Zhu J: Quantile regression in reproducing kernel Hilbert spaces. Journal of the American Statistical Association 2007, 102(477):255–268. 10.1198/016214506000000979MathSciNetView ArticleMATHGoogle Scholar
  7. Tong H, Chen D-R, Peng L: Analysis of support vector machines regression. Foundations of Computational Mathematics 2009, 9(2):243–257. 10.1007/s10208-008-9026-0MathSciNetView ArticleMATHGoogle Scholar
  8. Tong H, Chen D-R, Peng L: Learning rates for regularized classifiers using multivariate polynomial kernels. Journal of Complexity 2008, 24(5–6):619–631. 10.1016/j.jco.2008.05.008MathSciNetView ArticleMATHGoogle Scholar
  9. Zhou D-X, Jetter K: Approximation with polynomial kernels and SVM classifiers. Advances in Computational Mathematics 2006, 25(1–3):323–344.MathSciNetView ArticleMATHGoogle Scholar
  10. Chen D, Xiang D-H: The consistency of multicategory support vector machines. Advances in Computational Mathematics 2006, 24(1–4):155–169.MathSciNetView ArticleMATHGoogle Scholar
  11. De Vito E, Rosasco L, Caponnetto A, Piana M, Verri A: Some properties of regularized kernel methods. Journal of Machine Learning Research 2004, 5: 1363–1390.MathSciNetMATHGoogle Scholar
  12. Aronszajn N: Theory of reproducing kernels. Transactions of the American Mathematical Society 1950, 68: 337–404. 10.1090/S0002-9947-1950-0051437-7MathSciNetView ArticleMATHGoogle Scholar
  13. Micchelli CA, Xu Y, Zhang H: Universal kernels. Journal of Machine Learning Research 2006, 7: 2651–2667.MathSciNetMATHGoogle Scholar
  14. Wu Q, Ying Y, Zhou D-X: Multi-kernel regularized classifiers. Journal of Complexity 2007, 23(1):108–134. 10.1016/j.jco.2006.06.007MathSciNetView ArticleMATHGoogle Scholar
  15. Bergh J, Löfström J: Interpolation Spaces. Springer, New York, NY, USA; 1976.View ArticleMATHGoogle Scholar
  16. Berens H, Lorentz GG: Inverse theorems for Bernstein polynomials. Indiana University Mathematics Journal 1972, 21(8):693–708. 10.1512/iumj.1972.21.21054MathSciNetView ArticleMATHGoogle Scholar
  17. Berens H, Li LQ: The Peetre -moduli and best approximation on the sphere. Acta Mathematica Sinica 1995, 38(5):589–599.MathSciNetMATHGoogle Scholar
  18. Berens H, Xu Y: -moduli, moduli of smoothness, and Bernstein polynomials on a simplex. Indagationes Mathematicae 1991, 2(4):411–421. 10.1016/0019-3577(91)90027-5MathSciNetView ArticleMATHGoogle Scholar
  19. Chen W, Ditzian Z: Best approximation and -functionals. Acta Mathematica Hungarica 1997, 75(3):165–208. 10.1023/A:1006543020828MathSciNetView ArticleMATHGoogle Scholar
  20. Chen W, Ditzian Z: Best polynomial and Durrmeyer approximation in . Indagationes Mathematicae 1991, 2(4):437–452. 10.1016/0019-3577(91)90029-7MathSciNetView ArticleMATHGoogle Scholar
  21. Dai F, Ditzian Z: Jackson inequality for Banach spaces on the sphere. Acta Mathematica Hungarica 2008, 118(1–2):171–195. 10.1007/s10474-007-6206-3MathSciNetView ArticleMATHGoogle Scholar
  22. Ditzian Z, Zhou X: Optimal approximation class for multivariate Bernstein operators. Pacific Journal of Mathematics 1993, 158(1):93–120.MathSciNetView ArticleMATHGoogle Scholar
  23. Ditzian Z, Runovskii K: Averages and -functionals related to the Laplacian. Journal of Approximation Theory 1999, 97(1):113–139. 10.1006/jath.1997.3262MathSciNetView ArticleMATHGoogle Scholar
  24. Ditzian Z: A measure of smoothness related to the Laplacian. Transactions of the American Mathematical Society 1991, 326(1):407–422. 10.2307/2001870MathSciNetMATHGoogle Scholar
  25. Ditzian Z, Totik V: Moduli of Smoothness, Springer Series in Computational Mathematics. Volume 9. Springer, New York, NY, USA; 1987:x+227.Google Scholar
  26. Ditzian Z: Approximation on Banach spaces of functions on the sphere. Journal of Approximation Theory 2006, 140(1):31–45. 10.1016/j.jat.2005.11.013MathSciNetView ArticleMATHGoogle Scholar
  27. Ditzian Z: Fractional derivatives and best approximation. Acta Mathematica Hungarica 1998, 81(4):323–348. 10.1023/A:1006554907440MathSciNetView ArticleMATHGoogle Scholar
  28. Schumaker LL: Spline Functions: Basic Theory. John Wiley & Sons, New York, NY, USA; 1981:xiv+553. Pure and Applied Mathematics Pure and Applied MathematicsMATHGoogle Scholar
  29. Wang KY, Li LQ: Harmonic Analysis and Approximation on the Unit Sphere. Science Press, Beijing, China; 2000.Google Scholar
  30. Xu Y: Approximation by means of -harmonic polynomials on the unit sphere. Advances in Computational Mathematics 2004, 21(1–2):37–58.MathSciNetView ArticleMATHGoogle Scholar
  31. Smale S, Zhou D-X: Estimating the approximation error in learning theory. Analysis and Applications 2003, 1(1):17–41. 10.1142/S0219530503000089MathSciNetView ArticleMATHGoogle Scholar
  32. Sheng B: Estimates of the norm of the Mercer kernel matrices with discrete orthogonal transforms. Acta Mathematica Hungarica 2009, 122(4):339–355. 10.1007/s10474-008-8037-2MathSciNetView ArticleMATHGoogle Scholar
  33. Loustau S: Aggregation of SVM classifiers using Sobolev spaces. Journal of Machine Learning Research 2008, 9: 1559–1582.MathSciNetMATHGoogle Scholar
  34. Mhaskar HN, Micchelli CA: Degree of approximation by neural and translation networks with a single hidden layer. Advances in Applied Mathematics 1995, 16(2):151–183. 10.1006/aama.1995.1008MathSciNetView ArticleMATHGoogle Scholar
  35. Sheng BH: Approximation of periodic functions by spherical translation networks. Acta Mathematica Sinica. Chinese Series 2007, 50(1):55–62.MathSciNetMATHGoogle Scholar
  36. Sheng B: On the degree of approximation by spherical translations. Acta Mathematicae Applicatae Sinica. English Series 2006, 22(4):671–680. 10.1007/s10255-006-0341-4MathSciNetView ArticleMATHGoogle Scholar
  37. Sheng B, Wang J, Zhou S: A way of constructing spherical zonal translation network operators with linear bounded operators. Taiwanese Journal of Mathematics 2008, 12(1):77–92.MathSciNetGoogle Scholar
  38. Mhaskar HN, Narcowich FJ, Ward JD: Approximation properties of zonal function networks using scattered data on the sphere. Advances in Computational Mathematics 1999, 11(2–3):121–137.MathSciNetView ArticleMATHGoogle Scholar
  39. Mhaskar HN: Weighted quadrature formulas and approximation by zonal function networks on the sphere. Journal of Complexity 2006, 22(3):348–370. 10.1016/j.jco.2005.10.003MathSciNetView ArticleMATHGoogle Scholar
  40. Groemer H: Geometric Applications of Fourier Series and Spherical Harmonics, Encyclopedia of Mathematics and Its Applications. Volume 61. Cambridge University Press, Cambridge, Mass, USA; 1996:xii+329.View ArticleMATHGoogle Scholar
  41. Müller C: Spherical Harmonics, Lecture Notes in Mathematics. Volume 17. Springer, Berlin, Germany; 1966:iv+45.Google Scholar
  42. Lu SZ, Wang KY: Bochner-Riesz Means. Beijing Normal University Press, Beijing, China; 1988.Google Scholar
  43. Wang Y, Cao F: The direct and converse inequalities for jackson-type operators on spherical cap. Journal of Inequalities and Applications 2009, 2009:-16.Google Scholar
  44. Wendland H: Scattered Data Approximation, Cambridge Monographs on Applied and Computational Mathematics. Volume 17. Cambridge University Press, Cambridge, Mass, USA; 2005:x+336.MATHGoogle Scholar
  45. Mhaskar HN, Narcowich FJ, Sivakumar N, Ward JD: Approximation with interpolatory constraints. Proceedings of the American Mathematical Society 2002, 130(5):1355–1364. 10.1090/S0002-9939-01-06240-2MathSciNetView ArticleMATHGoogle Scholar
  46. Narcowich FJ, Ward JD: Scattered data interpolation on spheres: error estimates and locally supported basis functions. SIAM Journal on Mathematical Analysis 2002, 33(6):1393–1410. 10.1137/S0036141001395054MathSciNetView ArticleMATHGoogle Scholar
  47. Brown G, Dai F: Approximation of smooth functions on compact two-point homogeneous spaces. Journal of Functional Analysis 2005, 220(2):401–423. 10.1016/j.jfa.2004.10.005MathSciNetView ArticleMATHGoogle Scholar
  48. Brown G, Feng D, Sheng SY: Kolmogorov width of classes of smooth functions on the sphere . Journal of Complexity 2002, 18(4):1001–1023. 10.1006/jcom.2002.0656MathSciNetView ArticleMATHGoogle Scholar
  49. Dai F: Multivariate polynomial inequalities with respect to doubling weights and weights. Journal of Functional Analysis 2006, 235(1):137–170. 10.1016/j.jfa.2005.09.009MathSciNetView ArticleMATHGoogle Scholar
  50. Mhaskar HN, Narcowich FJ, Ward JD: Spherical Marcinkiewicz-Zygmund inequalities and positive quadrature. Mathematics of Computation 2001, 70(235):1113–1130.MathSciNetView ArticleMATHGoogle Scholar
  51. Belinsky E, Dai F, Ditzian Z: Multivariate approximating averages. Journal of Approximation Theory 2003, 125(1):85–105. 10.1016/j.jat.2003.09.005MathSciNetView ArticleMATHGoogle Scholar
  52. Kamzolov AI: Approximation of functions on the sphere . Serdica 1984, 10(1):3–10.MathSciNetMATHGoogle Scholar
  53. Dai F, Ditzian Z: Cesàro summability and Marchaud inequality. Constructive Approximation 2007, 25(1):73–88. 10.1007/s00365-005-0623-8MathSciNetView ArticleMATHGoogle Scholar
  54. Dai F: Some equivalence theorems with -functionals. Journal of Approximation Theory 2003, 121(1):143–157. 10.1016/S0021-9045(02)00059-XMathSciNetView ArticleMATHGoogle Scholar