Let \(X=\{x_{j}: j \in \mathbb{N}\}\) be a sequence of independent and identically distributed (i.i.d.) random variables, each of which is uniformly drawn from Ω, and \(y_{j}=f(x_{j}) +\epsilon _{j}\) with \(\epsilon _{j}\) being a random noise. In this section, we investigate the probability that any \(f\in H_{\gamma , \varOmega }\) can be recovered from its samples \(\{(x_{j}, y_{j})\}_{j=1}^{n}\) stably. We will prove that if the random noise satisfies some mild conditions, then, with overwhelming probability, the sampling inequalities (1.1) hold uniformly for all functions in \(H_{\gamma , \varOmega }\) with (noisy) sampling values.
For every fixed \(f \in H_{\gamma , \varOmega }\), we define the random variable
$$ X_{j}(f):= \bigl\vert f(x_{j}) \bigr\vert ^{p}- \int _{\varOmega } \bigl\vert f(x) \bigr\vert ^{p} \,dx, $$
(3.1)
where \(\{x_{j} \}\) is uniformly drawn from Ω. Then one can check that the sequence \(\{X_{j}(f): j\in \mathbb{N}\}\) is independent and the expectation \(\mathbb{E}(X_{j}(f))=0\). Besides, for the variance of \(X_{j}(f)\), we have
$$\begin{aligned} \operatorname{Var} \bigl(X_{j}(f)\bigr) &=\mathbb{E} \bigl(X_{j}(f)^{2}\bigr)-\bigl(\mathbb{E} \bigl(X_{j}(f)\bigr)\bigr)^{2} \\ &=\mathbb{E}\bigl( \bigl\vert f(x_{j}) \bigr\vert ^{2p} \bigr)- \biggl( \int _{\varOmega } \bigl\vert f(x) \bigr\vert ^{p}\,dx \biggr) ^{2} \\ &\le \mathbb{E}\bigl( \bigl\vert f(x_{j}) \bigr\vert ^{2p}\bigr) \\ &= \int _{\varOmega } \bigl\vert f(x) \bigr\vert ^{2p}\,dx \\ &\le \|f\|_{L_{\infty }(\varOmega )}^{2p}. \end{aligned}$$
The following Bernstein inequality plays an important role in probability theory, which gives bounds on the probability that the sum of independent random variables deviates from the expectation.
Lemma 3.1
([20])
Let
\(\xi _{1}, \xi _{2}, \ldots , \xi _{n}\)
be independent random variables. Assume that
\(\mathbb{E} (\xi _{j})=0\), \(\operatorname{Var} (\xi _{j}) \le \sigma ^{2}\)
and
\(|\xi _{j}|\le K\)
almost surely for all
j. Then, for any
\(\lambda \ge 0\),
$$ \operatorname{Prob} \Biggl( \Biggl\vert \frac{1}{n} \sum _{j=1}^{n} \xi _{j} \Biggr\vert \ge \lambda \Biggr)\le 2\exp \biggl(-\frac{n {\lambda }^{2}}{2\sigma ^{2}+\frac{2}{3} K \lambda } \biggr). $$
Lemma 3.2
Let
\(\{x_{j}:j=1,2, \ldots , n \}\)
be a sequence of i.i.d. random variables that are uniformly drawn from
\(\varOmega =(0, 1)^{d}\). Let
\(H_{\gamma , \varOmega }^{\ast }\)
be defined by (2.1), and
\(X_{j}(f)\)
be given by (3.1). Then, for any
\(\lambda \ge 0\)
and
\(n\in \mathbb{N}\),
$$ \operatorname{Prob} \Biggl(\sup_{f\in H_{\gamma , \varOmega }^{\ast } } \Biggl\vert \frac{1}{n} \sum_{j=1}^{n} X_{j}(f) \Biggr\vert \ge \lambda \Biggr)\le 2 \mathcal{N} \biggl(H_{\gamma , \varOmega }^{\ast }, \frac{\lambda }{2p} \biggr) \exp \biggl( - \frac{ 3n \lambda ^{2}}{24+ 4 \lambda } \biggr). $$
Proof
Let \(\{f_{\ell }\}_{\ell =1, \ldots , L}\), where \(L=\mathcal{N} (H _{\gamma , \varOmega }^{\ast }, \frac{\lambda }{2p} )\), be a sequence in \(H_{\gamma , \varOmega }^{\ast }\) such that \(H_{\gamma , \varOmega }^{ \ast }\) can be covered by the \(L_{\infty }\) balls centered at \(f_{\ell }\) with radius \(\frac{\lambda }{2p}\). For each fixed \(f_{\ell } \in H_{\gamma , \varOmega }^{\ast }\), since \(\|f_{\ell }\| _{L_{\infty }(\varOmega )} = 1\), we have \(\operatorname{Var} (X_{j}(f_{\ell })) \le 1\) and \(|X_{j}(f_{\ell })| \le 1\). By Lemma 3.1, we obtain
$$ \operatorname{Prob} \Biggl( \Biggl\vert \frac{1}{n} \sum _{j=1}^{n} X_{j}(f_{\ell }) \Biggr\vert \ge \lambda \Biggr)\le 2\exp \biggl(-\frac{n {\lambda }^{2}}{2+\frac{2}{3} \lambda } \biggr). $$
(3.2)
For any given \(f\in H_{\gamma , \varOmega }^{\ast }\), there exists some \(\ell \in \{1, 2, \ldots , L\}\) such that \(\|f-f_{\ell }\|_{L_{\infty }} \le \frac{\lambda }{2p}\). Thus, by the mean value theorem,
$$\begin{aligned} & \Biggl\vert \frac{1}{n} \sum_{j=1}^{n} X_{j}(f) -\frac{1}{n} \sum_{j=1}^{n} X _{j}(f_{\ell }) \Biggr\vert \\ &\quad = \Biggl\vert \frac{1}{n} \sum_{j=1}^{n} \bigl( \bigl\vert f(x_{j}) \bigr\vert ^{p} - \bigl\vert f_{\ell }(x_{j}) \bigr\vert ^{p} \bigr) \Biggr\vert \\ &\quad \le p\bigl(\max \bigl\{ \Vert f \Vert _{L_{\infty }(\varOmega )}, \Vert f_{\ell } \Vert _{L_{\infty }(\varOmega )} \bigr\} \bigr)^{p-1} \Vert f-f_{\ell } \Vert _{L_{\infty }(\varOmega )} \\ &\quad \le p \Vert f-f_{\ell } \Vert _{L_{\infty }(\varOmega )} \\ &\quad \le \frac{\lambda }{2}. \end{aligned}$$
Combining this with (3.2), we conclude that, for each fixed ℓ, \(1\le \ell \le L\),
$$\begin{aligned} & \operatorname{Prob} \Biggl\{ \sup_{ \{f: \|f-f_{\ell }\|_{L_{\infty }} \le \frac{\lambda }{2p} \}} \Biggl\vert \frac{1}{n} \sum_{j=1}^{n} X_{j}(f) \Biggr\vert \ge \lambda \Biggr\} \\ &\quad \le \operatorname{Prob} \Biggl\{ \Biggl\vert \frac{1}{n} \sum _{j=1}^{n} X_{j}(f_{ \ell }) \Biggr\vert \ge \lambda - \frac{\lambda }{2} \Biggr\} \\ &\quad \le 2 \exp \biggl( -\frac{ 3n \lambda ^{2}}{24+ 4 \lambda } \biggr) . \end{aligned}$$
Besides, since
$$ H_{\gamma , \varOmega }^{\ast } \subseteq \bigcup_{\ell =1}^{L} \biggl\{ f: \|f-f _{\ell } \|_{L_{\infty }} \le \frac{\lambda }{2p} \biggr\} , $$
we obtain
$$ \operatorname{Prob} \Biggl(\sup_{f\in H_{\gamma , \varOmega }^{\ast } } \Biggl\vert \frac{1}{n} \sum_{j=1}^{n} X_{j}(f) \Biggr\vert \ge \lambda \Biggr)\le \sum _{\ell =1} ^{L} \operatorname{Prob} \Biggl\{ \sup _{ \{f: \|f-f_{\ell }\|_{L_{\infty }} \le \frac{\lambda }{2p} \}} \Biggl\vert \frac{1}{n} \sum _{j=1}^{n} X_{j}(f) \Biggr\vert \ge \lambda \Biggr\} . $$
Therefore, noting that \(L=\mathcal{N} (H_{\gamma , \varOmega }^{\ast }, \frac{ \lambda }{2p} )\), we conclude that
$$ \operatorname{Prob} \Biggl(\sup_{f\in H_{\gamma , \varOmega }^{\ast } } \Biggl\vert \frac{1}{n} \sum_{j=1}^{n} X_{j}(f) \Biggr\vert \ge \lambda \Biggr)\le 2 \mathcal{N} \biggl(H_{\gamma , \varOmega }^{\ast }, \frac{\lambda }{2p} \biggr) \exp \biggl( - \frac{ 3n \lambda ^{2}}{24+ 4 \lambda } \biggr) . $$
□
Theorem 3.3
Let
\(H_{\gamma , \varOmega }\)
be defined by (1.2). Assume that
\(\{x_{j}: j=1, 2, \ldots , n\}\)
is a sequence of i.i.d. random variables that are uniformly drawn from
Ω. Then, for any
\(0< \lambda <\frac{1}{(p+1)^{d} \gamma ^{d}}\), the following sampling inequalities:
$$ n \bigl(1- (p+1)^{d} \gamma ^{d} \lambda \bigr) \int _{\varOmega } \bigl\vert f(x) \bigr\vert ^{p} \,dx \le \sum_{j=1}^{n} \bigl\vert f(x_{j}) \bigr\vert ^{p} \le n \bigl(1+ (p+1)^{d} \gamma ^{d} \lambda \bigr) \int _{\varOmega } \bigl\vert f(x) \bigr\vert ^{p} \,dx $$
(3.3)
hold uniformly for all functions in
\(H_{\gamma , \varOmega }\)
with probability at least
$$ 1- 2 \exp \biggl( \ln 2 + \frac{4p}{\lambda } +\biggl( \frac{8p \gamma }{ \lambda }\biggr) ^{d} \ln 3 -\frac{ 3n \lambda ^{2}}{24+ 4 \lambda } \biggr). $$
Proof
Obviously that every \(f\in H_{\gamma , \varOmega }\) satisfies (3.3) if and only if \(f/\|f\|_{L_{\infty }}\) does. Thus, we assume that \(\|f\|_{L_{\infty }}=1\) and \(f \in H_{\gamma , \varOmega }^{\ast }\).
Let \(X_{j}(f)\) be defined by (3.1), and one can check that the event
$$ \mathcal{E}=\Biggl\{ \sup_{f\in H_{\gamma , \varOmega }^{\ast } } \Biggl\vert \frac{1}{n} \sum_{j=1}^{n} X_{j}(f) \Biggr\vert \ge \lambda \Biggr\} $$
is the complement of
$$ \tilde{\mathcal{E}}=\Biggl\{ n \int _{\varOmega } \bigl\vert f(x) \bigr\vert ^{p} \,dx- \lambda n \le \sum_{j=1}^{n} \bigl\vert f(x_{j}) \bigr\vert ^{p}\le n \int _{\varOmega } \bigl\vert f(x) \bigr\vert ^{p} \,dx+ \lambda n, \forall f\in H_{\gamma , \varOmega }^{\ast } \Biggr\} . $$
For an arbitrary \(f\in H_{\gamma , \varOmega }^{\ast }\), there exists an \(x^{\star } \in \overline{\varOmega }\) such that \(|f(x^{\star })|= \|f\| _{L_{\infty }}=1\). Without loss of generality, we assume that \(f(x^{\star })=1\). Then, for \(x\in \varOmega ^{\ast }\), where \(\varOmega ^{ \ast }:=\{x\in \varOmega , |x-x^{\star } |\le \frac{1}{\gamma } \}\), we have
$$ f\bigl(x^{\star }\bigr)-f(x)= \bigl\vert f(x)- f\bigl(x^{\star } \bigr) \bigr\vert \le \gamma \bigl\vert x-x^{\star } \bigr\vert \le 1. $$
It follows that
$$ f(x) \ge 1- \gamma \bigl\vert x-x^{\star } \bigr\vert \ge 0 \quad \mbox{for } x \in \varOmega ^{\ast } $$
and
$$ \int _{\varOmega } \bigl\vert f(x) \bigr\vert ^{p} \,dx \ge \int _{\varOmega ^{\ast }} \bigl(1- \gamma \bigl\vert x-x^{\star } \bigr\vert \bigr)^{p} \,dx \ge \frac{1}{(p+1)^{d} \gamma ^{d}} . $$
Therefore, the event
$$\begin{aligned} \bar{\mathcal{E}} =& \Biggl\{ n \bigl(1- (p+1)^{d} \gamma ^{d} \lambda \bigr) \int _{\varOmega } \bigl\vert f(x) \bigr\vert ^{p} \,dx \le \sum_{j=1}^{n} \bigl\vert f(x_{j}) \bigr\vert ^{p} \le n \bigl(1+ (p+1)^{d} \gamma ^{d} \lambda \bigr) \int _{\varOmega } \bigl\vert f(x) \bigr\vert ^{p} \,dx, \\ &{} \forall f\in H_{\gamma , \varOmega } ^{\ast } \Biggr\} \end{aligned}$$
contains the event \(\tilde{\mathcal{E}}\).
Thus, by Lemma 3.2 and Proposition 2.1, the inequalities (3.3) hold uniformly for all functions in \(H_{\gamma , \varOmega }\) with probability
$$\begin{aligned} \operatorname{Prob} (\bar{\mathcal{E}}) \ge& \operatorname{Prob} (\tilde{ \mathcal{E}})=1- \operatorname{Prob} ({\mathcal{E}}) \\ \ge& 1- 2 \exp \biggl( \ln 2 + \frac{4p}{ \lambda } +\biggl( \frac{8p \gamma }{\lambda }\biggr) ^{d} \ln 3 - \frac{ 3n \lambda ^{2}}{24+ 4 \lambda } \biggr) . \end{aligned}$$
□
Corollary 3.4
Under the same conditions of Theorem 3.3, let
\(y_{j}=f(x _{j}) +\epsilon _{j}\), \(j=1, 2, \ldots , n\)
be the sampling of
f. Suppose that the random noise
\(\{\epsilon _{j}\}\)
are independent with
\(E(|\epsilon _{j}|^{p})=\sigma ^{p}\)
and
\(| |\epsilon _{j}|^{p} - \sigma ^{p} | \le M \|f\|_{L_{p}}^{p}\)
for all
\(\epsilon _{j}\). In addition, we assume that
\(\frac{\sigma ^{p} }{\|f\|_{L_{p}}^{p} } \le \rho \ll 1\). Then, for any
\(0< \lambda <\frac{2^{1-p} -\rho }{2^{1-p} (p+1)^{d} \gamma ^{d} +1}\), the inequalities
$$\begin{aligned} & n \bigl(2^{1-p}-\rho - \bigl( 2^{1-p} (p+1)^{d} \gamma ^{d} +1 \bigr) \lambda \bigr) \int _{\varOmega } \bigl\vert f(x) \bigr\vert ^{p} \,dx \\ &\quad \le \sum_{j=1}^{n} \bigl\vert f(x_{j})+ \epsilon _{j} \bigr\vert ^{p} \\ &\quad \le 2^{p-1} n \bigl(1+\rho + \bigl( (p+1)^{d} \gamma ^{d}+1 \bigr) \lambda \bigr) \int _{\varOmega } \bigl\vert f(x) \bigr\vert ^{p} \,dx \end{aligned}$$
(3.4)
hold uniformly for all functions in
\(H_{\gamma , \varOmega }\)
with probability at least
$$ \biggl( 1-2 \exp \biggl(-\frac{n \lambda ^{2} }{2 M^{2}}\biggr) \biggr) \biggl( 1- 2 \exp \biggl( \ln 2 + \frac{4p}{\lambda } +\biggl( \frac{8p \gamma }{\lambda }\biggr) ^{d} \ln 3 -\frac{ 3n \lambda ^{2}}{24+ 4 \lambda } \biggr) \biggr). $$
Proof
One can check that every \(f\in H_{\gamma , \varOmega }\) satisfies the inequalities of (3.4) if and only if \(f/\|f\|_{L_{p}}\) does. Thus, we assume that \(\|f\|_{L_{p}}=1\). By Hoeffding’s inequality [21], we have
$$ \operatorname{Prob} \Biggl\{ \Biggl\vert \frac{1}{n} \sum _{j=1}^{n} |\epsilon _{j} |^{p} - \sigma ^{p} \Biggr\vert \ge \lambda \Biggr\} \le 2 \exp \biggl(- \frac{n \lambda ^{2}}{2 M^{2}}\biggr). $$
So, with probability \(1-2 \exp (-\frac{n \lambda ^{2} }{2 M^{2}})\),
$$ \frac{1}{n} \sum_{j=1}^{n} |\epsilon _{j} |^{p} \le \sigma ^{p}+ \lambda \|f \|_{L_{p}}^{p}. $$
For \(1\le p <\infty\), since \(t^{p}\) is a convex function of t on \([0, + \infty )\), by Jensen’s inequality, we have
$$ \bigl\vert f(x_{j})+ \epsilon _{j} \bigr\vert ^{p} \le \bigl( \bigl\vert f(x_{j}) \bigr\vert + \vert \epsilon _{j} \vert \bigr)^{p} \le 2^{p-1} \bigl( \bigl\vert f(x_{j}) \bigr\vert ^{p} + \vert \epsilon _{j} \vert ^{p} \bigr) $$
and
$$ \bigl\vert f(x_{j})+ \epsilon _{j} \bigr\vert ^{p} \ge \bigl( \bigl\vert f(x_{j}) \bigr\vert - \vert \epsilon _{j} \vert \bigr)^{p} \ge 2^{1-p} \bigl\vert f(x_{j}) \bigr\vert ^{p} - \vert \epsilon _{j} \vert ^{p} . $$
Hence, with the same probability, we have
$$ \sum_{j=1}^{n} \bigl\vert f(x_{j})+ \epsilon _{j} \bigr\vert ^{p} \le 2^{p-1} \sum_{j=1} ^{n} \bigl\vert f(x_{j}) \bigr\vert ^{p} + 2^{p-1} n \bigl( \sigma ^{p}+ \lambda \|f\|_{L_{p}} ^{p}\bigr) $$
and
$$ \sum_{j=1}^{n} \bigl\vert f(x_{j})+ \epsilon _{j} \bigr\vert ^{p} \ge 2^{1-p} \sum_{j=1} ^{n} \bigl\vert f(x_{j}) \bigr\vert ^{p} - n\bigl( \sigma ^{p}+ \lambda \|f\|_{L_{p}}^{p}\bigr). $$
Combining this with Theorem 3.3, we conclude that
$$\begin{aligned} & n \bigl(2^{1-p}-\rho - \bigl( 2^{1-p} (p+1)^{d} \gamma ^{d} +1 \bigr) \lambda \bigr) \int _{\varOmega } \bigl\vert f(x) \bigr\vert ^{p} \,dx \\ &\quad \le \sum_{j=1}^{n} \bigl\vert f(x_{j})+ \epsilon _{j} \bigr\vert ^{p} \\ &\quad \le 2^{p-1} n \bigl(1+\rho + \bigl( (p+1)^{d} \gamma ^{d}+1 \bigr) \lambda \bigr) \int _{\varOmega } \bigl\vert f(x) \bigr\vert ^{p} \,dx \end{aligned}$$
holds with probability at least
$$ \biggl( 1-2 \exp \biggl(-\frac{n \lambda ^{2} }{2 M^{2}}\biggr) \biggr) \biggl( 1- 2 \exp \biggl( \ln 2 + \frac{4p}{\lambda } +\biggl( \frac{8p \gamma }{\lambda }\biggr) ^{d} \ln 3 -\frac{ 3n \lambda ^{2}}{24+ 4 \lambda } \biggr) \biggr). $$
□
We remark that ρ in Corollary 3.4 is connected with the signal-to-noise ratio (SNR) of f, and the signal can be recovered from its noisy samples stably only if the noise level is relatively small.