Some Shannon-McMillan Approximation Theorems for Markov Chain Field on the Generalized Bethe Tree
© W. Kangkang and D. Zong 2011
Received: 26 September 2010
Accepted: 7 January 2011
Published: 23 February 2011
A class of small-deviation theorems for the relative entropy densities of arbitrary random field on the generalized Bethe tree are discussed by comparing the arbitrary measure with the Markov measure on the generalized Bethe tree. As corollaries, some Shannon-Mcmillan theorems for the arbitrary random field on the generalized Bethe tree, Markov chain field on the generalized Bethe tree are obtained.
1. Introduction and Lemma
Let be a tree which is infinite, connected and contains no circuits. Given any two vertices , there exists a unique path from to with distinct. The distance between and is defined to , the number of edges in the path connecting and . To index the vertices on , we first assign a vertex as the "root" and label it as . A vertex is said to be on the th level if the path linking it to the root has edges. The root is also said to be on the 0th level.
Let be a tree with root , and let be a sequence of positive integers. is said to be a generalized Bethe tree or a generalized Cayley tree if each vertex on the th level has branches to the th level. For example, when and ( ), is rooted Bethe tree on which each vertex has neighboring vertices (see Figure 1, ), and when ( ), is rooted Cayley tree on which each vertex has branches to the next level.
The convergence of in a sense ( convergence, convergence in probability, or almost sure convergence) is called the Shannon-McMillan theorem or the entropy theorem or the asymptotic equipartition property (AEP) in information theory. The Shannon-McMillan theorem on the Markov chain has been studied extensively (see [2, 3]). In the recent years, with the development of the information theory scholars get to study the Shannon-McMillan theorems for the random field on the tree graph (see ). The tree models have recently drawn increasing interest from specialists in physics, probability and information theory. Berger and Ye (see ) have studied the existence of entropy rate for G-invariant random fields. Recently, Ye and Berger (see ) have also studied the ergodic property and Shannon-McMillan theorem for PPG-invariant random fields on trees. But their results only relate to the convergence in probability. Yang et al. [7–9] have recently studied a.s. convergence of Shannon-McMillan theorems, the limit properties and the asymptotic equipartition property for Markov chains indexed by a homogeneous tree and the Cayley tree, respectively. Shi and Yang (see ) have investigated some limit properties of random transition probability for second-order Markov chains indexed by a tree.
In this paper, we study a class of Shannon-McMillan random approximation theorems for arbitrary random fields on the generalized Bethe tree by comparison between the arbitrary measure and Markov measure on the generalized Bethe tree. As corollaries, a class of Shannon-McMillan theorems for arbitrary random fields and the Markov chains field on the generalized Bethe tree are obtained. Finally, some limit properties for the expectation of the random conditional entropy are discussed.
Proof (see ).
2. Main Results
We take , then . It implies that (2.2) always holds when . Therefore holds. Equation (2.27) follows from (2.3) and (2.6).
3. Some Shannon-McMillan Approximation Theorems on the Finite State Space
Imitating the proof of (2.5), (3.2) follows from (1.5), (1.10), (2.2), and (3.10).
Corollary 3.2 (see ).
Set , then . It implies (2.2) always holds when . Therefore holds. Equation(3.11) follows from (3.12).
We have the following conclusion.
4. Some Limit Properties for Expectation of Random Conditional Entropy on the Finite State Space
Lemma 4.1 (see ).
Equation(4.2) follows from (4.4).
Therefore (4.5) also holds.
The work is supported by the Project of Higher Schools' Natural Science Basic Research of Jiangsu Province of China (09KJD110002). The author would like to thank the referee for his valuable suggestions. Correspondence author: K. Wang, main research interest is strong limit theory in probability theory. D. Zong main research interest is intelligent algorithm.
- Chung KL: A Course in Probability Theory. 2nd edition. Academic Press, New York, NY, USA; 1974:xii+365.MATHGoogle Scholar
- Wen L, Weiguo Y: An extension of Shannon-McMillan theorem and some limit properties for nonhomogeneous Markov chains. Stochastic Processes and their Applications 1996,61(1):129–145. 10.1016/0304-4149(95)00068-2MATHMathSciNetView ArticleGoogle Scholar
- Liu W, Yang W: Some extensions of Shannon-McMillan theorem. Journal of Combinatorics, Information & System Sciences 1996,21(3–4):211–223.MATHMathSciNetView ArticleGoogle Scholar
- Ye Z, Berger T: Information Measures for Discrete Random Fields. Science Press, Beijing, China; 1998:iv+160.MATHGoogle Scholar
- Berger T, Ye ZX: Entropic aspects of random fields on trees. IEEE Transactions on Information Theory 1990,36(5):1006–1018. 10.1109/18.57200MATHMathSciNetView ArticleGoogle Scholar
- Ye Z, Berger T: Ergodicity, regularity and asymptotic equipartition property of random fields on trees. Journal of Combinatorics, Information & System Sciences 1996,21(2):157–184.MATHMathSciNetGoogle Scholar
- Yang W: Some limit properties for Markov chains indexed by a homogeneous tree. Statistics & Probability Letters 2003,65(3):241–250. 10.1016/j.spl.2003.04.001MATHMathSciNetView ArticleGoogle Scholar
- Yang W, Liu W: Strong law of large numbers and Shannon-McMillan theorem for Markov chain fields on trees. IEEE Transactions on Information Theory 2002,48(1):313–318. 10.1109/18.971762MATHView ArticleGoogle Scholar
- Yang W, Ye Z: The asymptotic equipartition property for nonhomogeneous Markov chains indexed by a homogeneous tree. IEEE Transactions on Information Theory 2007,53(9):3275–3280.MATHMathSciNetView ArticleGoogle Scholar
- Shi Z, Yang W: Some limit properties of random transition probability for second-order nonhomogeneous markov chains indexed by a tree. Journal of Inequalities and Applications 2009., 2009:Google Scholar
- Liu W, Yang W: Some strong limit theorems for Markov chain fields on trees. Probability in the Engineering and Informational Sciences 2004,18(3):411–422.MATHMathSciNetView ArticleGoogle Scholar
- Doob JL: Stochastic Processes. John Wiley & Sons, New York, NY, USA; 1953:viii+654.MATHGoogle Scholar
- Gray RM: Entropy and Information Theory. Springer, New York, NY, USA; 1990:xxiv+332.MATHView ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.