# Some Shannon-McMillan Approximation Theorems for Markov Chain Field on the Generalized Bethe Tree

- Kangkang Wang
^{1}Email author and - Decai Zong
^{2}

**2011**:470910

https://doi.org/10.1155/2011/470910

© W. Kangkang and D. Zong 2011

**Received: **26 September 2010

**Accepted: **7 January 2011

**Published: **23 February 2011

## Abstract

A class of small-deviation theorems for the relative entropy densities of arbitrary random field on the generalized Bethe tree are discussed by comparing the arbitrary measure with the Markov measure on the generalized Bethe tree. As corollaries, some Shannon-Mcmillan theorems for the arbitrary random field on the generalized Bethe tree, Markov chain field on the generalized Bethe tree are obtained.

## 1. Introduction and Lemma

Let be a tree which is infinite, connected and contains no circuits. Given any two vertices , there exists a unique path from to with distinct. The distance between and is defined to , the number of edges in the path connecting and . To index the vertices on , we first assign a vertex as the "root" and label it as . A vertex is said to be on the th level if the path linking it to the root has edges. The root is also said to be on the 0th level.

Definition 1.1.

Let be a tree with root , and let be a sequence of positive integers. is said to be a generalized Bethe tree or a generalized Cayley tree if each vertex on the th level has branches to the th level. For example, when and ( ), is rooted Bethe tree on which each vertex has neighboring vertices (see Figure 1, ), and when ( ), is rooted Cayley tree on which each vertex has branches to the next level.

Definition 1.2.

Then will be called a Markov chain field on the tree determined by the stochastic matrix and the distribution .

The convergence of in a sense ( convergence, convergence in probability, or almost sure convergence) is called the Shannon-McMillan theorem or the entropy theorem or the asymptotic equipartition property (AEP) in information theory. The Shannon-McMillan theorem on the Markov chain has been studied extensively (see [2, 3]). In the recent years, with the development of the information theory scholars get to study the Shannon-McMillan theorems for the random field on the tree graph (see [4]). The tree models have recently drawn increasing interest from specialists in physics, probability and information theory. Berger and Ye (see [5]) have studied the existence of entropy rate for G-invariant random fields. Recently, Ye and Berger (see [6]) have also studied the ergodic property and Shannon-McMillan theorem for PPG-invariant random fields on trees. But their results only relate to the convergence in probability. Yang et al. [7–9] have recently studied a.s. convergence of Shannon-McMillan theorems, the limit properties and the asymptotic equipartition property for Markov chains indexed by a homogeneous tree and the Cayley tree, respectively. Shi and Yang (see [10]) have investigated some limit properties of random transition probability for second-order Markov chains indexed by a tree.

In this paper, we study a class of Shannon-McMillan random approximation theorems for arbitrary random fields on the generalized Bethe tree by comparison between the arbitrary measure and Markov measure on the generalized Bethe tree. As corollaries, a class of Shannon-McMillan theorems for arbitrary random fields and the Markov chains field on the generalized Bethe tree are obtained. Finally, some limit properties for the expectation of the random conditional entropy are discussed.

Lemma 1.3.

Proof (see [11]).

Hence can be look on as a type of measures of the deviation between the arbitrary random fields and the Markov chain fields on the generalized Bethe tree.

## 2. Main Results

Theorem 2.1.

where is the natural logarithmic, is expectation with respect to the measure .

Proof.

It is easy to see that (2.20) also holds if from (2.21).

Therefore (2.5) follows from (2.26). Set in (2.4) and (2.5), (2.6) holds naturally.

Corollary 2.2.

Proof.

We take , then . It implies that (2.2) always holds when . Therefore holds. Equation (2.27) follows from (2.3) and (2.6).

## 3. Some Shannon-McMillan Approximation Theorems on the Finite State Space

Corollary 3.1.

Proof.

Imitating the proof of (2.5), (3.2) follows from (1.5), (1.10), (2.2), and (3.10).

Corollary 3.2 (see [9]).

Proof.

Set , then . It implies (2.2) always holds when . Therefore holds. Equation(3.11) follows from (3.12).

Corollary 3.3.

Proof.

It can be obtained that . holds if (see Gray 1990 [13]), therefore . Equation (3.13) follows from (3.12).

Let the initial distribution and joint distribution of with respect to the measure be defined as (1.4) and (1.5), respectively.

We have the following conclusion.

Corollary 3.4.

Proof.

It follows from (2.2) and (3.21) that ; therefore (3.18), (3.19) follow from (3.1), (3.2).

## 4. Some Limit Properties for Expectation of Random Conditional Entropy on the Finite State Space

Lemma 4.1 (see [8]).

where is the stationary distribution determined by .

Theorem 4.2.

Proof.

Equation(4.2) follows from (4.4).

Theorem 4.3.

Proof.

Therefore (4.5) also holds.

## Declarations

### Acknowledgments

The work is supported by the Project of Higher Schools' Natural Science Basic Research of Jiangsu Province of China (09KJD110002). The author would like to thank the referee for his valuable suggestions. Correspondence author: K. Wang, main research interest is strong limit theory in probability theory. D. Zong main research interest is intelligent algorithm.

## Authors’ Affiliations

## References

- Chung KL:
*A Course in Probability Theory*. 2nd edition. Academic Press, New York, NY, USA; 1974:xii+365.MATHGoogle Scholar - Wen L, Weiguo Y:
**An extension of Shannon-McMillan theorem and some limit properties for nonhomogeneous Markov chains.***Stochastic Processes and their Applications*1996,**61**(1):129–145. 10.1016/0304-4149(95)00068-2MATHMathSciNetView ArticleGoogle Scholar - Liu W, Yang W:
**Some extensions of Shannon-McMillan theorem.***Journal of Combinatorics, Information & System Sciences*1996,**21**(3–4):211–223.MATHMathSciNetView ArticleGoogle Scholar - Ye Z, Berger T:
*Information Measures for Discrete Random Fields*. Science Press, Beijing, China; 1998:iv+160.MATHGoogle Scholar - Berger T, Ye ZX:
**Entropic aspects of random fields on trees.***IEEE Transactions on Information Theory*1990,**36**(5):1006–1018. 10.1109/18.57200MATHMathSciNetView ArticleGoogle Scholar - Ye Z, Berger T:
**Ergodicity, regularity and asymptotic equipartition property of random fields on trees.***Journal of Combinatorics, Information & System Sciences*1996,**21**(2):157–184.MATHMathSciNetGoogle Scholar - Yang W:
**Some limit properties for Markov chains indexed by a homogeneous tree.***Statistics & Probability Letters*2003,**65**(3):241–250. 10.1016/j.spl.2003.04.001MATHMathSciNetView ArticleGoogle Scholar - Yang W, Liu W:
**Strong law of large numbers and Shannon-McMillan theorem for Markov chain fields on trees.***IEEE Transactions on Information Theory*2002,**48**(1):313–318. 10.1109/18.971762MATHView ArticleGoogle Scholar - Yang W, Ye Z:
**The asymptotic equipartition property for nonhomogeneous Markov chains indexed by a homogeneous tree.***IEEE Transactions on Information Theory*2007,**53**(9):3275–3280.MATHMathSciNetView ArticleGoogle Scholar - Shi Z, Yang W:
**Some limit properties of random transition probability for second-order nonhomogeneous markov chains indexed by a tree.***Journal of Inequalities and Applications*2009.,**2009:**Google Scholar - Liu W, Yang W:
**Some strong limit theorems for Markov chain fields on trees.***Probability in the Engineering and Informational Sciences*2004,**18**(3):411–422.MATHMathSciNetView ArticleGoogle Scholar - Doob JL:
*Stochastic Processes*. John Wiley & Sons, New York, NY, USA; 1953:viii+654.MATHGoogle Scholar - Gray RM:
*Entropy and Information Theory*. Springer, New York, NY, USA; 1990:xxiv+332.MATHView ArticleGoogle Scholar

## Copyright

This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.