Some inequalities on generalized entropies

Abstract

We give several inequalities on generalized entropies involving Tsallis entropies, using some inequalities obtained by the improvements of Young’s inequality. We also give a generalized Han’s inequality.

MSC:26D15, 94A17.

1 Introduction

Generalized entropies have been studied by many researchers (we refer the interested reader to [1, 2]). Rényi [3] and Tsallis [4] entropies are well known as one-parameter generalizations of Shannon’s entropy, being intensively studied not only in the field of classical statistical physics [5–7], but also in the field of quantum physics in relation to the entanglement [8–11]. The Tsallis entropy is a natural one-parameter extended form of the Shannon entropy, hence it can be applied to known models which describe systems of great interest in atomic physics [12]. However, to our best knowledge, the physical relevance of a parameter of the Tsallis entropy was highly debated and it has not been completely clarified yet, the parameter being considered as a measure of the non-extensivity of the system under consideration. One of the authors of the present paper studied the Tsallis entropy and the Tsallis relative entropy from the mathematical point of view. Firstly, fundamental properties of the Tsallis relative entropy were discussed in [13]. The uniqueness theorem for the Tsallis entropy and Tsallis relative entropy was studied in [14]. Following this result, an axiomatic characterization of a two-parameter extended relative entropy was given in [15]. In [16], information theoretical properties of the Tsallis entropy and some inequalities for conditional and joint Tsallis entropies were derived. These entropies are again used in the present paper, to derive the generalized Han’s inequality. In [17], matrix trace inequalities for the Tsallis entropy were studied. And in [18], the maximum entropy principle for the Tsallis entropy and the minimization of the Fisher information in Tsallis statistics were studied. Quite recently, we provided mathematical inequalities for some divergences in [19], considering that it is important to study the mathematical inequalities for the development of new entropies. In this paper, we define a further generalized entropy based on Tsallis and Rényi entropies and study mathematical properties by the use of scalar inequalities to develop the theory of entropies.

We start from the weighted quasilinear mean for some continuous and strictly monotonic function $\psi :I\to \mathbb{R}$, defined by

$$M_{\psi }(x_1,x_2,\dots ,x_n)\equiv {\psi }^{-1}(\sum _{j=1}^n{p}_j\psi (x_j)),$$

(1)

where ${\sum }_{j=1}^n{p}_j=1$, $p_j>0$, $x_j\in I$ for $j=1,2,\dots ,n$ and $n\in \mathbb{N}$. If we take $\psi (x)=x$, then $M_{\psi }(x_1,x_2,\dots ,x_n)$ coincides with the weighted arithmetic mean $A(x_1,x_2,\dots ,x_n)\equiv {\sum }_{j=1}^n{p}_j{x}_j$. If we also take $\psi (x)=log(x)$, then $M_{\psi }(x_1,x_2,\dots ,x_n)$ coincides with the weighted geometric mean $G(x_1,x_2,\dots ,x_n)\equiv {\prod }_{j=1}^n{x}_j^{p_j}$.

If $\psi (x)=x$ and $x_j={ln}_q\frac{1}{p_j}$, then $M_{\psi }(x_1,x_2,\dots ,x_n)$ is equal to the Tsallis entropy [4]:

$$H_q(p_1,p_2,\dots ,p_n)\equiv -\sum _{j=1}^n{p}_j^q{ln}_q{p}_j=\sum _{j=1}^n{p}_j{ln}_q\frac{1}{p_j}(q\ge 0,q\ne 1),$$

(2)

where $\{p_1,p_2,\dots ,p_n\}$ is a probability distribution with $p_j>0$ for all $j=1,2,\dots ,n$ and the q-logarithmic function for $x>0$ is defined by ${ln}_q(x)\equiv \frac{x^{1-q}-1}{1-q}$ which uniformly converges to the usual logarithmic function $log(x)$ in the limit $q\to 1$. Therefore, the Tsallis entropy converges to the Shannon entropy in the limit $q\to 1$:

$$\underset{q\to 1}{lim}{H}_q(p_1,p_2,\dots ,p_n)=H_1(p_1,p_2,\dots ,p_n)\equiv -\sum _{j=1}^n{p}_{j}log{p}_j.$$

(3)

Thus, we find that the Tsallis entropy is one of generalizations of the Shannon entropy. It is known that the Rényi entropy [3] is also a generalization of the Shannon entropy. Here, we review a quasilinear entropy [1] as another generalization of the Shannon entropy. For a continuous and strictly monotonic function ϕ on $(0,1]$, the quasilinear entropy is given by

$$I^{\varphi }(p_1,p_2,\dots ,p_n)\equiv -log{\varphi }^{-1}(\sum _{j=1}^n{p}_j\varphi (p_j)).$$

(4)

If we take $\varphi (x)=log(x)$ in (4), then we have $I^{log}(p_1,p_2,\dots ,p_n)=H_1(p_1,p_2,\dots ,p_n)$. We may redefine the quasilinear entropy by

$$I_1^{\psi }(p_1,p_2,\dots ,p_n)\equiv log{\psi }^{-1}\left(\sum _{j=1}^n{p}_j\psi \left(\frac{1}{p_j}\right)\right)$$

(5)

for a continuous and strictly monotonic function ψ on $(0,\mathrm{\infty })$. If we take $\psi (x)=log(x)$ in (5), we have $I_1^{log}(p_1,p_2,\dots ,p_n)=H_1(p_1,p_2,\dots ,p_n)$. The case $\psi (x)=x^{1-q}$ is also useful in practice, since we recapture the Rényi entropy, namely $I_1^{x^{1-q}}(p_1,p_2,\dots ,p_n)=R_q(p_1,p_2,\dots ,p_n)$ where the Rényi entropy [3] is defined by

$$R_q(p_1,p_2,\dots ,p_n)\equiv \frac{1}{1-q}log\left(\sum _{j=1}^n{p}_j^q\right).$$

(6)

From a viewpoint of application on source coding, the relation between the weighted quasilinear mean and the Rényi entropy has been studied in Chapter 5 of [1] in the following way.

Theorem A ([1])

For all real numbers $q>0$ and integers $D>1$, there exists a code $(x_1,x_2,\dots ,x_n)$ such that

$$\frac{R_q(p_1,p_2,\dots ,p_n)}{logD}\le M_{D^{\frac{1-q}{q}x}}(x_1,x_2,\dots ,x_n)<\frac{R_q(p_1,p_2,\dots ,p_n)}{logD}+1,$$

(7)

where the exponential function $D^{\frac{1-q}{q}x}$ is defined on $[1,\mathrm{\infty })$.

By simple calculations, we find that

$$\underset{q\to 1}{lim}{M}_{D^{\frac{1-q}{q}x}}(x_1,x_2,\dots ,x_n)=\sum _{j=1}^n{p}_j{x}_j$$

and

$$\underset{q\to 1}{lim}\frac{R_q(p_1,p_2,\dots ,p_n)}{logD}=-\sum _{j=1}^n{p}_j{log}_D{p}_j.$$

Therefore, Theorem A appears as a generalization of the famous Shannon’s source coding theorem:

$$-\sum _{j=1}^n{p}_j{log}_D{p}_j\le \sum _{j=1}^n{p}_j{x}_j<-\sum _{j=1}^n{p}_j{log}_D{p}_j+1.$$

Motivated by the above results and recent advances on the Tsallis entropy theory, we investigate the mathematical results for generalized entropies involving Tsallis entropies and quasilinear entropies, using some inequalities obtained by improvements of Young’s inequality.

Definition 1.1 For a continuous and strictly monotonic function ψ on $(0,\mathrm{\infty })$ and two probability distributions $\{p_1,p_2,\dots ,p_n\}$ and $\{r_1,r_2,\dots ,r_n\}$ with $p_j>0$, $r_j>0$ for all $j=1,2,\dots ,n$, the quasilinear relative entropy is defined by

$$D_1^{\psi }(p_1,p_2,\dots ,p_n\parallel r_1,r_2,\dots ,r_n)\equiv -log{\psi }^{-1}\left(\sum _{j=1}^n{p}_j\psi \left(\frac{r_j}{p_j}\right)\right).$$

(8)

The quasilinear relative entropy coincides with the Shannon relative entropy if $\psi (x)=log(x)$, i.e.,

$$D_1^{log}(p_1,p_2,\dots ,p_n\parallel r_1,r_2,\dots ,r_n)=-\sum _{j=1}^n{p}_{j}log\frac{r_j}{p_j}=D_1(p_1,p_2,\dots ,p_n\parallel r_1,r_2,\dots ,r_n).$$

We denote by $R_q(p_1,p_2,\dots ,p_n\parallel r_1,r_2,\dots ,r_n)$ the Rényi relative entropy [3] defined by

$$R_q(p_1,p_2,\dots ,p_n\parallel r_1,r_2,\dots ,r_n)\equiv \frac{1}{q-1}log\left(\sum _{j=1}^n{p}_j^q{r}_j^{1-q}\right).$$

(9)

This is another particular case of the quasilinear relative entropy, namely for $\psi (x)=x^{1-q}$we have

$$\begin{array}{rcl}{D}_1^{x^{1-q}}(p_1,p_2,\dots ,p_n\parallel r_1,r_2,\dots ,r_n)& =& -log{\left(\sum _{j=1}^n{p}_j{\left(\frac{r_j}{p_j}\right)}^{1-q}\right)}^{\frac{1}{1-q}}=\frac{1}{q-1}log\left(\sum _{j=1}^n{p}_j^q{r}_j^{1-q}\right)\\ =& R_q(p_1,p_2,\dots ,p_n\parallel r_1,r_2,\dots ,r_n).\end{array}$$

We denote by

$$D_q(p_1,p_2,\dots ,p_n\parallel r_1,r_2,\dots ,r_n)\equiv \sum _{j=1}^n{p}_j^q({ln}_q{p}_j-{ln}_q{r}_j)=-\sum _{j=1}^n{p}_j{ln}_q\frac{r_j}{p_j}$$

(10)

the Tsallis relative entropy which converges to the usual relative entropy (divergence, K-L information) in the limit $q\to 1$:

$$\begin{array}{rcl}\underset{q\to 1}{lim}{D}_q(p_1,p_2,\dots ,p_n\parallel r_1,r_2,\dots ,r_n)& =& D_1(p_1,p_2,\dots ,p_n\parallel r_1,r_2,\dots ,r_n)\\ \equiv & \sum _{j=1}^n{p}_j(log{p}_j-log{r}_j).\end{array}$$

(11)

See [2, 5–7, 13–20] and references therein for recent advances and applications on the Tsallis entropy. We easily find that the Tsallis relative entropy is a special case of Csiszár f-divergence [21–23] defined for a convex function f on $(0,\mathrm{\infty })$ with $f(1)=0$ by

$$D_f(p_1,p_2,\dots ,p_n\parallel r_1,r_2,\dots ,r_n)\equiv \sum _{j=1}^n{r}_{j}f\left(\frac{p_j}{r_j}\right),$$

(12)

since $f(x)=-x{ln}_q(1/x)$ is convex on $(0,\mathrm{\infty })$, vanishes at $x=1$ and

$$D_{-x{ln}_q(1/x)}(p_1,p_2,\dots ,p_n\parallel r_1,r_2,\dots ,r_n)=D_q(p_1,p_2,\dots ,p_n\parallel r_1,r_2,\dots ,r_n).$$

Furthermore, we define the dual function with respect to a convex function f by

$$f^{\ast }(t)=tf\left(\frac{1}{t}\right)$$

(13)

for $t>0$. Then the function $f^{\ast }(t)$ is also convex on $(0,\mathrm{\infty })$. In addition, we define the f-divergence for incomplete probability distributions $\{a_1,a_2,\dots ,a_n\}$ and $\{b_1,b_2\dots ,b_n\}$ where $a_i>0$ and $b_i>0$, in the following way:

$$\widetilde{D_{f^{\ast }}}(a_1,a_2,\dots ,a_n\parallel b_1,b_2,\dots ,b_n)\equiv \sum _{j=1}^n{a}_j{f}^{\ast }\left(\frac{b_j}{a_j}\right).$$

(14)

On the other hand, the studies on refinements for Young’s inequality have given a great progress in the papers [24–35]. In the present paper, we give some inequalities on Tsallis entropies applying two types of inequalities obtained in [29, 32]. In addition, we give the generalized Han’s inequality for the Tsallis entropy in the final section.

2 Tsallis quasilinear entropy and Tsallis quasilinear relative entropy

As an analogy with (5), we may define the following entropy.

Definition 2.1 For a continuous and strictly monotonic function ψ on $(0,\mathrm{\infty })$ and $q\ge 0$ with $q\ne 1$, the Tsallis quasilinear entropy (q-quasilinear entropy) is defined by

$$I_q^{\psi }(p_1,p_2,\dots ,p_n)\equiv {ln}_q{\psi }^{-1}\left(\sum _{j=1}^n{p}_j\psi \left(\frac{1}{p_j}\right)\right),$$

(15)

where $\{p_1,p_2,\dots ,p_n\}$ is a probability distribution with $p_j>0$ for all $j=1,2,\dots ,n$.

We notice that if ψ does not depend on q, then ${lim}_{q\to 1}{I}_q^{\psi }(p_1,p_2,\dots ,p_n)=I_1^{\psi }(p_1,p_2,\dots ,p_n)$.

For $x>0$ and $q\ge 0$ with $q\ne 1$, we define the q-exponential function as the inverse function of the q-logarithmic function by ${exp}_q(x)\equiv {\{1+(1-q)x\}}^{1/(1-q)}$, if $1+(1-q)x>0$, otherwise it is undefined. If we take $\psi (x)={ln}_q(x)$, then we have $I_q^{{ln}_q}(p_1,p_2,\dots ,p_n)=H_q(p_1,p_2,\dots ,p_n)$. Furthermore, we have

$$\begin{array}{rcl}{I}_q^{x^{1-q}}(p_1,p_2,\dots ,p_n)& =& {ln}_q{\left(\sum _{j=1}^n{p}_j{p}_j^{q-1}\right)}^{\frac{1}{1-q}}={ln}_q{\left(\sum _{j=1}^n{p}_j^q\right)}^{\frac{1}{1-q}}\\ =& \frac{{[{({\sum }_{j=1}^n{p}_j^q)}^{\frac{1}{1-q}}]}^{1-q}-1}{1-q}=\frac{{\sum }_{j=1}^n(p_j^q-p_j)}{1-q}=H_q(p_1,p_2,\dots ,p_n).\end{array}$$

Proposition 2.2 The Tsallis quasilinear entropy is nonnegative:

$$I_q^{\psi }(p_1,p_2,\dots ,p_n)\ge 0.$$

Proof We assume that ψ is an increasing function. Then we have $\psi (\frac{1}{p_j})\ge \psi (1)$ from $\frac{1}{p_j}\ge 1$ for $p_j>0$ for all $j=1,2,\dots ,n$. Thus, we have ${\sum }_{j=1}^n{p}_j\psi (\frac{1}{p_j})\ge \psi (1)$ which implies ${\psi }^{-1}({\sum }_{j=1}^n{p}_j\psi (\frac{1}{p_j}))\ge 1$, since ${\psi }^{-1}$ is also increasing. For the case that ψ is a decreasing function, we can prove it similarly. □

We note here that the q-exponential function gives us the following connection between the Rényi entropy and Tsallis entropy [36]:

$$exp{R}_q(p_1,p_2,\dots ,p_n)={exp}_q{H}_q(p_1,p_2,\dots ,p_n).$$

(16)

We should note here ${exp}_q{H}_q(p_1,p_2,\dots ,p_n)$ is always defined, since we have

$$1+(1-q)H_q(p_1,p_2,\dots ,p_n)=\sum _{j=1}^n{p}_j^q>0.$$

From (16), we have the following proposition.

Proposition 2.3 Let $\mathcal{A}\equiv \{{\mathcal{A}}_i:i=1,2,\dots ,k\}$ be a partition of $\{1,2,\dots ,n\}$ and put $p_i^{\mathcal{A}}\equiv {\sum }_{j\in {\mathcal{A}}_i}{p}_j$. Then we have

(17)

(18)

Proof We use the generalized Shannon additivity (which is often called q-additivity) for the Tsallis entropy (see [14] for example):

$$H_q(x_{11},\dots ,x_{n{m}_n})=H_q(x_1,\dots ,x_n)+\sum _{i=1}^n{x}_i^q{H}_q(\frac{x_{i1}}{x_i},\dots ,\frac{x_{i{m}_i}}{x_i}),$$

(19)

where $x_{ij}\ge 0$, $x_i={\sum }_{j=1}^{m_i}{x}_{ij}$ ($i=1,\dots ,n$; $j=1,\dots ,m_i$). Thus, we have

$$H_q(p_1,p_2,\dots ,p_n)\ge H_q(p_1^{\mathcal{A}},p_2^{\mathcal{A}},\dots ,p_k^{\mathcal{A}}),$$

(20)

since the second term of the right-hand side in (19) is nonnegative because of nonnegativity of the Tsallis entropy. Thus, we have

$$\begin{array}{rcl}exp{R}_q(p_1,p_2,\dots ,p_n)& =& {exp}_q{H}_q(p_1,p_2,\dots ,p_n)\\ \ge & {exp}_q{H}_q(p_1^{\mathcal{A}},p_2^{\mathcal{A}},\dots ,p_k^{\mathcal{A}})\\ =& exp{R}_q(p_1^{\mathcal{A}},p_2^{\mathcal{A}},\dots ,p_k^{\mathcal{A}}),\end{array}$$

since ${exp}_q$ is a monotone increasing function. Hence, the inequality

$$R_q(p_1,p_2,\dots ,p_n)\ge R_q(p_1^{\mathcal{A}},p_2^{\mathcal{A}},\dots ,p_k^{\mathcal{A}})$$

(21)

holds, which proves the present proposition. □

Definition 2.4 For a continuous and strictly monotonic function ψ on $(0,\mathrm{\infty })$ and two probability distributions $\{p_1,p_2,\dots ,p_n\}$ and $\{r_1,r_2,\dots ,r_n\}$ with $p_j>0$, $r_j>0$ for all $j=1,2,\dots ,n$, the Tsallis quasilinear relative entropy is defined by

$$D_q^{\psi }(p_1,p_2,\dots ,p_n\parallel r_1,r_2,\dots ,r_n)\equiv -{ln}_q{\psi }^{-1}\left(\sum _{j=1}^n{p}_j\psi \left(\frac{r_j}{p_j}\right)\right).$$

(22)

For $\psi (x)={ln}_q(x)$, the Tsallis quasilinear relative entropy becomes Tsallis relative entropy, that is,

$$D_q^{{ln}_q}(p_1,p_2,\dots ,p_n\parallel r_1,r_2,\dots ,r_n)=-\sum _{j=1}^n{p}_j{ln}_q\frac{r_j}{p_j}=D_q(p_1,p_2,\dots ,p_n\parallel r_1,r_2,\dots ,r_n),$$

and for $\psi (x)=x^{1-q}$, we have

$$\begin{array}{rcl}{D}_q^{x^{1-q}}(p_1,p_2,\dots ,p_n\parallel r_1,r_2,\dots ,r_n)& =& -{ln}_q{\left(\sum _{j=1}^n{p}_j{\left(\frac{r_j}{p_j}\right)}^{1-q}\right)}^{\frac{1}{1-q}}=-{ln}_q{\left(\sum _{j=1}^n{p}_j^q{r}_j^{1-q}\right)}^{\frac{1}{1-q}}\\ =& \frac{-\{{[{({\sum }_{j=1}^n{p}_j^q{r}_j^{1-q})}^{\frac{1}{1-q}}]}^{1-q}-1\}}{1-q}=\frac{{\sum }_{j=1}^n(p_j-p_j^q{r}_j^{1-q})}{1-q}\\ =& D_q(p_1,p_2,\dots ,p_n\parallel r_1,r_2,\dots ,r_n).\end{array}$$

(23)

We give a sufficient condition on nonnegativity of the Tsallis quasilinear relative entropy.

Proposition 2.5 If ψ is a concave increasing function or a convex decreasing function, then we have nonnegativity of the Tsallis quasilinear relative entropy:

$$D_q^{\psi }(p_1,p_2,\dots ,p_n\parallel r_1,r_2,\dots ,r_n)\ge 0.$$

Proof We firstly assume that ψ is a concave increasing function. The concavity of ψ shows that we have $\psi ({\sum }_{j=1}^n{p}_j\frac{r_j}{p_j})\ge {\sum }_{j=1}^n{p}_j\psi (\frac{r_j}{p_j})$ which is equivalent to $\psi (1)\ge {\sum }_{j=1}^n{p}_j\psi (\frac{r_j}{p_j})$. From the assumption, ${\psi }^{-1}$ is also increasing so that we have $1\ge {\psi }^{-1}({\sum }_{j=1}^n{p}_j\psi (\frac{r_j}{p_j}))$. Therefore, we have $-{ln}_q{\psi }^{-1}({\sum }_{j=1}^n{p}_j\psi (\frac{r_j}{p_j}))\ge 0$, since ${ln}_{q}x$ is increasing and ${ln}_q(1)=0$. For the case that ψ is a convex decreasing function, we can prove similarly nonnegativity of the Tsallis quasilinear relative entropy. □

Remark 2.6 The following two functions satisfy the sufficient condition in the above proposition.

• (i) $\psi (x)={ln}_{q}x$ for $q\ge 0$, $q\ne 1$.

• (ii) $\psi (x)=x^{1-q}$ for $q\ge 0$, $q\ne 1$.

It is notable that the following identity holds:

$$exp{R}_q(p_1,p_2,\dots ,p_n\parallel r_1,r_2,\dots ,r_n)={exp}_{2-q}{D}_q(p_1,p_2,\dots ,p_n\parallel r_1,r_2,\dots ,r_n).$$

(24)

We should note here ${exp}_{2-q}{D}_q(p_1,p_2,\dots ,p_n\parallel r_1,r_2,\dots ,r_n)$ is always defined, since we have

$$1+(q-1)D_q(p_1,p_2,\dots ,p_n\parallel r_1,r_2,\dots ,r_n)=\sum _{j=1}^n{p}_j^q{r}_j^{1-q}>0.$$

We also find that (24) implies the monotonicity of the Rényi relative entropy.

Proposition 2.7 Under the same assumptions as in Proposition 2.3 and $r_i^{\mathcal{A}}\equiv {\sum }_{j\in {\mathcal{A}}_i}{r}_j$, we have

$$R_q(p_1,p_2,\dots ,p_n\parallel r_1,r_2,\dots ,r_n)\ge R_q(p_1^{\mathcal{A}},p_2^{\mathcal{A}},\dots ,p_k^{\mathcal{A}}\parallel r_1^{\mathcal{A}},r_2^{\mathcal{A}},\dots ,r_k^{\mathcal{A}}).$$

(25)

Proof We recall that the Tsallis relative entropy is a special case of f-divergence so that it has the same properties with f-divergence. Since ${exp}_{2-q}$ is a monotone increasing function for $0\le q\le 2$ and f-divergence has a monotonicity [21, 23], we have

$$\begin{array}{rcl}exp{R}_q(p_1,p_2,\dots ,p_n\parallel r_1,r_2,\dots ,r_n)& =& {exp}_{2-q}{D}_q(p_1,p_2,\dots ,p_n\parallel r_1,r_2,\dots ,r_n)\\ \ge & {exp}_{2-q}{D}_q(p_1^{\mathcal{A}},p_2^{\mathcal{A}},\dots ,p_k^{\mathcal{A}}\parallel r_1^{\mathcal{A}},r_2^{\mathcal{A}},\dots ,r_k^{\mathcal{A}})\\ =& exp{R}_q(p_1^{\mathcal{A}},p_2^{\mathcal{A}},\dots ,p_k^{\mathcal{A}}\parallel r_1^{\mathcal{A}},r_2^{\mathcal{A}},\dots ,r_k^{\mathcal{A}}),\end{array}$$

which proves the statement. □

3 Inequalities for Tsallis quasilinear entropy and f-divergence

In this section, we give inequalities for the Tsallis quasilinear entropy and f-divergence. For this purpose, we review the results obtained in [29] as one of generalizations of refined Young’s inequality.

Proposition 3.1 ([29])

For two probability vectors $\mathbf{p}=\{p_1,p_2,\dots ,p_n\}$ and $\mathbf{r}=\{r_1,r_2,\dots ,r_n\}$ such that $p_j>0$, $r_j>0$, ${\sum }_{j=1}^n{p}_j={\sum }_{j=1}^n{r}_j=1$ and $\mathbf{x}=\{x_1,x_2,\dots ,x_n\}$ such that $x_i\ge 0$, we have

$$\underset{1\le i\le n}{min}\left\{\frac{r_i}{p_i}\right\}T(f,\mathbf{x},\mathbf{p})\le T(f,\mathbf{x},\mathbf{r})\le \underset{1\le i\le n}{max}\left\{\frac{r_i}{p_i}\right\}T(f,\mathbf{x},\mathbf{p}),$$

(26)

where

$$T(f,\mathbf{x},\mathbf{p})\equiv \sum _{j=1}^n{p}_{j}f(x_j)-f\left({\psi }^{-1}(\sum _{j=1}^n{p}_j\psi (x_j))\right),$$

(27)

for a continuous increasing function $\psi :I\to I$ and a function $f:I\to J$ such that

$$f\left({\psi }^{-1}((1-\lambda )\psi (a)+\lambda \psi (b))\right)\le (1-\lambda )f(a)+\lambda f(b)$$

(28)

for any $a,b\in I$ and any $\lambda \in [0,1]$.

We have the following inequalities on the Tsallis quasilinear entropy and Tsallis entropy.

Theorem 3.2 For $q\ge 0$, a continuous and strictly monotonic function ψ on $(0,\mathrm{\infty })$ and a probability distribution $\{r_1,r_2,\dots ,r_n\}$ with $r_j>0$ for all $j=1,2,\dots ,n$, we have

$$\begin{array}{rcl}0& \le & n\underset{1\le i\le n}{min}\{r_i\}\{{ln}_q\left({\psi }^{-1}\left(\frac{1}{n}\sum _{j=1}^n\psi \left(\frac{1}{r_j}\right)\right)\right)-\frac{1}{n}\sum _{j=1}^n{ln}_q\frac{1}{r_j}\}\\ \le & I_q^{\psi }(r_1,r_2,\dots ,r_n)-H_q(r_1,r_2,\dots ,r_n)\\ \le & n\underset{1\le i\le n}{max}\{r_i\}\{{ln}_q\left({\psi }^{-1}\left(\frac{1}{n}\sum _{j=1}^n\psi \left(\frac{1}{r_j}\right)\right)\right)-\frac{1}{n}\sum _{j=1}^n{ln}_q\frac{1}{r_j}\}.\end{array}$$

Proof If we take the uniform distribution $\mathbf{p}=\{\frac{1}{n},\dots ,\frac{1}{n}\}\equiv \mathbf{u}$ in Proposition 3.1, then we have

$$n\underset{1\le i\le n}{min}\{r_i\}{T}_n(f,\mathbf{x},\mathbf{u})\le T_n(f,\mathbf{x},\mathbf{r})\le n\underset{1\le i\le n}{max}\{r_i\}{T}_n(f,\mathbf{x},\mathbf{u})$$

(29)

(which coincides with Theorem 3.3 in [29]). In the inequalities (29), we put $f(x)=-{ln}_q(x)$ and $x_j=\frac{1}{r_j}$ for any $j=1,2,\dots ,n$, then we obtain the statement. □

Corollary 3.3 For $q\ge 0$ and a probability distribution $\{r_1,r_2,\dots ,r_n\}$ with $r_j>0$ for all $j=1,2,\dots ,n$, we have

$$\begin{array}{rcl}0& \le & n\underset{1\le i\le n}{min}\{r_i\}\{{ln}_q\left(\frac{1}{n}\sum _{j=1}^n\frac{1}{r_j}\right)-\frac{1}{n}\sum _{j=1}^n{ln}_q\frac{1}{r_j}\}\le {ln}_{q}n-H_q(r_1,r_2,\dots ,r_n)\\ \le & n\underset{1\le i\le n}{max}\{r_i\}\{{ln}_q\left(\frac{1}{n}\sum _{j=1}^n\frac{1}{r_j}\right)-\frac{1}{n}\sum _{j=1}^n{ln}_q\frac{1}{r_j}\}.\end{array}$$

(30)

Proof Put $\psi (x)=x$ in Theorem 3.2. □

Remark 3.4 Corollary 3.3 improves the well-known inequalities $0\le H_q(r_1,r_2,\dots ,r_n)\le {ln}_{q}n$. If we take the limit $q\to 1$, the inequalities (30) recover Proposition 1 in [25].

We also have the following inequalities.

Theorem 3.5 For two probability distributions $\mathbf{p}=\{p_1,p_2,\dots ,p_n\}$ and $\mathbf{r}=\{r_1,r_2,\dots ,r_n\}$, and an incomplete probability distribution $\mathbf{t}=\{t_1,t_2,\dots ,t_n\}$ with $t_j\equiv \frac{p_j^2}{r_j}$, we have

$$\begin{array}{rcl}0& \le & \underset{1\le i\le n}{min}\left\{\frac{r_i}{p_i}\right\}(\widetilde{D_{f^{\ast }}}(\mathbf{t}\parallel \mathbf{p})-f\left(\sum _{j=1}^n{t}_j\right))\\ \le & D_f(\mathbf{p}\parallel \mathbf{r})\le \underset{1\le i\le n}{max}\left\{\frac{r_i}{p_i}\right\}(\widetilde{D_{f^{\ast }}}(\mathbf{t}\parallel \mathbf{p})-f\left(\sum _{j=1}^n{t}_j\right)).\end{array}$$

(31)

Proof Put $x_j=\frac{p_j}{r_j}$ in Proposition 3.1 with $\psi (x)=x$. Since we have the relation

$$\sum _{j=1}^n{p}_{j}f\left(\frac{p_j}{r_j}\right)=\sum _{j=1}^n{p}_j\frac{p_j}{r_j}{f}^{\ast }\left(\frac{r_j}{p_j}\right)=\sum _{j=1}^n{t}_j{f}^{\ast }\left(\frac{p_j}{t_j}\right),$$

we have the statement. □

Corollary 3.6 ([25])

Under the same assumption as in Theorem 3.5, we have

$$\begin{array}{rcl}0& \le & \underset{1\le i\le n}{min}\left\{\frac{r_i}{p_i}\right\}(log\left(\sum _{j=1}^n{t}_j\right)-D_1(\mathbf{p}\parallel \mathbf{r}))\\ \le & D_1(\mathbf{r}\parallel \mathbf{p})\le \underset{1\le i\le n}{max}\left\{\frac{r_i}{p_i}\right\}(log\left(\sum _{j=1}^n{t}_j\right)-D_1(\mathbf{p}\parallel \mathbf{r})).\end{array}$$

Proof If we take $f(x)=-log(x)$ in Theorem 3.5, then we have

$$D_f(\mathbf{p}\parallel \mathbf{r})=-\sum _{j=1}^n{r}_{j}log\frac{p_j}{r_j}=\sum _{j=1}^n{r}_{j}log\frac{r_j}{p_j}=D_1(\mathbf{r}\parallel \mathbf{p}).$$

Since $f^{\ast }(x)=xlog(x)$ and $t_j=\frac{p_j^2}{r_j}$, we also have

$$\begin{array}{rcl}\widetilde{D_{f^{\ast }}}(\mathbf{t}\parallel \mathbf{p})-f\left(\sum _{j=1}^n{t}_j\right)& =& \sum _{j=1}^n{t}_j\frac{p_j}{t_j}log\frac{p_j}{t_j}+log\left(\sum _{j=1}^n{t}_j\right)\\ =& \sum _{j=1}^n{p}_{j}log\frac{r_j}{p_j}+log\left(\sum _{j=1}^n{t}_j\right)\\ =& -\sum _{j=1}^n{p}_{j}log\frac{p_j}{r_j}+log\left(\sum _{j=1}^n{t}_j\right)=log\left(\sum _{j=1}^n{t}_j\right)-D_1(\mathbf{p}\parallel \mathbf{r}).\end{array}$$

 □

4 Inequalities for Tsallis entropy

We firstly give Lagrange’s identity [37], to establish an alternative generalization of refined Young’s inequality.

Lemma 4.1 (Lagrange’s identity)

For two vectors $\{a_1,a_2,\dots ,a_n\}$ and $\{b_1,b_2,\dots ,b_n\}$, we have

$$\begin{array}{rcl}\left(\sum _{k=1}^n{a}_k^2\right)\left(\sum _{k=1}^n{b}_k^2\right)-{\left(\sum _{k=1}^n{a}_k{b}_k\right)}^2& =& \frac{1}{2}\sum _{i=1}^n\sum _{j=1}^n{(a_i{b}_j-a_j{b}_i)}^2\\ =& \sum _{1\le i<j\le n}{(a_i{b}_j-a_j{b}_i)}^2.\end{array}$$

(32)

Theorem 4.2 Let $f:I\to \mathbb{R}$ be a twice differentiable function such that there exist real constants m and M so that $0\le m\le f^{\mathrm{\prime }\mathrm{\prime }}(x)\le M$ for any $x\in I$. Then we have

$$\begin{array}{rcl}\frac{m}{2}\sum _{1\le i<j\le n}{p}_i{p}_j{(x_j-x_i)}^2& \le & \sum _{j=1}^n{p}_{j}f(x_j)-f\left(\sum _{j=1}^n{p}_j{x}_j\right)\\ \le & \frac{M}{2}\sum _{1\le i<j\le n}{p}_i{p}_j{(x_j-x_i)}^2,\end{array}$$

(33)

where $p_j>0$ with ${\sum }_{j=1}^n{p}_j=1$ and $x_j\in I$ for all $j=1,2,\dots ,n$.

Proof We consider the function $g:I\to \mathbb{R}$ defined by $g(x)\equiv f(x)-\frac{m}{2}{x}^2$. Since we have $g^{\mathrm{\prime }\mathrm{\prime }}(x)=f^{\mathrm{\prime }\mathrm{\prime }}(x)-m\ge 0$, g is a convex function. Applying Jensen’s inequality, we thus have

$$\sum _{j=1}^n{p}_{j}g(x_j)\ge g\left(\sum _{j=1}^n{p}_j{x}_j\right),$$

(34)

where $p_j>0$ with ${\sum }_{j=1}^n{p}_j=1$ and $x_j\in I$ for all $j=1,2,\dots ,n$. From the inequality (34), we have

$$\begin{array}{rcl}\sum _{j=1}^n{p}_{j}f(x_j)-f\left(\sum _{j=1}^n{p}_j{x}_j\right)& \ge & \frac{m}{2}\{\sum _{j=1}^n{p}_j{x}_j^2-{\left(\sum _{j=1}^n{p}_j{x}_j\right)}^2\}\\ =& \frac{m}{2}\{\left(\sum _{j=1}^n{p}_j\right)\left(\sum _{j=1}^n{p}_j{x}_j^2\right)-{\left(\sum _{j=1}^n{p}_j{x}_j\right)}^2\}\\ =& \frac{m}{2}\sum _{1\le i<j\le n}{(\sqrt{p_i}\sqrt{p_j}{x}_j-\sqrt{p_j}\sqrt{p_i}{x}_i)}^2\\ =& \frac{m}{2}\sum _{1\le i<j\le n}{p}_i{p}_j{(x_j-x_i)}^2.\end{array}$$

In the above calculations, we used Lemma 4.1. Thus, we proved the first part of the inequalities. Similarly, one can prove the second part of the inequalities putting the function $h:I\to \mathbb{R}$ defined by $h(x)\equiv \frac{M}{2}{x}^2-f(x)$. We omit the details. □

Lemma 4.3 For $\{p_1,p_2,\dots ,p_n\}$ with $p_j>0$ and ${\sum }_{j=1}^n{p}_j=1$, and $\{x_1,x_2,\dots ,x_n\}$ with $x_j>0$, we have

$$\sum _{1\le i<j\le n}{p}_i{p}_j{(x_j-x_i)}^2=\sum _{j=1}^n{p}_j{(x_j-\sum _{i=1}^n{p}_i{x}_i)}^2.$$

(35)

Proof We denote

$$\overline{x}=\sum _{i=1}^n{p}_i{x}_i.$$

The left-side term becomes

$$\begin{array}{rcl}\sum _{1\le i<j\le n}{p}_i{p}_j{(x_j-x_i)}^2& =& \frac{1}{2}\sum _{i=1}^n\sum _{j=1}^n{p}_i{p}_j{(x_j-x_i)}^2\\ =& \frac{1}{2}\sum _{i=1}^n\sum _{j=1}^n{p}_i{p}_j(x_j^2+x_i^2-2x_j{x}_i)\\ =& \frac{1}{2}\sum _{i=1}^n\sum _{j=1}^n{p}_i{p}_j{x}_j^2+\frac{1}{2}\sum _{i=1}^n\sum _{j=1}^n{p}_i{p}_j{x}_i^2-\sum _{i=1}^n\sum _{j=1}^n{p}_i{p}_j{x}_j{x}_i\\ =& \frac{1}{2}\sum _{i=1}^n{p}_i\sum _{j=1}^n{p}_j{x}_j^2+\frac{1}{2}\sum _{i=1}^n{p}_i{x}_i^2\sum _{j=1}^n{p}_j-\sum _{i=1}^n{p}_i{x}_i\sum _{j=1}^n{p}_j{x}_j\\ =& \sum _{j=1}^n{p}_j{x}_j^2-{\overline{x}}^2.\end{array}$$

Similarly, a straightforward computation yields

$$\begin{array}{rcl}\sum _{j=1}^n{p}_j{(x_j-\sum _{i=1}^n{p}_i{x}_i)}^2& =& \sum _{j=1}^n{p}_j(x_j^2-2x_j\overline{x}+{\overline{x}}^2)\\ =& \sum _{j=1}^n{p}_j{x}_j^2-2{\overline{x}}^2+{\overline{x}}^2\\ =& \sum _{j=1}^n{p}_j{x}_j^2-{\overline{x}}^2.\end{array}$$

This concludes the proof. □

Corollary 4.4 Under the assumptions of Theorem 4.2, we have

$$\begin{array}{rcl}\frac{m}{2}\sum _{j=1}^n{p}_j{(x_j-\sum _{i=1}^n{p}_i{x}_i)}^2& \le & \sum _{j=1}^n{p}_{j}f(x_j)-f\left(\sum _{j=1}^n{p}_j{x}_j\right)\\ \le & \frac{M}{2}\sum _{j=1}^n{p}_j{(x_j-\sum _{i=1}^n{p}_i{x}_i)}^2.\end{array}$$

(36)

Remark 4.5 Corollary 4.4 gives a similar form with Cartwright-Field’s inequality [38]:

$$\begin{array}{rcl}\frac{1}{2M^{\mathrm{\prime }}}\sum _{j=1}^n{p}_j{(x_j-\sum _{i=1}^n{p}_i{x}_i)}^2& \le & \sum _{j=1}^n{p}_j{x}_j-\prod _{j=1}^n{x}_j^{p_j}\\ \le & \frac{1}{2m^{\mathrm{\prime }}}\sum _{j=1}^n{p}_j{(x_j-\sum _{i=1}^n{p}_i{x}_i)}^2,\end{array}$$

(37)

where $p_j>0$ for all $j=1,2,\dots ,n$ and ${\sum }_{j=1}^n{p}_j=1$, $m^{\mathrm{\prime }}\equiv min\{x_1,x_2,\dots ,x_n\}>0$ and $M^{\mathrm{\prime }}\equiv max\{x_1,x_2,\dots ,x_n\}$.

We also have the following inequalities for the Tsallis entropy.

Theorem 4.6 For two probability distributions $\{p_1,p_2,\dots ,p_n\}$ and $\{r_1,r_2,\dots ,r_n\}$ with $p_j>0$, $r_j>0$ and ${\sum }_{j=1}^n{p}_j={\sum }_{j=1}^n{r}_j=1$, we have

(38)

where $m_q$ and $M_q$ are positive numbers depending on the parameter $q\ge 0$ and satisfying $m_q\le q{r}_j^{-q-1}\le M_q$ and $m_q\le q{p}_j^{-q-1}\le M_q$ for all $j=1,2,\dots ,n$.

Proof Applying Theorem 4.2 for the convex function $-{ln}_q(x)$ and $x_j=\frac{1}{r_j}$, we have

$$\begin{array}{rcl}\frac{m_q}{2}\sum _{1\le i<j\le n}{p}_i{p}_j{(\frac{1}{r_j}-\frac{1}{r_i})}^2& \le & -\sum _{j=1}^n{p}_j{ln}_q\frac{1}{r_j}+{ln}_q\left(\sum _{j=1}^n\frac{p_j}{r_j}\right)\\ \le & \frac{M_q}{2}\sum _{1\le i<j\le n}{p}_i{p}_j{(\frac{1}{r_j}-\frac{1}{r_i})}^2,\end{array}$$

(39)

since the second derivative of $-{ln}_q(x)$ is $q{x}^{-q-1}$. Putting $r_j=p_j$ for all $j=1,2,\dots ,n$ in the inequalities (39), it follows

$$\begin{array}{rcl}\frac{m_q}{2}\sum _{1\le i<j\le n}{p}_i{p}_j{(\frac{1}{p_j}-\frac{1}{p_i})}^2& \le & -\sum _{j=1}^n{p}_j{ln}_q\frac{1}{p_j}+{ln}_{q}n\\ \le & \frac{M_q}{2}\sum _{1\le i<j\le n}{p}_i{p}_j{(\frac{1}{p_j}-\frac{1}{p_i})}^2.\end{array}$$

(40)

From the inequalities (39) and (40), we have the statement. □

Remark 4.7 The first part of the inequalities (40) gives another improvement of the well-known inequalities $0\le H_q(r_1,r_2,\dots ,r_n)\le {ln}_{q}n$.

Corollary 4.8 For two probability distributions $\{p_1,p_2,\dots ,p_n\}$ and $\{r_1,r_2,\dots ,r_n\}$ with $p_j>0$, $r_j>0$ and ${\sum }_{j=1}^n{p}_j={\sum }_{j=1}^n{r}_j=1$, we have

(41)

where $m_1$ and $M_1$ are positive numbers satisfying $m_1\le r_j^{-2}\le M_1$ and $m_1\le p_j^{-2}\le M_1$ for all $j=1,2,\dots ,n$.

Proof Take the limit $q\to 1$ in Theorem 4.6. □

Remark 4.9 The second part of the inequalities (41) gives the reverse inequality for the so-called information inequality [[39], Theorem 2.6.3]

$$0\le \sum _{j=1}^n{p}_{j}log\frac{1}{r_j}-\sum _{j=1}^n{p}_{j}log\frac{1}{p_j}$$

(42)

which is equivalent to the non-negativity of the relative entropy

$$D_1(p_1,p_2,\dots ,p_n\parallel r_1,r_2,\dots ,r_n)\ge 0.$$

Using the inequality (42), we derive the following result.

Proposition 4.10 For two probability distributions $\{p_1,p_2,\dots ,p_n\}$ and $\{r_1,r_2,\dots ,r_n\}$ with $0<p_j<1$, $0<r_j<1$ and ${\sum }_{j=1}^n{p}_j={\sum }_{j=1}^n{r}_j=1$, we have

$$\sum _{j=1}^n(1-p_j)log\frac{1}{1-p_j}\le \sum _{j=1}^n(1-p_j)log\frac{1}{1-r_j}.$$

(43)

Proof In the inequality (42), we put $p_j=\frac{1-p_j}{n-1}$ and $r_j=\frac{1-r_j}{n-1}$ which satisfy ${\sum }_{j=1}^n\frac{1-p_j}{n-1}={\sum }_{j=1}^n\frac{1-r_j}{n-1}=1$. Then we have the present proposition. □

5 A generalized Han’s inequality

In order to state our result, we give the definitions of the Tsallis conditional entropy and the Tsallis joint entropy.

Definition 5.1 ([16, 40])

For the conditional probability $p(x_i|y_j)$ and the joint probability $p(x_i,y_j)$, we define the Tsallis conditional entropy and the Tsallis joint entropy by

$$H_q(\mathbf{x}|\mathbf{y})\equiv -\sum _{i,j}p{(x_i,y_j)}^q{ln}_{q}p(x_i|y_j)(q\ge 0,q\ne 1)$$

(44)

and

$$H_q(\mathbf{x},\mathbf{y})\equiv -\sum _{i,j}p{(x_i,y_j)}^q{ln}_{q}p(x_i,y_j)(q\ge 0,q\ne 1).$$

(45)

We summarize briefly the following chain rules representing relations between the Tsallis conditional entropy and the Tsallis joint entropy.

Proposition 5.2 ([16, 40])

Assume that x, y are random variables. Then

$$H_q(\mathbf{x},\mathbf{y})=H_q(\mathbf{x})+H_q(\mathbf{y}|\mathbf{x}).$$

(46)

Proposition 5.2 implied the following propositions.

Proposition 5.3 ([16])

Suppose ${\mathbf{x}}_1,{\mathbf{x}}_2,\dots ,{\mathbf{x}}_n$ are random variables. Then

$$H_q({\mathbf{x}}_1,{\mathbf{x}}_2,\dots ,{\mathbf{x}}_n)=\sum _{i=1}^n{H}_q({\mathbf{x}}_i|{\mathbf{x}}_{i-1},\dots ,{\mathbf{x}}_1).$$

(47)

Proposition 5.4 ([16, 40])

For $q\ge 1$, two random variables x and y, we have the following inequality:

$$H_q(\mathbf{x}|\mathbf{y})\le H_q(\mathbf{x}).$$

(48)

Consequently, we have the following self-bounding property of the Tsallis joint entropy.

Theorem 5.5 (Generalized Han’s inequality)

Let ${\mathbf{x}}_1,{\mathbf{x}}_2,\dots ,{\mathbf{x}}_n$ be random variables. Then for $q\ge 1$, we have the following inequality:

$$H_q({\mathbf{x}}_1,\dots ,{\mathbf{x}}_n)\le \frac{1}{n-1}\sum _{i=1}^n{H}_q({\mathbf{x}}_1,\dots ,{\mathbf{x}}_{i-1},{\mathbf{x}}_{i+1},\dots ,{\mathbf{x}}_n).$$

Proof Since the Tsallis joint entropy has a symmetry $H_q(\mathbf{x},\mathbf{y})=H_q(\mathbf{y},\mathbf{x})$, we have

$$\begin{array}{rcl}{H}_q({\mathbf{x}}_1,\dots ,{\mathbf{x}}_n)& =& H_q({\mathbf{x}}_1,\dots ,{\mathbf{x}}_{i-1},{\mathbf{x}}_{i+1},\dots ,{\mathbf{x}}_n)+H_q({\mathbf{x}}_i|{\mathbf{x}}_1,\dots ,{\mathbf{x}}_{i-1},{\mathbf{x}}_{i+1},\dots ,{\mathbf{x}}_n)\\ \le & H_q({\mathbf{x}}_1,\dots ,{\mathbf{x}}_{i-1},{\mathbf{x}}_{i+1},\dots ,{\mathbf{x}}_n)+H_q({\mathbf{x}}_i|{\mathbf{x}}_1,\dots ,{\mathbf{x}}_{i-1})\end{array}$$

by the use of Proposition 5.2 and Proposition 5.4. Summing both sides on i from 1 to n, we have

$$\begin{array}{rcl}n{H}_q({\mathbf{x}}_1,\dots ,{\mathbf{x}}_n)& =& \sum _{i=1}^n{H}_q({\mathbf{x}}_1,\dots ,{\mathbf{x}}_{i-1},{\mathbf{x}}_{i+1},\dots ,{\mathbf{x}}_n)+\sum _{i=1}^n{H}_q({\mathbf{x}}_i|{\mathbf{x}}_1,\dots ,{\mathbf{x}}_{i-1},{\mathbf{x}}_{i+1},\dots ,{\mathbf{x}}_n)\\ \le & \sum _{i=1}^n{H}_q({\mathbf{x}}_1,\dots ,{\mathbf{x}}_{i-1},{\mathbf{x}}_{i+1},\dots ,{\mathbf{x}}_n)+H_q({\mathbf{x}}_1,\dots ,{\mathbf{x}}_n),\end{array}$$

due to Proposition 5.3. Therefore, we have the present proposition. □

Remark 5.6 Theorem 5.5 recovers the original Han’s inequality [41, 42] if we take the limit as $q\to 1$.

6 Conclusion

We gave an improvement of Young’s inequalities for scalar numbers. Using this result, we gave several inequalities on generalized entropies involving Tsallis entropies. We also provided a generalized Han’s inequality, based on the conditional Tsallis entropy and the joint Tsallis entropy.