## Maximal Maximum-Entropy Sets

September 7, 2015 Leave a comment

Consider a probability distribution on a space . Suppose we want to construct a set of probability distributions on such that is the maximum-entropy distribution over :

where is the entropy. We call such a set a *maximum-entropy set for *. Furthermore, we would like to be as large as possible, subject to the constraint that is convex.

Does such a maximal convex maximum-entropy set exist? That is, is there some convex set such that is the maximum-entropy distribution in , and for any satisfying the same property, ? It turns out that the answer is yes, and there is even a simple characterization of :

Proposition 1For any distribution on , the setis the maximal convex maximum-entropy set for .

To see why this is, first note that, clearly, , and for any we have

so is indeed the maximum-entropy distribution in . On the other hand, let be any other convex set whose maximum-entropy distribution is . Then in particular, for any , we must have . Let us suppose for the sake of contradiction that , so that . Then we have

Since , for sufficiently small this will exceed , which is a contradiction. Therefore we must have for all , and hence , so that is indeed the maximal convex maximum-entropy set for .