Progressive encoding of a signal generally involves an estimation step, designed to reduce the entropy of the residual of an observation over the entropy of the observation itself. Oftentimes the conditional distributions of an observation, given already-encoded observations, are well fit within a class of symmetric and unimodal distributions (e.g., the two-sided geometric distributions in images of natural scenes, or symmetric Paretian distributions in models of financial data). It is common practice to choose an estimator that centers, or aligns, the modes of the conditional distributions, since it is common sense that this will minimize the entropy, and hence the coding cost of the residuals. But with the exception of a special case, there has been no rigorous proof. Here we prove that the entropy of an arbitrary mixture of symmetric and unimodal distributions is minimized by aligning the modes. The result generalizes to unimodal and rotation-invariant distributions in $R^{n}$. We illustrate the result through some experiments with natural images.
展开▼
机译：信号的渐进编码通常包含一个估计步骤，该步骤被设计为相对于观测值本身的熵来减少观测值残差的熵。在给定已经编码的观测值的情况下，观测值的条件分布通常很适合一类对称和单峰分布（例如，自然场景图像中的两侧几何分布，或金融数据模型中的对称Paretian分布） 。通常的做法是选择一个使条件分布的模式居中或对齐的估计器，这是常识，因为这将使熵最小，从而使残差的编码成本最小。但是，除了特殊情况外，没有严格的证据。在这里，我们证明了通过对齐模式可使对称和单峰分布的任意混合的熵最小化。结果推广到$ R ^ {n} $中的单峰分布和旋转不变分布。我们通过一些使用自然图像的实验来说明结果。
展开▼
