• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文会议>IEEE International Symposium on Information Theory >Maximum Likelihood Estimation of information measures
【24h】

Maximum Likelihood Estimation of information measures

机译：信息量度的最大似然估计

获取原文
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

The Maximum Likelihood Estimator (MLE) is widely used in estimating information measures, and involves “plugging-in” the empirical distribution of the data to estimate a given functional of the unknown distribution. In this work we propose a general framework and procedure to analyze the nonasymptotic performance of the MLE in estimating functionals of discrete distributions, under the worst-case mean squared error criterion. We show that existing theory is insufficient for analyzing the bias of the MLE, and propose to apply the theory of approximation using positive linear operators to study this bias. The variance is controlled using the well-known tools from the literature on concentration inequalities. Our techniques completely characterize the maximum L risk incurred by the MLE in estimating the Shannon entropy H(P) = ∑ −pln p, and F(P) = ∑p up to a multiplicative constant. As a corollary, for Shannon entropy estimation, we show that it is necessary and sufficient to have n ≪ S observations for the MLE to be consistent, where S represents the support size. In addition, we obtain that it is necessary and sufficient to consider n ≪ S samples for the MLE to consistently estimate F(P); 0 <α < 1. The minimax rate-optimal estimators for both problems require S/ln S and S / ln S samples, which implies that the MLE is strictly sub-optimal. When 1 < α < 3/2, we show that the maximum L rate of convergence for the MLE is n for infinite support size, while the minimax L rate is (n ln n). When α ≥ 3/2, the MLE achieves the minimax optimal L convergence - ate n regardless of the support size.
机译：最大似然估计器（MLE）被广泛用于估计信息量度，并且涉及“插入”数据的经验分布以估计未知分布的给定功能。在这项工作中，我们提出了一个通用的框架和程序来分析在最坏情况的均方误差准则下，MLE在估计离散分布函数时的非渐近性能。我们证明现有理论不足以分析MLE的偏差，并建议应用使用正线性算子的近似理论来研究该偏差。使用文献中有关浓度不均的众所周知的工具来控制方差。我们的技术完全刻画了MLE在估计香农熵H（P）= ∑ -pln p和F（P）= ∑p直到乘法常数时所引起的最大L风险。作为推论，对于香农熵估计，我们表明，对于MLE保持一致，有必要满足n≪ S个观测值，其中S表示支持规模。此外，我们认为对于MLE而言，考虑n个S样本以一致地估计F（P）是必要且充分的; 0 <α<1.两个问题的最小最大速率最优估计器都需要S / ln S和S / lnS样本，这意味着MLE严格是次优的。当1 <α<3/2时，我们表明对于无限支持大小，MLE的最大L收敛率为n，而最小最大L率为（n ln n）。当α≥3/2时，无论支撑大小如何，MLE都将达到最小最大最优L收敛-n。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号