• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文期刊>IEEE Transactions on Information Theory >Maximum Likelihood Estimation of Functionals of Discrete Distributions
【24h】

Maximum Likelihood Estimation of Functionals of Discrete Distributions

机译：离散分布函数的最大似然估计

获取原文
获取原文并翻译 | 示例
           
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

We consider the problem of estimating functionals of discrete distributions, and focus on a tight (up to universal multiplicative constants for each specific functional) nonasymptotic analysis of the worst case squared error risk of widely used estimators. We apply concentration inequalities to analyze the random fluctuation of these estimators around their expectations and the theory of approximation using positive linear operators to analyze the deviation of their expectations from the true functional, namely their bias. We explicitly characterize the worst case squared error risk incurred by the maximum likelihood estimator (MLE) in estimating the Shannon entropy , and the power sum , up to universal multiplicative constants for each fixed functional, for any alphabet size and sample size for which the risk may vanish. As a corollary, for Shannon entropy estimation, we show that it is necessary and sufficient to have observations for the MLE to be consistent. In addition, we establish that it is necessary and sufficient to consider samples for the MLE to consistently estimate . The minimax rate-optimal estimators for both problems require and samples, which implies that the MLE has a strictly sub-optimal sample complexity. When , we show that the worst case squared error rate of convergence for the MLE is for infinite alphabet size, while the minimax squared error rate is . When , the MLE achieves the minimax optimal rate regardless of the alphabet size. As an application of the general theory, we analyze the Dirichlet prior smoothing techniques for Shannon entropy estimation. In this context, one approach is to plug-in the Dirichlet prior smoothed distribution into the entropy functional, while the other one is to calculate the Bayes estimator for entropy under the Dirichlet prior for squared error, which is the conditional expectation. We show that in general such estimators do not improve over the maximum likelihood estimator. No matter how we tune the parameters in the Dirichlet prior, this approach cannot achieve the minimax rates in entropy estimation. The performance of the minimax rate-optimal estimator with samples is essentially at least as good as that of Dirichlet smoothed entropy estimators with samples.
机译：我们考虑了估计离散分布函数的问题，并着重于对广泛使用的估计器的最坏情况平方误差风险的严格（最多为每个特定函数的通用乘法常数）非渐近分析。我们使用集中不等式来分析这些估计量围绕其期望值的随机波动，并使用正线性算子来逼近该理论，以分析其期望值与真实函数的偏差，即偏差。我们明确地描述了最大似然估计器（MLE）在估计Shannon熵和幂和以及每个固定函数的幂乘和直到通用乘法常数时对于任何字母大小和样本大小所造成的最坏情况平方误差风险的特征可能消失。作为推论，对于香农熵估计，我们表明对MLE进行观测具有一致性是必要和充分的。另外，我们确定有必要考虑足以使MLE一致估计的样本。针对问题需求和样本的最小最大速率最优估计器，这意味着MLE具有严格次优的样本复杂度。当时，我们表明MLE收敛的最坏情况平方错误率是无限字母大小，而minimax平方错误率是。当为时，无论字母大小如何，MLE都会达到minimax最佳速率。作为一般理论的应用，我们分析了狄利克雷先验平滑技术用于香农熵估计。在这种情况下，一种方法是将Dirichlet先验平滑分布插入到熵泛函中，而另一种方法是在Dirichlet先验平方误差下计算熵的贝叶斯估计量，这是条件期望。我们证明，一般而言，此类估计量不会超过最大似然估计量。无论我们如何先在Dirichlet中调整参数，该方法都无法在熵估计中实现minimax速率。带样本的minimax速率最优估计器的性能至少与带样本的Dirichlet平滑熵估计器的性能至少一样好。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号