...
  • 主题
高级搜索  >
历史搜索 清空历史
首页> 外文期刊>Information Theory, IEEE Transactions on >On the Minimax Risk of Dictionary Learning
【24h】

On the Minimax Risk of Dictionary Learning

机译：论字典学习的极小极大风险

获取原文
获取原文并翻译 | 示例
           
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

We consider the problem of learning a dictionary matrix from a number of observed signals, which are assumed to be generated via a linear model with a common underlying dictionary. In particular, we derive lower bounds on the minimum achievable worst case mean squared error (MSE), regardless of computational complexity of the dictionary learning (DL) schemes. By casting DL as a classical (or frequentist) estimation problem, the lower bounds on the worst case MSE are derived following an established information-theoretic approach to minimax estimation. The main contribution of this paper is the adaption of these information-theoretic tools to the DL problem in order to derive lower bounds on the worst case MSE of any DL algorithm. We derive three different lower bounds applying to different generative models for the observed signals. The first bound only requires the existence of a covariance matrix of the (unknown) underlying coefficient vector. By specializing this bound to the case of sparse coefficient distributions and assuming the true dictionary satisfies the restricted isometry property, we obtain a lower bound on the worst case MSE of DL methods in terms of the signal-to-noise ratio (SNR). The third bound applies to a more restrictive subclass of coefficient distributions by requiring the non-zero coefficients to be Gaussian. Although the applicability of this bound is the most limited, it is the tightest of the three bounds in the low SNR regime. A particular use of our lower bounds is the derivation of necessary conditions on the required number of observations (sample size), such that DL is feasible, i.e., accurate DL schemes might exist. By comparing these necessary conditions with sufficient conditions on the sample size such that a particular DL technique is successful, we are able to characterize the regimes, where those algorithms are optimal in terms of required sample size.
机译：我们考虑了从多个观察到的信号中学习字典矩阵的问题，这些信号被认为是通过具有共同基础字典的线性模型生成的。尤其是，无论字典学习（DL）方案的计算复杂度如何，我们都会得出最小可实现的最坏情况均方误差（MSE）的下界。通过将DL转换为经典（或频繁出现的）估计问题，最坏情况下的MSE的下界可遵循已建立的信息理论方法进行极小极大估计。本文的主要贡献是这些信息理论工具适用于DL问题，以便得出任何DL算法的最坏情况MSE的下限。我们推导了三个不同的下限，它们适用于所观察信号的不同生成模型。第一个边界仅要求存在（未知）基础系数向量的协方差矩阵。通过专门针对稀疏系数分布情况下的绑定并假设真实字典满足受限的等距特性，就信噪比（SNR）而言，我们获得了DL方法最坏情况MSE的下限。通过要求非零系数为高斯，第三个界限适用于更具限制性的系数分布子类。尽管此界限的适用性是最有限的，但在低SNR方案中，它是三个界限中最严格的。下限的一种特殊用法是在所需观察数（样本大小）上推导必要条件，以使DL可行，即可能存在准确的DL方案。通过将这些必要条件与样本量的足够条件进行比较，从而成功使用特定的DL技术，我们能够表征那些算法在所需样本量方面最优的方案。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号