...
  • 主题
高级搜索  >
历史搜索 清空历史
首页> 外文期刊>Neural processing letters >Competitive Cross-Entropy Loss: A Study on Training Single-Layer Neural Networks for Solving Nonlinearly Separable Classification Problems
【24h】

Competitive Cross-Entropy Loss: A Study on Training Single-Layer Neural Networks for Solving Nonlinearly Separable Classification Problems

机译：竞争交叉熵损失：训练单层神经网络，用于解决非线性可分离分类问题的单层神经网络

获取原文
获取原文并翻译 | 示例
           
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

After Minsky and Papert (Perceptrons, MIT Press, Cambridge, 1969) showed the inability of perceptrons in solving nonlinearly separable problems, for several decades people misinterpreted it as an inherent weakness that is common to all single-layer neural networks. The introduction of the backpropagation algorithm reinforced this misinterpretation as its success in solving nonlinearly separable problems passed through the training of multilayer neural networks. Recently, Conaway and Kurtz (Neural Comput 29(3):861-866, 2017) proposed a single-layer network in which the number of output units for each class is the same as input units and showed that it could solve some nonlinearly separable problems. They used the MSE (Mean Square Error) between the input units and the output units of the actual class as the objective function for training the network. They showed that their method could solve the XOR and M&S'81 problems, but it could not do any better than random guessing on the 3-bit parity problem. In this paper, we use a soft competitive approach to generalize the CE (Cross-Entropy) loss, which is a widely accepted criterion for multiclass classification, to networks that have several output units for each class, calling the resulting measure the CCE (Competitive cross-entropy) loss. In contrast to Conaway and Kurtz (2017), in our method, the number of output units for each class can be chosen arbitrarily. We show that the proposed method can successfully solve the 3-bit parity problem, in addition to the XOR and M&S'81 problems. Furthermore, we perform experiments on several datasets for multiclass classification, comparing a single-layer network trained with the proposed CCE loss against LVQ, linear SVM, a single-layer network trained with the CE loss, and the method of Conaway and Kurtz (2017). The results show that the CCE loss performs remarkably better than existing algorithms for training single-layer neural networks.
机译：在Minsky和Papert（Perceptrons，MIT Press，剑桥，1969）之后，在解决非线性可分离的问题方面表明，几十年来，人们将其误解为所有单层神经网络的固有弱点。背部衰退算法的引入加强了这种误解，因为它在解决了通过培训多层神经网络来源的非线性可分离问题方面的成功。最近，Conaway和Kurtz（神经计算机29（3）：861-866,2017）提出了一种单层网络，其中每个类的输出单元数与输入单元相同，并显示它可以解决一些非线性可分离的问题。它们在输入单元和实际类的输出单元之间使用了MSE（均方误差）作为培训网络的目标函数。他们表明，他们的方法可以解决XOR和M＆S'81问题，但它不能比三位奇偶校验问题更好地进行随机猜测。在本文中，我们使用软竞争方法概括CE（跨熵）丢失，这是一个广泛接受的对多条输出单元的多条输出单元的多级传输单位的标准，调用CCE的结果（竞争交叉熵）损失。与Conaway和Kurtz（2017）相比，在我们的方法中，可以任意选择每个类的输出单元数量。除了XOR和M＆S'81问题之外，我们表明该方法可以成功解决3位奇偶校验问题。此外，我们对多种数据集进行实验，用于多款分类，比较用提出的CCE损耗对LVQ，线性SVM，用CE损失训练的单层网络进行训练，以及Conaway和Kurtz的方法（2017年）。结果表明，CCE损耗比现有算法训练单层神经网络的现有算法更好。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号