• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文期刊>Embedded Systems Letters, IEEE >Evolutionary Recurrent Neural Architecture Search
【24h】

Evolutionary Recurrent Neural Architecture Search

机译：进化经常性神经结构搜索

获取原文
获取原文并翻译 | 示例
           
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

Deep learning has promoted remarkable progress in various tasks while the effort devoted to these hand-crafting neural networks has motivated so-called neural architecture search (NAS) to discover them automatically. Recent aging evolution (AE) automatic search algorithm turns to discard the oldest model in population and finds image classifiers beyond manual design. However, it achieves a low speed of convergence. A nonaging evolution (NAE) algorithm tends to neglect the worst architecture in population to accelerate the search process whereas it obtains a lower performance compared with AE. To address this issue, in this letter, we propose to use an optimized evolution algorithm for recurrent NAS (EvoRNAS) by setting a probability epsilon to remove the worst or oldest model in population alternatively, which can balance the performance and time length. Besides, parameter sharing mechanism is introduced in our approach due to the heavy cost of evaluating the candidate models in both AE and NAE. Furthermore, we train the sharing parameters only once instead of many epochs like ENAS, which makes the evaluation of candidate models faster. On Penn Treebank, we first explore different epsilon in EvoRNAS and find the best value suited for the learning task, which is also better than AE and NAE. Second, the optimal cell found by EvoRNAS can achieve state-of-the-art performance within only 0.6 GPU hours, which is 20 x and 40 x faster than ENAS and DARTS. Moreover, the transferability of the learned architecture to WikiText-2 also shows strong performance compared with ENAS or DARTS.
机译：深入学习在各种任务中促进了显着进展，而致力于这些手工制作的神经网络的努力是有动机的所谓神经结构搜索（NAS）自动发现它们。最近的老化进化（AE）自动搜索算法转向丢弃人口中最古老的模型，并找到手动设计之外的图像分类器。然而，它实现了低的收敛速度。非营利演进（NAE）算法倾向于忽略人口中最糟糕的架构，以加速搜索过程，而它与AE相比获得较低的性能。为了解决此问题，在这封信中，我们建议通过设置概率epsilon来使用优化的进化算法来使用概率epsilon来消除群体中最差或最古老的模型，这可以平衡性能和时间长度。此外，由于在AE和NAE中评估候选模型的沉重成本，我们的方法引入了参数共享机制。此外，我们只训练共享参数一次，而不是许多时代，这使得候选模型的评估更快。在Penn TreeBank上，我们首先在埃文斯探索不同的epsilon，并找到适合学习任务的最佳价值，这也比AE和NAE更好。其次，Evornas发现的最佳单元可以在仅0.6 GPU小时内实现最先进的性能，这比ZHAS和飞镖快20 x和40 x。此外，与enas或飞镖相比，学习架构与Wikitext-2的可转换性也表现出强烈的表现。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号