...
  • 主题
高级搜索  >
历史搜索 清空历史
首页> 外文期刊>Journal of Computers >The Large-scale Dynamic Data Rapid Reduction Algorithm Based on Map-Reduce
【24h】

The Large-scale Dynamic Data Rapid Reduction Algorithm Based on Map-Reduce

机译：基于Map-Reduce的大规模动态数据快速约简算法

获取原文
           
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

With the advent of the era of “Big Data”, the application of the large-scale data is becoming popular. Efficiently using and analyzing the data has become an interesting research topic. Traditional knowledge reduction algorithms read small data samples once into a computer main memory for reduction, but it is not suitable for large-scale data. This paper takes large-scale sensor monitoring dynamic data as the research object and puts forward an incremental reduction algorithm based on Map-Reduce. Using a Hash fast partitioning strategy this algorithm divides the dynamic data set into multiple subdatasets to compute, which has greatly reduced the calculation time and space complexity of each node. Finally，experiments are conducted on the data from UCI Machine Learning Repository using Hadoop platform to prove that the algorithm is efficient and suitable for large-scale dynamic data. Compared to the traditional algorithms, the highest speedup of the parallel algorithm can be increased up to 1.55 times.
机译：随着“大数据”时代的到来，大规模数据的应用正变得越来越流行。有效地使用和分析数据已成为一个有趣的研究主题。传统的知识约简算法一次将小的数据样本读取到计算机主存储器中以进行约简，但是它不适用于大规模数据。本文以大规模传感器监测动态数据为研究对象，提出了一种基于Map-Reduce的增量约简算法。该算法使用哈希快速分区策略，将动态数据集分为多个子数据集进行计算，从而大大减少了每个节点的计算时间和空间复杂度。最后，利用Hadoop平台对UCI机器学习库中的数据进行了实验，证明了该算法高效且适用于大规模动态数据。与传统算法相比，并行算法的最高速度可以提高到1.55倍。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号