...
  • 主题
高级搜索  >
历史搜索 清空历史
首页> 外文期刊>IEEE transactions on big data >Anomaly Detection in Catalog Streams
【24h】

Anomaly Detection in Catalog Streams

机译：目录流中的异常检测

获取原文
获取原文并翻译 | 示例
           
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

Detecting anomalies with high accuracy and real time from large amounts of streaming data is a challenge for many real-world applications, such as smart city, astronomical observations, and remote sensing. This article focuses on a special kind of stream, catalog stream, whose high-level catalog structure can be used to analyze the stream effectively. We first formulate the anomaly detection in catalog streams as a constrained optimization problem based on a catalog stream matrix. Then, a novel filtering-identifying based anomaly detection algorithm ( FIAD ) is proposed, which includes two complementary strategies, true event identifying and false alarm filtering, data-oriented general method and domain-oriented specific method together, to detect truly valuable anomalies. Furthermore, different kinds of attention windows are developed to provide corresponding data for various algorithm components. A scalable and lightweight catalog stream processing framework CSPF is designed to support and implement the proposed method efficiently. A prototype system is developed to evaluate the proposed algorithm. Extensive experiments are conducted on the catalog stream data sets from an operational super large field-of-view high-cadence astronomy observation. The experimental results show that the proposed method can achieve a false-positive rate as low as 0.04, reduces the false alarms by 98.6 compared with the existing methods, and the latency to handle each catalog is 2.1 seconds (much less than the required 15 seconds). Furthermore, a total of 36 transient candidates, including seven microlensing events, 27 superflares, and two dual-superflares, are detected from 21.67 million stars (involving 1.09 million catalogs) from one observation season.
机译：从大量流数据中高精度和实时检测异常对于许多现实世界应用（如智慧城市、天文观测和遥感）来说都是一个挑战。本文重点介绍一种特殊的流，即目录流，其高级目录结构可用于有效地分析流。我们首先将目录流中的异常检测表述为基于目录流矩阵的约束优化问题。然后，提出了一种基于滤波-识别的异常检测算法（FIAD），该算法将真事件识别和误报过滤两种互补的策略、面向数据的通用方法和面向领域的特定方法结合在一起，以检测真正有价值的异常。此外，还开发了不同类型的注意力窗口，为各种算法组件提供相应的数据。CSPF是一种可扩展的轻量级目录流处理框架，旨在有效地支持和实现所提出的方法。开发了一个原型系统来评估所提出的算法。对来自操作超大视场高节奏天文观测的目录流数据集进行了广泛的实验。实验结果表明，所提方法能够实现低至0.04%的误报率，与现有方法相比，误报减少98.6%，处理每个目录的延迟为2。1 秒（远小于所需的 15 秒）。此外，在一个观测季节的2167万颗恒星（涉及109万个目录）中共探测到36个瞬态候选星，包括7个微透镜事件、27个超级耀斑和2个双超耀斑。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号