High energy collider produces several billions of events in the whole life time. Physical analysis is to select thousands of meaningful events from them and it is a typical big data processing and data mining application. Therefore, it is significantly important to design an efficient data structure, storage and access mechanism, so that the meaningful events can be selected quickly. This paper introduces event data structure,storage and processing technology in popular. This paper analyses the features of high energy physics analysis and proposes a new technology of data storing and processing for high energy physics. This paper fertilizes HBase to store data, uses MapReduce to implement parallel processing and selects ROOT and BEAN as high energy physics analysis frame. This paper also describes the specific design and implementation of the new platform. Test result shows that compared with traditional data storage system of high energy physics, the system has quick data processing speed, it can use effectively I/O and CPU resources when reselection goes into effect.%高能物理对撞机产生数百亿计的物理事例,而物理分析则是从中选取几千个有意义的事例,该分析过程是一个典型的大数据处理及数据挖掘应用。由此,设计高效的数据结构、存储及访问机制,快速挑选出有意义的物理事例十分重要。介绍事例的数据结构、存储和处理技术,分析高能物理数据的特点,提出一种以HBase,ROOT,BEAN及MapReduce为基础的新型高能物理数据存储及处理技术系统。利用HBase存储数据、MapReduce实现并行处理,选择ROOT和BEAN作为高能物理分析框架,并给出具体设计与实现方案。测试结果表明,与传统高能物理数据存储系统相比,该系统具有更快的数据处理速度,当预筛选服务生效时能够更加有效地利用I/O和CPU资源。
展开▼
