原始体检数据存在信息模糊、有噪声、不完整和冗余的问题,无法直接用于疾病的风险评估与预测.由于体检数据在结构和格式等方面的不足,不适合采用传统的数据预处理方法.为了充分挖掘体检数据中有价值的信息,从多角度提出了针对体检数据的预处理方法:通过基于压缩方法的数据归约,降低了体检数据预处理的时间及空间复杂度;通过基于分词和权值的字段匹配算法,完成了体检数据的清洗,解决了体检数据不一致的问题;通过基于线性函数的数据变换,实现了历年体检数据的一致性和连续性.实验结果表明,基于分词和权值的字段匹配算法,相对于传统算法具有更高的准确性.%The original physical examination data has many problems,including ambiguity,noise,incomplete and redundancy information,so it cannot be used for disease risk assessment and prediction directly.Traditional processing methods are not suitable for physical examination data because of its special structure and format.In order to solve these problems and make full use of the valuable information in the data,this paper proposed several methods.It used a compression-based data reduction method to reduce the time and space complexity of the data,and used a field matching algorithm based on segmentation and weights to complete the data cleaning and solved the problem of inconsistency.It used a data transformation method based on linear function to get the consistency and continuity of the history data.It also proves that the proposed filed matching algorithm is more accurate than the traditional method.
展开▼
