With the continuous upgrading of hardware in the terminal equipment, how to provide high-performance computing for low-tech threshold users has become a current research hotspot. In the era of green high-performance computing, the heterogeneous computing system can provide good versatility, performance, and efficiency and has broad development prospects. This article provides an in-depth analysis and research on the construction and application of improved models using the artistic design pattern of heterogeneous tasks and parallel computing. Based on the hardware resources in the existing desktop system, this article optimizes the original heterogeneous parallel technology from the aspects of task division and data transmission to reduce the complexity of data allocation and processing for users. Based on the analysis and study of the multicore CPU and GPU architectures in the desktop system, as well as the original CPU-GPU heterogeneous parallel technology, this article optimizes the solution of heterogeneous parallel computing, designs a heterogeneous parallel computing architecture, and deploys a heterogeneous parallel computing architecture. The nodes of the desktop system constitute the parallel computing system. In terms of task allocation, the computing system divides tasks according to the parallelism of tasks. According to the computing resources and bandwidth conditions of each heterogeneous node, starting from the parallel execution time, the task scheduling algorithm is optimized, and the load balancing scheduling scheme is designed to achieve the optimal allocation of resources. In terms of storage resources, the computing system adopts distributed storage as a whole. The CPU-GPU heterogeneous parallel in the desktop system adopts virtual unified storage. Global distributed storage and local shared storage are used to balance overall performance and programming complexity. This article introduces the design and implementation of JTangSync, a distributed heterogeneous data synchronization system. The system adopts a distributed architecture, and each node is organized by a data source module, a data transmission module, a processor module, etc. The data source module is responsible for extracting data, the data transmission module is mainly responsible for efficient data transmission, and the processor module is responsible for data processing. More importantly, each module is designed as a replaceable plug-in, which is convenient for secondary expansion. Each node relies on ZooKeeper to form a cluster, which realizes distributed functions such as centralized management of distributed resources, failover, and resumed transmission. Compared with the mainstream scheduling algorithms HEFT, CPOP, PEFT, and HSIP on heterogeneous systems participating in the experimental evaluation, the scheduling length ratio of DONF series algorithms is reduced by 36.3-67.5 and the parallelism is increased by 17-125 in terms of efficiency. Compared with the existing database synchronization system, the JTangSync system has built-in multiple heterogeneous database data sources and supports the synchronization of complex heterogeneous databases. The system supports users to develop and customize their own data sources and data processing programs, to promote secondary development. By adopting the custom compressed data exchange format and network optimization methods such as packet merging, caching, and adaptive compression algorithm, the system has high performance.
展开▼
