Development of Operational Technology for Meteorological High Performance Computing ()
1. Introduction
High-performance computing addresses the need for supercomputing performance through parallel computing. Science and technology personnel have carried out many studies on this. Research by Reshmi & Poongodi (2018) shows that high-performance computing becomes the most important field in various industries and organizations, which needs to be concentrated more today due to their massive resource requirements. A new generation of computational scientists is expected to further increase the impact of advanced computing in conceiving creative, powerful ways to understand the universe and solve complex problems (Esteban, 2018) . Also, high-throughput, large-scale medical image computing demands tight integration of high-performance computing infrastructure for data storage, job distribution, and image processing. The Vanderbilt University Institute for Imaging Science (VUIIS) Center for Computational Imaging (CCI) has constructed large-scale image storage and processing infrastructure with HPC (Huo et al., 2018) .
Meteorology has always been one of the important application areas for high performance computing. With the rapid development of meteorological numerical forecasting business and scientific research work, the demand for high-performance meteorological data has surged. Since the 1990s, the high-performance computing capabilities of the Chinese meteorological department have made great progress, and the application support capability has been continuously improved. The level of informationization of meteorological services has made important contributions. However, there are few studies and summaries for the application of high performance computing in meteorology, so we will fill the gaps or adds to the existing literature.
In the rest of the paper, we will elaborate on the development of high-performance computing technology, the application and development of high-performance meteorological computing, and future prospects.
2. Development of High Performance Computing Technology
High Performance Computing (HPC) mainly refers to computationally intensive and fast and efficient operations (Chen & Sun, 2017; Sharma et al., 2018; Wei & Wang, 2017) . High-performance computers are often referred to as supercomputers. High-performance computing has become an important branch of the discipline of computer science and technology. It refers to the research and development of high-performance computers from the aspects of architecture, parallel algorithms and software development. High-performance computing constitutes the important support technology of scientific research, also with theoretical and experimental science.
The field of high-performance computing applications continues to expand and its impact is growing. Early high-performance computing was mainly used to solve computational problems in the military field. After the 1980s and 1990s, high performance computing was widely used in scientific computing disciplines such as weather forecasting, oil exploration, nuclear explosion simulation, and life sciences. High-performance computing technology has become a strategic high-point technology for national development, and its development level has become an important indicator to measure a country’s overall national strength (Wang et al., 2008) . High-performance computing plays an indispensable role in national science and technology, national defense, industry, finance, service, and life.
According to the Flynn’s macroscopic classification (Flynn, 1966; Flynn, 1972; Flynn & Rudd, 1996) , most modern high- performance computer systems belong to the MIMD (Multiple Instruction Stream Multiple Data Stream) type of computer.
The hardware components of high-performance computer systems produced by various manufacturers are basically the same. The architectures mainly include Cluster and MPP. The processors include X86 and RISC. The internal interconnection network includes Infiniband, OPA, Ethernet, and proprietary networks. The mainstream operating system includes Linux and UNIX systems, the compiler uses Intel/GNU/PGI compiler, and uses parallel environment such as MVAPICH/OpenMPI. The job management software uses commercial software (LSF, PBSpro, LoadLeveler, etc.) or open source software (Slurm, OpenPBS, Torque, etc.).
Currently, high-performance computer systems are mainly cluster system architectures. Cluster refers to a group of independent computers, including compute nodes, storage nodes, service nodes, pre-processing nodes and other types of servers. They are connected by a high-speed communication network according to a certain structure to form a single computer system to achieve unified management scheduling. Parallel processing is more efficient. The biggest feature of the cluster system is to provide high reliability, scalability and catastrophic (Ji, 2012) .
At present, the key performance indicators for measuring high-performance computer systems mainly include system architecture, machine scale, number of processors (cores), peak speed, measured speed, operating efficiency, memory access speed, network interconnection performance, and performance per watt.
The global supercomputer TOP500 ranking (Roudas et al., 2012; Strohmaier & Meuer, 2000) is a regular evaluation of global high-performance computer systems by international academic organizations. It is published once every six months and has become the industry-recognized de facto standard for measuring high-performance computing. It has become the vane of development status and trends.
The latest data of the world supercomputer TOP500 (June 2018) shows that there are 273 systems over PFLOPS, and the open system technology represented by X86+ Linux has the upper hand. It also used the CPU+ accelerator device. The number and proportion of hybrid architectures continues to increase. The architecture is dominated by the Cluster structure, and the internal node interconnection network is dominated by Infiniband technology. The X86 processor chip has a large advantage, and the Linux operating system still dominates.
While the performance of high-performance computers has improved dramatically, the demands of computationally intensive, data-intensive applications are becoming increasingly apparent. High-performance computing systems face the challenges of low cost, low power consumption, availability of parallel program support, and efficient use of existing legacy code.
In the past few decades, the speed of computer processors has been increasing in accordance with Moore’s Law. Applications get better performance as processor frequencies increase. However, a higher main frequency leads to an increase in energy consumption, resulting in a physical bottleneck of a single CPU chip such as heat dissipation and leakage. In the “post-Moore era”, especially the peak speed of high-performance computers from Terascale to Petascale, only relying on the increase of CPU chip clock frequency cannot meet the application requirements, for this CPU+ acceleration component hybrid architecture is gradually emerging, relying on acceleration components The rapid development of technology and better cost performance support.
Current mainstream acceleration component technologies include GPU (Graphic Processing Unit) and many cores. The number and proportion of hybrid architecture building systems in the TOP500 are constantly improving. Five of the top 10 systems have adopted an accelerated component hybrid architecture, ranking first in the world and ranking second in the world in Shenwei Taihu Light supercomputer system (Duan et al., 2017) . The computer uses domestic Shenwei multi-core technology, with a total of 10.64 million processor cores.
3. Meteorological High Performance Computing Applications and Development
Meteorological applications have always been an important area of high performance computing. Meteorological high-performance computing applications are always the focus of high-performance computing in terms of scale and scope. Advances in high-performance computing technology have played a key role in the development of numerical weather prediction applications. At almost any time, numerical weather prediction systems used the fastest high-performance computers at the time, and numerical weather prediction systems were the fastest. High-performance computers can demonstrate their outstanding predictive application value (Zheng, 2001) .
3.1. High-Performance Computer Systems in China National Level Meteorological Department
Based on the rapid development of computing technology, the meteorological high-performance computing business has been established from scratch, which has effectively promoted the development of meteorological numerical forecasting services. High-performance computer systems have become a core component of the China Meteorological Administration’s IT system, and computing power has become an important indicator of the level of meteorological modernization (Wang et al., 2010a) . In November 1978, the China Meteorological Administration introduced a Hitachi M-170 computer with a million operations per second, 4 MB of memory, and 2.1 GB of disk. This was the most comprehensive computer system in China at that time, mainly used for meteorology. Data processing and running MOS numerical forecasting mode have ended the history of China without numerical forecasting business. In July 1980, the Eurasian regional model was put into operation. In August 1983, the Asian regional model was put into operation. Since then, China’s numerical weather forecast has entered the practical stage.
Since the mid-1990s, in order to adapt to the rapid development of meteorological numerical forecasting business and scientific research work, the meteorological high-performance computer system has been continuously upgraded and its capabilities have been significantly enhanced. National Meteorological Business Center has introduced Galaxy II, CRAY J90, CRAY EL98, CRAY C92, IBM SP2, IBM SP, Dawning 1000A, Galaxy III, Shenwei I, Shenwei New Century-32I, Shenwei New Century-32P, IBM Cluster 1600 High-performance computer systems such as SGI Altix4700, Shenwei 4000A, and IBM Flex P460 support the operation of numerical forecasting services, research and development, and data processing services (see Figure 1).
Since the early 1990s, national meteorological computing capabilities have grown by an order of magnitude every five years. In 2014, after more than trillions of IBM Flex P460 systems were put into operation, computing power was 1 billion times higher than in 1978, and nearly 30 times higher than during the 11th Five-Year Plan period.
From the perspective of the model, the initial stage mainly used general-purpose supercomputers, and since the mid-1990s, it has turned to massively parallel architecture computers.
From the development history, imported high-performance computer systems dominated before 2000, and domestic and imported high-performance computers went hand in hand after 2000.
The main high-performance computer system currently operating in the national meteorological department is the IBM Flex P460. The system is mainly composed of P460 server. The total computing capacity of the system is 1054.2 TFLOPS, the storage physical capacity exceeds 4.2 PB, the total system has a total of 37,120 CPU cores, and the total memory is 163,584 GB. The IBM Flex P460 high-performance computer system can be divided into two subsystems, each with a computing power of 527.1 TFLOPS and a storage physical capacity of 2109.38 TB. The system is installed in the high-performance computer room of the National Meteorological Information Center. As a national-level computing business R&D platform, the IBM Flex P460 high-performance computer system undertakes numerical weather forecasting business, short-term climate forecasting business, numerical forecasting quasi-business and R&D. Each subsystem of the IBM Flex P460 uses an InfiniBand network interconnect with a non-blocking fat tree structure to form a high-speed data network within the system. The IBM Flex P460 high-performance computer system uses cooling water cooling. All cabinets are equipped with water-cooled back doors, and most of the heat is carried away by cooling water.
![]()
Figure 1. China Meteorological Administration high-performance computer development process.
3.2. High-Performance Computer Systems in Provincial Meteorological Departments
With the continuous development of regional center numerical forecasting business research, since 2000, some regional and provincial meteorological departments have built high-performance computer systems of different scales according to their own development needs. The Northeast, East China, Central China, South China, Southwest, Northwest, and Xinjiang Regional Meteorological Centers installed the IBM Flex P460 high-performance computer system in 2014, which greatly improved the high- performance computing power of the regional center.
High-performance computing resources are always scarce resources for the meteorological field. Due to the imbalance between the national level and the local level and the development of numerical forecasting services between local and local, the current scarce resources are distributed within the department (including geographical distribution). Therefore, high-performance computing resources distributed in national bureaus and regional centers will be shared and used to achieve unified management, monitoring and resource sharing, which will solve the problems of resource integration, sharing and collaborative management in meteorological countries and local departments.
The high-performance computer system interconnected network bandwidth in different regions, the numerical forecast mode input data environment, the numerical forecast mode output product mode and the data volume, and the meteorological service system use the numerical forecast mode product to jointly determine the shared scheduling strategy of high-performance computing resources in different places. According to the network interconnection conditions of the existing meteorological high-performance computing resources and the current situation of the model application data environment, the existing high-performance computing resource management of the China Meteorological Administration adopts the “resource pre-allocation” strategy based on the principle of localization priority. For high- performance computer systems at national and regional centers, a pre-allocated unified resource application scheme is adopted to establish a sophisticated resource statistics, allocation and audit mechanism, and corresponding resource pre-allocation and specification means are provided to unify high-performance computing resources. Combined with the needs of each unit and user each year and the usage of the previous year, the corresponding annual resource allocation amount is determined, the resources of each system are uniformly allocated, and the use of high-performance computing resources at the national level and regional centers is reasonably planned. Establish a unified monitoring platform for meteorological high-performance computing in the National Bureau, realize unified real-time monitoring of the operational status and resource usage of high-performance computer systems at the national and regional levels, and realize unified real-time monitoring of the national regional two-level numerical forecasting service system.
According to the different characteristics of the numerical forecasting business model and scientific research mode operation, different application methods are adopted. According to the current situation of computing resource load of each system, local resources are preferentially used.
We establish a unified operation platform for numerical models at the national level, which can coordinate the planning of national and regional model operations, and achieve unified scheduling and operation and maintenance management of numerical forecasting services. Since the business model application is relatively mature and the input and output data flow is fixed, the numerical forecasting service will mainly run on local high-performance computers. The regional numerical forecasting business model that cannot be satisfied by the regional center local system is deployed to the national high-performance computer system through resource pre-allocation management. Business backup is carried out in a cold backup mode. In special periods such as major meteorological services or major meteorological disasters, the application of hot backup mode is used to simultaneously run a certain business model to ensure reliable operation of the business. Due to the diversity and uncertainty of the application of the R&D model, resources are used according to the principle of local priority and off-site deployment (Wang et al., 2010b) .
4. Conclusion and Discussion
With the emergence of new technologies such as multi-core/many-core architecture processors, hyper-scale parallel processing, heterogeneous programming languages, GPU computing, big data, cloud computing, etc., scientific computing will undergo changes in both technology and thinking. The industry is at a turning point in the development of computing technology, and the future development trend must be the integration of multiple technologies. The development of high performance computing presents a trend of polymorphism, heterogeneity and extreme scale parallelism. Domestic and foreign manufacturers are stepping up the development of high-end computer systems. It is foreseeable that heterogeneous acceleration architecture computers will usher in greater development. Lightweight nuclear technology will promote the integration of processors and acceleration devices, and the programming software tool chain is maturing.
High-performance computer system construction, resource management, and technology development will better support the development of meteorological numerical model software business and scientific research. In response to new trends in technology development, we have added applications for new technologies such as large-scale core and GPU computing. We must pay attention to the parallel computing of cross-cut talents, promote the development of the meteorological business model to the new parallel technology architecture platform, and improve parallel scalability. We also need to coordinate the layout, construction and management of high-performance computing resources in the meteorological department, and gradually reduce the small-scale system with geographically dispersed. In order to meet the needs of the numerical weather and climate forecasting model business operation and scientific research work, we will build a new generation of domestic high-performance computer systems, alleviate the shortage of computing resources, and support the business research work of numerical weather forecasting, climate prediction and climate change.
In the future, high-performance computing resource management will gradually transit from the current local pre-allocation mode to the local remote unified scheduling and shared use. Based on the national level and regional unified central meteorological information sharing platform, it realizes dynamic and unified scheduling of cross-systems for limited applications such as numerical forecasting service application and GRAPES mode research and development, and realizes off-site backup and reliable operation of business applications.