Transitioning compute-intensive applications from prototype to industry-ready production software for high-end local clusters of thousands of compute elements is a significant undertaking. Computational complexity must be organized on the available architecture, optimizing parallelization and data movement strategies (mpi, peer-to-peer, thread-based), while taking into account the granularity of the computational infrastructure (fiber, nodes, cores, file systems) and avoiding competition in accessing the resources. Reliability, fault tolerance and recovery mechanisms are topics of special concern in HPC software design, especially when the processing of a single job may require several weeks even on powerful supercomputers.
Further, software customization to industrial applications requires modulating the dataflow of the implementation in terms of data input/output requirements, format conversions, and human-machine interfaces to satisfy all production requisites and facilitate the daily use of the HPC infrastructure.
Every architectural change in the infrastructure affects the application performance, making necessary to port the software so that an optimal executable program can be created in the new compute environment. To pursue the best performance on the target hardware sometimes means the re-design of algorithms and their parallelization pattern or, in the extreme case, the re-formulation of the physical model to obtain a significant speedup using state-of-the-art acceleration hardware at the cost of a controlled loss of accuracy.
Altogether, these activities make the difference between a research prototype and a stable HPC product devised for breakthrough science and industrial engineering.
The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.