빅데이터 분석 및 머신러닝 서비스 제공을 위한 컨테이너 기반 HPC 클러스터 구축 방안

Container-based HPC Cluster System for BigData Analysis and Machine Learning Applications

Several fields of basic sciences such as meteorology, chemistry, and high-energy physics have required large-scale workloads. To support it, a high-performance system (HPC), which is optimized for parallel computing is widely used. Recently, due to the fourth industrial revolution, HPC computing resources are increasing demands in computer science fields such as big data analysis and machine learning. To support these new services, the existing HPC system has big troubles such as rigid SW configuration, high maintenance cost, and limited job type support. In this paper, we propose a container-based HPC cluster system architecture to solve these problems. The presented system architecture is used Docker for container execution and kubernetes as the container orchestration toolkit.