- Open Access
Exploring the support for high performance applications in the container runtime environment
© The Author(s) 2018
- Received: 13 July 2017
- Accepted: 20 December 2017
- Published: 8 January 2018
Cloud computing is the driving power behind the current technological era. Virtualization is rightly referred to as the backbone of cloud computing. Impacts of virtualization employed in high performance computing (HPC) has been much reviewed by researchers. The overhead in the virtualization layer was one of the reasons which hindered its application in the HPC environment. Recent developments in virtualization, especially the OS container based virtualization provides a solution that employs a lightweight virtualization layer and promises lesser overhead. Containers are advantageous over virtual machines in terms of performance overhead which is a major concern in the case of both data intensive applications and compute intensive applications. Currently, several industries have adopted container technologies such as Docker. While Docker is widely used, it has certain pitfalls such as security issues. The recently introduced CoreOS Rkt container technology overcomes these shortcomings of Docker. There has not been much research on how the Rkt environment is suited for high performance applications. The differences in the stack of the Rkt containers suggest better support for high performance applications. High performance applications consist of CPU-intensive and data-intensive applications. The High Performance Linpack Library and the Graph500 are the commonly used computation intensive and data-intensive benchmark applications respectively. In this work, we explore the feasibility of this inter-operable Rkt container in high performance applications by running the HPL and Graph500 applications and compare its performance with the commonly used container technologies such as LXC and Docker containers.
- Cloud computing
- High performance computing
- Core OS Rkt
- Graph 500
Cloud computing is being used for innumerable applications these days. The end-users vary from naive clients to expertised technicians. Cloud is a pool of resources shared among number of users . Presently, in the world of cloud computing, it is the era of XaaS (Anything-as-a-Service) which means that the providers offer a wide variety of services [2, 3]. One of the most recent services provided through the cloud is high performance computing (HPC) environments for the complex applications. Virtualization is the technology which enables users to share a single entity among a group of users. Based on the position of the virtualization layer, virtualization can be of different types like full virtualization, paravirtualization and OS level virtualization.
Traditional HPC clusters are composed of many separate dedicated servers called nodes and may be shared among different organizations. The requirements of each user or organization will be different, which demands the creation of customized environments without affecting others. This is not an easy task in traditional HPC systems. As a solution for this, virtualization was adopted for HPC. Virtualization materializes the task by creating separate customized virtual environments of the system based on the requirements of each user.
The Hypervisor based virtualization employs a full guest OS in each virtual machine along with the necessary binaries and libraries for the applications. Containers will hold only the necessary binaries required by any application to run . Containers possess packaged, ready to deploy applications or parts of applications, and if necessary middleware and business logic to run those applications . Figures 1 and 2 shows the two different virtualization architectures .
There exists different container management technologies for managing the entire life cycle of containers. This includes creating, deleting and performing modifications on images and tools associated with it. Linux LXC, Docker and the latest release Rkt are the major managing technologies. All the technologies work on the same base principle that the application space should be isolated within the operating system.
Investigate the existing works which aim at comparing and contrasting the various container technologies in a performance-oriented perspective.
Explore the features which differentiate the Rkt container runtime from the other runtimes.
Implement archetypes to assess the variation in support provided by the Rkt container runtime for applications with high performance requirements.
Analyze how the different features impact the performance of computationally challenging tasks.
Analyze the impact of the features specific to the Rkt container runtime on the data intensive tasks.
The rest of this paper is structured as follows: “Background and related work” section contains the history of containers and similar works in this area. Detailed description and specification of experiment setup and various benchmarking tools is elaborated in “Experimental setup and benchmarking tools” section. “Results and discussions” section contains the obtained results and analysis done based on it. Finally, the conclusion and future scope of research is described in “Conclusion” section.
Values of the input parameters
# of values
Problems size (N)
10000 11000 12000 13000 15000
Block size (NBs)
100 92 104 98
Process grids (P × Q)
P = 4, Q = 8
Right left crout
Recursive stopping criterium (NBMin)
Recursive panel fact (RFACTs)
Panels in recursion (NDIVs)
Our experiments were performed on an HP EliteDesk 800G1 Tower system with two Intel 4th Generation Core i7 Processors for a total of 16 cores. We used Ubuntu 16.04.2 LTS (Xenial Xerus) 64 bit with Linux kernel 4.4.0-62-generic, Docker container 1.3.0, LXC 2.0.7, and Rkt container 1.24.0. For consistency, we used the same Ubuntu base image on these different platforms.
CPU throttling was disabled in all the experiments because of the usage of BLAS (Basic Linear Algebra Subprogram) specification in the ATLAS library. The ATLAS library requires disabling of CPU Throttling for its proper execution.
High performance computing applications can be either Computation Intensive or Data Intensive. We evaluated the performance of Rkt container in both scenarios. HPL benchmarking tool (computing intensive application) and Graph500 (data intensive applications) are used for analysing the suitability of respective container platform and for measuring the performance in high performance computing applications. For analysis, the Linpack 2.1 CPU-intensive benchmark and Graph 500 2.1.4 Data-intensive benchmarks were executed and results were collected.
Computing intensive applications
Linpack is the benchmarking tool which we used for analysing computing intensive applications. It has two implementations. One is optimized only for Intel machines and the other one is supported by all machines. HPL is the portable implementation of Linpack, is written in C and requires an MPI implementation, BLAS interface implementation ATLAS library. HPL tests the performance of a system by generating and solving a system of equations using LU decompositions .
Lower upper factorization of a random matrix.
Lower upper factorization used to solve the linear system consisting of the random matrix and a scalar.
Varying the problem size (N).
Varying block size (NB).
Varying the rfact and panel fact.
With these input parameter configurations around 120 tests were performed in all the different container runtimes and the averaged experiment results on several runs were used for the purpose of analyzing the performance.
Data intensive applications
generates edge list.
construction of graph from the generated edge list.
BFS (Breadth First Search) on the constructed graph
randomly selects 64 unique search keys whose degree is greater than or equivalent to one.
parent array of each key is computed and check whether it is a valid BFS search tree.
There exists different problem classes based on the size of the problem. They are toy (17 GB) called as level 10, mini (140 GB)-level 11, small (1 TB)-level 12, medium (17 TB)-level 13, large (140 TB)-level 14, huge (1.1 PB)-level 15.
We explore the performance of CoreOS Rkt container in HPC environment and compare it with existing popular container runtimes such as Docker and Linux LXC. HPC environment mainly comprises of applications which are data intensive or computation intensive in nature. We considered both of these important scenarios for our experimental analysis. The tests were carried out in single node environment and later in a cluster environment. We have tuned the HPL benchmark and tests were carried out.
Exploring computing intensive applications
Rising zone The low problem size wont invoke memory and processor to their maximum performance.
Flat zone Problem invokes only processor to its maximum performance.
Decaying zone Problem size is too large and cache memory is not large enough to keep all the necessary data, because processor is running in top speed.
The Rkt container runtime runs for a longer time in the flat zone implying that the load experienced by the containers are less challenging. Even when the Docker container subsides to the decaying zone, the Rkt container continues in the flat zone. This is supporting the intuition that Rkt container engines provide better HPC support. This can be attributed to the absence of a daemon process in the Rkt runtime stack. The Rkt containers are started based on command clients. This also resolves compatibility issues with init systems (which are existing in Docker Runtime). The absence of such a daemon increases the startup time slightly, but greatly lowers the overhead experienced by the containers. This is the major reason why the Rkt containers fare better in the high performance scenarios.
Exploring data intensive applications
Graph500 represents a data intensive application benchmark. This benchmark attempts to obtain the BFS of a large graph and the performance is evaluated. Data traceability is the criteria which is used for evaluation, it is the ability to keep track of data and accessing on a system. We examined the performance of Graph500 with varying Edge factors. We took edge factors varying from 16 to 36. Results are measured in TEPS which is the Traversed Edges Per Second. We explored the simple graph problem size in the Graph500 benchmark. The results given in Fig. 7a–d show that LXC, Docker and Rkt containers give near native performance because they are not emulating an entire system like the host and they are virtualized only at the level of operating system. Containers are sharing the same kernel of the host system where it is running and this enables them to access data in a very fast manner. Compared to virtual machines, containers fare better in data intensive applications and Rkt container gives similar performance results along with other containers such as Docker and LXC.
From the results shown in Fig. 8, it can be deduced that the Rkt container performs well in data intensive applications, but, the performance is lower when compared to the other container environments. The LXC container has the performance closest to the native performance in data-intensive applications.
The results shows that Rkt container performs well in both computation intensive and data intensive high performance application environments. However, the Rkt containers are better suited for computation-intensive applications. The improved support for computation-intensive applications coupled with advanced security features connotes that the Rkt containers offer a viable option for HPC. There are more challenges to be resolved to deliver the Rkt environment as the best option for HPC applications. Being a successor of Docker, the Rkt containers are better in some aspects, on the other hand, being an emerging technology still in its evolving stage, there remains more scope for improvement.
Container technology is becoming a widespread platform used in Cloud computing. High performance computing centers are now using containers for their complex applications because of the flexibility and gain in productivity. Adopting containers is not an easy task. In this work, we explore the feasibility of CoreOS Rkt container in the HPC world. We performed evaluations in two scenarios related to HPC. One is the data intensive application environment and the other is the computation intensive environment. We explored the widely used containers such as Docker and LXC in the same scenarios and provided a comparison of the results.
The results of the experiments show that the CoreOS Rkt container gives near native performance in computational intensive and data intensive high performance application environment. The results are promising and indicate that Rkt is well suited to run HPC applications as well, especially for computation-intensive workloads. Future research can experiment various HPC applications, measure the communication performance and tune the algorithms accordingly. The enhanced security features, interoperability and better performance will lead Rkt to a prominent position in HPC environments in the near future.
JPM conducted the experiments, analysed the results and drafted the manuscript. AK and KC provided valuable suggestions on improving the standards of the manuscript. All authors read and approved the final manuscript.
The authors thank the reviewers for their suggestions which helped in improving the quality of the paper.
The authors declare that they have no competing interests.
Availability of data and materials
Consent for publication
Ethics approval and consent to participate
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
- Moon Y, Yu H, Gil J-M, Lim J (2017) A slave ants based ant colony optimization algorithm for task scheduling in cloud computing environments. Hum-centric Comput Inf Sci 7(1):28View ArticleGoogle Scholar
- Zhu W, Lee C (2016) A security protection framework for cloud computing. J Inf Process Syst 12(3):538–547Google Scholar
- Kar J, Mishra MR (2016) Mitigate threats and security metrics in cloud computing. J Inf Process Syst 12(2):226–233Google Scholar
- Huh J-H, Seo K (2016) Design and test bed experiments of server operation system using virtualization technology. Hum-centric Comput Inf Sci 6(1):1View ArticleGoogle Scholar
- Yu H-E, Huang W (2015) Building a virtual hpc cluster with auto scaling by the docker. arXiv preprint arXiv:1509.08231
- Louati T, Abbes H, Cérin C, Jemni M (2018) Lxcloud-cr: towards linux containers distributed hash table based checkpoint-restart. J Parallel Distrib Comput 111:187–205View ArticleGoogle Scholar
- Ciuffoletti A (2015) Automated deployment of a microservice-based monitoring infrastructure. Procedia Comput Sci 68:163–172View ArticleGoogle Scholar
- Babu A, Hareesh M, Martin JP, Cherian S, Sastri Y (2014) System performance evaluation of para virtualization, container virtualization, and full virtualization using xen, openvz, and xenserver. In: 2014 fourth international conference on advances in computing and communications (ICACC). IEEE, New York, pp 247–250Google Scholar
- CoreOS. https://coreos.com/rkt. Accessed 23 June 2017
- Julian S, Shuey M, Cook S (2016) Containers in research: initial experiences with lightweight infrastructure. In: Proceedings of the XSEDE16 conference on diversity, big data, and science at scale. ACM, New York, p 25Google Scholar
- Jacobsen DM, Canon RS (2015) Contain this, unleashing docker for hpc. In: Proceedings of the Cray User GroupGoogle Scholar
- Felter W, Ferreira A, Rajamony R, Rubio J (2015) An updated performance comparison of virtual machines and linux containers. In: 2015 IEEE international symposium on performance analysis of systems and software (ISPASS). IEEE, New York, pp 171–172Google Scholar
- Linux LXC. https://linuxcontainers.org/lxc/. Accessed 23 June 2017
- Docker Hub. https://hub.docker.com/. Accessed 23 June 2017
- Kniep C (2014) Containerization of high performance compute workloads using docker. doc.qnib.orgGoogle Scholar
- de Alfonso C, Calatrava A, Moltó G (2017) Container-based virtual elastic clusters. J Syst Softw 127:1–11View ArticleGoogle Scholar
- Morabito R, Kjällman J, Komu M (2015) Hypervisors vs. lightweight virtualization: a performance comparison. In: 2015 IEEE international conference on cloud engineering (IC2E). IEEE, New York, pp 386–393Google Scholar
- Xavier MG, Neves MV, Rossi FD, Ferreto TC, Lange T, De Rose CA (2013) Performance evaluation of container-based virtualization for high performance computing environments. In: 2013 21st euromicro international conference on parallel, distributed and network-based processing (PDP). IEEE, New York, pp 233–240Google Scholar
- Chung MT, Quang-Hung N, Nguyen M-T, Thoai N (2016) Using docker in high performance computing applications. In: 2016 IEEE sixth international conference on communications and electronics (ICCE). IEEE, New York, pp 52–57Google Scholar
- Kozhirbayev Z, Sinnott RO (2017) A performance comparison of container-based technologies for the cloud. Fut Gener Comput Syst 68:175–182View ArticleGoogle Scholar
- Varma PCV, Kumari VV, Raju SV et al (2016) Analysis of a network io bottleneck in big data environments based on docker containers. Big Data Res 3:24–28View ArticleGoogle Scholar
- Chung MT, Le A, Quang-Hung N, Nguyen D-D, Thoai N (2016) Provision of docker and infiniband in high performance computing. In: 2016 international conference on advanced computing and applications (ACOMP). IEEE, New York, pp 127–134Google Scholar
- Peinl R, Holzschuher F, Pfitzer F (2016) Docker cluster management for the cloud-survey results and own solution. J Grid Comput 14(2):265–282View ArticleGoogle Scholar
- HPL—a portable implementation of the high-performance Linpack benchmark for distributed-memory computers. http://www.netlib.org/benchmark/hpl/. Accessed 4 July 2017
- Top 10 (2016) http://www.graph500.org/. Accessed 4 July 2017