Today, Sergi Girona, Operations Director at BSC, inform as that on July 1st MareNostrum 4 will come in real production for the first research projects that uses the general purpose block of Marenostrum 4. Great new! I want to thanks the Operations department for their maratonian work.
MareNostrum IV peak power is 11.15 Petaflops, or what is the same, it is able to perform more than eleven thousand trillion operations per second, ten times more than the MareNostrum3, which was installed between 2012 and 2013. Although its power is ten times greater than that of its predecessor, it only increases energy consumption by a 30% and now is of 1.3 MWatt/year.
It have two distinct parts: General Purpose part and Emerging Technologies part.
MareNostrum IV is a supercomputer based on Intel Xeon Platinum processors from the Skylake generation. It is a Lenovo system composed of SD530 Compute Racks, an Intel Omni-Path high performance network interconnect and running SuSE Linux Enterprise Server as operating system. Its current Linpack Rmax Performance is 6.2272 Petaflops.
This general-purpose block consists of 48 racks housing 3456 nodes with a grand total of 165,888 processor cores and 390 Terabytes of main memory. Compute nodes are equipped with:
- 2 sockets Intel Xeon Platinum 8160 CPU with 24 cores each @ 2.10GHz for a total of 48 cores per node
- L1d 32K; L1i cache 32K; L2 cache 1024K; L3 cache 33792K
- 96 GB of main memory 2GB/core (216 nodes high memory, 10368 cores with 8GB/core)
- 100 Gbit/s Intel Omni-Path HFI Silicon 100 Series PCI-E adapter
- 10 Gbit Ethernet
- 200 GB local SSD available as temporary storage during jobs
The processors support well-known vectorization instructions such as SSE, AVX up to AVX–512.
More technical detail of this part could be found here.
The second element of MareNostrum 4 is formed of clusters of three different technologies that will be added and updated as they become available. These are technologies currently being developed in the US and Japan to accelerate the arrival of the new generation of pre-exascale supercomputers.
- A cluster consists of IBM POWER9 processors and NVIDIA Tesla GPUs, which are the same components that IBM and NVIDIA will use for the Summit and Sierra supercomputers that the US Department of Energy has commissioned for the Oak Ridge and Lawrence Livermore National Laboratories. Computing power over 1.5 Petaflop/s.
- A cluster made up of Intel Knights Hill (KNH) processors. They are the same processors that will be inside the Theta and Aurora supercomputers purchased by the US Department of Energy for the Argonne National Laboratory. Computing power in excess of 0.5 Petaflop/s.
- A cluster formed of 64 bit ARMv8 processors in a prototype machine, using state-of-the-art technologies from the Japanese Post-K supercomputer. Computing power over 0.5 Petaflop/s.
The goal of the progressive incorporation of these emerging technologies into MareNostrum4 is to enable BSC to operate with what are expected to be some of the most state-of-the-art developments of the coming years and to test if they are suitable for future versions of MareNostrum.
MareNostrum4 has a disk storage capacity of 14 Petabytes and is connected to the Big Data infrastructures of BSC-CNS, which have a total capacity of 24.6 Petabytes. Like its predecessors, MareNostrum4 is also connected to the network of European research centres and universities through the RedIris and Geant networks.
(*) source of image: https://www.bsc.es/innovation-and-services/supercomputers-and-facilities/marenostrum/technical-information