EuroHPC JU
The supercomputer Leonardo is co-funded by EuroHPC Joint Undertaking, a joint initiative between the EU, European countries and private partners created to lead the way in European supercomputing.Its mission is to develop a European supercomputing ecosystem, deploying world-class supercomputers across Europe, and developing a full European supply chain: from processors and software to applications to be run on these supercomputers and know-how to develop strong European expertise. Some of these supercomputers will be exascale systems, able to perform more than one trillion operations per second. The EuroHPC supercomputers are made accessible to European researchers, industry and businesses, to develop new applications in areas such as artificial intelligence and personalised medicine, drug and material design, bio-engineering, weather forecasting, and combating climate change. The EuroHPC JU was established on 28 September 2018 and is currently regulated by Council Regulation (EU) 2021/1173.
about
Hosting Entity
CINECA, established in 1969 as an Interuniversity Consortium for Automatic Computing in North Eastern Italy, is today the largest computing center in Italy, one of the most important in the world.
CINECA, made up of 103 public entities, operates under the control of the Ministry of University and Research, and offers support to the activities of the scientific community through supercomputing and its applications, creates management systems for university administrations and the MUR, designs and develops information systems for public administration, health care and businesses.
As hosting entity of the project, CINECA, faithful to its mission of promoting the use of the most advanced high-performance scientific computing systems for the benefit of the public and private national research system, will take care of all those actions necessary to host and make Leonardo functional, from the construction of the installation site to its management; in addition, it will provide documentation and support to users, public or private, who will use Leonardo for their projects.
Financing Entities
The project was made possible thanks to the joint commitment of various entities: we present them below.
The avant-garde research in the field of the high performance computing is an asset of strategic importance at an international level for the country system, and it is with this spirit that the Ministry of University and Research supported Italy’s candidacy to host the Leonardo project, in collaboration with the EuroHPC Joint Undertaking, in order to realize the first European HPC network
The project is funded by the MUR with 120 million euros distributed over seven years (2019-2025) and by the EuroHPC Joint Undertaking with another 120 million euros, and will have repercussions in various areas: from the management of related risks to natural disasters and pandemics, to predictive medicine; from the development of new materials to cybersecurity; from weather forecasts to artificial intelligence.
The project is part of the actions that European Union is implementing to support the spread of high performance computing as a driving force for growth and innovation. In fact, 50 percent of the computing power generated by Leonardo will be available to research institutes, universities, but also companies in the host country; the remainder, on the other hand, will be used by the countries participating in the Joint Undertaking.
Other Partners
The Emilia Romagna Region always been attentive to technological innovation in its territory, and envisages it as a priority axis aimed at strengthening the regional system of industrial research and technology transfer.
During the last years the Emilia Romagna Region worked to strength the regional network of laboratories on the territory, through a development program of technopoles, i.e. infrastructures dedicated to scientific research and of industrial interest that will establish laboratories, research centers and structures for the incubation of new innovative and high-tech companies.
The Leonardo project received the patronage of the Emilia Romagna Region, which, as part of the expansion of the regional network of Technopoles, provided the building of the Ex Manifattura Tabacchi as the seat of the new Cineca and INFN Data Center.
The National Institute for Nuclear Physics (INFN) is the Italian research agency dedicated to the study of the fundamental constituents of matter and the laws that govern them, under the supervision of the Ministry of Universities and Research (MUR). It conducts theoretical and experimental research in the fields of subnuclear, nuclear and astroparticle physics. All of the INFN’s research activities are undertaken within a framework of international competition, in close collaboration with Italian universities based on solid academic partnerships spanning decades
INFN provides the project with its expertise regarding the aspects of handling large volumes of data and binding of high-speed networks.
SISSA – Scuola Internazionale Superiore di Studi Avanzati – was founded in 1978 and is a scientific center of excellence > within the national and international academic scene.
SISSA’s activities focus on three main areas: Physics, Neuroscience and Mathematics.
SISSA is at the international forefront of numerical simulation and modeling across disciplines ranging from the infinitely small, nano-sciences and materials science, to the infinitely large, astrophysics and cosmology, and withinthe project makes available its scientific skills for the creation of a national and international ecosystem for HPC.
Leonardo Supercomputer
The rapid growth of the amount of data coming from diverse sources, including numerical models and sensors, asks for capable and reliable processing and storage services.
Leonardo is one of the most powerful supercomputing systems worldwide, capable of executing more than 250 Petaflops, i.e., more than 250 million billion operations per second. It is remarkable that each single GPU based rack provides the same computational power the entire BlugeneQ Fermi system, decommissioned just in 2015.
Leonardo is a key enabling technology, aiming to raise the competitiveness of Italian and European research, paving the way to the forthcoming exascale era in Europe.
Technical Features Overview
Leonardo has two main computing modules, named Booster and Data Centric, that will cover a wide variety of computing requests and user needs. The modules are based on two state-of-the-art processing units. A customized version of NVIDIA Ampere GPUs, called A100 SXM6 64GB , equips the Booster module, whileevwry blade of the Data Centric module features two Intel new generation Sapphire Rapids 56 cores processors. Both are extremely powerful computing engines and key factors in enabling such high system performance.
These processor units are optimized to perform computationally intensive workloads on high-performance computing systems like Leonardo. They are built to offer the same performance over time while constantly working at a usage close to 100%. Notably, the system operates to offer a 24/7 high performance computing service for its entire 5-year lifetime.
The data network performance is a key aspect that impacts the capability of a supercomputer to processing large-scale, inter-node simulations, possibly scaling over hundreds of computing servers.
Leonardo features a Dragonfly+ topology based on NVIDIA Mellanox HDR 200Gb/s InfiniBand interconnection, with smart in-network computing acceleration engines that enable extremely low latency and high data throughput to provide the highest AI and HPC application performance and scalability.
For the data network, over 25000 fiber cables and 800 network switches have been used to interconnect the whole Leonardo infrastructure.
The extreme processing capacity provided with Leonardo needs to be supported by cutting-edge storage infrastructure. Leonardo storage is composed of two tiers. A fast tier – based on flash storage – providing a remarkable aggregated bandwidth of 1,4 TB/s and able to support high-IOPS workloads. Besides, a capacity tier ensures a storage space of 106 PB and an aggregated bandwidth of over 600 GB/s.
Leonardo system architecture is designed to provide a storage/main memory capacity ratio of 66, well in the range of similar top ranked HPC systems.
Leonardo is based on the BullSequana XH2000 solution, a technology designed, developed, produced and tested by Eviden. This technology is able to integrate GPU and CPU on extremely dense computing blades and therefore to provide a modern infrastructure that offers maximum performance for computing, data processing, High Performance Data Analytics (HPDA), Artificial Intelligence (AI) and the development of Machine Learning models.
Remarkable energy efficiency is achieved through direct liquid cooling circulating through cold plates that reside on top of blade computing motherboards. This cooling system allows to extract 95% of the heat dissipated inside the rack. Direct liquid cooling is used to cool all critical components within the rack, such as processing blades, interconnect and management switches, and PSUs.
Processing unit
Leonardo has two main computing modules, named Booster and General Purpose, that will cover a wide variety of computing requests and user needs. The modules are based on two state-of-the-art processing units. A customized version of NVIDIA Ampere GPUs, called A100 SXM6 64GB , equips the Booster module, while every blade of the Data Centric module features each two Intel new generation Sapphire Rapids 56 cores processors. Both are extremely powerful computing engines and key factors in enabling such high system performance.
These processor units are optimized to perform computationally intensive workloads on high-performance computing systems like Leonardo. They are built to offer the same performance over time while constantly working at a usage close to 100%. Notably, the system operates to offer a 24/7 high performance computing service for its entire 5-year lifetime.
Network
The data network performance is a key aspect that impacts the capability of a supercomputer to processing large-scale, inter-node simulations, possibly scaling over hundreds of computing servers.
Leonardo features a Dragonfly+ topology based on NVIDIA Mellanox HDR 200Gb/s InfiniBand interconnection, with smart in-network computing acceleration engines that enable extremely low latency and high data throughput to provide the highest AI and HPC application performance and scalability.
For the data network, over 25000 fiber cables and 800 network switches have been used to interconnect the whole Leonardo infrastructure.
Storage
The extreme processing capacity provided with Leonardo needs to be supported by cutting-edge storage infrastructure. Leonardo storage is composed of two tiers. A fast tier – based on flash storage – providing a remarkable aggregated bandwidth of 1,4 TB/s and able to support high-IOPS workloads. Besides, a capacity tier ensures a storage space of 106 PB and an aggregated bandwidth of over 600 GB/s.
Leonardo system architecture is designed to provide a storage/main memory capacity ratio of 66, well in the range of similar top ranked HPC systems.
integration
Leonardo is based on the BullSequana XH2000 solution, a technology designed, developed, produced and tested by Eviden. This technology is able to integrate GPU and CPU on extremely dense computing blades and therefore to provide a modern infrastructure that offers maximum performance for computing, data processing, High Performance Data Analytics (HPDA), Artificial Intelligence (AI) and the development of Machine Learning models.
Remarkable energy efficiency is achieved through direct liquid cooling circulating through cold plates that reside on top of blade computing motherboards. This cooling system allows to extract 95% of the heat dissipated inside the rack. Direct liquid cooling is used to cool all critical components within the rack, such as processing blades, interconnect and management switches, and PSUs.
Leonardo Consortium
The Leonardo Consortium, led by Italy, is made up of six signatory countries of the European Declaration on High Performance Computing.
The mission of the Declaration is to participate in a joint effort to build and deliver a world-class computing and data infrastructure in Europe, and sustain the computing capability for the consortium and European researchers.
The six Countries of the Consortium are:
- Italy
- Austria
- Greece
- Hungary
- Slovakia
- Slovenia
- Countries which have signed the EuroHPC Declaration
- Leonardo Consortium
The six countries of the consortium, wich all boast a long and fruitful tradition in computational sciences, committed themselves to make their HPC expertise available to the project, providing high-level technical and engineering support to the user’s communities.
