The Jean-Zay supercomputer is recycling its heat
Corporate
Digital
Work was carried out in December 2023 to connect the Jean-Zay supercomputer to the Saclay plateau heating network to recover the waste heat this scientific equipment generates. This work is part of the CNRS's broader plan to reduce energy consumption.
In December 2023 strange pipes began to appear, snaking around the Saclay plateau. The pipes connect the Jean Zay supercomputer's cooling system to the network of the Paris-Saclay Établissement public d'aménagement (EPAPS, Public Development Body)1 . France's leading academic research supercomputer is equipped and coordinated by the French very large-scale computing centre (GENCI) and operated by the CNRS's Institute for Development and Resources in Intensive Scientific Computing (IDRIS). The steel and copper creature-like pipes are there in response to a major challenge – reusing the waste heat from one of Europe's most powerful supercomputers (French link) to heat the equivalent of 1000 new homes on the Saclay plateau.
- 1The EPAPS is responsible for regional planning, economic and property development. It is under the joint supervisory authority of France's ministers for town planning, higher education and research. Its mission is to unite academic stakeholders on one of the world's largest campuses, strengthen the links between public research and companies to boost innovation, and finally to develop infrastructure (housing, transport, facilities, services) intended to enhance the quality of life in this region of Greater Paris.
The CNRS energy efficiency plan is making great steps forward
The CNRS's energy sobriety plan was launched in 2022 in application of ministerial directives. Its aim is a 10% reduction to the organisation's 2019 energy consumption by the end of 2024. There has been rapid progress. In barely a year, the CNRS had reduced its energy consumption by nearly 8% (see key figures) by the summer of 2023.
This significant reduction can firstly be explained by the insulation work (French link) that financial support from the State enabled the CNRS to carry out on its buildings. The Laboratory for Analysis and Architecture of Systems (LAAS-CNRS) provides an emblematic example. Until 2023, 80% of the energy consumed by this Toulouse computing laboratory was down to its clean room. Compared with the 2017 figures, the LAAS-CNRS has now reduced its electricity consumption by 37% and its gas consumption by 80%. It achieved this through the installation of a heat recovery system and photovoltaic panels combined with modulating dampers on the ventilation systems while also more effectively supervising the laboratory's energy expenditure.
As well as action on insulation and so forth, professional networks of technical building correspondents and sustainable development advisors have been set up combined with initiatives to raise energy awareness among members of the scientific community. These have made it easier to share experiences and best practices in the field of energy efficiency. Communication campaigns have also been run to promote eco-actions.
Recycling waste heat
It is difficult to reduce energy consumption so another valid idea is to re-inject excess energy into other systems such as the management of IDRIS, which hosts and operates the Jean Zay supercomputer, has chosen to do. This supercomputer is made up of a very dense assembly of servers stacked on top of each other and federated by a very fast network. It processes problems that cannot be solved with desktop computers by parallelizing calculations (giving each server a small part of a problem to process). This type of method can be applied to a variety of research themes. Jean Zay has trained an artificial intelligence to play bridge (French link), helped model the coronavirus (French link) and in 2022 contributed to the development of Bloom (French link), one of the largest multilingual and open science language models ever created.
However this digital calculation champion has an energy consumption of 17 GWh/year which makes it one of the CNRS's most energy-intensive pieces of equipment. On top of the electricity consumption required to reach the peak power of 36.85 petaflops, the cost of cooling the machine can represent 20 to 50% more electricity, depending on the type of cooling used. All of this injected energy ends up as what is known as 'waste heat'. From the outset, this level of energy consumption led IDRIS to focus its research on the eco-responsibility of its equipment. "IDRIS is among the CNRS's most energy-intensive laboratories so we are very sensitive to the issue of energy costs. That's why we aim to optimise the science we can produce with 17 GWh/year", explains Pierre-François Lavallée, the unit director.
To achieve this, the laboratory used a special investment made by the Ministry of Higher Education and Research to install a liquid cooling system at 30-36°C, compared with 12°C for the previous supercomputer. This temperature is sufficient to capture heat calories from the supercomputer's internal network which heats up to 42°C. Cooling less means consuming less energy.
Also, in 2011 IDRIS installed a system to recover waste heat from Jean Zay and reuse it to heat its own buildings and part of the premises of the neighbouring Interdisciplinary Computer Sciences Laboratory1 . This installation has a far greater capacity and can now heat the equivalent of 1000 new homes on the Saclay plateau. Rafael Medeiros, the IDRIS's deputy systems and operations manager in charge of technical infrastructure is heading the project to connect the laboratory's network to that of the EPAPS. When thinking back on the project he recalls that after the CNRS and the EPAPS officially signed an agreement in February 2021, the EPAPS "asked us to contribute to their temperature-regulated water network. It was an opportunity for them to have a waste heat supplier and for the CNRS to have a consumer, the Saclay plateau, in this case".
New purpose-built machines that limit electricity consumption
The future extension to the supercomputer is currently being installed and will also be cooled with hot water, enabling even more heat to be captured and more buildings to be heated. These new partitions are designed to balance out computing processors and graphics processors while also increasing computing power without increasing energy consumption.
Rafael Medeiros assures us that the amount of waste heat produced by the Jean Zay supercomputer means it would have no difficulty providing more heating because "right from the design stage, we make sure our facilities are compatible with recycling heat by including the possibility of connections to the hydraulic network". The Saclay plateau hosts many energy-intensive scientific facilities but the combination of recycled waste heat from Jean Zay and the local use of geothermal energy means the site can be proud of using recovered or renewable energy for 60% of overall consumption.
This unique approach makes the Jean Zay supercomputer something of a pioneer in eco-responsibility. Pierre-François Lavallée is enthusiastic when telling us that this machine "is now one of the world's most eco-responsible supercomputers thanks to this waste heat recovery. If this factor were taken into account when calculating its Power Usage Effectiveness2 , our score would be around 0.8 compared to 1.2. Jean Zay is the first and only supercomputer of this scale in France with this type of configuration and one of the very few worldwide.' The director of IDRIS believes that Jean Zay's performances are being taken into account in "ongoing discussions about the future French exaflop machine".
- 1CNRS / Université Paris-Saclay.
- 2Power Usage Effectiveness is an indicator developed to measure a data centre's energy efficiency. It is calculated by dividing the total energy a data centre consumes by the total energy used by its IT equipment (server, storage, network). The closer it is to 1, the more efficient the energy consumption. As mentioned above, this calculation does not take heat recovery into account.
A 7,8 % overall reduction in energy consumption by CNRS facilities
A 12 % reduction in natural gas consumption and heating networks
A 20 % reduction in fuel oil consumption which now represents just 0.92% of the energy mix
62 % of the 50 sites with the highest electricity consumption recorded reductions in this
