It has been a while again that we reduced the amount of electrical energy with 86% consumed by the cooling installations and you might be asking: why am I writing an article after all those years.
Between 2009 and 2012 we replaced our cooling installation and achieved an energy saving of 86% and in 2013 when we became a finalist at the Uptime Institute GEIT Awards.
Our cooling installation is still unique for two reasons: the cooling method ( cooling towers and Aquifer Thermal Energy Storage ) and because it is a retrofit. It’s much easier to build a new data center then optimize an older data center. We were convinced that things would change in the data center industry but it didn’t: not enough innovation, the industry is still doing a “copy paste” so I want to share our project and the optimizing afterwards.
Before the project
It started with the design of the cooling installation in 2000 which was great at that moment but in 2009 we knew things could be smarter and more efficient. The design existed of the following main components:
- 750 kW chillers
- 28 pumps ( A & B )
- More than 40 in room cooling units
The cooling installation between 2000 and 2009 was not dynamic. The chillers had several steps of operating ( 30, 50, 70 and 100 % ) but together with the 3-way valve system, the pumps and the in-room units, both without a variable-frequency drive. Not efficient at all.
We concluded that the combination of Aquifer Thermal Energy Storage with Cooling towers would be the best solution for of our data center.
The new design existed of the following main components, next to that we insisted in a 2N distribution system as we always had:
- 3 Aquifer Thermal Energy Storage systems ( we can add three systems more )
- 2 Cooling towers
This combination of cooling on the most efficient way would only work if all other components in the cooling installation would be optimized for this set-up, so we focused on the following:
- Keep it simple, as less as components as possible
- Optimal distribution networks
- Optimal data room setup
- The installation should operate fully automatic based on the outside climate conditions ( temperature and humidity ) so we would have an efficient installation during the entire year. For this we needed a Building Management System were we could change all variables ourselves, we shouldn’t be dependent on a contractor.
Generating, transporting and using no more energy then necessary
During the year we make use of the cooling towers as much as possible because this is the most efficient way of cooling for us.
Cold period of the year ( Cooling towers ) During the cold period of the year we cool the data center with the cooling towers.
Very cold period during the year ( Cooling towers & Aquifer Thermal Energy Storage ) During the very cold period of the year we cool the data center with the cooling towers and when the outside wet bulb temperature < 8° C we also store the additional generated cooling capacity with the cooling towers in aquifers approx. 175 meters below in the ground.
Warm period of the year ( Cooling towers & Aquifer Thermal Energy Storage ) During the warm period of the year we cool the data center with the cooling towers. When the cooling towers can not generate enough cooling capacity the Aquifer Thermal Energy Storage system provides the necessary cooling capacity. In this situation the cooling towers and aquifer thermal energy storage systems work together.
Very warm period of the year ( Aquifer Thermal Energy Storage ) During the very warm period of the year we only use the aquifer thermal energy storage for cooling the data center. The cooling capacity stored during the very cold period of the year will be extracted. During this type of operation the cooling towers are not in use.
The cooling capacity we use from the Aquifer Thermal Energy Storage systems needs to be stored in the ground again. The amount of energy in and out should be in balance, this is regulated by the government. Our installation is in balance. The last years we were able to store more cooling capacity in the ground then we extracted during the summer.
Optimal distribution network
With installing an energy efficient cooling generation we still have to transport the cooling capacity to the data rooms on a efficient way. For this we defined the following
- Minimal flow based, demand driven pumps
- Selecting the pump for the right work area
- we “programmed” the entire mechanical installation in a software application which resulted in the most optimal specifications of the pumps. The amount of power used by the pumps for the distribution reduced from 120 to 2 kW!
- Transport only the energy you need
- Measuring all required data, temperature, temperature delta t, flow, pressure etc.
- Open header for optimal efficiency so that we have
- no valves which are controlled by the Building Management System
- cooling the data center and storing cooling capacity during the winter period in the Aquifer Thermal Energy Storage systems at the same time.
- use water in stead of a mixture of water and glycol.
- water can absorb more energy than a mixture of water and glycol
- pumps use less power transporting water in stead of a mixture of water and glycol
Before and after
In-room cooling units:
The goal was: create a as high as possible Δ T ( difference between in and outlet water temperature ) on the cooling unit and make sure the power used by the in-room cooling unit would be brought to a minimum:
In total we optimized more than 40 in-room cooling units in stead of replacing them and by optimizing them we did the following:
- removing the old cabling and electronics we don’t really need for the cooling process together with the humidification unit.
- Keep it simpler
- replacing the old controller for a new controller
- the old controller worked based on return temperature
- the new controller works based on outlet temperature. Based on the outlet temperature the Δ T from the flow and return of the cooling water will increase compared with based on the return temperature
- modify the 3-way valve to an 2-way valve
- Only warm water will return to the cooling generation. The Δ T from the flow and return of the cooling water of the in-room cooling unit will increase
- installing a variable-frequency drive for bring down the the frequency of the fan
- the fan is consuming less energy
- the water flowing through the in-room cooling unit can absorb energy back to the cooling generation
- connecting the in-room cooling unit controller to the Building Management System for optimal monitoring, alarming and improving of the efficiency
- Unit in alarm ( reading )
- Outlet and return temperature ( reading )
- Actual fan frequency ( reading )
- Actual water valve position
- Outlet temperature ( set point )
- Alarm threshold on the outlet and return temperature ( set point )
- Minimal frequency the fan in the in room unit should operate ( set point )
- Fan frequency when another unit in the same data room and header is in alarm ( set point )
- Fan frequency when the cooling installation is in Emergency mode ( set point )
Data room optimization
As mentioned above it was necessary to increase the return temperature at the in-room cooling unit. Modifying the in-room cooling unit was not enough, we completely separated the in- and outlet airflow by doing the following:
- installing closed corridors
- making use of blind panels
After this we were able to increase the outlet temperature of the in-room cooling unit to 28 ° C in stead of the average of 19 ° C.
Before and after
Building Management System
Many different technology’s are used in data centers and for availability and efficiency you need to have a Building Management System in place. Not only for alarming but also for improving of the complete installation. Next to that the Building Management System collects lots of data of all different sensors you need for the daily operations
On all distribution boards providing power to the data center we installed power meters which we connected to the Building Management System. With doing this we not only have a Building Management System bu also an Energy Management System.
Emergency Response System
With the Building Management System we created an “Automatic Response System”. During normal conditions we have a very efficient cooling system and the system ( Controller A & Controller B ) are monitoring ten important values. If one or more of those are not within specification the system will change from very efficient mode to and “Emergency Response mode”. When this mode is activated all available cooling capacity will be activated which gives the operations team the time to find the source of the problem and fix it. The cooling capacity activated during this “Emergency Reponse Mode” is available withing 2 minutes automatically.
Keep it simple!
Moving from a not efficient to a very efficient installation
The 86% energy saving is achieved in a live data center, several IT infrastructures from energy trading and financial company’s were installed in our data center which meant that during the retrofit no-downtime was allowed.
An new header was installed and before taking over the cooling generation and data rooms several test were performed, think about:
Calculations and dry-run tests of:
- half of the available cooling capacity in the data rooms
- the air circulation in the data rooms
- the velocity of the water through the system
- the changes in the pressure
- we used to have more pressure in the system, with bringing the pressure down we could get leakages we wouldn’t have with the high pressure in the system
During the take-over from the cooling generation and data rooms we had additional emergency cooling installed outside the building and prepared to have it operational during the take-over in case of problems. During the entire project we had enough cooling capacity available and our customers didn’t have cooling problems at all.
In total we have spend six weeks for the complete take-over. After one take-over we prepared a perfect fitting connection for the next take-over. We also wanted to have enough time in between so we could test the new installation on a structured way
The team was the most important “part” during the project, the team achieved the 86% energy saving.What I am trying to explain: another project with the same cooling components but with another team wouldn’t achieve the 86%.
Achieving the 86% was possible because of several things:
- everybody trusted each others knowledge and experience so we were able to help each other as one team. During this project a member not only looked after his own responsibility but also how and where it was connected to.
- the members understood each others “technical language“
- if needed we took a time-out if adjustments were needed
- understand the details of the entire installation, complete involvement of all members
- let the engineers talk with each other
- the project was not installed by the department “new buildings” but by the department “service and maintenance” who have the experience you need for operating and maintaining an installation for a long period
- dare for taking the time to optimize
- awareness of energy use of each individual component
- don’t choose the easiest way
- open communication and discussions
- discuss potential problems on a early stage in the project
- create space for innovative ideas
- together GO for the best result
- sharing knowledge with each other
- in business critical processes you can do innovative things like creating a reliable Aquifer Thermal Energy Storage system
- think in solutions, don’t be negative
During and after the project we implemented also the following technical stuff
During the project
- older installations can be optimized, you don’t need to replace them
- the installed valves we use to create a 2N situation ( two different cooling circuits ) are equipped with position sensor which means that while working from remote we know exactly in which mode we are operating.
- the cooling installation and the Building Management System are both true 2N which means we can operate on one header without problems.
- make use of open protocols between all installations
- all control signals are make use of individual IO ports on the controllers, management information is retrieved through a industrial bus connection
- make use of proven technology, many times this technology can be used more efficient or in another set-up
After the project
- keep optimizing based on what you have measured and learned
- if needed we can isolate the source of a mechanical problem with operating the manual valves. These are easily to find because of the red signs on the floor and not for example on the wall ( difficult to find in a technical room most of the time )
- integrating of the industrial PLC’s from our Dynamic Rotating UPS systems from Hitec Power Protection using the mod-bus communication protocol. For every system we monitor almost 250 values ( digital and analog ) in the Building Management System. Together with the power meters it gives a complete overview of all critical infrastructure systems in the data center
- integration of the leak detection system in the Building Management System
- installation of diffusers on top of the cooling towers for maximum cooling.
- the project was designed with water temperatures of 12 °C – 20 °C. Keep on optimizing resulted in a 18 °C-24 °C water temperatures. Because of this we can use the cooling towers more which is the most energy efficient mode of cooling in our data center
Keep improving after the commissioning of the project with the project members
Together we achieved great results during and after the project
- we saved 86% of electrical energy for the cooling installation, these our hundreds of kilowatts 24×7 365 days a year
- the saving of electrical energy from the cooling installation can be used to power more IT equipment
- we still make use of several installation components more than 15 years old, we made them energy efficient.
- we proved that energy inefficient mission critical facilities can be changed in energy efficient facilities without interruption
- because of the improved efficiency we created more redundancy with the existing cooling installation components
We are not there yet
There is always room for improvement and on a very short period we will publish the results from our new water treatment installation which is operational since several months.We saved more than 90% of the discharge water of the cooling towers! An article about this project will be published soon
Next to that we going to optimize our Dynamic Rotating UPS systems from Hitec Power Protection, think about:
- reuse the heat from the generator and induction coupling to preheat the diesel engine
- don’t cool and preheat in the same room
- adjust the induction coupling kinetic energy to the actual requested power. Store not more kinetic energy than needed which saves energy
- reuse the heat from the data center for heating the office
If you want to know more about our de-icing during the winter, you should read this article:Using heat from a data center to remove ice from cooling tower
Can this done in other data centers ?
Yes, off course! There are a lot of things which can be used in other data centers, except for the Aquifer Thermal Energy Storage systems. For this you are dependent of the soil.
The challenge was continuous finding the balance between energy efficiency and availability
It’s now the opposite!
Before: low temperatures, high air circulation and high pressure cooling circuit
After: high temperatures, low air circulation and low pressure cooling circuit
This project was a case story of the “Innovators aan het woord 2014” by the RVO ( Rijksdienst voor Ondernemend Nederland )
Note: the images used in this publication are created after the commissioning. In the period after wards we kept optimizing, these images are not optimized.