Data centers are notoriously intensive energy consumers. From 2000 to 2006, the total electricity consumption of data centers more than doubled; it now represents an estimated 1.5 percent of the United States' total electricity use. Moreover, energy consumption among HVAC systems in similar-sized data centers can vary by as much as fivefold. Typically, the computer load and HVAC system represent the bulk of a data center's total energy use (Figure 1), making those systems the best targets for energy savings.

In order to better manage your data center's energy costs, it helps to understand how you are charged for those costs. Most utilities charge commercial buildings for natural gas based on the amount of fuel delivered. Electricity, on the other hand, can be charged based on two measures: consumption and demand.

The consumption component of the electricity bill is based on the amount of electricity, in kilowatt-hours (kWh), that the building consumes during a month. The demand component is the peak demand in kilowatts (kW) occurring within the month or, for some utilities, during the previous 12 months (Figure 2). Demand charges can range from a few dollars per kilowatt-month to upwards of $20 per kilowatt-month. Because demand can be a considerable percentage of your bill, care should be taken to reduce peak demand whenever possible. As you read the following energy cost-management recommendations, keep in mind how each one will affect both your consumption and your demand.

The Bottom Line

Almost all of the conservation measures discussed for the short and longer term represent good investments. Most will not only save money but will also help to reduce energy use, improve power reliability, and slow power density growth. To assess the energy consumption of your data center, visit Lawrence Berkeley National Laboratory's (LBNL's) online self-benchmarking guide: http://hightech.lbl.gov/documents/DATA_CENTERS/Self_benchmarking_guide-2.pdf.

Quick Fixes

Many data centers have high utility costs, so low- or no-cost reductions in energy expenditures are especially important.

Turning Things Off

Although it may seem like a simple measure to take, remember that for every 1,000 kWh you save by turning things off, you save $100 on your utility bill (assuming an average electricity cost of 10 cents/kWh).

Lights. Turn off lights when they are not in use. Occupancy sensors can help, but a less-expensive alternative is to train your staff to turn off lights in unoccupied rooms and at the end of the day.

Turning Things Down

Some equipment cannot be turned off entirely, but turning it down to minimum levels where possible can save energy.

Decrease server power levels. Data center operators can take advantage of the power management (PM) features that are typically built into servers. These PM features adjust power usage in response to the processor's activity level, enabling the data center to run at the minimum power level necessary to perform necessary tasks. Most servers in data centers run below 50 percent of their maximum utilization. Powering some servers down or even off when workloads decrease (at night and on weekends, for many corporate servers) or using continuous PM strategies that scale microprocessor operating voltage or frequency in response to server demand could reduce annual energy consumption by about 30 percent.

Broaden temperature and humidity ranges. ASHRAE (the American Society of Heating, Refrigerating and Air-Conditioning Engineers) recently updated its guidelines for data centers and broadened the allowable temperature and humidity ranges, which can result in significant energy savings. These standards are included in the 2004 ASHRAE publication "Thermal Guidelines for Data-Processing Environments," which can be purchased online at www.ashrae.org.

Check common-area lighting. It is often possible to dim the lights in hallways by as much as 30 percent during daytime hours, thus reducing demand charges and energy consumption.

Cleaning and Maintenance

Making sure that your HVAC system is regularly cleaned and serviced can greatly reduce heating and cooling bills.

Check the economizer. Many air-conditioning systems use a dampered vent called an economizer to draw in cool outside air to reduce the need for mechanically cooled air. If not regularly checked and maintained, the linkage on the damper can seize up or break. An economizer that is stuck in the fully opened position can add as much as 50 percent to a building's annual energy bill by allowing hot air in during the air-conditioning season and cold air in during the heating season. Have a licensed technician clean, calibrate, and lubricate your economizer about once a year and repair it if necessary.

Check the temperature of return and supply air. With a thermometer, check the temperature of the return air going to your air conditioner and then check the temperature of the air coming out of the register nearest the air-conditioning unit. A small temperature differential is a good indication that hot and cold air are mixing in your data center, whereas a large temperature differential indicates separation of hot and cold air. Additionally, low return-air temperatures can make computer room air conditioners (CRACs) operate at a lower capacity.

Optimize controls. Ask your HVAC maintenance company to inspect your cooling system to verify correct operating conditions. CRACs are commonly found to be simultaneously heating and cooling or humidifying and dehumidifying. Also, have specialists set operating parameters, or setpoints, for optimum efficiency.

Change filters. Filters should be changed on a monthly basis—or more often if you are located next to a highway or construction site where the air is particularly dirty.

Clean condenser coils. Check condenser coils quarterly for either man-made or natural debris that can collect there. At the beginning and end of each loading season, thoroughly wash the coils.

Check for airflow. Hold your hand up to air registers to ensure that there is adequate airflow. If there is little airflow or if dirt and dust are found at the register, have a technician inspect the CRAC unit and ductwork.

Longer-Term Solutions

Although the actions covered in this section require more extensive implementation, they can dramatically increase the efficiency of your facility without compromising speed or reliability. Ask your local utility's representative for more information about initiating such projects.

Commissioning

Commissioning is a process in which engineers observe a building and perform a tune-up to ensure that its systems are operating appropriately and efficiently. Studies have shown that continuously monitoring a building's energy systems can lead to reductions of 10 to 15 percent in annual energy bills. Savings typically result from resetting existing controls to reduce HVAC waste while maintaining the necessary temperature and humidity conditions for your IT equipment. Commissioning usually costs between 5 and 40 cents per square foot.

Lighting Controls

Data centers typically have rooms that remain unoccupied during most of the day. Occupancy sensors, time switches, and local override controls can ensure that the lights are kept off while the rooms are unoccupied and still accommodate people who have to work in them.

Monitors

Work with HVAC experts to identify key locations in your data center to install temperature and humidity monitors. Monitors help to ensure that IT equipment is kept within the appropriate temperature and humidity ranges in order to perform reliably. They can also help detect problems with HVAC systems.

More-Efficient IT Equipment

Because nearly the entire electrical load in a data center exists to serve IT (either directly providing energy for the servers or removing heat from the servers), upgrading the efficiency of IT equipment can have a big impact on energy consumption. Data center operators and other customers, however, still lack standard testing procedures to determine which servers are worth buying to improve energy performance and whether they are worth their incremental cost. To remedy the lack of a standard, the U.S. Environmental Protection Agency is working with industry leaders to create an Energy Star rating for servers and other equipment in data centers. You can learn about the current status of the Energy Star rating system at www.energystar.gov.

In addition, industry competitors and researchers have come together to form the Standard Performance Evaluation Corporation (SPEC) Power-Performance Committee. Contributors expect the committee's first energy and performance benchmark for small to medium server platforms to be issued in the second half of 2007. You can learn about the current status of the SPEC benchmark at www.spec.org/specpower.

Finally, high-efficiency emerging technologies might greatly reduce energy consumption in data centers. One such technology, a massive array of idle disks (MAID) system, conserves energy by keeping hard-drive discs that contain rarely accessed data idle and spinning only frequently accessed disks.

Better Airflow Management

Poor airflow management in data centers can dramatically decrease the efficiency of the HVAC system and increase energy costs. Check out the following ways to keep hot and cool air separate in your facility.

Create efficient airflow. Most data centers suffer from poor airflow management, which has two detrimental effects. First, the mixed air recirculates around and above servers, warming as it rises, making servers higher up on the racks less reliable. Second, data center operators must set supply-air temperatures lower and airflows higher to compensate for air mixing, thus wasting energy. Setting up servers in alternating hot and cold aisles is the key to managing airflow. This allows delivery of cold air to the fronts of two rows of servers and concentration and collection of the waste heat from the backs of the racks (Figure 3). Along with this configuration, operators can close off gaps within the rack to minimize airflow through the rack. With cold-hot isolation, LBNL researchers found that the air-conditioner fan could maintain proper temperature while operating at a lower speed, resulting in 75 percent energy savings for the fan alone.

Reduce bypass-airflow losses. Bypass airflow is when cold air returns into a hot air return duct. A recent study found that up to 60 percent of the total cold air supply can be lost via bypass airflow. The main causes of bypass-airflow losses are unsealed cable cutout openings and mislocated perforated tiles in hot aisles. This type of problem can be easily eliminated by identifying the bypasses through a study of the cooling system's airflow patterns.

Economizers and Other Cooling Options

Cooling systems represent one of the largest energy consumers within a data center. The following are some energy-efficient cooling options to consider.

Air-side economizers. When the outdoor temperature and humidity are mild, air-side economizers can save energy by bringing in outside air to cool a building rather than using refrigeration equipment to cool the building's return air. Air-side economizers have two benefits: They have lower capital costs than many conventional systems, and they reduce energy consumption. Using standard, low-cost equipment, an air-side economizer can achieve a 60 percent cut in cooling costs at a data center. A recent study performed by LBNL researchers found that the particulate matter concentrations at servers in data centers that used air-side economizers were well below the point of potential harm. The study also stated that although the humidity control equipment used by most data centers was usually sufficient to address humidity concerns, an assessment of local climate factors during the design process is recommended.

Water-side economizers. When conditions are right, water can be evaporatively cooled enough in a cooling tower so that the building's compressor and refrigerant loops can be bypassed. In northern climates, the opportunity for free cooling with a water-side economizer typically exceeds 75 percent of the total annual operating hours. In southern climates, such free cooling may only be available during 20 percent of the operating hours. While they are operating, the free cooling that water-side economizers provide can reduce the energy consumption of a chilled-water plant by up to 75 percent.

Direct cabinet cooling. Although some facility managers have a reflexive aversion to having water anywhere near their computer rooms, new cooling systems bring the water very close—all the way to the base of the server cabinets or racks. This practice allows the cabinets to be cooled much more directly and efficiently than by cooling the entire room (Figure 4). Hewlett-Packard and IBM now offer direct-cooled computer racks, and many industry experts believe that this is the wave of the future.

Improved chilled-water system. There are many best practices for chilled-water systems that apply to cooling systems for data centers as well as other commercial buildings. These include using variable-speed chillers or pumping and optimizing chilled-water temperatures. If your facility isn't already taking advantage of these techniques, consult with an HVAC expert—you may find that highly cost-effective savings are available.

Right-Sized HVAC

Adequate cooling is essential to a data center's continued performance, reliability, and operation. The power intensities of data centers, however, are frequently overestimated, and the sizes of HVAC systems are often based on expectations of future load growth. Significant energy savings can be achieved by sizing the cooling capacity based on current needs and by choosing a modular design that can be scaled up to serve future computer load additions.