Energy efficiency in data centres

A power saving is a number plus a place in the electrical chain. Move the instrument and the same saving becomes a different percentage.

At the accelerator die, you read what the chip itself draws, through the hardware vendor's own interface. At the rack input, you see everything in the rack: accelerators, host processors, memory, networking and conversion losses.

At a power distribution unit or a meter, you see the rack plus whatever else shares its circuit. On the bill, you see the meter plus the site overhead, the tariff and a month of time.

A percentage at one of those places is not a larger or smaller copy of a percentage at another. The same saving reads as a smaller percentage of the rack, and of the site, because each total includes equipment the saving never touches. Two numbers of your own, the accelerator share and your PUE, convert one percentage into the other.

So the first question for any efficiency figure is where it was measured. The second is what it was compared against.

The tools fall into two families, and they are easiest to compare by what they act on.

Building-side tools act on the overhead. They include cooling control and airflow management, warmer supply temperatures, liquid cooling, more efficient power conversion and distribution, and facility monitoring that finds waste. Their results show up in the facility's overhead.

Load-side tools act on the equipment. Placement and scheduling decide where work runs, so fewer machines sit idle. Consolidation retires or merges underused servers. Newer silicon does more work per watt. Per-accelerator power control sets what each GPU draws while the work runs.

The two families are not rivals. A site can work on its cooling and its load at the same time, and less heat from the load leaves less for the cooling to remove. The guide to reducing GPU power consumption goes further into the load-side levers.

Lists of the most efficient data centres usually rank sites by PUE: total facility power divided by IT power. A value near 1.0 means little overhead.

PUE grades the building, not what the equipment produces for the power it takes. The PUE guide covers what a good value looks like and what the ratio cannot see.

For the load, the useful measure is output per unit of energy. For AI inference that is tokens per watt.

An efficient site does well on both numbers, and keeps its equipment on useful work rather than idling at a high draw. Neither number replaces the other.

Take one sentence carrying one number and check it on its own. It should state four things: the value, the place it was measured, the hardware, and the window of time.

It should also say what kind of number it is. Measured means an instrument recorded it. Derived means it was calculated from measured inputs. Modelled means at least one input is an assumption. The weakest input sets the word, as the glossary sets out.

Then ask two questions. Whose instrument produced the figure? And did throughput and latency hold on the same run? A comparison arm running at the same time, on the same hardware, under the same power limit, is the design to look for.