A Comprehensive Analysis of Process Energy Consumption on Multi-Socket Systems with GPUs

Server computers are widely used for cloud services, intensive computations, and network services, forming the foundation of modern computing. These computers often include powerful processing devices such as server-grade CPUs, large amounts of RAM, hardware accelerators, and cooling systems. With the arrival and rapid growth of Artificial Intelligence (AI) applications, server computers are in more demand, converting AI into one of the most demanding HPC workloads as the models become more complex and compute-hungry. It inspires a transition of HPC to Acceleration as a Service (XaaS) using microservices [12], which implies sharing the computational resources amongst several services. This also raises an increasing concern about AI energy consumption and the carbon emissions produced to power these systems.

The current tools for measuring energy consumption in HPC are mostly limited to the node level. They are incapable of distinguishing the computation impacts, communication, and energy consumption distribution among the running processes. The absence of detailed granularity impedes our comprehension of the energy consumption characteristics of the applications and the identification of their most energy-intensive components. This research seeks to address this knowledge gap, thereby facilitating the development of more efficient and energy-conscious optimization strategies.

This work aims to comprehensively analyse the computing devices while executing a running process on a multi-socket computer. We then propose a novel energy model for estimating energy consumption while running a process of interest (PoI), focusing on CPU and GPU execution, pioneering in this research field. This model does not require execution isolation, making it ideal for shared computing nodes and the next generation of supercomputers based on XaaS.

Quantifying the energy consumption of computational systems is an open field that has been the focus of research across the wide spectrum of HPC, from embedded HPC to supercomputers. Most systems contain sensors that allow direct or indirect computation of the power drained by the various subsystems of the computer. This includes the processors, main memory, power supply units (PSU) and acceleration cards [26, 5, 21], covering measurements of the entire system at a course level.

A finer level of energy quantification involves measuring the energy consumption of the system’s running processes (or software tasks). Currently, there are still limitations in collecting this information. The hardware sensors cannot assign the power consumption to a running process, as this requires a software component that needs analysis.

Fine-grained energy consumption analyses typically focus on embedded systems, as they are simpler and often powered by batteries. These studies have analysed the impact of executing assembly instructions, such as load/storage [7], Very Long Instruction Words (VLIW) [25, 30], including some outdated GPUs [29]. Some proposals include power models to extract energy consumption based on the dispatched instructions, while others utilise heuristics at compile time [17]. These approaches are feasible given that the computing hardware (i.e. CPU, DSP or GPUs) consumes most of the power.

Particularly, from the GPU perspective, some studies evaluate the effect of memory transactions depending on the type of access [4], where some interesting insights have been found regarding the sequential and strided memory accesses. There are also studies analysing the kernel dispatching, suggesting that the multiple kernel execution also plays a role in how the energy is consumed [28]. Others have benchmarked several workloads to get power profiles [9] and developed static analysis tools after a profile for predictions [2, 3], which are suitable for execution in isolation.

The scenario changes significantly for server-grade computers with increased complexity, where some fine-grained energy strategies are insufficient, unsuitable, or require further study. The study of the energy consumption in these computers must include cooling, storage, networking, memory and multi-socket CPU consumption. A survey from 2016 suggests that 50% is used for cooling, 10% for storage and 10% for networking [10]. Another survey from 2020 reported that the CPU takes 32% of the power consumption [15], changing the scenario concerning the embedded system case, implying more components than only analysing the CPU’s power.

The community has addressed estimating the energy consumed by server computers mainly by using models whose parameters include CPU utilization and frequency, memory and disk utilization, temperature, fan speed, and throughput (bandwidths) [15]. Others have been working on optimisation based on execution isolation and global utilisation for CPU and GPUs [16]. Our key finding during our state-of-the-art study is that most models try to estimate energy at a system-wide level, and there has been little recent work on models that use the instructions executed by workloads as parameters, as found in embedded systems. Moreover, in the case of GPU energy accounting, models use similar ideas from the CPU, given its composition of multiple stream multiprocessors (or compute units) [10].

Additionally, the models proposed by related work are classified into different types, depending on the formulation. The most fundamental is the additive model, which describes the total power as the sum of all the loads [10, 27, 1]. The Baseline + Active (BA) models decompose the power load into base power (idle), active power (during computation), and a correction term, which can be interpreted as static power [10, 8]. Regression models are fitted using training data. The model can be linear, non-linear, or based on an existing power model [10]. However, all these models focus on general system-wide prediction rather than single-process estimation.

Therefore, there is an opportunity to contribute to the field of process-level energy consumption (fine-grained) and to use executed instructions as a parameter to enhance energy consumption estimation. This work will focus primarily on studying the energy impact of different types of instructions (scalar, vector, branching, and memory), the use of other computational resources, and how these can be used as parameters for energy consumption estimation of a single process by combining additive, BA and regression approaches.

The instantaneous power consumption of a server computer can be modelled as an additive model, including the power consumption of the CPU, hardware accelerators (i.e. GPUs), RAM, storage, network interface cards (NIC), cooling (fans), and other electronic components [1], which involves the following power model at an instant $t$:

All these power components can vary their values over time based on their utilisation and power domain status (on or off). For instance, CPU-intensive workloads will cause an increase in the $P_{\text{CPU,t}}$, as well for HW-accelerator workloads, which increase the $P_{\text{Accel,t}}$. Nevertheless, the increase starts from a base power value, which corresponds to the idle state of the device that matches the static power. In this case, the first proposal is to decompose each power component into its static and dynamic components, where the latter depends on the activity of the device:

This corresponds to a BA model [8], that can be even more aware of the status of the hardware, taking into consideration the existence of power domains that can be switched off for energy savings:

Some workloads utilise more than one device at a time. For instance, a CPU-based matrix multiplication uses the CPU and the RAM (if all operands are loaded into memory). Therefore, we propose classifying the workloads based on their computational nature, i.e. CPU-based workloads, GPU-based workloads, storage operations and network communication. This is possible due to the conservation of energy, which states that the power supplied by a power source is equal to the sum of all the power loads. Moreover, it is possible to characterise any electrical circuit using the superposition principle, meaning that exercising parts of the circuit gives a characterisation of their loads [1].

This section will be divided into the computational natures (CPU and GPU), analysing each component and using actual measurements to define the behaviours of the loads, using a set of benchmarks that can exercise the several parts of the computer. For the CPU, we are specifically using the hardware from Table 1, and for the GPU, the hardware from Table 2. Network and disk activity are not within the scope of this work and will be addressed in future analysis.

This section explores the fitting of the linear regression models for the CPU and the GPU, represented by equations (16) and (20), respectively. For this purpose, we use the hardware presented in Table 1 for the CPU experiment and Table 2 for the GPU. Moreover, we use a dataset created from the latter hardware, taking 50 samples per experiment run (number of threads).

Fig. 3 shows the fitted models for the CPU and GPU. The CPU regression model (Fig. 3(a)) has a root mean square error (RMSE) of 9.69 Watts for the dataset, showing outstanding prediction performance for the whole system when relating the instructions and the CPU usage. The relative error of this RMSE value with respect to the midpoint of the dataset (500 W) corresponds to 1.93%. This leads to an opportunity to contribute to fine-grained energy prediction using runtime available data, enabling measurements of the energy footprint of the computations on the CPU side.

In contrast, the GPU model has an RMSE of 12.3 Watts, around 1.27 times less effective than the CPU model. This corresponds to a 9.7% of relative error with respect to the central point of the dataset. Besides, Fig. 3(b) shows that most of the points concentrate on the bisection line, which represents the optimal line of the model. This suggests that the outliers play a role in affecting the overall model performance. The outliers are generated due to the runtime metrics, influenced mainly by the sampling time of the kernel execution, the power accounting tool and the profiler used to take the measurements.

Table 3 shows the parameters obtained after the Least Squares Linear Regression for the models described by equations (16) and (20). The model’s intercept gives the system’s static power, which in idle is 336 Watts, whereas, for the GPU card, it is 34.98 Watts.

In terms of relevance, the CPU model places more importance on vector logic operations, followed by vector arithmetic, which are linear factors. It is also impacted by memory, which is represented as the product of the probability and the logarithm of usage. This is consistent with the fact that vector instructions are the most power-consuming in CPUs, followed by the memory transactions, which involve changes in the DRAM and various queries along the memory hierarchy [13, 14].

In the GPU model, the most relevant instructions are the scalar, involving the CUDA cores and memory transactions. The vector arithmetic seems less relevant given that the Tensor Cores are more optimised than the CUDA cores [22]. It is also consistent with the observations from Fig. 2(a), suggesting the hardware optimisations in the GPU under study. Likewise, the vector memory loads are more relevant than the scalar version because they can spend more cycles filling different registers.

This work has presented a novel framework for instruction-driven process energy consumption analysis in multi-socket computers with GPU accelerators in shared computational environments. We developed two mathematical models (one for CPU and another for GPU) to estimate the power consumption of individual processes. These models are based on process usage, instruction histograms, and power meter data, making them highly adaptable for real-world HPC applications. The CPU model was trained on a dual-socket compute node using a set of likwid benchmarks, while the GPU model was trained on an NVIDIA Tesla V100 using three different matrix multiplication implementations.

Our experimental results showed outstanding performance, with the CPU model achieving a relative error of 1.93% and the GPU model achieving a relative error of 9.7%. These results highlight the effectiveness of our models and their potential to significantly advance energy accounting in HPC systems. By enabling measurement of the energy consumption of individual processes, our models isolate the effects of different processes running on the same system, paving the way for more efficient and energy-conscious optimisation strategies.

In future work, we plan to integrate the frequency and fan speed as variables in the model, making it more adequate in real scenarios these variables fluctuate due to runtime energy optimisations. Our approach holds great promise for developing new tools and methodologies that enhance energy efficiency in shared supercomputing environments.