Understanding Power Consumption Metric on Heterogeneous Memory Systems

Nowadays, High Bandwidth Memory (HBM) modules offers higher bandwidth and have becomed a part of other high-performance computing systems, including high-end GPUs and nodes equipped with A64FX processors.

NVM, is a type of memory that entered the market as a result of Intel’s decision to bridge the growing gap between DRAM and NAND SSD. Their flexibility enables them to coexist with DRAM, fostering a heterogeneous environment that harmonizes the strengths of different memory technologies.

CXL, which stands for Compute Express Link, instead of being a distinct form of memory, CXL is, in fact, an industry-endorsed Cache-Coherent Interconnect meticulously engineered to seamlessly integrate processors, memory, and accelerators. This standard is underpinned by an innovative protocol known as CXL.mem [1].

A notable contrast to traditional DRAM DIMMs is the use of LPDDR (Low-Power Double Data Rate) memory in NVIDIA Grace CPUs [2]. However, there are still gaps in the memory-storage continuum. The need for these gaps depends on the requirements of applications to reach new levels of performance and power efficiency.

The task of identifying memory system targets holds significant importance for both developers and applications. Particularly with the advent of HMS that incorporate multiple memory types, each characterized by distinct properties like bandwidth, latency, and power consumption.

A tool that greatly facilitates this task is $\tt{hwloc}$ [8]. This tool streamlines the discovery of hardware topologies in HPC platforms.

It is crucial to highlight that NVM possesses the flexibility for two distinct applications: serving as storage or functioning as a memory device. In our study, our focus lies on utilizing NVM as a memory device to establish a HMS that incorporates both NVM and DRAM.

The configuration of our test machine features two sockets housing second-generation Intel Xeon Scalable processors and distinct types of memory local to each socket. This configuration results in a total of four NUMA nodes.

The characterization of our memory system encompasses various metrics, including bandwidth, latency, capacity, power performance, and others.

In our study, the power consumption metrics are primarily gathered through hardware counters during application profiling. Given the absence of dedicated memory-kind power consumption counters, we resort to employing the global memory power counter. To differentiate power consumption, we bind the entire process to each memory type, allowing us to work around this limitation.

In the context of power-limited environments, the power consumption metric assumes vital importance. This is exemplified in [9], where NVM showcases higher power consumption efficiency compared to DRAM. Given our utilization of an HMS featuring three distinct memory types, the power performance ranking’s outcome remains less clear-cut.

In Table I, the disparities in power consumption across various applications are apparent (further elaborated in VI). These variations underline the potential for power savings when opting for NVM over DRAM.

In the scope of our study, we deliberately opted for a variety of memory-bound applications. This selection includes two proxy applications from the Exascale Computing Project (ECP) - namely, miniFE and XSBench. We also incorporated the NASA Advanced Supercomputing (NAS) benchmark suite, encompassing applications such as Integer Sort (IS), Conjugate Gradient (CG), Multi-Grid (MG), Discrete 3D Fourier Transform (FT), and Block Tri-diagonal solver (BT). In addition, we subjected the applications to testing using the High Performance Conjugate Gradient (HPCG).

In conducting our analysis, it is essential to take into account that the evaluations provide insights into power consumption across the complete memory system. This is due to the inherent limitation of tools in distinguishing between power consumption stemming from DRAM and that originating from NVM. However, by leveraging the capabilities of $\tt{hwloc}$, we are able to allocate the entire process to various memory types, thereby facilitating a differentiation of power consumption among them.

To comprehensively understand the behavior of applications, particularly in terms of bandwidth, total power consumption, and memory system power consumption, we have employed performance counters managed by profilers. Our approach involves using two specific tools: The Intel Performance Counter Monitor (PCM) and Linux perf. In our study, we have utilized several PCM command-line utilities, including pcm-numa, pcm-mem, and pcm-power which retrieves information related to memory such as accessesm throughput and power respectively [10].

The Linux perf tool enables the retrieval of main memory power consumption through the MSR_DRAM_ENERGY_STATUS register. It should be used with $\tt{hwloc-bind}$ to be capable to differentiate the power of each kind of memory.

This methodology empowers users to not only discern power consumption within memory but also to delineate this consumption based on memory types. The ability to differentiate between them becomes paramount a task achieved in our scenario through power consumption analysis.

Our evaluation encompasses two memory types: DRAM and NVM. To enhance the diversity of our assessment, we have introduced a third memory type through emulation. Through a clever mechanism, we have successfully replicated this third type by leveraging one of CPU0’s remote access memories, such as CPU1’s local DRAM as indicated in Figure 2.

This strategic approach yields a memory module with distinct attributes compared to CPU0’s local memories.

In Figure 3 we present the power consumption of the Heterogeneous Memory System (HMS) using DRAM and NVM memories and thread counts. Our observations based on this figure include:

• •

  In many cases, the power consumption of Remote DRAM is actually lower than that of Local DRAM. To reinforce this notion, it is crucial to reemphasize the concept of simulating memory through the utilization of the NUMA system.

• •

  A key characteristic of an HPC application is its utilization of multithreaded executions. While this might result in heightened consumption within components like CPU cores, it is also evident that this phenomenon contributes to an escalation in memory power consumption. This effect arises from the concurrent access to memory facilitated by the multithreading nature of the application.

• •

  There is a correlation of memory power consumption when increasing the parallelization level in all kinds of memory, i.e., there is a tendency for having more power consumption when increasing the number of threads. Which memory should be chosen will depend on the application used and also on the memory system. We could say that the generated ordering depends on the memory system and the application.

• •

  To achieve a equilibrium between power efficiency and performance, the clarity of the situation might not be immediate. It’s important to recognize that the second rank in the hierarchy may not consistently correspond to a single memory type.