NVPC: A Transparent NVM Page Cache

Depicted in Figure 3, the fundamental design of the sync absorbing area is a series of logs on NVM. The main task of the log is to persist the index of the data, so that we can still retrieve the data after a power outage. For each log series, the 64B log entries are first sequentially organized in a 4KB page. When the log exceeds the current page, another page is allocated. The page is then attached to the previous page by a linked list. Page allocation is served by the free page manager of NVPC, which will be introduced in Section 4.3. The traversal operation on the log is accelerated by prefetching.

There are two types of logs in the sync absorbing area. The first one is called a super log that contains pointers that link to the log heads of all the inodes managed by NVPC. There is only one global super log in NVPC, and its first log page (log head) is the physical 0 address of the NVM device, in order that NVPC can find it again directly after power failure. The super log is the root of all logs, from which all the persistent-domain data can be found and replayed. Another type is called an inode log. Each NVPC-managed file has an inode log, and all sync writes and metadata updates on it are recorded by such inode log.

Each entry in the super log describes an inode log. Specifically, there are four fields for each entry: an s_dev field and an i_ino field to find a specific file, a head_log_page pointer that points to the first log page of this inode log, and a committed_log_tail field to indicate the inode’s current log tail. Whenever a new inode is delegated to NVPC, a new super log entry pointing to a new inode log is created. Then when this file is accessed again, NVPC will find the super log entry to retrieve its inode log.

An inode log describes all the sync-relevant update operations on a file, and the most important operation on this log is sync writes. NVPC manages NVM in pages, so the write is naturally separated into parts no larger than a page, and we may need to record them $n$ times if a write crosses $n-1$ page boundaries. For inode logs, each entry represents a write part. Two log entry types are designed to persist data. The first one is the out-of-place log entry (OOP entry), meaning that the data it refers to is in a stand-alone persistent domain page. The OOP entry contains a page_index field that points to the address of the whole-page data in NVM. NVPC uses OOP entries for large page-aligned shadow-paging writes, which offers convenience for data page allocating and reclaiming. The new OOP data page is filled with new data, so the old data doesn’t need to be copied to the new page at all. The second type is the in-place log entry (IP entry), meaning that the data is included inside the log zone just next to the entry. IP entry suits for small unaligned write parts with arbitrary sizes, in this case NVPC can take the advantage of NVM’s byte-addressable characteristic (principle 4). Besides these differences, both the OOP entry and the IP entry have the following fields: a file_offset field to indicate the position in the file that the current entry is writing to; a data_len field to indicate the length of the current write entry; a last_write field to find the previous write on the same position for backtracking (Section 4.1.5); a tid field to mark a transaction; and a flag field to mark the state of an entry. Apart from two kinds of write entries, there is also a metadata update entry to record the change of the inode’s metadata, and a write-back record entry that will be discussed in Section 4.1.3.

With the log structure defined, it is time to give a glimpse of the process of sync write. When a sync write is performed (O_SYNC write or stand-alone sync), the data is always written to the non-persistent page cache first (no matter to DRAM or NVM). Then the data is written to the persistent domain again. We are not reusing pages between the non-persistent domain and the persistent domain (yellow and green data pages in Figure 3), even if the cached page is in NVM, because further writes on the non-persistent domain may introduce partial writes to the persistent domain, which violates the transactional consistency on the sync absorbing area (principle 2). Now we focus on the write process of the persistent domain:

Each sync write operation is regarded as a transaction in NVPC. The write transaction is broken into parts according to the page boundaries it crosses. For each part of the write, a log entry will be appended to the end of the inode log of the current file. Aligned whole-page parts are recorded by OOP entries, with the data copied from the non-persistent domain to a newly allocated persistent domain data page (① in Figure 3). Note that even if there is a previous persistent domain data page found for the same offset (Data Page B in Figure 3), we cannot reuse it, otherwise, we may lose the data of the previous transaction if a crash happens in this transaction (principle 2). Unaligned parts are recorded by IP entries, and the data is copied to the entry’s data zone. A simple case is the stand-alone sync operation (e.g. fsync), for which we just need to persist all dirty pages with OOP entries. The transaction division and the entry type selection policies are also depicted in Figure 4

The entries are filled as follows: For OOP entries, the page_index field of the entry points to the data page, as ② in Figure 3 shows. The file_offset and data_len fields are set as is. The tid field is set to an auto-increment id that increases on each transaction. For a write operation, all entries that describe its parts have the same tid. To ensure the integrity of a write transaction, the committed_log_tail of the inode log is only atomically updated after all the parts in a transaction finish.

To further ensure principle 2, several techniques are applied. First, due to the existence of CPU caches, writes to NVM may return before they are eventually persisted to the NVM device. To solve the inconsistency risk, NVPC uses cache line write-back (e.g. Intel’s clwb) to explicitly instruct the CPU to flush data back to NVM. Note that with the help of eADR (EADRNewOpportunities, ), the cache line write-back process can be omitted, in which case NVPC can achieve a better performance. Further, memory barriers (sfence) are performed to maintain the ordering of store operations before and after consistency-critical points. There are only two barriers used in the sync absorbing area. The first one is put after all transaction parts are logged and before the committed_log_tail is updated, to make sure that a transaction is complete before we can see it. Then another barrier is put after the commit before the start of the next transaction to maintain the order between transactions.

We now have two separate write paths to NVM and disk. Writes with sync indications are persisted directly to the persistent domain of NVM, and may have bytes less than a page. On the other hand, any kind of write, sync or not, dirties some pages in the non-persistent domain page cache, leading to asynchronous write-backs from the non-persistent domain to disk. Put simply, let’s assume that the block size of the disk is equal to the page size. Pages on the underlying disk of NVPC always persist the absolutely right data with absolutely right intra-page write sequences. This is because that a disk page is written as a checkpoint of the non-persistent cached page, and the cached page is always guaranteed to be right by the file system. However, since we have changed all sync writes to async on disk, the page on disk may persist an older data version if a power failure happens before a synchronized dirty page is written back to the disk. Fortunately, NVPC guarantees that we have a fresher version data persisted on NVM. But since we only persist the necessary bytes of sync writes, but not all page bytes of all dirty pages, we may face inconsistency problems here, which violate principle 2.

Figure 5 illustrates a typical scenario of the inconsistency between the data on disk and NVM. At the point of $t2$, the page cache, disk, and NVM reach a synchronous status of $V1$. At the point of $t7$, the disk maintains the latest version $V3$, but the NVM has the previous version $V2$, which can be rebuilt from $V1$ and $O1$. The lag of the data version on NVM is because $O2$ is not a sync write, and we have no reason to persist it to NVM. So the first problem is that if we crash at $t7$, the recovery process will rebuild $V2$ from the NVM and overwrite the $V3$ on the disk, meaning that we encounter an unexpected data rollback. This is an annoying problem but does not violate the sync semantics, because $O2$ didn’t require itself to be synced. Even if we can bear this problem, we may also face a problem at $t10$. At this point, a new sync write $O3$ was performed and completed with itself successfully recorded by NVM, and the new data $V4$ is not written back to the disk yet. However, if we have a crash here, we can only rebuild abcxyz from the NVM, which is not any version we expect. The data has been messed up on NVM at this point, and the disk only has the previous version $V3$ instead of the latest sync version $V4$. The key to these frustrating problems is that we don’t have the exact time sequence of disk writes and NVM writes. Now we persist the disk write-back events on NVM (the yellow bubble on the NVM timeline in Figure 5) to maintain a global clock. Whenever a disk write-back happens, if there is a valid previous entry, a write-back entry is appended to the inode log, implying that the previous writes on this page are expired. Then when we rebuild the page after crash, we only replay unexpired operations to the page version on the disk. E.g. at $t10$ we replay $O3$ (NVM) to $V3$ (disk) and get a31xyz, which is exactly what we want (the lost $V4$). By this mechanism NVPC ensures that it will always rebuild the latest data without rollback or mistake.

There are three types of file operations that cause direct I/O on a disk file system: the O_SYNC flag that indicates that a file or a mount point should always be synchronously written; the O_DIRECT flag that attempts to minimize cache effects of I/O on a file which also causes sync writes; the explicit sync operation (e.g. fsync) that claims an immediate data persistence. Both the O_SYNC and the O_DIRECT flags are prior instructions, meaning that we can immediately persist the written data once that write operation is performed. Differently, the sync operation is a posterior instruction after the data of previous write operations is written to the non-persistent page cache, which means that we may need to reposition previous writes to the persistent domain. However, when a sync operation is performed individually, we have lost the data position and length information of all the previous writes. In such case, if we have multiple small writes across many pages before a sync, the sync operation will write all the dirtied full-pages to the persistent-domain NVM, causing severe write amplification. E.g. in Figure 4, the pink fragments will cause all contents of the red dirty pages written to NVM.

To solve the write amplification problem, we introduce active sync (principle 4). The idea is to predict how many following write operations on a file are going to be synchronized. The prediction is made by the previous write-sync pattern. If there are small scattered writes, we actively mark that file as O_SYNC, thereby the following writes will be synchronized immediately inside the write system call, at which time we still have the data position and length information. Vice versa, if the writes are predicted better to be absorbed by the DRAM page cache, we clear the O_SYNC flag. This procedure is described by Algorithm 1. We need the record of written bytes and dirtied pages since the last sync to calculate whether it is better to persist every write directly to the NVM or to wait for a whole page. The $sensitivity$ is used to tune the algorithm on different workloads to prevent thrashing, and is empirically set to 2 to suit most workloads.

To recover the data on NVPC to the disk after a power failure, a crash replay procedure is introduced. The procedure walks multiple times through each inode log to recover the file. First, it walks along the whole inode log. In this pass, log entries on the same data page offset are linked together in sequence through the $last\_write$ field of each entry, and the latest write entry is also picked out for each data page. Then for each page, the rebuilder walks from its latest entry towards the earliest via the $last\_write$ field. Once a write-back entry or an OOP entry is found, the walk halts, because the previous data are either expired or overwritten. The data of the traversed entries are then replayed to the data page on the disk.

To comprehensively illustrate the performance superiority of NVPC’s sync-absorbing strategy, we first deploy 4KB random r/w tests with different r/w ratios (0/10, 3/7, 5/5, 7/3) and various sync write percentages (0% to 100% step by 20%). The experimental result is shown in Figure 6. Due to our DRAM-NVM cooperative design, NVPC outperforms NVM file systems, disk file systems, and NVM-based FS accelerators in the majority of cases. In non-sync workloads, NVPC performs the same as a heated Ext-4, which is up to 3.72x, 2.93x, and 1.24x faster than NOVA, P2CACHE, and SPFS. For all-sync workloads, NVPC performs the same as P2CACHE, which is up to 1.70x, 4.44x, and 5.59x faster than NOVA, Ext-4, and SPFS. Meanwhile, in moderate sync level, NVPC outperforms both Ext-4 and P2CACHE. The results indicate that NVPC can always have a good trade-off between DRAM and NVM access over different sync levels.

The blue shadow in Figure 6 is the performance NVPC gains from Ext-4, and the red shadow indicates that any data point within it means a performance degradation to Ext-4. It is obvious that NVPC accelerates Ext-4 efficiently while causing no slowdown at all, which meets our principle 3. For other candidates, however, there are always varying degrees of performance degradation. NOVA and P2CACHE both suffer from slow NVM writes under low sync level, due to their strong consistency. NOVA is also hindered by slow NVM reads when the read proportion rises. SPFS only shows its advantage to Ext-4 in write-only and high sync level scenarios, and falls behind NVPC in all cases, because of its two-tier design and read-after-sync slowdown.

To demonstrate the sync performance of NVPC on different data sizes, we then perform sync append, sequential overwrite, and random overwrite tests with multiple write lengths. Except for NVPC-fsync, tests are performed on an O_SYNC-enabled file. As shown in Figure 7, NVPC accelerates Ext-4 in all cases for up to 15.19x due to the fast NVM sync write path. Note that NVPC can even achieve a higher speedup ratio on slower block storage devices like SATA SSDs and HDDs. NVPC also outperforms SPFS in all cases due to our concise monolayer design. Compared to NOVA, NVPC has a better performance in small sync writes (100B and 1KB), since NVPC adopts a free-length log design that introduces no write amplification, while NOVA manages data by pages and needs copy-on-write to ensure crash consistency for small writes. For larger sync workloads, NOVA performs better than NVPC, because 1) NVPC cannot eliminate the software overhead of lower file systems and page cache, and 2) NVPC has to write to both DRAM and NVM, while NOVA only writes to NVM. However, these overheads are necessary, since our goal is to 1) transparently accelerate current disk file systems, while still utilizing their large capacity and requiring no data migration (principle 1), and to 2) maintain the advantage of DRAM fast path access and refuse slowdown to any use cases (principle 3). Sync writes only account for a small proportion of real-world workloads, and will not be the major bottleneck anymore if they reach the performance of the same order of magnitude as normal r/w. Hence, our moderate but not aggressive acceleration here can bring more benefits to a vast range of mixed scenarios, which has been proven by our previous mixed test. Note that P2CACHE is omitted in this evaluation, because it has the same behavior with NVPC under full-sync workloads. The reason for the absence of P2CACHE in the following evaluations will be the same.

We also evaluate NVPC’s performance with fsync operations instead of O_SYNC flags. The performance is shown as NVPC-fsync and NVPC-fsync +Optm in Figure 7, for vanilla NVPC and NVPC with active sync semantic optimization (Section 4.1.4) respectively. The result shows that the optimization increases the performance of vanilla NVPC by up to 59%, because of the elimination of write amplification. With the optimization, fsync operations in NVPC outperform Ext-4, NOVA, and SPFS under O_SYNC in all tested cases below 4KB.

We notice that crash consistency checkers for file systems (leblancChipmunkInvestigatingCrashConsistency2023, ; martinezCrashMonkeyFrameworkAutomatically2017, ) have been proposed. However, these checkers either only intercept writes from the block layer for disk file systems, or raise the persistent semantics to each single write only for strong consistency PM file systems. None of these checkers is suitable under NVPC’s heterogeneous, loose-consistency work conditions. To prove that NVPC provides a proper consistency guarantee, we manually trial the system with crashes injected to the critical points of the code. For log transactions, crashes are added between the memory barriers described in Section 4.1.2. These tests demonstrate whether the log of NVPC is updated with atomicity and isolation. For NVM-disk page consistency presented in Section 4.1.3, crashes are added around the write-back event. These tests demonstrate whether NVPC can provide sync acceleration with eventual correctness. No FS semantics violation is found during our limited test.

To illustrate the effectiveness of our NVM-extended page cache design, we shrink the capacity of DRAM to 16GB and limit the size of NVM to 32GB. We perform a random read test with uniform distribution under 32GB file size and 160GB total I/O size, to demonstrate the performance of the page cache under some certain workloads with uniform access patterns. We also deploy a test with Zipfian distribution under 160GB file size and 1000GB total I/O size to study the performance under daily workloads with hot-spotted working sets and long access terms. We then compare NVPC’s throughput and cost-effectiveness to Ext-4 with 16GB and 48GB DRAM page cache. Since Intel stopped to offer Optane PM modules anymore, we choose the price recorded by previous work (ruanPersistentMemoryDisaggregation2023, ), i.e. $419.0 for 128GB PM and $877.0 for 128GB DDR4 DRAM, to calculate the cost-effectiveness.

Figure 8 shows the results. In the uniform test, NVPC’s DRAM-NVM combined page cache achieves 3.34x faster than 16GB DRAM page cache, and reaches 97.07% of 48GB all DRAM page cache. In the Zipfian test, NVPC performs 2.69x faster than 16GB DRAM and reaches 82.04% of 48GB DRAM. For DRAM-only settings, the per-byte cost is $6.85, while the tested 16GB-DRAM plus 32GB-NVM configuration for NVPC only costs $4.47 for each byte. Meanwhile, our approach achieves the highest cost-effectiveness in all tests, i.e. 48.92% and 25.85% higher than 48GB DRAM-only setting in uniform and Zipfian tests respectively.

To measure the scalability of NVPC, we perform a 4KB random r/w test with multiple threads accessing different files, and the thread number varies from 1 to 16. The read-write ratio is set to 1:1, and the writes are all synchronous, to demonstrate the scalability of NVPC’s DRAM-NVM cooperative design. Before the test, a pre-read of each file is performed to heat up the cache and minimize the influence of the lower file system and storage device. The result in Figure 9 shows that with the increment of threads, NVPC scales well and outperforms all competitors. NVPC has a performance up to 1.94x, 3.11x, and 8.87x compared with NOVA, Ext-4, and SPFS. The advantage of NVPC lies in its ability to fully utilize DRAM and NVM to separately serve read and write requests with high efficiency. In comparison, reads on NOVA are served with slower NVM. For SPFS, its second index and read-after-sync slowdown make it inferior to Ext-4 under 2 or more threads. The performance degradation from 8 threads to 16 is caused by the saturation of NVM write bandwidth (since our testbed only has 2 PM modules interleaved) and the contention between threads.

Filebench (Filebench, ) provides 3 representative macrobenchmark scripts to simulate server workloads: file- server is a non-sync write intensive workload with 1:2 r/w ratio; webserver is a read intensive workload with 10:1 r/w ratio; varmail is a balanced read/sync-write 1:1 test with small I/O size. The detailed settings of these workloads are listed in Table 1.

The result of the test is shown in Figure 10. In fileserver and webserver workloads, NVPC, SPFS, and Ext-4 show similar performance, significantly leading NOVA, due to the utilization of fast DRAM page cache. E.g. NVPC shows 3.55x and 2.10x performance as NOVA in fileserver and webserver respectively. NVPC is also 2.32x faster than P2CACHE in fileserver, because P2CACHE forces all writes to be persisted by NVM, while NVPC doesn’t. In varmail workload, NVPC is 2.84x and 2.65x faster than Ext-4 and SPFS, while 25.98% slower than NOVA. SPFS fails to effectively accelerate Ext-4 in varmail because it demands a prediction period before absorbing sync writes to NVM. However, varmail synchronously writes to scattered files for only twice on each file, making SPFS fail to predict and absorb most of these scattered sync writes. The relatively lower speed of NVPC compared to NOVA in varmail is attributed to the double-write to DRAM and NVM. Again, we believe the retention of DRAM cache helps to provide a more balanced performance in daily workloads, in which sync operations only account for a small portion.

RocksDB is an LSM-tree-based key-value database. Data written to RocksDB are first recorded by a write-ahead log (WAL). Then the logged data are asynchronously written to the LSM tree (SST files). Reading data from RocksDB will cause reads on the SST file. RocksDB provides db_bench as its test suite. We choose sequential write (fillseq), sequential read (readseq), and multi-thread r/w mixed (readrandomwriterandom) tests to demonstrate the performance of NVPC under different cases. The read- randomwriterandom benchmark is also separately performed with a small and a large working set. The small test runs on the original testbed with 4GB working set size that can fit into the DRAM cache. The large test runs with the DRAM size limited to 16GB while the working set size set to 32GB, in which the working set cannot fit into the DRAM, to illustrate the united speed-up from NVPC’s sync absorbing area and NVM-extended page cache. We pre-deploy a fillseq to create the database and then switch on sync mode for each test, and the database is removed after each test. Note that SPFS encounters several crashes when we try to clean up the existing database. It also fails to run under RocksDB’s O_DIRECT mode. Figure 11 shows the performance of RocksDB.

For fillseq, SPFS, NOVA, and NVPC achieves 5.83x, 4.33x, and 5.23x faster than Ext-4. The reason for the low performance of Ext-4 is that its WAL sync write suffers from the low speed of the disk. NOVA has a lower speed than SPFS and NVPC due to its write amplification for small metadata writes, which is caused by its copy-on-write design.

For readseq, Ext-4 and NVPC perform similarly and both outperform NOVA, because in NVPC and Ext-4, read operations are served by the fast-path DRAM, while NOVA can only perform reads on NVM. Meanwhile, SPFS also provides the same high speed, because SPFS avoids its read-after-write slowdown in RocksDB. Specifically, RocksDB reads from SST files, which are previously written to disk with large-bulk (tens or hundreds of MB) sync. However, SPFS doesn’t absorb sync writes with more than 4MB data, thus it can still serve RocksDB reads with the lower DRAM page cache instead of the slow NVM.

For small readrandomwriterandom, NVPC performs 1.38x and 1.24x faster than Ext-4 and NOVA. The advantage of NVPC comes from its DRAM-NVM cooperative design. SPFS achieves similar performance as NVPC, again, due to its skip of large bulk sync. On the whole, NVPC and SPFS both achieve a balanced performance on RocksDB in small working sets, with a higher speed than Ext-4 on writes and an equal speed on reads, while NOVA provides higher write speed but lower read speed.

For large readrandomwriterandom, NVPC outperforms both Ext-4, SPFS, and NOVA. Note that under this test the workload is larger than the DRAM size, meaning that part of the workload cannot be served with the DRAM page cache. Compared with Ext-4 and SPFS, NVPC can utilize more NVM as the extra page cache, thus relieving the slow reads from the disk. Compared with NOVA, NVPC can serve some of the reads with the limited DRAM at a higher speed.