Memory Interface and Controller Designs and Optimizations for Next Generation Memory Systems

DRAM has been used as the conventional main memory for decades and cannot catch up with the increasing demands on high bandwidth and large capacity of modern-day applications such as server virtualization, machine learning, generative artificial intelligence and image classification. DRAM is volatile memory that requires periodic refresh to retain stored data and is prone to soft errors. Non-Volatile Memory (NVM) technologies promise significantly higher densities, zero refresh power, and lower cost-per-bit compared to DRAM. In this thesis, we analyze the applications' run-time memory access pattern and propose novel memory interface and controller designs that enable hybrid memory configuration and improve system performance by filtering unnecessary memory accesses. First, we present POMI, a polling-based memory interface framework that enables multiple memory technologies to be housed under the same system. POMI is a decoupled memory organization which uses polling-based bus protocol for communications between the memory controller and DIMMs, instead of the conventional DDRx bus protocol. The POMI design adds additional bus lines dedicated to transfer communication packets. To avoid the cost, we further propose an improvement that utilizes the Command/Address bus to transfer the communication packets. We propose two novel bus scheduling policies called static grouping and dynamic grouping, which schedule multiple packets together before flipping the bus' direction. Next, we present MSF, a Memory-Side perceptron-based prefetching Filtering which is an accuracy- and memory-state-aware prefetch filtering mechanism that can adaptively learn memory access patterns of the underlying prefetcher and filter useless prefetches from further polluting the cache hierarchy. A good prediction by MSF effectively prevents unnecessary prefetches from further polluting the caches. Unlike other filtering schemes which only examine core-side metadata, MSF monitors both core-side and memory-side features which can convey information that has not been analyzed by the underlying prefetcher Our evaluation shows that these proposed schemes can successfully enable heterogeneous memory configuration and further improve performance and energy efficiency of the underlying hardware-based prefetcher.