Defeat the Heap: Zero-Copy Data Movement in AXI4MLIR
This paper optimizes data transfer between host and accelerator in machine learning workloads by eliminating redundant staging operations.
This work is novel in its ability to eliminate redundant staging operations in data transfer between host and accelerator, improving performance in machine learning workloads.
Before reading this…
Applications
- →matrix-matrix multiplication
- →machine learning accelerators
To understand this paper, make sure you know these concepts first:
- understanding of machine learning workloadsfind papers →
- knowledge of hardware acceleratorsfind papers →
Abstract
More Like ThisAs custom hardware accelerators become increasingly central to machine learning workloads, efficient data transfer is critical for maximizing accelerator performance on linear algebra kernels. AXI4MLIR, an extension of the Multi-Level Intermediate Representation (MLIR) compiler framework for automated generation of host-accelerator driver code, incurs significant runtime overhead due to non-zero-copy CPU-accelerator data movement. During transfers from the host to the accelerator, data is copied from heap-allocated memory buffers into contiguous Direct Memory Access (DMA)-mapped buffers. This work identifies this copy as a redundant staging operation and eliminates it through zero-copy data movement. The optimization extends accel, an MLIR dialect introduced by AXI4MLIR, and implements lowering support that allocates buffers directly within DMA-mapped memory, thereby omitting the staging copy. We evaluate the proposed scheme using a configurable matrix-matrix multiplication accelerator and show that the zero-copy optimization reduces main memory data movement by up to 2x, increasing overall accelerator utilization.