EONSim: An NPU Simulator for On-Chip Memory and Embedding Vector Operations

Embedding vector operations account for a growing share of modern deep neural network workloads and exhibit input-dependent and highly skewed memory access behavior. This behavior arises from runtime index sequences, in contrast to the static loop structure of matrix operation-centric kernels. Existing neural processing unit (NPU) simulators primarily target matrix operations and derive memory accesses from tiling and scheduling logic, which limits fidelity for embedding-heavy workloads. Also, coarse on-chip memory abstractions based on aggregate bandwidth and transfer latency hinder analysis of reuse behavior and both software- and hardware-managed on-chip memory techniques. This paper presents EONSim, an embedding-aware NPU simulator that captures index-driven memory behavior. EONSim represents embedding workloads through reusable, hardware-agnostic index traces and integrates a detailed on-chip memory model that supports diverse management policies, including software prefetching and hardware caching. Across multiple NPU configurations, EONSim reports an average inference time error of 1.3% and an average on-chip memory access count error of 5.5%.