Reviewer: The Guardian

---

Summary

The authors present WSC-LLM, a co-exploration framework designed to optimize Large Language Model (LLM) serving on wafer-scale chip architectures. The framework proposes to jointly explore hardware parameters (such as DRAM capacity and interconnect bandwidth) and software scheduling strategies (such as resource partitioning, placement, and memory management for prefill and decode phases). The core contributions include a Central Scheduler for resource allocation, a Memory Scheduler for KV cache management, and an evaluation methodology based on an extended version of the ASTRA-sim simulator. The paper claims that a wafer-scale architecture with moderate DRAM capacity yields the best performance and that their proposed framework significantly outperforms a state-of-the-art GPU-based serving system (Splitwise) by an average of 3.12x.

Strengths

1. The paper addresses a timely and significant problem at the intersection of LLM serving and next-generation hardware design. The challenges posed by LLM inference on wafer-scale systems are non-trivial, and the authors correctly identify key trade-offs.

2. The conceptual separation of the Central Scheduler and Memory Scheduler is logical, addressing distinct optimization problems (computation/placement vs. data management) that arise in disaggregated LLM inference.

3. The ablation study presented in Section 5.4 provides some insight into the relative contributions of the proposed scheduling components, suggesting that memory management becomes increasingly critical for larger models.

Weaknesses

The paper's conclusions are built upon a methodological foundation that lacks the rigor required for this venue. The claims, while significant, are not substantiated by the evidence provided.

1. Critically Flawed Evaluation Methodology: The entire quantitative analysis hinges on a simulator described in Section 4.6. The authors state they extend ASTRA-sim and, more critically, use a DNN to create a "mapping lookup table" to estimate performance metrics. The paper provides zero validation for this performance model. There is no information on the model's accuracy, error bounds, or how it was trained and tested against a ground truth (e.g., a cycle-accurate model or hardware measurement). The authors merely cite that other works [37, 89] have found this approach feasible. This is insufficient. Without this validation, every performance number, graph, and conclusion in Section 5 is speculative and cannot be trusted.

2. Confounded and Misleading Baseline Comparison: The central claim of a 3.12x performance improvement (Section 5.3, Figure 11) is derived from comparing the proposed simulated wafer-scale system to a real-world A100 GPU cluster. This is a fundamentally confounded comparison. The simulated wafer-scale chip is described with a total interconnect bandwidth of 6 TB/s per die (Section 5.1.1), orders of magnitude higher than the 400 GB/s inter-node bandwidth of the GPU cluster. The reported performance gains are almost certainly dominated by these vastly superior, and perhaps optimistic, hardware assumptions rather than the novelty of the scheduling framework itself. The paper fails to isolate the contribution of its scheduling algorithms from the contribution of the hypothetical hardware, making the headline claim unsubstantiated. The SW-Wafer experiment is a step in the right direction but is insufficient to fully de-confound these factors.

3. Overstated "Co-Exploration" Scope: The paper frames itself as an "architecture co-exploration framework." However, the actual exploration in Section 5.2 is a simple parameter sweep across just four pre-selected hardware configurations (Table 1). The fundamental architectural choices—a 2D-mesh topology, Dojo-style compute cores, and a specific die structure—are fixed. This is not a general exploration of the architectural design space but rather a tuning of a few parameters within a highly constrained template. The claims of generality are therefore not supported by the experiments.

4. Unjustified Heuristics and Parameters: The resource placement strategy in Section 4.2.2 relies on minimizing a TransferCost function which includes a hyperparameter α. The paper provides no details on how α is determined, its sensitivity, or its impact on the final placement. This introduces an element of arbitrariness into a key part of the methodology, potentially suggesting the results are cherry-picked based on a favorable but unexplained tuning.

Questions to Address In Rebuttal

The authors must address the following critical points to establish the validity of their work:

1. Simulator Validation: Please provide rigorous validation data for the DNN-based performance model described in Section 4.6. What is its prediction accuracy (e.g., Mean Absolute Percentage Error) against a ground truth for latency, DRAM access, and communication overhead? Without this, the results are not credible.

2. De-confounding Performance Gains: How can the authors de-confound the performance gains attributed to their scheduling algorithm from the gains stemming from the assumed hardware's massive bandwidth advantage? A more convincing experiment would be to implement a simplified version of the WSC-LLM scheduler on the GPU baseline to demonstrate its benefits on a fixed hardware platform.

3. Hyperparameter Justification: Please provide a justification and sensitivity analysis for the hyperparameter α used in the resource placement strategy (Section 4.2.2). How does the system's performance change with different values of α, and how was the value used in the experiments chosen?

4. Clarification of Constraints: Algorithm 1 (Section 4.2.1) iterates through "all feasible instance sizes." What are the constraints that determine feasibility? The text mentions a die limit and memory capacity, but is the search space truly exhaustive, and what is the typical runtime of this "offline" algorithm for the configurations tested?

WSC-LLM: Efficient LLM Service and Architecture Co-exploration for Wafer-scale Chips

Review Form: The Synthesizer

---

Summary

This paper presents WSC-LLM, a novel co-exploration framework designed to optimize both the architecture of wafer-scale chips and the scheduling of Large Language Model (LLM) inference workloads. The authors correctly identify that the unique characteristics of wafer-scale hardware—namely, vast on-wafer bandwidth but a fixed area that forces trade-offs between compute, memory, and communication resources—create a complex, coupled optimization problem. The core contribution is a holistic framework that systematically explores this design space. It features a sophisticated Central Scheduler that partitions and places prefill/decode instances with awareness of the 2D-mesh topology, and a novel Memory Scheduler that leverages high Die-to-Die (D2D) bandwidth to manage KV cache storage across the entire wafer. Through a comprehensive design space exploration and comparison against state-of-the-art LLM serving systems, the paper demonstrates that a balanced wafer-scale architecture with moderate DRAM capacity yields the best performance, and that its scheduling strategies significantly outperform existing methods.

---

Strengths

The true strength of this paper lies in its ambitious and well-executed synthesis of two traditionally separate domains: computer architecture and distributed systems software.

1. Excellent Problem Formulation and Contextualization: The authors have done an outstanding job articulating the core tensions in designing wafer-scale systems for LLMs. Figure 1 (page 2) provides a remarkably clear and concise visualization of the fundamental architectural trade-off (DRAM capacity vs. DRAM costs) and the key inefficiencies in existing disaggregated scheduling approaches. This demonstrates a deep understanding of the problem that goes beyond a superficial application of known techniques. The work correctly positions itself at the intersection of wafer-scale computing (e.g., Cerebras, Tesla Dojo), chiplet integration, and disaggregated LLM inference systems (e.g., Splitwise).

2. Holistic Co-exploration Framework: The central idea of a unified framework for co-exploration is the paper's most significant contribution. Instead of proposing a fixed hardware architecture and then designing a scheduler for it, WSC-LLM provides a methodology for discovering a near-optimal hardware/software pairing. This is a crucial step forward for the field. As we move into an era of specialized hardware, such co-design frameworks will become indispensable, and this paper provides a strong early example in a highly relevant domain.

3. Actionable Architectural Insights: The study produces results that are not merely academic but provide genuine guidance for hardware architects. The conclusion in Section 5.2 (page 10) that a moderate DRAM capacity (Case 3) is superior to both lower (Cases 1, 2) and higher (Case 4) capacity designs is a powerful, non-obvious finding. It beautifully illustrates the law of diminishing returns, showing that beyond a certain point, the D2D bandwidth lost to accommodating more DRAM becomes the primary performance bottleneck. This is a critical insight for the future design of AI accelerators.

4. Novelty in Scheduling for Wafer-Scale Topologies: The paper successfully adapts and extends concepts from disaggregated inference to the unique 2D-mesh topology of wafer-scale systems. The topology-aware placement strategy (Section 4.2.2, page 6) and the wafer-wide Memory Scheduler (Section 4.4, page 7) are key innovations that directly exploit the hardware's strengths (high D2D bandwidth) to mitigate its constraints (communication locality). The ablation study in Section 5.4 (page 11) convincingly shows that the Memory Scheduler's contribution becomes increasingly vital for larger models, which is another significant finding.

---

Weaknesses

While the core ideas are strong, the paper could be improved by addressing the implications of some of its methodological abstractions and exploring the boundaries of its proposed solution.

1. Fidelity of the Performance Evaluator: The entire co-exploration process hinges on the accuracy of the Evaluator module (Section 4.6, page 9), which uses a DNN to predict performance and avoid full, slow simulations. While this is a standard and necessary technique for tractable design space exploration, its implications are not fully discussed. The validity of the paper's central claims rests on this model's fidelity. A brief discussion of the model's validation, its potential sources of error (e.g., modeling network contention), and the sensitivity of the final architectural choice to this error would significantly strengthen the work's foundations.

2. Static Nature of Resource Allocation: The resource partitioning and placement strategies (Section 4.2.1 and 4.2.2) are performed offline. This is a reasonable simplification for demonstrating the framework's potential. However, real-world serving environments are highly dynamic, with fluctuating request loads and potentially evolving model popularity. The current framework does not address how the system might adapt to such long-term changes, which could lead to suboptimal static partitioning. Acknowledging this as a scope limitation and an avenue for future work (e.g., dynamic re-partitioning) would provide a more complete picture.

3. Limited Scope of Explored Architectural Parameters: The design space exploration in Section 5.1.2 (page 9) focuses primarily on the number of DRAM chiplets, which in turn affects DRAM capacity/bandwidth and D2D bandwidth. While this is arguably the most critical trade-off, a true co-exploration could extend to other parameters like the size and number of compute cores per die, the on-die NoC bandwidth, or the SRAM capacity per core. The current work serves as an excellent proof-of-concept, but its conclusions are conditioned on the fixed compute die design.

---

Questions to Address In Rebuttal

1. On the Evaluator's Accuracy: Could the authors provide more detail on the validation of the DNN-based performance model used in the Evaluator (Section 4.6)? Specifically, what is the reported prediction error against a cycle-accurate or full-system simulation for key metrics like latency and throughput, and how does this potential error margin impact the confidence in identifying Case 3 as the definitive optimal architecture?

2. On the Generality of the Optimal Architecture: The conclusion that the balanced Case 3 architecture is optimal is a key result. How sensitive is this finding to the workload characteristics? For example, if faced with a workload composed exclusively of very long-context requests (e.g., summarizing entire books), which are heavily prefill-bound and generate massive KV caches, would the optimal point in the design space shift towards Case 4 (higher DRAM bandwidth/capacity)?

3. On Network Contention from the Memory Scheduler: The Memory Scheduler (Section 4.4) is a powerful concept that leverages the entire wafer's DRAM pool for KV cache. As multiple, distributed decoding instances access these remote KV cache blocks, this could create "hotspots" or significant contention on the 2D-mesh network. Is this cross-instance network traffic and potential congestion fully modeled within the Evaluator? And how does the system arbitrate or manage this contention in practice?

Title: WSC-LLM: Efficient LLM Service and Architecture Co-exploration for Wafer-scale Chips Reviewer: The Innovator

---

Summary

The authors present WSC-LLM, a framework designed to co-explore hardware architecture parameters and software scheduling strategies for serving Large Language Models (LLMs) on wafer-scale chips. The framework aims to navigate the complex trade-offs between computation, memory capacity, and communication bandwidth inherent in such systems. It proposes strategies for resource partitioning, placement, and memory management tailored to the 2D-mesh topology of wafer-scale systems. The core claim of novelty rests on this co-exploration framework and its constituent algorithms for optimizing disaggregated LLM inference in this specific hardware context.

My analysis concludes that while the paper addresses a timely problem with a comprehensive engineering effort, its foundational novelty is limited. The primary contribution is the application-specific synthesis and refinement of existing concepts from distributed systems, scheduling, and hardware/software co-design, rather than the introduction of a fundamentally new algorithmic or architectural paradigm. The most notable novel component is a greedy algorithm for remote KV cache placement.

Strengths

1. Problem-Specific Algorithmic Contribution: The memory scheduling algorithm for KV cache placement (Algorithm 2, Page 8) is a concrete and novel contribution. While the concept of memory disaggregation is not new, this algorithm provides a specific, topology-aware greedy strategy to utilize distributed idle DRAM for KV cache, which is a key challenge in disaggregated LLM serving.

2. Holistic System Integration: The framework's strength lies in its comprehensive integration of multiple optimization layers (architectural DSE, instance partitioning, physical placement, memory management) into a single toolchain. While the concept of a co-design framework is not novel, creating a functional one for this complex domain is a non-trivial engineering achievement.

3. Context-Specific Heuristic: The "decoding-centered" resource placement strategy (Section 4.2.2, Page 6) is a simple and logical heuristic tailored to the producer-consumer dataflow of prefill and decoding phases on a 2D mesh. It is a novel, albeit incremental, heuristic for this specific problem.

Weaknesses

1. Limited Conceptual Novelty of the Core Framework: The central claim of a "co-exploration framework" is an overstatement of novelty. The field of hardware/software co-design has long used automated Design Space Exploration (DSE) to find optimal system configurations. WSC-LLM appears to be a well-executed but conceptually standard DSE framework applied to a new and important domain. It does not introduce a new theory or methodology for co-design itself.

2. Heavy Reliance on Assembling Prior Art: The framework's scheduling engine is built upon a foundation of well-established techniques from recent LLM serving literature, which the authors correctly cite. These include:

   • Disaggregated Inference: The core idea of separating prefill and decoding resources was popularized by systems like Splitwise [62].
   • Continuous Batching: A key technique from systems like Orca [91] and vLLM [45].
   • Chunked Prefill: A strategy to handle long prompts, proposed in works like DeepSpeed-FastGen [10]. The novelty in WSC-LLM is the integration of these techniques for a wafer-scale target, not the techniques themselves.

3. Algorithmic Contributions are Largely Heuristic Search: The "Optimal Resource Partition Algorithm" (Algorithm 1, Page 7) is essentially a structured grid search over instance sizes and pre-defined Tensor Parallelism (TP) strategies. While systematic, this is a standard methodology for performance exploration and does not represent a novel algorithmic paradigm for optimization. It formalizes an exhaustive search within a constrained space.

Questions to Address In Rebuttal

1. The term "co-exploration framework" suggests a novel methodology. Can the authors precisely articulate what makes the WSC-LLM framework conceptually different from standard hardware/software Design Space Exploration (DSE) frameworks, beyond its application to the specific domain of wafer-scale LLM inference?

2. The decoding-centered placement strategy (Section 4.2.2) is a key contribution for handling the 2D-mesh topology. This producer-consumer placement problem (prefill instances producing KV cache for decoding consumers) is analogous to problems in NoC design and general parallel task mapping. Can the authors contrast their heuristic with prior work on topology-aware task placement for producer-consumer patterns and clarify the novelty?

3. The KV cache placement algorithm (Algorithm 2) is a greedy strategy. While this is the paper's strongest novel component, its simplicity warrants discussion. Could you elaborate on the conditions under which this greedy approach is sufficient and identify scenarios (e.g., highly fragmented memory, complex communication patterns) where it might lead to suboptimal placements compared to more complex graph-based or ILP formulations?

4. If you had to isolate the single most significant and novel conceptual contribution of this work, what would it be? Is it the framework itself, one of the specific algorithms, or the insight that moderate DRAM is optimal? Please be specific.

Reviewer: The Guardian

Summary

This paper introduces LightML, a photonic co-processor architecture designed for general-purpose machine learning acceleration. The authors claim to present the first complete "system-level" photonic crossbar design, including a dedicated memory and buffer architecture. The core of the accelerator is a photonic crossbar that performs matrix-matrix multiplication (MMM) using coherent light interference. The paper also proposes methods for implementing other necessary ML functions, such as element-wise operations and non-linear activation functions, directly on the photonic hardware. The headline claims are a peak performance of 325 TOP/s at only 3 watts of power and significant latency improvements (up to 4x) over an NVIDIA A100 GPU for certain models.

Strengths

The paper is well-written and addresses a compelling long-term research direction. The core strengths are:

• Detailed Physical-Layer Analysis: The paper provides a thorough and convincing analysis of the physical error sources in the proposed photonic crossbar (Section 3.2, Pages 3-4). The modeling of noise from beam splitters, modulators, phase shifts, and detectors is detailed and grounded in prior work, lending credibility to the feasibility of the core computational unit.
• Clear Articulation of Photonic Advantages: The authors do an excellent job of explaining why photonics is a promising substrate for computation, correctly identifying the fundamental advantages over electronic and resistive-memory-based crossbars, such as the ability to perform true MMM instead of MVM and the circumvention of slow, high-power reprogramming (Section 4, Page 4).

Weaknesses

While the physical foundation is solid, the paper's system-level claims are undermined by significant methodological flaws, questionable assumptions, and a disconnect between the simulated components and the real-world baseline comparisons.

• Unsubstantiated Performance Claims due to Inequitable Comparison: The headline performance claims (e.g., 325 TOP/s, 4x shorter latency than an A100) are built on a foundation of inconsistent and unfair comparisons. LightML's performance is derived from a custom simulator using a 12 GHz clock frequency, while the baseline GPU and TPU measurements are from real hardware running at much lower frequencies (e.g., 765 MHz for the A100) (Table 4, Page 10). Furthermore, LightML is simulated using 5-bit integer precision (Int5), while the GPU/TPU baselines use 16-bit floating-point (FP16). Comparing a specialized, low-precision, high-frequency simulated accelerator to a general-purpose, high-precision, lower-frequency real-world chip is not a valid methodology for claiming superior performance. The reported speedups are more likely an artifact of these disparate parameters than a true architectural advantage.
• Overstated "System-Level" Contribution and Unrealistic Buffer Architecture: The paper claims to be the "first system-level photonic crossbar architecture" with a "novel memory and buffer design" (Abstract, Page 1). However, the memory solution is not novel; it is a standard hierarchy of SRAM buffers whose parameters are estimated using CACTI. There is no evidence of a co-designed memory architecture that fundamentally addresses the unique data-delivery challenges of a photonic core. The claim that this buffer design achieves "over 80% utilization" (Abstract, Page 1 and Figure 13, Page 11) is not adequately supported. Figure 13 shows utilization for specific models but does not demonstrate that this holds for a general workload. It is highly likely that for many operations, particularly those that are not dense matrix multiplications, the photonic core would be severely data-starved by a conventional SRAM buffer, making the high utilization claim suspect.
• Impractical Implementation of Non-Linear Functions: The proposal for implementing non-linear functions (e.g., Sigmoid, tanh) via Fourier Series using the existing optical modulators is theoretically interesting but practically flawed (Section 6.2, Page 6; Table 5, Page 11). The authors claim this requires "no extra hardware" and is therefore highly efficient. This is misleading. This method requires two full passes through the crossbar and multiple ADC readouts, which introduces significant latency and energy overhead that is not properly accounted for in the performance analysis. Claiming it has no area or power overhead (Table 5, Page 11) is incorrect because it consumes the primary compute resource for multiple cycles. This approach is far less efficient than a small, dedicated digital logic unit, as is standard in electronic accelerators.
• Contradictory and Insufficient Precision Analysis: The paper makes conflicting claims about precision. It targets a conservative 5-bit precision (4-bit magnitude, 1-bit sign) for its primary design (Section 3.3, Page 4) and claims this results in a minimal 1-2% accuracy drop (Table 6). However, the error modeling itself shows that to achieve 5-bit precision, fabrication non-idealities must be controlled to an extremely high degree (e.g., $\delta_{k}^{2}<0.5/2^{5}$) (Section 3.3, Page 4). The paper hand-waves this away by appealing to "SOTA technology" and "one-time calibration" without providing sufficient evidence that such calibration is feasible or sufficient for a 128x128 array operating at 12 GHz. The claim that the accuracy drop is only 1-2% is based on injecting Gaussian noise during inference (Section 3.2, Page 4), which may not accurately model the complex, correlated noise profiles of a real photonic system.

Questions to Address In Rebuttal

1. Please justify your direct performance comparison to the NVIDIA A100. Provide a new comparison where the baseline is either simulated with the same high clock frequency and low precision as LightML, or where LightML is simulated with the clock speed and precision (FP16) of the A100.
2. Your claim of ">80% utilization" is based on a few selected models. Provide a more rigorous analysis of the memory and buffer architecture. What is the utilization for less ideal workloads, such as sparse matrix operations or models with many element-wise additions, and how does your "novel" buffer design specifically address the data-delivery challenges of a 12 GHz photonic core?
3. Please provide a detailed cycle- and energy-cost analysis for your proposed non-linear function implementation. How many cycles does it take to compute a single Sigmoid activation for a vector of 128 elements, and what is the total energy consumption, including all data movement and ADC conversions? How does this compare to a standard digital implementation?
4. The paper states that achieving the required precision depends on post-fabrication calibration (Section 3.3, Page 4). Can you provide evidence from prior work demonstrating that such per-unit-cell calibration is viable for a large (128x128) array and remains stable across varying temperatures and operating conditions at 12 GHz? Without this, the claimed precision is purely theoretical.

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper proposes LightML, an architecture for a photonic co-processor aimed at general-purpose machine learning acceleration. The central contribution is an attempt to define a complete system-level architecture around a photonic crossbar, which performs matrix-matrix multiplication (MMM) in the optical domain. The authors move beyond the core optical computation—which has been demonstrated in prior work—to address the surrounding necessary components, including a memory and buffering hierarchy and methods for implementing non-linear activation functions. The work positions itself as a bridge between foundational physics-level research and practical, high-performance ML accelerators, claiming significant performance and power efficiency gains over conventional electronic hardware like GPUs.

---

Strengths

This paper represents a valuable and necessary step in the evolution of photonic computing for AI. Its primary strength is its holistic, system-level perspective, which moves the conversation forward from isolated device physics to architectural reality.

• Tackling the "System Problem": The field of photonic ML acceleration is maturing. Foundational work, such as Shen et al. (2017), demonstrated the feasibility of on-chip optical neural networks using Mach-Zehnder Interferometer (MZI) arrays. However, much of the subsequent research has focused on optimizing the core photonic devices. This paper correctly identifies that the next major hurdle is the "system problem": how do you feed data to, and get data from, these incredibly fast cores efficiently? By proposing and modeling a complete data path, including an SRAM buffer hierarchy (Section 5, Page 5), the paper connects the photonic core to the rest of the computing world. This is a crucial and often-overlooked aspect of analog and non-traditional accelerator design.

• Pragmatic Approach to General-Purpose Operations: The paper thoughtfully considers how to implement the full suite of operations required for modern ML models, not just matrix multiplication. The proposed methods for element-wise operations (Section 6.1, Page 6) and, more ambitiously, non-linear functions using Fourier Series decomposition (Section 6.2, Page 6), show a commitment to creating a truly general-purpose accelerator. While the efficiency of these specific methods can be debated, the attempt to solve these problems within the photonic domain is a significant contribution that pushes the field to think beyond simple linear algebra.

• Connects to Broader Analog Computing Trends: This work fits squarely within the broader landscape of research into analog and in-memory computing accelerators. Like resistive RAM (ReRAM) crossbars, LightML leverages a physical property (light interference) to perform computation in-place, promising massive energy savings by avoiding data movement. By detailing the challenges of noise, precision, and the analog-to-digital interface (Section 3, Page 3), this paper contributes to the shared knowledge base of this entire research thrust, and its findings will be relevant even to those working on non-photonic analog systems.

---

Weaknesses

While the system-level ambition is a strength, the paper's current weaknesses stem from a disconnect between its theoretical potential and the practical realities of both hardware implementation and software integration.

• The "Memory Wall" Reappears in a New Form: The paper proposes a standard SRAM buffer hierarchy to feed its 12 GHz photonic core. However, the analysis does not fully grapple with the immense data-delivery challenge this creates. The core can consume data far faster than conventional SRAM, even in a stacked configuration, can provide it. This creates a new kind of "memory wall," where the incredible speed of the photonic computation is likely to be bottlenecked by the electronic memory system. The high utilization figures reported (Figure 13, Page 11) are for specific, dense models and may not reflect the reality of more diverse or sparse workloads, where data starvation could become a dominant issue.

• The Software/Compiler Challenge is Understated: The paper focuses on the hardware architecture but does not deeply address the software and compiler stack required to make such an accelerator usable. Mapping complex modern models (like Transformers) onto this architecture, managing the limited precision, and deciding when to use the inefficient on-chip non-linear functions versus offloading to a digital co-processor are all non-trivial compiler problems. Without a clear path to a programmable software stack, LightML remains more of a specialized engine than a "general purpose" accelerator.

• Positioning Relative to Commercial Efforts: The academic landscape is not the only context. Companies like Lightmatter are already building and shipping silicon photonics products for AI. While their architecture is different (focusing more on interconnects), a discussion of how LightML's "all-in-one" compute-and-memory approach compares to these commercial strategies would provide valuable context and highlight the paper's unique contributions more clearly.

---

Questions to Address In Rebuttal

1. Your work takes an important step towards a full system design. Could you elaborate on the long-term vision for the memory architecture? How can the data delivery gap between the electronic SRAM and the photonic core be sustainably bridged as both technologies scale?

2. Thinking about the software stack, how would a compiler decide to map operations onto LightML? For instance, for a given model, how would it weigh the cost of using the on-chip Fourier-based ReLU against the latency and energy cost of sending the data off-chip to a digital unit?

3. The paper focuses on a "co-processor" model. Can you discuss the future integration of LightML with a host CPU? What would be the ideal interface (e.g., PCIe, CXL), and what new challenges would arise in a tightly-coupled heterogeneous system?

4. How do you see the trade-offs made in LightML (e.g., low precision, complex non-linearities) influencing the design of future machine learning models? Could this architecture drive the development of new, "photonics-friendly" model architectures that are inherently more robust to analog noise and rely more on efficient linear operations?

Reviewer: The Innovator (Novelty Specialist)

Summary

The authors present LightML, an architecture for a photonic co-processor. The paper's primary claims to novelty are: 1) Being the "first system-level photonic crossbar architecture" by including a co-designed memory and buffer system (Abstract, Page 1), and 2) A novel method for implementing non-linear activation functions directly in the optical domain by approximating them with a Fourier Series, which they claim requires "no extra hardware" (Abstract, Page 1; Section 6.2, Page 6). The architecture is based on a photonic crossbar that performs matrix-matrix multiplication (MMM) using coherent light interference in an array of Mach-Zehnder interferometers.

---

Strengths

From a novelty standpoint, the paper's strength lies in its attempt to synthesize existing concepts into a more complete system, rather than inventing entirely new foundational components.

• System-Level Integration as a Novelty Claim: The primary novel contribution is the argument for a "system-level" design. While individual components have been demonstrated before, this work is among the first in the academic literature to explicitly model and present an end-to-end architecture that includes the photonic core, ADCs/DACs, and a multi-level SRAM buffer hierarchy (Section 5, Page 5). The "delta" here is the integration itself—proposing a complete blueprint where prior work often focused on demonstrating a single component in isolation.

---

Weaknesses

While the system-level claim is a step forward, the novelty of the underlying components and techniques is highly questionable when examined against prior art.

• Core Computational Method is Well-Established: The fundamental concept of using a mesh of Mach-Zehnder interferometers (MZIs) to perform matrix multiplication is not new. This was famously demonstrated by Shen et al., Nature Photonics (2017), which laid the groundwork for this entire line of research. The use of homodyne detection for optical MAC operations is also a known technique. LightML uses this established foundation (Section 2, Page 2), and while the specific array size and configuration are implementation details, the core computational principle is not a novel contribution of this paper.

• "Novel" Memory Architecture is Standard Practice: The paper claims a "novel memory and buffer design" (Abstract, Page 1). However, the described architecture is a standard two-level hierarchy consisting of off-chip HBM and on-chip SRAM buffers (Section 5, Page 5). This is a conventional memory architecture used in virtually all modern digital accelerators (GPUs, TPUs). There is no novel mechanism presented that is uniquely tailored to the challenges of photonic data delivery. The use of double-buffering (Section 5.2, Page 6) is a standard technique to hide latency and is not a novel invention of this work. Therefore, the claim of a novel memory architecture is unsubstantiated.

• Non-Linear Function Implementation is Not Novel and Impractical: The paper proposes using the Fourier Series to implement non-linear functions by leveraging the inherent sine/cosine response of the phase modulators (Section 6.2, Page 6). This idea is not entirely new; the general concept of function approximation via basis functions is a staple of signal processing. More importantly, the claim that this requires "no extra hardware" is misleading. The implementation described requires two full passes through the crossbar and multiple ADC conversions: one pass to calculate the intermediate products of the input with the Fourier frequencies, and a second pass to multiply those results by the Fourier coefficients. This consumes the primary, most powerful computational resource in the entire accelerator to perform what is typically a simple, low-cost operation in digital logic. This is not a novel, efficient solution but rather a highly inefficient re-purposing of existing hardware that incurs significant latency and energy overhead, which is not adequately accounted for in the analysis.

• Incremental Advance, Not a Breakthrough: When viewed in the context of the field, LightML appears to be an incremental, academic synthesis of pre-existing ideas. The core computation is from Shen et al., the concept of system-level photonic integration is being actively pursued commercially by companies like Lightmatter, and the proposed "novel" solutions for memory and non-linearities are either standard digital designs or inefficient adaptations. The performance gains reported are largely a product of comparing a highly-specialized, low-precision (5-bit) simulated design (Section 3.3, Page 4) running at an optimistic 12 GHz to general-purpose, high-precision (FP16) real-world hardware running at much lower clock speeds (Table 4, Page 10), which is not a valid basis for claiming a novel performance breakthrough.

---

Questions to Address In Rebuttal

1. Please clarify the novelty of your memory architecture (Section 5, Page 5). Given that it is a standard HBM + SRAM hierarchy, what specific mechanism or design choice is fundamentally new and not considered standard practice in digital accelerator design?
2. Can you defend the claim that your Fourier-based non-linear function implementation (Section 6.2, Page 6) has "no extra hardware overhead"? Please provide a detailed breakdown of the cycle count and energy consumption for this operation, including all data movement to and from the crossbar and all ADC/DAC conversions, and compare it to the cost of a small, dedicated digital ALU for the same function.
3. The foundational work of Shen et al. (2017) demonstrated optical matrix multiplication with MZI arrays. What is the fundamental, conceptual "delta" of the LightML computational core (Section 2, Page 2) that you would consider a significant advance over this and other subsequent works that have used the same principle?
4. How does your proposed "system-level" architecture differ from the integrated photonic systems being developed commercially by companies like Lightmatter? What is the unique architectural insight in LightML that is not already being pursued in industry?

Reviewer: The Guardian

Summary

This paper identifies the communication inefficiencies of standard 2D Mesh interconnects for supporting the complex, multi-dimensional parallelism strategies (e.g., 3D parallelism) used in large-scale DNN training. To address this, the authors propose FRED, a wafer-scale fabric architecture. FRED's design is a hierarchical, switch-based topology (specifically, a fat-tree) composed of small, recursive "FRED Switches" that feature in-network collective support. The authors claim that by providing flexible, non-blocking connectivity, FRED significantly improves end-to-end training time—by up to 1.87x—for large models compared to a conventional 2D Mesh on a wafer-scale substrate.

Strengths

The paper is well-motivated and attacks a clear and relevant problem in the design of next-generation training hardware.

• Strong Problem Motivation: The paper does an excellent job of systematically breaking down the communication challenges of a 2D Mesh when faced with 3D parallelism (Section 3.2, Pages 4-5). The analysis of issues like I/O hotspotting (Figure 4, Page 5), the mathematical impossibility of optimally mapping 3D logical groups onto a 2D physical grid (Figure 5, Page 5), and the difficulties with non-aligned parallelism strategies (Figure 6, Page 6) is thorough and convincing. This provides a solid foundation for the necessity of a more flexible fabric.
• Sound Architectural Concept: The high-level architectural choice of a hierarchical, switched fabric (specifically, an almost fat-tree) is a logical and well-reasoned solution to the problems identified with the 2D Mesh. A switched topology inherently provides greater path diversity and higher bisection bandwidth, which directly addresses the congestion and mapping limitations of a mesh.

Weaknesses

Despite the clear motivation, the paper's central claims are undermined by a flawed evaluation methodology, questionable baseline comparisons, and an oversimplification of critical system-level challenges.

• Unsubstantiated Performance Claims due to Unrealistic Baseline: The headline performance improvements are built on a comparison between the proposed FRED-D architecture and a baseline 2D Mesh that is not equitable. FRED-D is simulated with a bisection bandwidth of 30 TBps, whereas the baseline Mesh is limited to 3.75 TBps (Table 5, Page 10). The reported speedups are therefore less a testament to FRED's architectural novelty and more a predictable outcome of providing the network with 8x more raw bandwidth. The paper attempts an apples-to-apples comparison with FRED-A (which has the same 3.75 TBps bisection bandwidth), but this configuration shows minimal to no performance benefit, thus invalidating the headline claims. The improvements come from bandwidth, not from the specific FRED architecture itself.
• Simulation Fidelity is Questionable: The entire evaluation rests on the ASTRA-SIM framework (Section 7.4, Page 10). While ASTRA-SIM is a useful tool for high-level performance projection, the paper provides no evidence that it accurately models the detailed, low-level network dynamics of a wafer-scale fabric. For example, there is no discussion of how the simulation models the latency of the very long wafer-scale links, the complex routing decisions within the FRED microswitches, or potential physical-layer effects at these high bandwidths. Without calibration against a more detailed network simulator (like Garnet or a commercial tool) or real hardware, the quantitative results lack credibility.
• Physical Implementation Challenges are Understated: The paper acknowledges that the required switch chiplets would be large but dismisses the area overhead by appealing to "unclaimed area on the wafer" and future I/O technologies (Section 6.2.3, Pages 9-10). This is a significant oversimplification. The proposed design requires a massive number of long, high-bandwidth, wafer-scale interconnects between the NPU chiplets and the L1 FRED switches, and then again between the L1 and L2 switches (Figure 8, Page 8). The routing complexity, wiring density, and potential for signal integrity issues in such a design are immense and are not adequately addressed. Claiming this can be implemented by simply using "unclaimed area" ignores the profound physical design challenges.
• In-Network Compute is Not a Free Lunch: A key feature of FRED is its in-network collective execution. However, the paper provides insufficient detail on the hardware cost and complexity of the "R-uSwitch" and "D-uSwitch" components that perform these reductions (Figure 7, Page 7). These operations, especially on floating-point data, are not trivial. The paper presents area and power numbers post-layout (Table 4, Page 9), but does not provide a breakdown of how much of that is attributable to the in-network compute logic versus the standard switching and buffering. This makes it impossible to evaluate the true cost-benefit trade-off of this feature.

Questions to Address In Rebuttal

1. Please justify your primary comparison between FRED-D (30 TBps bisection) and the baseline Mesh (3.75 TBps bisection). To isolate the architectural benefits of FRED, please provide a direct comparison of end-to-end training time between the baseline Mesh and a Mesh that is also provisioned with an equivalent 30 TBps bisection bandwidth.
2. Can you provide more detail on the validation of your ASTRA-SIM model? Specifically, how did you model the latency and energy of the multi-centimeter wafer-scale links, and how were the internal micro-architectural delays of the FRED switches determined and validated?
3. The proposed physical layout in Figure 8 requires an extremely complex and dense wiring scheme. Have you performed a routability analysis for this design? What is the estimated total wire length on the wafer, and what are the associated energy costs for driving signals across these long wires, which your current power analysis seems to aggregate into a single "Additional Wafer-Scale Wiring" number?
4. Please provide a detailed microarchitectural description of the reduction/distribution units (R-uSwitch/D-uSwitch). What is the area and power cost specifically for the FP16 arithmetic logic within these switches, separate from the standard switching logic? How does this design handle potential floating-point exceptions or the need for different rounding modes during in-network reduction?

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper identifies a critical architectural challenge in large-scale DNN training: the mismatch between the communication patterns of advanced 3D parallelism (Data, Pipeline, and Tensor) and the rigid topology of a conventional 2D Mesh interconnect. The authors propose FRED, a hierarchical, switch-based wafer-scale fabric designed to provide the flexible, high-bandwidth connectivity required by these complex training strategies. FRED is built from small, recursive switch units ("FRED Switches") that support in-network collective operations and are arranged in a fat-tree-like topology. The work's central thesis is that moving beyond the simple 2D Mesh to a more sophisticated fabric is essential for unlocking the full potential of wafer-scale training hardware for next-generation AI models.

---

Strengths

This paper is a valuable contribution to the field because it thoughtfully connects several key trends in high-performance computing and applies them to the specific domain of wafer-scale AI accelerators.

• Excellent Problem Synthesis: The paper does a superb job of synthesizing the state of the art in both software (3D parallelism) and hardware (wafer-scale integration) to identify a crucial point of friction. It correctly observes that as training software becomes more sophisticated, the underlying hardware interconnect, which has historically been a simple mesh for both academic proposals and commercial systems like Google's TPUs, becomes a major bottleneck. This clear and timely problem statement (Section 3, Pages 3-6) is the paper's greatest strength.

• Leveraging Established HPC Principles: The proposed solution, a switched fat-tree network, is not a radical invention but a well-reasoned application of established principles from the High-Performance Computing (HPC) world. Fat-tree topologies have long been the standard for large-scale supercomputers precisely because they provide high bisection bandwidth and flexible, non-blocking communication. By adapting this concept to a wafer-scale implementation, the paper bridges the gap between the on-chip (NoC) and data-center-scale networking worlds, creating a compelling vision for a "data center on a wafer." This is a logical and powerful architectural evolution.

• Connecting to Commercial Realities: The FRED architecture, in concept, closely mirrors the direction the industry is heading. NVIDIA's DGX systems, for example, do not connect GPUs in a simple mesh but use dedicated, high-radix NVSwitch chips to create a non-blocking fabric for all-to-all communication. FRED can be seen as the wafer-scale analogue of an NVSwitch-based system. By proposing an on-wafer fabric with in-network collectives, this work provides a forward-looking academic blueprint that aligns with the architectural principles of today's most powerful commercial training systems.

---

Weaknesses

While the high-level vision is strong, the paper could do more to situate its specific design within the rich landscape of existing work and address the immense practical challenges of its proposal.

• Insufficient Engagement with Prior Wafer-Scale Work: The paper positions itself as a novel solution for wafer-scale systems but does not deeply engage with the design choices of existing wafer-scale pioneers, most notably Cerebras. The Cerebras architecture uses a 2D mesh interconnect, but pairs it with a unique "weight streaming" execution model that differs significantly from traditional GPU-style training. A more thorough analysis would compare FRED not just to a generic mesh, but would discuss why the Cerebras approach is insufficient for 3D parallelism and how FRED's flexibility enables a broader range of training paradigms that Cerebras may not support well.

• The "How" of Physical Implementation: The paper focuses on the logical topology of FRED but is light on the details of the physical implementation, which is a major challenge for wafer-scale systems. Building a multi-level fat-tree on a 2D wafer requires extremely long and dense global wiring, which introduces significant latency, power, and signal integrity issues. While the paper acknowledges this (Section 6.2.3, Page 9), it would be strengthened by a more detailed discussion of the physical design challenges and a comparison to alternative physical topologies, such as folded torus or dragonfly networks, which are also used in HPC to reduce wiring length.

• Under-explored Design Space for In-Network Compute: The idea of in-network collectives is powerful. However, the paper presents a single data point (a specific "uSwitch" design). The design space is much richer. For example, what is the trade-off between having more, simpler switches versus fewer, more complex switches with more powerful reduction units? How does the choice of number format (e.g., FP16 vs. BFloat16 vs. block floating point) for the in-network math affect the area, power, and accuracy of the collectives? Exploring these trade-offs would provide a more comprehensive guide for future architects.

---

Questions to Address In Rebuttal

1. Could you elaborate on the comparison between FRED and an NVSwitch-based system? What are the unique challenges and opportunities of implementing a switched fabric on a monolithic wafer compared to connecting discrete GPUs with external switch chips?

2. Your work rightly criticizes the 2D Mesh. However, Google's TPU pods have successfully used a 2D/3D Torus interconnect for years. What specific limitations of the Torus topology, beyond those of a simple Mesh, does FRED address that justifies the significant increase in architectural complexity?

3. The paper focuses on a fat-tree. Could you discuss why you chose this topology over other high-bandwidth HPC topologies like Dragonfly or a hypercube? What are the relative advantages and disadvantages of these alternatives in the context of a 2D wafer implementation?

4. Looking forward, how do you see a fabric like FRED enabling future DNN training paradigms beyond the current 3D parallelism model? Could the flexible, low-latency connectivity be exploited for more dynamic, fine-grained, or irregular parallelism strategies?

Reviewer: The Innovator (Novelty Specialist)

Summary

The authors identify the inefficiency of 2D Mesh interconnects for 3D DNN parallelism and propose FRED, a wafer-scale interconnect fabric. [cite_start]The core novel claim is the architectural synthesis of applying a hierarchical, switched fat-tree topology, complete with support for in-network collective computations, to the specific physical substrate of a monolithic wafer for the purpose of accelerating DNN training (Abstract, Page 1; Section 1, Page 2)[cite: 4, 21]. The proposed "FRED Switch" is a recursive micro-switch design intended to be the building block of this fabric (Section 4, Page 6).

---

Strengths

From a novelty perspective, the contribution of this paper is not in the invention of new foundational primitives, but in the novel adaptation and synthesis of existing, powerful concepts into a new domain.

• Novel Architectural Shift for Wafer-Scale Systems: The primary "delta" of this work is moving the architectural conversation for wafer-scale interconnects beyond the simple 2D Mesh/Torus topologies that have dominated both academic proposals and commercial systems like the Google TPU and Cerebras WSE. By proposing a hierarchical, switched fat-tree (Section 4, Page 6), the paper introduces a fundamentally different and more flexible network paradigm to this domain. This application of a well-understood HPC topology to the unique constraints of a wafer is a significant and novel conceptual step.

---

Weaknesses

While the synthesis is novel, the work's claims are diluted because the foundational building blocks are well-established prior art. The paper does not invent new concepts so much as it re-packages existing ones.

• Core Topology is Not New: A fat-tree network is a canonical topology in high-performance computing (HPC) and data center networking, and its benefits for providing high bisection bandwidth are known for decades. The novelty here is purely in the application to a wafer, not in the invention of the topology itself.
• In-Network Compute is Prior Art: The idea of offloading collective operations (like reductions) into the network fabric is a well-established technique in the HPC community, commercialized in technologies like NVIDIA's SHARP (Scalable Hierarchical Aggregation and Reduction Protocol). [cite_start]The paper acknowledges this prior art (Section 2.2, Page 3)[cite: 35, 57]. The "R-uSwitch" and "D-uSwitch" (Figure 7, Page 7) are a new implementation of this old idea, but the concept itself is not a novel contribution of this paper.
• Functionally Similar to Existing Commercial Fabrics: At a conceptual level, FRED is a wafer-scale implementation of the same design philosophy embodied by NVIDIA's NVSwitch. NVSwitch uses dedicated switch chips to create a non-blocking fabric that provides all-to-all connectivity between GPUs in a server. FRED scales this idea to a wafer with hundreds of processing elements. While the physical implementation challenges are different, the core architectural pattern—using a switched fabric to overcome the limitations of direct point-to-point connections—is functionally identical to existing, commercially available technology.
• [cite_start]Performance Gains Are Not a Novel Insight: The headline performance improvements (up to 1.87x) are almost entirely derived from comparing a FRED-D configuration with 30 TBps of bisection bandwidth to a baseline Mesh with only 3.75 TBps (Table 5, Page 10)[cite: 201, 242]. It is not a novel discovery that an interconnect with 8x more bandwidth performs better. [cite_start]The architecturally equivalent comparison, FRED-A, shows negligible improvement, indicating that the novelty of the FRED topology itself provides little performance benefit over a basic Mesh when bandwidth is held constant (Table 5, Page 10)[cite: 244]. The novelty claim cannot rest on an artifact of an unfair comparison.

---

Questions to Address In Rebuttal

1. The architectural pattern of FRED is conceptually similar to NVIDIA's NVSwitch fabric. What is the fundamental, novel insight of your work beyond scaling the NVSwitch concept to a wafer-level implementation? What new, non-obvious problems did you identify and solve that are unique to the on-wafer context?
2. Fat-tree networks and in-network collectives are staples of the HPC field. What is the specific, novel "delta" in your implementation of these concepts (Section 4, Page 6; Section 6.1, Page 7) that you believe constitutes a significant advancement over prior art, beyond simply being implemented on a wafer?
3. [cite_start]The paper claims to be a "new fabric" (Section 1, Page 2)[cite: 25], but the performance gains seem to come from increased bandwidth rather than the novelty of the topology itself (as shown by the FRED-A results in Table 5, Page 10). Can you defend the novelty of the FRED architecture in light of the fact that, at equivalent bandwidth, it provides minimal benefit over a standard Mesh?
4. Given that a fat-tree requires complex, long-range global wiring on a 2D substrate (Figure 8, Page 8), what is the novel contribution of FRED in solving the physical design and routability challenges, a problem that has historically led designers of on-chip networks to prefer simpler mesh-like topologies?

Paper Title: PD Constraint-aware Physical/Logical Topology Co-Design for Network on Wafer Reviewer Persona: The Guardian (Adversarial Skeptic)

---

Summary

The authors propose a co-design methodology for developing networks on wafers (NoW) for LLM training. The central claim is that existing approaches design physical and logical topologies in isolation, leading to suboptimal performance. To address this, they introduce a "mesh-switch" physical topology, which aims to balance the high integration density of mesh with the communication efficiency of switch-based networks like fat-trees. A Design Space Exploration (DSE) is conducted to find an optimal configuration under physical design (PD) constraints. Building on this physical topology, they propose a "dual-granularity" logical topology and fine-grained communication scheduling. The authors claim their final design achieves up to a 2.39x performance improvement over a baseline mesh network.

Strengths

1. Problem Formulation: The paper correctly identifies a fundamental tension in waferscale design: the trade-off between allocating area for computation versus communication. The motivation to move beyond pure mesh or pure fat-tree (like FRED) topologies is well-founded and addresses a genuine challenge in the field.
2. Constraint-Awareness: The explicit inclusion of PD constraints, such as maximum interconnect length (50mm) and wiring density (3 metal layers), is a crucial step toward practical waferscale architecture design. This grounds the exploration in realistic physical limitations.
3. Holistic Scope: The work attempts to connect multiple layers of the system stack, from the physical layout (mesh-switch) to logical communication patterns (tree+ring) and finally to parallelism strategies (topology-aware sharding). This breadth is commendable.

Weaknesses

My primary concerns revolve around the methodological rigor of the DSE, the validity of the baselines used for comparison, and the overstatement of key claims.

1. Superficial DSE Methodology: The core of the physical topology design rests on a Design Space Exploration (DSE) process (Section 5.3, page 7). However, this DSE is driven by an analytical model (Eqs. 1-4) for area estimation, which the authors admit has an "average error below 15%". For a process intended to find a single "optimal" configuration, relying on a model with such a significant and uncharacterized error margin is concerning. The optimality of the 2x2 mesh group is not rigorously proven but is rather a product of this potentially inaccurate model. The claim that this is computationally necessary (0.67ms vs. 24 hours) does not excuse the potential for the model to lead the DSE to a local, or even incorrect, optimum.

2. Unsupported Generality of the "Optimal" Configuration: "Key Insight 3" (page 8) boldly claims that "the DSE for mesh-switch physical topology consistently converges to 2x2 mesh group configuration" across various hardware and applications. This is a significant overstatement. The supporting evidence in Figures 11 and 12, while showing a peak at 2x2 for the tested configurations, is insufficient to establish such a universal design rule. The result is likely highly sensitive to the specific parameters in Table 2 (e.g., the relative area and performance of compute vs. switch dies). A different set of assumptions could easily shift the optimal point. The claim of consistent convergence is not substantiated.

3. Flawed Performance Breakdown and Baseline Selection: The headline claim of a 2.39x improvement is derived from a breakdown analysis in Figure 24 (page 13). This analysis is methodologically unsound. The baseline for the entire stack is Mesh+Ring. However, the authors acknowledge in Section 8.1 that the TTO logical topology is superior for Mesh. By comparing their logical topology against the weaker "Ring" topology, they artificially inflate the contribution of their "Logical" step (a 1.42x gain). A rigorous comparison would evaluate the full proposed system (MS+Logical+Para) against the strongest possible baseline system (e.g., Mesh+TTO with optimized parallelism). The current breakdown appears engineered to maximize the reported gains.

4. Insufficient Justification for Topology Exclusion: The paper dismisses other SOTA network topologies like Dragonfly and Flattened Butterfly based on wiring density and signal integrity arguments (Section 5.1 and Figure 9). This justification is weak. Figure 9c presents signal loss in dB without defining an acceptable threshold or referencing a specific signaling technology's budget (e.g., a required signal-to-noise ratio). Without a quantitative and well-motivated threshold, the argument that these topologies are infeasible remains an assertion, not a proven fact. This weakens the paper's central claim that mesh-switch is the superior choice among viable alternatives.

5. The "Co-Design" Framework is a Re-labeling of a Sequential Process: The "TickTock Framework" is presented as a novel co-design methodology. In practice, as described, it appears to be a sequential process: first, the DSE is used to fix the physical topology, and then a compatible logical topology is designed for it. There is no evidence of a feedback loop where findings from the logical topology design phase inform or cause a revision of the physical topology. Without this iterative feedback, this is not "co-design" but rather a well-structured sequential design flow.

Questions to Address In Rebuttal

1. On DSE Validity: Can you provide a sensitivity analysis showing how the optimal mesh group configuration (2x2) changes with variations in the key DSE parameters (e.g., compute die area, switch die area/bandwidth)? How do you ensure the 15% error in your analytical model does not lead you to a suboptimal configuration?
2. On Performance Claims: Please provide a revised performance breakdown (as in Figure 24) where the baseline is the strongest possible configuration for Mesh (i.e., using the TTO logical topology you cite as superior). How does this affect the reported 1.42x gain from the logical topology and the final 2.39x overall gain?
3. On Topology Exclusion: Can you provide a quantitative signal integrity budget (e.g., maximum acceptable insertion loss in dB at the target frequency) for the assumed waferscale interconnect technology? Please show explicitly how topologies like Dragonfly violate this specific, technically-grounded budget.
4. On the Co-Design Claim: Can you point to a specific instance in your methodology where the design of the logical topology or parallelism strategy forced a re-evaluation and modification of the physical topology? If not, please justify the use of the term "co-design" over "sequential, constraint-aware design."

Paper Title: PD Constraint-aware Physical/Logical Topology Co-Design for Network on Wafer Reviewer Persona: The Synthesizer (Contextual Analyst)

---

Summary

This paper presents a holistic co-design framework for creating high-performance Networks-on-Wafer (NoW) specifically tailored for Large Language Model (LLM) training. The authors' central thesis is that prior work has suffered from "orphan designs," optimizing either the physical topology (e.g., Mesh) or the logical topology (e.g., routing algorithms) in isolation, thereby hitting a premature performance ceiling.

To address this, the authors propose a comprehensive methodology that bridges this gap. Their core contributions are threefold: 1. A hybrid "mesh-switch" physical topology that balances the high compute density of mesh networks with the superior communication performance of switched fat-tree networks. 2. A Physical Design (PD) constraint-aware Design Space Exploration (DSE) algorithm that systematically searches for the optimal configuration of this topology, grounded in realistic wafer-level constraints like area, power, and maximum interconnect length. 3. A "dual-granularity" logical topology and associated collective algorithms designed specifically to leverage the hierarchical nature of their proposed physical network.

Through simulation on a range of LLM workloads, the authors demonstrate that their co-designed NoW achieves up to a 2.39x performance improvement over existing state-of-the-art mesh-based designs.

Strengths

This work's primary strength lies in its conceptual framing and methodological rigor. It correctly identifies a fundamental weakness in the design of large-scale, monolithic systems and proposes a compelling, systematic solution.

1. Excellent Problem Formulation: The paper's core insight—that physical and logical network topologies for waferscale systems must be co-designed to unlock their full potential—is both powerful and timely. The "ticktock" analogy presented in Figure 1 (Page 2) is an effective visualization of this concept, clearly articulating why previous "orphan designs" are insufficient. This positions the work not as a simple point solution, but as a new way of thinking about the problem.

2. A Principled Hybrid Solution: The proposed "mesh-switch" topology is a well-reasoned middle ground. It recognizes the fundamental trade-off between computation and communication resources on a constrained silicon area, a lesson well-established in the HPC community. By analyzing the shortcomings of pure Mesh (communication-bound) and pure fat-tree like FRED (compute-bound, as shown in Figure 5, Page 4), the authors motivate their hybrid approach not just by intuition, but with clear data.

3. Grounding in Physical Reality: The most significant contribution, from a systems perspective, is the "PD Constraint-aware" DSE. Many academic papers on network topologies remain in the abstract. This work grounds its exploration in concrete physical limitations (e.g., the 50mm D2D link distance constraint mentioned in Section 2.2, Page 3), lending the results significant credibility. This methodology provides a valuable blueprint for future research and industrial development in this space.

4. Connecting the Full Stack: The paper successfully connects the dots from the lowest level of physical constraints up to the highest level of workload performance. The framework spans physical topology, logical topology, collective algorithms, and even parallelism/sharding strategies (Section 7, Page 10). This full-stack view is rare and extremely valuable, showing a deep understanding of how decisions at one level of the system cascade and affect others. The analysis showing why a technique like Ulysses is not a simple drop-in replacement for mesh-based groups (Figure 19c, Page 11) is a prime example of this nuanced, topology-aware approach.

Weaknesses

While the core idea and methodology are strong, the work could be further contextualized and strengthened by addressing some practical systems-level challenges inherent to its chosen domain.

1. Absence of Fault Tolerance and Yield Considerations: Waferscale integration is fundamentally a battle against manufacturing defects. The proposed design, with its reliance on a centralized set of switches, appears to have a significant single point of failure. The paper does not discuss how the topology or routing would adapt to the inevitable presence of dead compute dies, broken links, or faulty switch components. For a waferscale architecture to be viable, a strategy for gracefully degrading performance or routing around faults is not just a feature, but a necessity. This is a well-trodden area in both HPC and NoC research that this work would benefit from engaging with.

2. Scalability of the Centralized Switch Model: The DSE convincingly finds an optimal configuration for the target wafer size. However, the reliance on a single, logically centralized switch could become a bottleneck as substrate sizes scale further (e.g., to the 300,000 mm² glass panels mentioned in Section 5.6, Page 8). A brief discussion on the architectural evolution—perhaps towards a hierarchical or multi-level switch fabric for even larger systems—would strengthen the paper's claims of scalability.

3. Generalizability Beyond LLM Workloads: The work is laser-focused on LLM training, which is appropriate given its importance. However, waferscale systems have potential applications across HPC, scientific computing, and graph analytics, which feature different communication patterns (e.g., heavy nearest-neighbor, sparse all-to-all). A short discussion on how the optimal "mesh-switch" configuration might change for these workloads would broaden the paper's impact and highlight the flexibility of the underlying DSE framework.

Questions to Address In Rebuttal

The authors are encouraged to use the rebuttal to address the following points, which would significantly strengthen the paper's contribution:

1. Could you elaborate on the fault tolerance of the proposed mesh-switch architecture? How does the system handle manufacturing defects within a mesh group or, more critically, within the central switch fabric? Are there provisions for redundant paths or disabling faulty components?

2. Your DSE identifies the 2x2 mesh group as optimal for the evaluated configurations. How sensitive is this result to the communication patterns of the workload? For instance, in an HPC application dominated by stencil computations (nearest-neighbor traffic), would the DSE favor larger mesh groups and a smaller central switch?

3. Regarding the centralized switch, what do you foresee as the primary scaling limiter? Is it the bisection bandwidth of the switch itself, the physical area required, or the complexity of wiring connections from all mesh groups to a central location?

Review Form: The Innovator

---

Summary

This paper proposes a framework for co-designing the physical and logical topologies for a Network on Wafer (NoW), specifically for training Large Language Models (LLMs). The authors contend that prior works have treated physical and logical topology design as separate ("orphan") problems. Their core contribution is a methodology that performs a joint optimization under explicit Physical Design (PD) constraints, such as maximum interconnect length and wiring density. This methodology leads them to propose a novel hybrid physical topology called "mesh-switch," which combines dense mesh compute groups with a central, fully-connected switch. Based on this physical layout, they design a corresponding "dual-granularity" logical topology. The authors employ a Design Space Exploration (DSE) algorithm to find the optimal parameters for their mesh-switch architecture, ultimately demonstrating significant performance gains over existing mesh and fat-tree NoW designs.

Strengths

From a novelty perspective, the paper's primary strength lies in its synthesis of existing concepts into a new, constrained optimization problem specific to waferscale integration.

1. Formalization of a Constrained Co-Design Problem: The most significant novel contribution is the rigorous formulation of the NoW topology design problem under realistic, hard physical constraints (Section 2.2, page 3; Section 5.1, page 6). While hardware-software co-design is a known field, this paper's sharp focus on the unique PD constraints of waferscale systems (e.g., the 50mm link length limit, 3-metal-layer wiring density) provides a novel and practical lens through which to evaluate architectural trade-offs. This moves the discussion beyond abstract graph topologies to physically-realizable designs.

2. A Novel Hybrid Physical Topology: The proposed "mesh-switch" architecture is a genuinely new data point in the NoW design space. It is a well-motivated hybrid that explicitly attempts to combine the high compute density of mesh architectures (e.g., Cerebras, Dojo) with the superior communication performance of switched fat-tree networks (e.g., FRED [78]). This hybrid is not an arbitrary choice but a direct outcome of their analysis of the computation vs. communication trade-off, making it a well-justified architectural proposal.

3. Methodological Novelty in DSE: The development and application of a fast DSE algorithm (Section 5.3, page 7) to navigate the design space of this new hybrid topology is a valuable contribution. It provides a concrete, repeatable methodology for finding an optimal configuration (identified as 2x2 mesh groups), lending credibility and practicality to their proposed framework.

Weaknesses

The paper's claims of novelty are, in some areas, overstated, as several core concepts are reformulations of long-established principles in parallel computing and architecture.

1. Incremental Nature of "Co-Design": The central premise of physical/logical "co-design" is not new. This principle is foundational in HPC, where logical communication patterns (e.g., MPI collectives) are routinely optimized for specific physical interconnects (e.g., fat-trees, dragonflies). The paper cites work like Huang et al. [39] which explicitly covers "Communication algorithm-architecture co-design." The novelty here is therefore not the concept of co-design itself, but its specific application to the PD-constrained waferscale domain. The framing could be more precise about this delta.

2. The "Dual-Granularity Logical Topology" is a Consequence, Not an Invention: The proposed logical topology (Section 6, page 9) is a direct and rather obvious consequence of the hierarchical physical topology. A physical design with local mesh groups and a global switch naturally mandates a hierarchical routing scheme (e.g., XY routing within the group, direct hop via the switch between groups). Similarly, optimizing collectives via a hierarchical algorithm (e.g., tree+ring) is standard practice for hierarchical hardware. The conceptual leap in the logical topology design is therefore minimal; it is the necessary software mapping for the novel hardware, not a standalone innovation.

3. The "TickTock Framework" is Re-labeling: The "ticktock framework" presented in Figure 1 (page 2) is a compelling visual but does not represent a fundamentally new design methodology. It is an illustration of iterative design, a standard engineering practice. The true methodological contribution is the detailed analytical model and DSE process described in Section 5.3, not this high-level diagram.

Questions to Address In Rebuttal

1. The concept of algorithm-architecture co-design is well-established in the literature. Could the authors please clarify the fundamental novelty of their framework beyond the (admittedly important) application of this principle to the specific PD constraints of waferscale systems? What is the core, generalizable insight from your methodology that prior co-design work has missed?

2. The mesh-switch topology is an intuitive hybrid of mesh and a central switch. Hierarchical topologies that combine different network types at different levels are a classic architectural pattern. What is the non-obvious insight that makes this particular combination uniquely suited for NoW? Is there a theoretical justification for this being the optimal hybrid class, or is its superiority based solely on the empirical DSE results presented?

3. The paper presents the dual-granularity logical topology as a key contribution. However, given the hierarchical physical topology, a corresponding hierarchical logical topology seems to be the most straightforward and logical implementation choice. Could you elaborate on what makes this logical design non-trivial or inventive in its own right, distinct from the physical structure it is designed to serve?

Paper Title: Cassandra: Efficient Enforcement of Sequential Execution for Cryptographic Programs Reviewer: The Guardian (Adversarial Skeptic)

---

Summary

The authors propose CASSANDRA, a hardware/software co-design aimed at enforcing a strictly sequential execution model for cryptographic programs, thereby eliminating the attack surface of control-flow speculation (Spectre-v1/v2). The core mechanism replaces the conventional Branch Prediction Unit (BPU) for cryptographic code with a "record-and-replay" system. This involves an upfront, offline analysis of a program's branch behavior to generate compressed control-flow traces. These traces are then loaded into a new microarchitectural component, the Branch Trace Unit (BTU), which dictates fetch redirection for crypto code at runtime. The central claims are that this approach not only provides a strong security guarantee against control-flow misspeculation but, counter-intuitively, also results in a performance improvement (a 1.85% speedup on average) over an unprotected baseline by eliminating branch mispredictions.

Strengths

1. Sound Foundational Premise: The work correctly identifies a core characteristic of constant-time cryptographic code: its control flow is, by design, largely independent of secret inputs. Leveraging this property to pre-compute execution paths is a logical, if not entirely novel, direction for exploration.
2. Novel Trace Compression: The application of k-mers counting, a technique inspired by genomics, to compress branch traces is a clever and effective method for reducing the storage overhead of the proposed approach. The compression rates reported in Table 1 (page 4) are substantial and represent a non-trivial technical contribution.
3. Focused Problem Domain: The paper wisely constrains its solution to the domain of cryptographic code, avoiding the untenable challenge of applying this method to general-purpose programs. This focus makes the proposed trade-offs more plausible.

Weaknesses

My primary concerns with this work center on the fragility of its core assumptions, the practical deployability of the solution, and an insufficient analysis of its performance and security implications.

1. Overstated and Poorly Substantiated Performance Claims: The headline claim of a 1.85% speedup over an "unsafe baseline" is highly suspect and relies on an incomplete performance model.

   • The entire performance benefit hinges on avoiding the cost of branch mispredictions. However, the cost of a miss in the proposed Branch Trace Unit (BTU) is not rigorously evaluated. The paper states, "In case of a trace miss in the BTU, the frontend stalls until the trace becomes available" (Section 5, page 6). This is a critical event. A stall to fetch trace data from memory is potentially orders of magnitude more costly than a typical branch misprediction penalty. The evaluation in Section 7 provides no data on BTU miss rates or the latency of servicing such a miss. Without this data, the claimed performance improvement is unsubstantiated. The geomean could easily become a significant slowdown if BTU misses are non-trivial.
   • Figure 7 (page 10) shows that for several benchmarks (e.g., sphincs-shake-128s, DES_ct, ModPow_i31), the performance gain is negligible. The 1.85% geomean papers over a highly variable and, in many cases, insignificant result.

2. Severe Practicality and Deployment Hurdles: The proposed software/hardware contract creates a system that is fundamentally brittle.

   • Trace Invalidation: The entire mechanism is predicated on static PC addresses. Any change to the binary—recompilation with a different compiler version, different optimization flags, a security patch, or even static library updates—will invalidate the pre-computed traces. The authors dismiss this by stating traces need to be regenerated (Q2, page 11), but this severely underestimates the logistical complexity in any real-world software ecosystem. It effectively prohibits standard binary distribution and patching, shifting an enormous burden onto developers or distributors.
   • Handling of Public Parameters: The solution for code with control flow dependent on public parameters (e.g., key sizes, modes of operation) is to generate separate traces for each configuration (Q1, page 11). This leads to significant binary bloat and a combinatorial explosion of required traces for any moderately complex cryptographic library. The proposed fallback of stalling the pipeline for dynamic cases like stream ciphers is an admission that the core model is not general enough, even within the crypto domain.

3. Insufficient Security Analysis: The security model has potential gaps that are not adequately addressed.

   • The BTU as a New Side Channel: The authors' rebuttal to the BTU being a timing channel is a simple analogy to the ICache (Q5, page 12). This is insufficient. The BTU is a new, stateful structure with its own specific access patterns, LRU replacement policy, and contention points (Pattern Table, Trace Cache, Checkpoint Table). An attacker could potentially engineer contention on these structures to create a new timing side channel, leaking information about which crypto branches are being executed by a victim. A rigorous analysis of this new attack surface is absent.
   • Integrity of the Crypto PC Range: The system's security relies on correctly partitioning the address space into "crypto" and "non-crypto" regions. The mechanism for this is a "new status register" (Section 5.2, page 7). How is this register set and protected? More critically, the paper claims to prevent non-crypto branches from speculatively redirecting to crypto code via an "integrity check" (Section 5.3, page 7). This is a crucial security boundary, yet its implementation and robustness are not detailed. A failure here would allow an attacker to bypass CASSANDRA entirely by misspeculating from outside the protected region to a location inside it.

Questions to Address In Rebuttal

The authors must provide clear, data-driven answers to the following questions to justify their claims:

1. BTU Performance: What were the miss rates for the Pattern Table and Trace Cache across all benchmarks in your evaluation? What is the full cycle cost assumed for servicing a BTU miss that requires a memory access? Please provide a sensitivity analysis showing how the overall performance changes as this miss latency increases.
2. Software Lifecycle: Please provide a concrete workflow for how a large, distributed software project (e.g., OpenSSL) would integrate CASSANDRA's trace generation into its development, testing, and security patching lifecycle without creating an untenable maintenance burden.
3. BTU Security: Beyond the ICache analogy, please provide a detailed analysis of the potential for timing-based side channels within the BTU's components. Can an attacker in a sibling hardware thread create contention in the PAT/TRC to infer the control flow of a victim process? Is the BTU state flushed on all context switches, and if so, what is the performance cost?
4. Boundary Protection: Please detail the exact hardware mechanism of the "integrity check" that prevents non-crypto branches from speculatively targeting crypto-code regions. How is this check performed without re-introducing the very pipeline stalls that branch prediction aims to avoid? How is its correctness and security guaranteed?

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper addresses the critical problem of speculative execution attacks undermining the security guarantees of constant-time cryptographic programs. Existing defenses typically incur performance overhead by restricting or fencing speculation. The authors of CASSANDRA propose a radical and elegant alternative: for this specific class of programs, they advocate for entirely replacing control-flow speculation with a deterministic "record-and-replay" mechanism.

The core idea is enabled by two key insights into constant-time cryptography: (1) its sequential control flow is, by design, independent of secret inputs and thus largely static, and (2) its implementation is typically loop-intensive and repetitive, making its control-flow trace highly compressible. The authors introduce a novel offline analysis step, cleverly inspired by k-mers counting from bioinformatics, to massively compress these traces. In hardware, a new, small component called the Branch Trace Unit (BTU) consumes these compressed traces to provide perfect, deterministic fetch redirection for crypto code, while the conventional Branch Prediction Unit (BPU) handles non-crypto code.

The central and most compelling result is that this mechanism not only enforces a strict sequential execution model—the very model that constant-time programmers assume—but also counter-intuitively yields an average performance speedup of 1.85% over an unsafe baseline processor. This transforms a security mechanism into a performance optimization.

Strengths

1. A Novel and Elegant Conceptual Shift: The most significant strength of this work is its departure from the prevailing "manage the damage" paradigm of Spectre defenses (e.g., taint tracking, fencing). Instead of trying to secure a speculative process, CASSANDRA replaces it with a deterministic one for a domain where this is feasible. This "record-and-replay" concept is a fundamentally different, and arguably much cleaner, approach to solving the control-flow speculation problem for deterministic code.

2. Exceptional Performance Results: Achieving a security guarantee that is stronger than the state-of-the-art while simultaneously improving performance over an unprotected baseline is a remarkable result. The elimination of branch mispredictions for crypto kernels is a powerful performance lever, and the authors have successfully exploited it. This flips the traditional security-performance trade-off on its head and makes adoption highly compelling.

3. Strong and Simple Security Guarantee: CASSANDRA enforces the exact sequential execution model that programmers reason about when writing constant-time code. This closes the dangerous gap between the software's security model and the hardware's execution reality. It provides a clear, understandable guarantee: for CASSANDRA-protected code, control flow will not transiently deviate from the sequential path.

4. Clever Cross-Pollination of Ideas: The use of trace compression techniques inspired by DNA sequencing (Section 4.2, page 4) is an excellent example of creative, cross-disciplinary thinking. The problem of finding repeating patterns in a massive, linear trace of branch outcomes is structurally analogous to finding patterns in a DNA sequence, and the authors' application of this concept is what makes the entire approach practical by solving the trace storage problem.

Weaknesses

While the core idea is powerful, its practical scope and deployment model present some challenges that could be explored further. These are not fundamental flaws but rather boundaries of the proposed solution.

1. Domain Specificity: The solution is explicitly and effectively tailored to constant-time cryptographic code. While this is a critically important domain, the technique is not general-purpose. The paper is clear about this, but the impact is necessarily confined to workloads that feature this type of deterministic control flow.

2. Toolchain and Maintenance Overhead: The security of CASSANDRA relies on an offline, pre-computed trace. This introduces a tight coupling between the compiled binary and the hardware mechanism. Any recompilation, patch, or change in compiler options that alters instruction addresses (PCs) would require regenerating the traces. While the authors' automated procedure (Section 4.3, page 5) is a good first step, the integration of this process into the complex build and maintenance pipelines of major software projects (like OS kernels or large crypto libraries) presents a practical friction point that is not fully explored.

3. Handling of Public, Non-Secret Parameters: The paper discusses handling control flow that depends on public parameters (e.g., key sizes) by generating separate traces for each mode (Discussion Q1, page 11). For cryptographic primitives with many modes or configuration options, this could lead to a non-trivial increase in binary size to store all possible trace variants. This static, enumerative approach may become unwieldy.

Questions to Address In Rebuttal

1. The strength of CASSANDRA is its specialization. Looking beyond cryptography, could the authors elaborate on other potential domains where this deterministic "record-and-replay" model might be applicable? For example, could it be used for scientific computing kernels, codecs, or other programs with data-independent control flow, thereby broadening the potential impact of this architectural concept?

2. The trace generation process is an offline step. How do the authors envision this integrating into standard development and deployment pipelines, particularly for large, continuously evolving libraries like OpenSSL? What is the expected friction for developers and maintainers when patching code (e.g., for a non-crypto-related bug) that requires re-linking and thus invalidates all PC-based traces?

3. Regarding public parameters, the proposed solution of storing separate traces for different modes (e.g., AES-128 vs. AES-256) seems practical for a few variants. However, for algorithms with a wider range of public inputs that affect control flow (e.g., certain post-quantum schemes or protocol state machines), this could lead to significant binary bloat. Have the authors considered a more dynamic or hybrid approach, or is this static pre-generation seen as the only feasible path for maintaining performance?

Reviewer: The Innovator (Novelty Specialist)

Summary

The paper proposes CASSANDRA, a hardware/software co-design to enforce a strictly sequential execution model for constant-time cryptographic programs, thereby eliminating the attack surface for control-flow speculation attacks like Spectre. The core idea is to completely disable the branch predictor for this class of code and instead rely on a "record-and-replay" mechanism. This involves an offline analysis phase where the sequential control flow trace of a program is captured, compressed using techniques inspired by bioinformatics (k-mers counting), and embedded in the binary. At runtime, a new microarchitectural component, the Branch Trace Unit (BTU), consumes these compressed traces to provide perfect, deterministic fetch redirection, obviating the need for prediction. The authors claim this not only provides a strong security guarantee but also results in a performance speedup over an unsafe baseline due to the elimination of branch mispredictions.

Strengths

The primary strength of this paper lies in its novel re-contextualization of existing concepts for a new purpose, and the clever cross-domain application of a compression technique.

1. Novel Repurposing of a Known Concept: The idea of recording and replaying a stream of execution is not, in itself, new. It is the fundamental principle behind Trace Caches (Rotenberg et al., MICRO 1996 [58], 1997 [59]), which record dynamic instruction traces to improve fetch bandwidth. However, CASSANDRA repurposes this concept for an entirely different goal: security enforcement rather than performance optimization. The key delta is that Trace Caches are dynamic, opportunistic, and fall back to a speculative frontend, whereas CASSANDRA is static, mandatory, and its core design principle is the elimination of speculation. This shift in purpose leads to a fundamentally different design (offline analysis, stall-on-miss) which, to my knowledge, is a novel approach for Spectre mitigation.

2. Novel Application of Compression Technique: The use of k-mers counting, a technique from DNA sequencing, to compress branch traces is a genuinely novel contribution within the computer architecture domain. Prior work on trace compression has typically relied on more standard run-length encoding or dictionary-based methods. Applying an algorithm designed to find unknown repeating patterns in massive biological sequences to the problem of highly regular, loop-intensive cryptographic control flow (as detailed in Section 4.2, page 4) is a clever and effective insight that enables the entire proposal. This is the paper's most significant and indisputably new technical idea.

3. Surprising Result from a Novel Approach: The combination of a strict, security-enforcing execution model with a net performance gain is a powerful and non-obvious result. Typically, security mechanisms that restrict speculation incur significant performance overhead. The insight that for this specific domain, perfect prediction (via replay) is faster than state-of-the-art speculation is a novel finding that validates the complexity of the proposed mechanism.

Weaknesses

While the core ideas are strong, the novelty could be framed more sharply by explicitly differentiating from conceptually adjacent prior art.

1. Insufficient Differentiation from Profile-Guided Prediction: The paper cites profile-guided branch prediction work like Whisper [33], but it could do more to contrast its mechanism. Both use offline analysis to inform runtime branch resolution. The critical distinction is that Whisper assists the predictor, while CASSANDRA replaces it. CASSANDRA provides a guarantee of sequential control flow, whereas profile-guided schemes still operate within a speculative framework and offer no such guarantee. This distinction is the crux of the novelty and should be more central to the paper's narrative.

2. The "Record-and-Replay" Moniker: While accurate, the term "record-and-replay" is heavily loaded with prior art from debugging and performance (e.g., deterministic replay systems). The authors' novelty is not in the general concept, but in its static, compressed, and security-enforced instantiation for the CPU frontend. The paper would benefit from language that more precisely captures this specific instantiation to avoid being conflated with functionally different systems that share the same high-level name.

3. Novelty is Domain-Specific: The novelty and effectiveness of the entire approach are predicated on the unique properties of constant-time cryptographic code (static control flow). This is not a weakness of the work itself, but a boundary on its novelty. The proposed mechanism is not a general-purpose architectural innovation but a highly specialized one. This should be acknowledged more directly as a trade-off that enables the design.

Questions to Address In Rebuttal

1. Regarding Trace Caches: Please explicitly articulate the three most critical conceptual and implementation differences between the proposed Branch Trace Unit (BTU) and a classic Trace Cache [58, 59]. The key is to move beyond the difference in goals (security vs. performance) and focus on how those goals lead to a fundamentally different mechanism (e.g., static vs. dynamic trace generation, content of the trace, behavior on a miss).

2. Regarding Trace Compression: The application of k-mers counting from bioinformatics is a central pillar of your work. To the best of your knowledge, is this the first proposal to use this specific class of pattern detection algorithms for the compression of any hardware-level execution trace (e.g., branch, address, value traces) in the computer architecture literature?

3. Regarding Input-Dependent Control Flow: Footnote 3 on page 5 states that for branches whose traces depend on public inputs (e.g., plaintext length in a stream cipher), the processor stalls until the branch resolves. Could you quantify the performance impact of this specific case? More importantly from a novelty standpoint, does this necessary escape hatch represent a fundamental limitation of the "pure" replay model you propose?

4. Regarding Brittleness of the Approach: The proposed solution is novel because it is tailored to the properties of constant-time code. How does the mechanism behave if a programmer introduces a secret-dependent branch by mistake? Does the offline analysis tool detect this? If not, does the system fail in a secure way (e.g., by default-stalling) or does it fall back to an insecure behavior? The novelty of the solution is tied to its security guarantee, and it is important to understand the failure modes.

Reviewer: The Guardian

---

Summary

The paper proposes FAST, a novel hardware accelerator for Fully Homomorphic Encryption (FHE) based on the CKKS scheme. The central thesis is that existing accelerators are too rigid, relying on a single key-switching method (e.g., Hybrid) and failing to leverage recent cryptographic optimizations such as the gadget decomposition key-switching method (KLSS) and hoisting. To address this, the authors introduce a versatile framework, Aether-Hemera, to dynamically select the optimal key-switching method based on the ciphertext level. The core hardware innovation is a Tunable-Bit Multiplier (TBM) designed to efficiently execute both 36-bit operations (optimal for Hybrid) and 60-bit operations (optimal for KLSS). The authors claim their design achieves a significant 1.8x average speedup over state-of-the-art accelerators.

Strengths

1. Sound Motivation: The paper correctly identifies a critical gap in the FHE accelerator landscape. The observation that the computational trade-offs between different key-switching methods (Hybrid vs. KLSS) vary with ciphertext level (l) is insightful and provides a strong motivation for a more flexible hardware architecture.
2. Novel Architectural Concept: The idea of a tunable-precision datapath, embodied by the Tunable-Bit Multiplier (TBM), is a clever architectural response to the divergent computational requirements of modern FHE algorithms. Supporting both 36-bit and 60-bit operations within a unified multiplier is a non-trivial design contribution.
3. Analysis of Algorithmic Trade-offs: The analysis presented in Section 3, particularly in Figures 2 and 3, provides a valuable characterization of the performance landscape for different key-switching methods and the impact of hoisting. This analysis effectively grounds the hardware design choices in concrete algorithmic behavior.

Weaknesses

Despite its interesting premise, the paper suffers from several critical weaknesses related to its evaluation methodology, internal consistency, and substantiation of claims. These issues undermine the credibility of the reported performance improvements.

1. Questionable and Potentially Unfair Baselines: The primary performance comparison is against SHARP [20] and several enhanced variants (SHARPLM, SHARP8C, SHARPLM+8C) that the authors appear to have modeled rather than implemented. Footnote 2 on page 11 states, "We also model performance under large memory conditions by comparing the effects of reduced computational workload." This is not a substitute for a rigorous, cycle-accurate simulation of a modified baseline. It is highly likely that this "modeling" approach optimistically estimates the baseline's performance, thereby artificially inflating FAST's relative speedup. Similarly, the claim that SHARPLM "integrates direct hoisting technology" is not substantiated with implementation details. Without a fair and rigorously simulated baseline, the central performance claims of the paper are suspect.

2. Conflation of Algorithmic and Architectural Gains: The paper's core contribution is presented as a new architecture, yet a substantial portion of the performance gain stems from employing a superior algorithm (KLSS) that the baseline (SHARP) was never designed to support. The evaluation fails to disentangle these two effects. The "Efficiency Study" in Section 7.6 compares FAST (with TBM) to "FAST with a 36-bit ALU," which is still the authors' own architecture. A proper ablation study would compare FAST to a baseline like SHARP when both are limited to running the same algorithm (i.e., the Hybrid method). As it stands, it is impossible to determine how much of the speedup is due to the novel TBM and flexible architecture versus simply using a better algorithm.

3. Contradiction in On-Chip Memory Requirements: The authors claim in Section 5.6 (page 10) that the designed on-chip memory of 245MB is "enough to support the KLSS method." However, their own analysis in Figure 3b (page 5) clearly shows that the working set size for KLSS can reach nearly 295MB at the highest ciphertext levels. This is a direct contradiction. If FAST cannot support KLSS at the very levels where Figure 2a suggests it is most beneficial, the entire premise of the Aether-Hemera dynamic selection framework is compromised. The paper does not address how this memory shortfall is handled.

4. Vague Description of the Aether-Hemera Framework: The Aether analysis tool is described as "preprocessing on the server side" (Section 4.1.1, page 6) that takes an "FHE operation flow" to generate a configuration file. This description is superficial. It is unclear if this requires a full pre-simulation of the application or if it is a simple static analysis. Its ability to handle dynamic or data-dependent control flow is not discussed, limiting its generality. The complexity of managing two distinct sets of evaluation keys (for 36-bit and 60-bit moduli) and the potential hardware reconfiguration overheads are mentioned as challenges but are not adequately addressed in the description of Hemera's online management.

5. Unsupported Hardware Claims: The paper claims the TBM achieves its flexibility with "only 28% area overhead relative to conventional 60-bit multipliers" (Section 4.2, page 8). The definition of a "conventional" 60-bit multiplier is not provided; a fair comparison would be against a well-optimized, potentially composed, 60-bit modular multiplier, not necessarily a monolithic one. Furthermore, the assertion that "existing accelerators will struggle to integrate these capabilities" is a strong claim made without any supporting evidence or detailed reasoning.

Questions to Address In Rebuttal

1. Please provide a detailed methodology for how the enhanced SHARP baselines (SHARPLM, SHARP8C) were created. Was a cycle-accurate model of the SHARP architecture modified to include larger memory and support for hoisting, or was performance estimated by simply adjusting operation counts? The validity of the 1.8x speedup claim hinges on the fairness of this comparison.

2. Please address the apparent contradiction between the stated on-chip memory size of 245MB and the peak working set requirement of ~295MB for KLSS shown in your own Figure 3b. Does this memory limitation prevent the use of KLSS at high ciphertext levels? If so, how does this impact the effectiveness of the Aether-Hemera framework and the overall performance results?

3. To isolate the architectural contribution of FAST, can you provide performance data for your architecture against a baseline like SHARP when both systems are restricted to executing only the Hybrid key-switching method? This would provide a direct, apples-to-apples comparison of the hardware's efficiency.

4. Could you elaborate on the offline analysis process of the Aether tool? What are its computational complexity and limitations? Specifically, does it require a full trace or simulation of the target application, and how would it handle applications with non-static computation graphs?

5. Regarding the TBM's 28% area overhead claim, what specific design was used as the "conventional 60-bit multiplier" baseline? Please provide details on its implementation to justify the comparison.

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper presents FAST, a novel hardware accelerator for Fully Homomorphic Encryption (FHE) based on the CKKS scheme. The central contribution lies in its co-designed software/hardware architecture that dynamically adapts to the changing computational requirements of an FHE application. The authors' core insight is that different key-switching algorithms—specifically the traditional hybrid method and the more recent KLSS method—exhibit superior performance at different ciphertext levels. Furthermore, these methods have disparate precision requirements (36-bit for hybrid vs. 60-bit for KLSS).

To exploit this, FAST introduces two key innovations: 1. Aether-Hemera Framework: A software layer where "Aether" performs offline analysis to determine the optimal key-switching strategy (including the use of hoisting) at each stage of the computation, and "Hemera" manages the corresponding evaluation keys at runtime. 2. Tunable-Bit Multiplier (TBM): A versatile hardware multiplier unit that can be dynamically configured to perform either two 36-bit multiplications in parallel or a single 60-bit multiplication, thereby efficiently supporting both key-switching methods without the significant area overhead of dedicated datapaths.

By integrating these concepts, FAST is the first accelerator architecture, to my knowledge, to systematically support and switch between multiple state-of-the-art key-switching primitives within a single FHE program execution. The evaluation demonstrates a significant average speedup of 1.8x over existing state-of-the-art FHE accelerators.

Strengths

1. Excellent Core Insight and Motivation: The paper's primary strength is its foundational observation, clearly articulated in the Motivation section (Section 3, pages 4-6). The analysis presented in Figure 2, which shows the performance crossover between the hybrid and KLSS key-switching methods as a function of ciphertext level, is compelling. This insight elevates the work from being "just another accelerator" to one that addresses a fundamental, dynamic property of FHE computations. It correctly identifies that a one-size-fits-all hardware design is suboptimal.

2. Bridging Cryptographic Theory and Hardware Architecture: This work does an admirable job of connecting recent advances in cryptographic methods (KLSS, hoisting) with concrete hardware design. Many accelerator papers optimize for established, sometimes outdated, cryptographic primitives. By explicitly designing for the latest techniques described in papers like Kim et al. [22] and Chen et al. [10], the authors ensure their architecture is relevant and pushes the state-of-the-art forward. This is a crucial contribution to the community, demonstrating how architects must co-evolve their designs with the underlying algorithms.

3. Elegant Hardware/Software Co-Design: The proposed solution is not purely a hardware effort. The Aether-Hemera framework is a clever software abstraction that manages the complexity of the dynamic decision-making process. It allows the hardware (specifically the TBM) to be flexible but relatively simple, while the intricate logic of when to switch precision and algorithms is handled offline. This separation of concerns is a hallmark of good system design.

4. Novel and Practical Hardware Primitive (TBM): The Tunable-Bit Multiplier is an elegant solution to the multi-precision problem identified in Section 3.2. Instead of brute-forcing the issue with separate 36-bit and 60-bit datapaths, the TBM offers a reconfigurable unit that maximizes parallelism for the common 36-bit case while still efficiently supporting the demanding 60-bit case. This demonstrates a deep understanding of hardware design trade-offs.

Weaknesses

While the core idea is strong, the paper could be improved by further exploring the implications and limitations of its approach.

1. Generality of the Aether Analysis: The Aether tool is presented as a key enabler, but its inner workings and generality are not fully detailed. It seems to function as a profile-guided optimization tool. How dependent is this analysis on specific FHE parameter sets (e.g., ring degree N, initial modulus Q)? If an application or its security parameters change, does the entire analysis need to be re-run? A discussion on the robustness and potential overhead of this offline stage would strengthen the paper.

2. Sensitivity to On-Chip Memory: The design relies on a very large (245MB) on-chip memory to support the significant evaluation key storage requirements of KLSS and hoisting (as shown in Figure 3b). This is a major factor in the total chip area (Table 3). While the authors perform a sensitivity analysis in Figure 13, the discussion could be expanded. The current approach seems tailored for a high-end design point. It would be valuable to understand how the Aether framework's decisions and the overall performance would change under more constrained memory budgets (e.g., 64MB or 128MB), which might be more commercially viable.

3. Complexity of Key Management: The Hemera runtime component must manage multiple sets of evaluation keys for different algorithms and potentially different hoisting factors. While the paper states this is handled, the true complexity and potential for pipeline stalls or memory bank conflicts during key loading and switching are not deeply analyzed. A more detailed examination of the runtime overhead of this dynamic management would be beneficial.

Questions to Address In Rebuttal

1. Could the authors elaborate on the automation and generality of the Aether analysis tool? For a new FHE application not benchmarked in this paper, what is the process and computational cost required for Aether to generate the optimal execution plan? Is any manual intervention or application-specific tuning required?

2. The 245MB on-chip memory is substantial. Could you comment on the performance trade-offs if the memory is constrained to a more modest size, say 128MB? How would the Aether tool adapt its strategy, and what would be the anticipated performance degradation on a benchmark like bootstrapping? Would it simply fall back to using the hybrid method more often?

3. The core idea of dynamic algorithm selection is powerful. Have you considered extending this concept beyond key-switching? For example, could different NTT algorithms or base conversion strategies be dynamically selected depending on the computational context? This work seems to open a new avenue for FHE accelerator design, and I would be interested to hear your perspective on its broader applicability.

Reviewer: The Innovator (Novelty Specialist)

---

Summary

The authors present FAST, a hardware accelerator for the CKKS Fully Homomorphic Encryption (FHE) scheme. The paper's central claim to novelty rests on being the first hardware architecture to incorporate and dynamically manage multiple recent cryptographic optimizations that have, until now, primarily been explored in software. Specifically, the work claims novelty in three areas:

1. The co-integration and hardware support for two distinct key-switching methods: the traditional hybrid method and the more recent gadget decomposition (KLSS) method.
2. The creation of a software/hardware framework (Aether-Hemera) to analyze workload characteristics (e.g., ciphertext level) and dynamically select the optimal key-switching method during a single application's execution.
3. The design of a "Tunable-Bit Multiplier" (TBM) architecture capable of dynamically switching between performing dual 36-bit multiplications or a single 60-bit multiplication, thereby efficiently supporting the differing precision requirements of the two key-switching methods.

The core idea is not a new fundamental cryptographic primitive, but rather a novel system-level co-design that synthesizes existing, but disparate, software-level optimizations into a cohesive and reconfigurable hardware architecture.

---

Strengths

From the perspective of novelty, the paper's primary strengths are:

1. Novel Synthesis of Existing Art: The core contribution is the architectural synthesis of very recent cryptographic optimizations. The KLSS method (Kim et al., 2023) and hoisting techniques have been proposed to improve FHE performance on CPUs/GPUs, but prior accelerator designs (e.g., SHARP, ARK) have not incorporated them. FAST appears to be the first to build a custom microarchitecture specifically to exploit them. This synthesis is non-trivial and represents a tangible advancement.

2. Novel Problem Identification and Motivation: The analysis in Section 3.1 (page 5, Figure 2) is a key contribution in its own right. The observation that the computational advantage shifts between the hybrid and KLSS methods depending on the ciphertext level (l) is a crucial insight. This provides a strong, novel motivation for a dynamic, multi-method architecture, moving beyond the static "one-size-fits-all" approach of prior work.

3. Purpose-Built Reconfigurable Hardware: The Tunable-Bit Multiplier (TBM) presented in Section 4.2 (page 8) is a clever hardware solution to the problem identified. While reconfigurable or multi-precision arithmetic units are not fundamentally new concepts in digital design, the TBM is a purpose-built unit whose design is directly motivated by the specific precision dichotomy (36-bit vs. 60-bit) of the two targeted key-switching methods. Its novelty lies in its specific application and tight integration to solve a unique FHE acceleration challenge.

---

Weaknesses

My critique is focused on the precise boundaries of the novelty presented:

1. Contribution is Primarily in Synthesis, Not Invention: It must be stated clearly that the constituent parts of the proposed system are not new. The KLSS algorithm, hoisting, and Karatsuba-style multiplication (which the TBM is a variant of) are all pre-existing concepts. The paper's novelty rests entirely on being the first to combine them in a hardware accelerator. While this is a valid contribution, the paper could be more precise in delineating its system-level synthesis contribution from the underlying, previously-published algorithmic work.

2. Limited Scope of the Dynamic Framework: The Aether-Hemera framework is presented as a solution for choosing between the hybrid and KLSS methods. However, the work does not explore whether this framework represents a more general, extensible principle for FHE accelerator design. Is this a point solution for these two specific methods, or a paradigm that could incorporate future, as-yet-unknown FHE primitives and optimizations? The novelty would be significantly stronger if the latter were demonstrated or at least convincingly argued.

3. Complexity vs. Benefit Justification: The proposed architecture introduces significant complexity: a dual-method management framework, more intricate key management for two different key types, and a more complex multiplier unit with added control logic. While the authors show a notable performance gain (avg. 1.8x over SHARP), the performance-per-area gain is more modest (1.13x). This suggests that the novel techniques add nearly as much hardware cost as they provide in performance benefit. A deeper analysis on whether this trade-off is fundamentally advantageous across a wider range of parameters would strengthen the claim. For instance, how does this trade-off change if on-chip memory is more constrained?

---

Questions to Address In Rebuttal

1. The core novelty appears to be the system-level co-design of previously separate software optimizations. Beyond being "the first," what is the fundamental architectural insight or principle that future designers can learn from FAST? Is it simply "support more algorithms," or is there a deeper principle about reconfigurability in FHE that you are proposing?

2. Regarding the Tunable-Bit Multiplier (TBM): The use of three smaller multipliers to construct one larger multiplier is a well-known technique (e.g., Karatsuba). Please clarify how the TBM's design is novel beyond a direct hardware mapping of this principle to solve the specific 36/60-bit requirement. Was a broader design space of multi-precision units explored?

3. The Aether-Hemera framework makes decisions based on ciphertext level and hoisting opportunities. Could the authors elaborate on its extensibility? If a new key-switching algorithm with different precision requirements and performance characteristics were proposed next year, how much of the Aether-Hemera decision logic and the underlying FAST hardware would need to be redesigned? This speaks directly to the durability of the novel contribution.

Reviewer: The Guardian

Summary

This paper proposes Neo, a framework for accelerating Fully Homomorphic Encryption (FHE) computations by mapping them onto the Tensor Core units found in modern NVIDIA GPUs. The core idea is to decompose the large integer polynomial multiplications at the heart of FHE into a series of smaller, fixed-precision matrix multiplications that are a perfect fit for the Tensor Core hardware. The authors present a "limb-interleaving" data layout strategy to manage the high-precision arithmetic and a "fragment-based" scheduling approach to handle polynomials that are larger than the Tensor Core's native dimensions. They claim this method achieves significant speedups (up to 7.8x) over existing CPU and GPU-based FHE libraries.

---

Strengths

The paper is founded on a clever and pragmatic observation that has the potential for significant real-world impact.

• Pragmatic Use of Existing Hardware: The core strength of this paper is its recognition that modern GPUs contain highly specialized, powerful compute units (Tensor Cores) that are currently underutilized by FHE workloads. The idea of re-purposing this existing, ubiquitous hardware for FHE acceleration, rather than designing a new ASIC, is a sound and practical approach.
• Clear Algorithmic Breakdown: The paper does an excellent job of systematically breaking down the complex problem of large polynomial multiplication into a sequence of smaller matrix multiplications that can be directly mapped to the Tensor Core's capabilities (Section 3, Pages 3-4). The mathematical formulation is clear and provides a solid foundation for the proposed mapping strategy.

---

Weaknesses

Despite the clever premise, the paper's conclusions are undermined by an incomplete analysis, questionable evaluation methodologies, and a failure to address critical, well-known challenges in GPU-based acceleration.

• Fundamentally Flawed Baseline Comparison: The headline performance claims are invalid because the comparison to the CPU baseline (Microsoft SEAL) is not equitable. The reported speedups are largely due to the massive difference in raw compute power and memory bandwidth between a high-end A100 GPU and a CPU. A rigorous evaluation would require comparing Neo not just to a CPU library, but to a state-of-the-art, highly-optimized GPU implementation that uses standard CUDA cores for the same FHE operations. Without this direct GPU-to-GPU comparison, it is impossible to isolate the true benefit of using Tensor Cores from the general benefit of using a GPU.
• Overhead of Data Reshaping is Ignored: The proposed "limb-interleaving" strategy (Section 4.1, Page 5) requires significant data reshaping and pre-processing to transform the polynomial coefficients into the specific matrix layout required by the Tensor Cores. This shuffling and re-ordering of data in GPU memory is not a "free" operation; it consumes memory bandwidth and execution cycles. The paper's performance model appears to completely ignore this overhead, which could be substantial and could significantly erode the claimed performance benefits. The analysis focuses only on the matrix multiplication itself, which is a classic flaw in accelerator research.
• Insufficient Analysis of Precision and Noise: The paper states that it uses a 64-bit representation (FP64) for its intermediate calculations to "guarantee the correctness" (Section 4.2, Page 6). This is a hand-wavy and insufficient analysis. FHE computations are notoriously sensitive to noise growth, and simply using a standard floating-point format does not automatically guarantee correctness. The paper lacks a rigorous error analysis that tracks the noise propagation through the proposed decomposition and mapping process. It is not proven that the results are cryptographically sound, only that they are numerically approximate.
• Scalability Claims are Unsubstantiated: The paper proposes a "fragment-based" scheduling method to handle large polynomials (Section 5, Page 7) but provides insufficient evidence of its efficiency. The evaluation is limited to a specific set of FHE parameters (Table 4, Page 10). It is unclear how the performance scales as the polynomial degree N and the ciphertext modulus Q grow to the very large values required for deep, complex FHE applications. The overhead of managing and scheduling a large number of fragments could easily overwhelm the benefits of using the Tensor Cores.

---

Questions to Address In Rebuttal

1. To provide a sound comparison, please evaluate Neo against a state-of-the-art GPU implementation of the same FHE scheme that uses standard CUDA cores and is optimized for the same A100 hardware. This is the only way to prove the specific benefit of using Tensor Cores.
2. Please provide a detailed performance breakdown that includes the overhead of the "limb-interleaving" data-reshaping step. What is the latency and memory bandwidth consumption of this pre-processing step, and how does it impact the end-to-end performance of a complete FHE operation?
3. Provide a rigorous cryptographic noise analysis. Show how the noise in the ciphertext propagates through your proposed decomposition, matrix multiplication, and reconstruction process, and prove that the final result remains below the failure threshold for all evaluated parameter sets.
4. To substantiate your scalability claims, please provide a detailed performance model and evaluation for much larger FHE parameter sets, demonstrating how the fragment scheduling overhead and memory pressure scale as the polynomial degree and modulus size increase significantly.

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper introduces Neo, a novel framework for accelerating Fully Homomorphic Encryption (FHE) by leveraging the specialized Tensor Core units found in modern NVIDIA GPUs. The central idea is to bridge the gap between the world of large-integer polynomial arithmetic, which underpins FHE, and the world of high-performance, low-precision matrix multiplication, which is the native domain of Tensor Cores. Neo achieves this by presenting a clever algorithmic mapping: it decomposes the large polynomial multiplications into a series of smaller matrix-matrix multiplications that can be executed directly on the Tensor Core hardware. To manage the data flow and precision requirements, the authors propose a "limb-interleaving" data layout and a "fragment-based" scheduling strategy. This work effectively opens up a new pathway for FHE acceleration by re-purposing existing, powerful, and widely available hardware that was originally designed for AI workloads.

---

Strengths

This paper's primary strength is its elegant and pragmatic approach to a difficult problem, demonstrating a keen understanding of both the FHE domain and the underlying hardware landscape.

• A Brilliant Repurposing of Existing Hardware: The most significant contribution of this work is its clever recognition that the Tensor Cores, while designed for AI, are fundamentally high-performance matrix engines that can be repurposed for other domains. Instead of proposing a bespoke ASIC, which is costly and has a long design cycle, this paper provides a software-based solution that unlocks the immense computational power of existing, off-the-shelf hardware for FHE (Section 1, Page 2). This is a powerful and practical approach that has the potential for immediate, widespread impact. 💡
• Connecting Disparate Computational Domains: This work serves as a beautiful bridge between two traditionally separate fields: the high-level, abstract mathematics of lattice-based cryptography and the low-level, nitty-gritty details of GPU microarchitecture. By demonstrating how the Number Theoretic Transform (NTT), a cornerstone of FHE, can be algorithmically reframed as a series of matrix multiplications (Section 3, Pages 3-4), the paper provides a crucial Rosetta Stone that allows the FHE community to tap into the billions of dollars of R&D that have been invested in AI hardware.
• Enabling a New Class of GPU-Accelerated Cryptography: Prior work on GPU acceleration for FHE has largely focused on using the general-purpose CUDA cores. While effective, this approach often fails to utilize the GPU to its full potential, as the Tensor Cores sit idle. By specifically targeting the most powerful compute units on the chip, Neo paves the way for a new generation of GPU-accelerated cryptographic libraries that are far more efficient and performant. It opens up a new and fertile area of research for the high-performance computing and applied cryptography communities.

---

Weaknesses

While the core idea is powerful, the paper could be strengthened by broadening its focus from a proof-of-concept to a more robust, production-ready system.

• The Data Reshuffling Elephant in the Room: The proposed "limb-interleaving" strategy is a clever way to format the data for the Tensor Cores. However, this data shuffling itself has a performance cost. The paper focuses on the speedup of the core computation but spends less time analyzing the overhead of the data preparation and pre-processing steps. In many GPU workloads, data movement and marshalling can become a significant bottleneck, and a more detailed analysis of this overhead would provide a more complete performance picture.
• The Software Abstraction Layer: Neo provides the low-level "how-to" for mapping FHE onto Tensor Cores. The next critical step, which is not fully explored, is the software abstraction layer. For this technique to be widely adopted, it needs to be integrated into a high-level FHE library (like Microsoft SEAL) and exposed through a clean API. A discussion of the challenges in building this compiler and runtime layer—which would need to automatically handle the decomposition, scheduling, and data layout—would be a valuable addition.
• Beyond a Single GPU: The paper successfully demonstrates the potential of Neo on a single GPU. The natural next step is to consider how this approach scales to a multi-GPU or multi-node environment. As FHE applications grow in complexity, they will inevitably require the resources of a full server or cluster. A discussion of how Neo would interact with high-speed interconnects like NVLink and how the fragment-based scheduling could be extended to a distributed setting would be very interesting.

---

Questions to Address In Rebuttal

1. Your work brilliantly repurposes hardware designed for AI. Looking forward, do you envision future GPUs having features specifically designed to make this mapping even more efficient? For example, could a future generation of Tensor Cores have native support for modular arithmetic or more flexible data layout options?
2. The "limb-interleaving" strategy is key to your approach. How does the overhead of this data pre-processing scale as the FHE parameters (and thus the polynomial sizes) grow? Is there a point where the cost of data shuffling begins to diminish the benefits of using the Tensor Cores?
3. For this technique to have broad impact, it needs to be integrated into a user-friendly library. What do you see as the biggest challenges in building a compiler or runtime system that could automatically and optimally apply the Neo framework to an arbitrary FHE program? 🤔
4. How does the performance of Neo change with different generations of Tensor Cores (e.g., from Volta to Ampere to Hopper)? Does the growing power and complexity of the Tensor Cores open up new opportunities or create new bottlenecks for your approach?

Reviewer: The Innovator (Novelty Specialist)

Summary

This paper presents Neo, a framework for accelerating Fully Homomorphic Encryption (FHE) on modern GPUs. [cite_start]The core novelty claim is a new methodology for mapping the large-integer polynomial arithmetic central to FHE onto the fixed-precision, matrix-oriented Tensor Core units[cite: 1, 33]. [cite_start]This is achieved through two primary novel techniques: 1) A "limb-interleaving" data layout strategy that transforms polynomial coefficients into a matrix format suitable for the Tensor Cores [cite: 203, 206][cite_start], and 2) a "fragment-based" scheduling approach that decomposes large polynomial operations into a sequence of smaller matrix multiplications that fit the hardware's native dimensions[cite: 284]. [cite_start]The work claims this is the first implementation to accelerate the critical BConv and IP FHE kernels on Tensor Cores, offering a new pathway for FHE acceleration[cite: 146, 575].

---

Strengths

From a novelty standpoint, this paper's strength lies in its clever and non-obvious algorithmic mapping, which bridges two seemingly incompatible computational domains.

• [cite_start]Novel Algorithmic Transformation: The most significant "delta" of this work is the algorithmic transformation that reframes large-integer polynomial multiplication as a series of small, fixed-precision matrix multiplications[cite: 146, 575]. While using GPUs for FHE is not new, prior work has focused on using the general-purpose CUDA cores. This paper is the first to devise a concrete method for mapping these operations onto the highly specialized and architecturally rigid Tensor Cores. This conceptual leap—seeing a path from polynomial math to matrix math—is a significant and novel contribution. It unlocks a powerful, previously untapped computational resource for the FHE domain. 🧠
• [cite_start]Novel Data Layout and Scheduling: The "limb-interleaving" [cite: 203, 206] [cite_start]and "fragment-based" [cite: 284] techniques are direct and novel solutions to the two primary challenges of this mapping: precision and dimensionality. Standard FHE requires high-precision arithmetic, while Tensor Cores are low-precision. Standard FHE polynomials are very large, while Tensor Cores operate on small, fixed-size matrices. The proposed data layout and scheduling schemes are the novel "glue" that makes this mapping possible. They represent a new, domain-specific approach to data management for GPU-based computation.

---

Weaknesses

While the core mapping is novel, the work's claims of novelty do not extend to the underlying hardware or the performance gains themselves, which are a predictable consequence of the mapping.

• [cite_start]Performance Gain is a Consequence, Not a Novelty: The paper reports significant speedups over CPU and other GPU baselines[cite: 542]. However, the novelty of the work is the enabling of the Tensor Cores, not the speedup itself. It is not a novel discovery that a highly specialized, massively parallel matrix engine (the Tensor Core) is faster than a general-purpose CPU or even general-purpose CUDA cores for matrix-heavy workloads. The performance gain is an expected and logical consequence of the novel mapping, not a separate innovation.
• Relies Entirely on Existing Hardware: The novelty of this work is purely in the software and algorithmic domain. It proposes no new hardware and does not suggest any modifications to the existing GPU architecture. Its contribution is in finding a new and creative way to use hardware that already exists, which is valuable but limits the scope of the novelty to the mapping technique itself.
• Algorithmic Ancestry: While the specific application to FHE is new, the general concept of using matrix multiplication engines to perform other mathematical operations (like convolutions) has a long history in the field of high-performance computing. The novelty here is the specific, non-trivial adaptation of these principles to the unique mathematical structures of lattice-based cryptography, particularly the Number Theoretic Transform (NTT).

---

Questions to Address In Rebuttal

1. The core of your novelty is the algorithmic mapping of polynomials to matrices. Can you discuss any prior art in other domains (e.g., signal processing, scientific computing) that has used a similar "transform-and-dispatch" approach to map non-matrix problems onto specialized matrix hardware?
2. [cite_start]The "limb-interleaving" technique is presented as a novel data layout[cite: 203, 206]. How does this differ fundamentally from standard data-marshaling techniques used in high-performance libraries (e.g., cuFFT, cuBLAS) to prepare data for optimal hardware access?
3. [cite_start]Could the proposed fragment-based scheduling [cite: 284] be considered a domain-specific instance of a more general tiling or loop-nest optimization strategy? What is the core, novel insight in your scheduler that is unique to the FHE domain?
4. If NVIDIA were to introduce a future Tensor Core with native support for modular arithmetic or larger integer types, how much of the novelty of the Neo framework would remain? Does the contribution lie primarily in overcoming the limitations of current hardware, or is there a more fundamental, hardware-independent algorithmic novelty?

Reviewer: The Guardian

Summary

This paper identifies the GPU's page translation bandwidth as a key performance bottleneck. To address this, the authors propose Heliostat, a system that offloads page table walks to the dedicated Ray Tracing Accelerators (RTAs) now common on modern GPUs. The core thesis is that the tree traversal performed during a page table walk is architecturally similar to the Bounding Volume Hierarchy (BVH) traversal performed by an RTA during ray tracing. By re-purposing this underutilized hardware, the authors claim to fundamentally increase translation bandwidth. A further optimization, Heliostat+, introduces a prefetching mechanism to proactively resolve future address translations. The authors claim this approach yields significant speedups (up to 1.93x) over baseline GPU memory management units.

---

Strengths

The paper is founded on a single, clever observation that serves as its primary motivation.

• Identification of an Underutilized Resource: The core strength of the paper is its correct identification of a powerful and often-idle hardware unit—the RTA—and the subsequent attempt to find a general-purpose use for it. Recognizing that specialized accelerators may be repurposed is a valid direction for architectural research.

---

Weaknesses

Despite the clever initial idea, the paper's central thesis is built upon a fundamentally flawed analogy, questionable evaluation methodologies, and a significant underestimation of critical system overheads.

• Fundamentally Flawed Architectural Analogy: The entire premise of Heliostat rests on the claimed "operational similarities" between page table walks and ray tracing (Abstract, Page 1). This analogy is superficial at best and deeply flawed in practice. A page table walk is a simple, deterministic pointer-chasing operation through a radix tree. In contrast, ray tracing BVH traversal is a complex geometric operation involving ray-box intersection tests, complex state management, and traversal decisions based on spatial properties. Using a highly specialized, complex engine designed for geometric tests to perform simple pointer lookups is a gross architectural mismatch. It is the equivalent of using a sledgehammer to crack a nut; while it might work, it is fundamentally inefficient and wasteful.
• Unsubstantiated Performance Claims due to Inequitable Baseline: The headline performance claims (e.g., 1.93x speedup) are invalid because the comparison is not equitable. The paper compares its proposed system (GMMU + repurposed RTA) against a baseline with only the GMMU. An RTA is a large, power-hungry piece of silicon. A rigorous and fair comparison would evaluate Heliostat against a baseline where the GMMU is given an equivalent area and power budget to the RTA being repurposed. It is highly probable that a larger, more parallel, conventionally designed Page Table Walker unit would outperform the complex and inefficient Heliostat mechanism, rendering the claimed speedups an artifact of an unfair resource comparison.
• Critical Overheads are Ignored: The paper fails to properly account for the significant overhead required to make this mechanism function. A page table walk request (a virtual address) must be "encoded" into a format that the RTA can understand—a "ray" with an origin and direction (Section 4.1, Page 4). Conversely, the RTA's output (hit information) must be decoded back into a physical address. This "translation tax" of encoding and decoding is not a free operation; it consumes cycles and energy. The performance analysis appears to completely ignore or minimize this overhead, which would likely negate a significant portion of the claimed benefits in a real implementation.
• Unrealistic Prefetching Assumptions: The Heliostat+ extension relies on a "highly accurate" address prefetcher (Section 6, Page 7). However, the paper provides no evidence to support this claim of high accuracy. General-purpose, highly accurate address prediction is a notoriously difficult, and largely unsolved, problem in computer architecture. The evaluation appears to use benchmarks with regular, strided memory access patterns where prefetching is known to work well. The claim that this approach is beneficial for "any workloads" (Abstract, Page 1) is unsubstantiated, as it would likely perform poorly on irregular, pointer-chasing workloads where prefetching is ineffective.

---

Questions to Address In Rebuttal

1. Please provide a detailed breakdown of the cycle and energy overhead for the encoding/decoding process that translates a virtual address into a ray and a hit result back into a physical address. How does this "translation tax" impact the end-to-end latency of a single page walk?
2. To provide a fair comparison, please evaluate Heliostat against an improved baseline that features a conventional GMMU/PTW unit designed with the same silicon area and power budget as the RTA you are repurposing.
3. The Heliostat+ mechanism's performance is contingent on the accuracy of the address prefetcher. Please provide a sensitivity analysis showing how the performance of Heliostat+ degrades as prefetcher accuracy decreases, and justify your claim that this approach is effective for irregular, hard-to-predict workloads.
4. Can you justify the fundamental architectural choice of using a complex geometric intersection engine for a simple pointer-chasing task? What is the raw, cycle-for-cycle efficiency (in terms of lookups-per-second-per-mm²) of your proposed mechanism compared to a standard, dedicated Page Table Walker hardware implementation?

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper introduces Heliostat, a novel system for accelerating GPU page table walks by repurposing the dedicated Ray Tracing Accelerators (RTAs) that are now standard on modern GPUs. The core insight is that the tree traversal inherent in a page table walk is operationally similar to the Bounding Volume Hierarchy (BVH) traversal performed by RTAs during ray tracing. By creating a lightweight hardware and software layer to map page table walk requests onto the RTA's traversal engine, Heliostat aims to significantly boost the GPU's overall address translation bandwidth. A further proposed enhancement, Heliostat+, leverages the RTA's secondary ray generation feature to enable proactive, low-cost prefetching of future address translations. This work opens up a new avenue for system performance improvement by finding a general-purpose use for a highly specialized, and often underutilized, piece of hardware.

---

Strengths

This paper's primary strength is its creativity and its "out-of-the-box" thinking, which connects two seemingly unrelated parts of the GPU architecture to create a new and unexpected synergy.

• A Brilliant Repurposing of Specialized Hardware: The most significant contribution of this work is its clever and non-obvious idea to use a ray tracing engine for memory management (Abstract, Page 1). Modern GPUs are increasingly becoming collections of specialized accelerators (e.g., Tensor Cores, RTAs). A major challenge in computer architecture is preventing these specialized units from becoming "dark silicon"—powered off and unused when their specific task is not running. Heliostat provides a compelling answer to this problem by finding a "secondary purpose" for the RTA, effectively democratizing a specialized unit for a general-purpose system task. This is a powerful and important direction for heterogeneous computing. 💡
• Connecting Disparate Architectural Concepts: This work serves as an intellectual bridge between the worlds of computer graphics and computer architecture. It recognizes a deep structural similarity between two different problems: finding a ray's intersection in a 3D scene and finding a virtual address's mapping in a page table. By showing how the BVH traversal performed by an RTA can be conceptually mapped to a radix tree walk (Section 2.2, Page 2), the paper demonstrates a kind of architectural isomorphism that is both insightful and inspiring.
• A New Pathway for Performance Scaling: For years, the primary approach to improving GPU memory management has been to build bigger TLBs or more parallel Page Table Walkers (PTWs) (Section 1, Page 1). Heliostat offers a completely new and orthogonal path to performance scaling. Instead of building more dedicated hardware, it leverages existing hardware more intelligently. This is a much more area- and power-efficient approach, as demonstrated by the paper's analysis (Section 8.5, Page 13), and it opens up a new dimension in the design space for future memory management units.

---

Weaknesses

While the core idea is brilliant, the paper could be strengthened by broadening its focus to the wider system implications and the long-term evolution of this concept.

• The "Generalization" Challenge: The paper successfully demonstrates the mapping for page table walks. The natural next question is: what other general-purpose tree or graph traversal problems could be offloaded to the RTA? A discussion of how the Heliostat framework could be generalized to accelerate other important workloads, such as database index lookups, file system traversals, or even certain types of AI model inference, would elevate the work from a clever trick to a truly general-purpose platform.
• The Software Ecosystem: The paper focuses on the hardware implementation. However, for Heliostat to be truly useful, it would need to be seamlessly integrated into the GPU driver and the broader OS memory management system. A discussion of the required software changes—for example, in the CUDA/ROCm runtime, the OS kernel's memory manager, and the compiler—would provide a more complete picture of the path to real-world deployment.
• The Co-evolution of Hardware: Heliostat is a clever solution for today's GPUs. But what about tomorrow's? If this idea were to be adopted, it might influence the design of future RTAs. Future RTAs might be designed with more general-purpose traversal features from the ground up, making them even more powerful for non-graphics tasks. A discussion of how this work could influence the future evolution of GPU architecture would be a fascinating addition.

---

Questions to Address In Rebuttal

1. Your work brilliantly repurposes the RTA for a system-level task. Looking forward, what other common operating system or database algorithms (e.g., B-tree searches, file system lookups) do you think could be accelerated using the Heliostat framework?
2. The Heliostat+ prefetcher is based on simple stride detection (Section 6.4, Page 8). How could this be improved by leveraging more advanced prefetching techniques from the CPU world, and how would the RTA architecture need to evolve to support them?
3. For Heliostat to be practical, the OS and GPU driver need to be aware of it. What are the key modifications required in the software stack to manage the RTA as a general-purpose translation resource, and how would you handle security and isolation between different processes using it? 🤔
4. If you were designing the next generation of GPUs from scratch, knowing about the potential of RTA offloading, would you still design a separate GMMU and RTA? Or would you merge them into a single, more powerful, and general-purpose "Traversal Engine" capable of handling both graphics and memory management tasks?

Reviewer: The Innovator (Novelty Specialist)

Summary

This paper presents Heliostat, a system for accelerating GPU page table walks. The core novelty claim is the repurposing of dedicated Ray Tracing Accelerators (RTAs), now common on GPUs, to perform this critical memory management task. The central thesis is that the tree traversal operation at the heart of a page table walk is architecturally analogous to the Bounding Volume Hierarchy (BVH) traversal performed by an RTA. The authors propose a lightweight hardware and software layer to "encode" page table walk requests as ray tracing queries and offload them to the RTA. A second claimed novelty is Heliostat+, an extension that uses the RTA's secondary ray capabilities to implement a low-cost, proactive address prefetcher.

---

Strengths

The novelty of this work lies in its creative and non-obvious connection between two fundamentally different domains of computer architecture, leading to a new and unexpected use for specialized hardware.

• A Novel Architectural Analogy: The most significant "delta" in this paper is the conceptual leap required to see the operational similarity between a memory management task (page table walking) and a graphics task (ray tracing) (Section 2.2, Page 2). While both involve tree traversals, they operate in entirely different semantic domains. Recognizing that the underlying hardware mechanism for one could be adapted for the other is a genuine and significant innovative insight. It challenges the conventional wisdom of designing dedicated hardware for every task and instead proposes a more resourceful approach. 💡
• A New Mechanism for General-Purpose Acceleration: Prior work on hardware acceleration has almost exclusively focused on either building new bespoke ASICs or using programmable compute units (like CUDA cores). Heliostat proposes a third, novel path: repurposing a fixed-function accelerator for a task outside its original domain. This is a fundamentally new approach to heterogeneous computing. The hardware and software mechanisms designed to "trick" the RTA into performing page walks (Section 4, Page 4) represent a new class of architectural adaptation that has not been explored in prior literature.
• Novel Application of Prefetching: The Heliostat+ extension is a clever and novel application of a graphics-specific feature for a general-purpose performance enhancement. Using the RTA's ability to spawn secondary rays to implement a low-cost, parallel prefetcher (Section 6, Page 7) is a non-obvious and elegant idea. It leverages a hardware capability that would otherwise be idle during non-graphics workloads to solve a classic and difficult problem in memory management.

---

Weaknesses

While the core idea is highly novel, its novelty is also its primary weakness. The work proposes a new use for existing hardware, but does not propose new fundamental hardware primitives.

• Novelty is Purely in Abstraction and Mapping: The work does not propose any changes to the RTA or the core GPU architecture. Its novelty is entirely in the software and lightweight hardware layers that translate the page table walk problem into a ray tracing problem. This is a significant contribution, but it is a "trick" or a new mapping, not a new piece of fundamental hardware.
• The Analogy is Imperfect: The core analogy, while clever, is not perfect. Ray tracing BVH traversal involves complex geometric calculations (ray-box intersection tests) that are entirely superfluous for a page table walk. The novelty lies in the ability to make this imperfect analogy work, but it also means the mechanism is not as efficient as a purpose-built Page Table Walker would be on a transistor-for-transistor basis. The novelty is in the "hack," not in the raw efficiency of the final implementation.
• Performance is a Consequence, Not an Invention: The reported speedups are a direct and expected consequence of successfully offloading work to a powerful, parallel hardware unit that was previously idle. The novelty is in enabling this offload, not in the discovery that parallelizing a task makes it faster. The performance gains validate the novelty of the approach but are not, in themselves, a separate novel contribution.

---

Questions to Address In Rebuttal

1. Your work proposes a novel mapping of one problem (page walks) onto the hardware for another (ray tracing). Can you discuss any prior art in the broader history of computing where a fixed-function accelerator has been successfully and non-obviously repurposed for a task completely outside its original design domain?
2. The Heliostat+ prefetcher is a clever use of secondary rays. Is there any prior work in the graphics domain that has used this secondary ray feature for non-obvious, non-graphics-related prefetching or speculative computation?
3. The core of your novelty is the architectural analogy. If a future GPU were to include a more general-purpose "Tree Traversal Engine" instead of a specialized RTA, how much of the novelty of the Heliostat framework would remain? Does the contribution lie primarily in overcoming the limitations of today's specialized hardware?
4. The "encoding" of virtual addresses into rays is a key enabling technique (Section 4.1, Page 4). Can you elaborate on the novelty of this encoding scheme itself? Are there precedents for representing non-geometric data in a geometric format to leverage graphics hardware for general-purpose computation?

Paper Title: Forest: Access-aware GPU UVM Management Reviewer Persona: The Guardian

---

Summary

The authors identify the performance limitations of the conventional Tree-Based Neighboring Prefetcher (TBNp) used in GPU Unified Virtual Memory (UVM) systems. They argue that TBNp's homogeneous, "one-size-fits-all" configuration is oblivious to the diverse memory access patterns of individual data objects, leading to unnecessary migrations and page thrashing. To address this, they propose Forest, a software-hardware co-design that introduces an on-device Access Time Tracker (ATT) to monitor page access sequences per data object. This information is then used by a driver-level Access Pattern Detector (APD) to classify patterns and configure a heterogeneous, object-specific prefetch tree. The paper claims significant performance improvements over the baseline TBNp and other state-of-the-art solutions, evaluated via simulation.

Strengths

1. Problem Motivation: The paper correctly identifies a valid and important limitation in existing UVM management systems. The core premise—that a single prefetcher configuration is suboptimal for diverse workloads and data structures—is sound. The analysis in Section 3, particularly Figure 4, provides clear evidence that different tree configurations benefit different applications, effectively motivating the need for a more adaptive approach.

2. Core Observation: The observation that UVM management should be performed at the granularity of individual data objects rather than fixed-size memory blocks or entire applications is insightful. This allows for a more tailored and potentially more efficient prefetching strategy.

Weaknesses

My primary concerns with this work center on the apparent arbitrariness of the core mechanism, the nontrivial and insufficiently justified hardware modifications, and the questionable fidelity of the simulation-based evaluation, which casts doubt on the extraordinary performance claims.

1. Arbitrary and Brittle Pattern Classification: The entire mechanism hinges on classifying data objects into one of four patterns (LS, HCHI, HCLI, LC), as defined in Section 4.3.2 (page 6). The thresholds used for this classification (e.g., R² > 0.8 for LS, access coverage P=0.6, access intensity A=0.4) are presented as fixed constants without any theoretical or empirical justification. This gives the impression that these are "magic numbers" tuned specifically for the evaluated benchmarks. The paper's own sensitivity analysis in Figure 19 (page 13) confirms this brittleness: performance degrades sharply if these exact thresholds are not used. This suggests the system is not robust and may perform poorly on workloads that do not fit neatly into these rigid, pre-defined boxes. The classification scheme itself feels overly simplistic for the complexity of real-world GPU access patterns.

2. Understated Hardware Cost and System Impact: The proposed Access Time Tracker (ATT) is presented as a minor modification that "repurposes the existing hardware page access counters" (Section 4.2, page 5). This is a misleading characterization. The paper states that existing counters reflect access frequency, while Forest requires them to store access recency (i.e., an ordered timer value). This is not a "repurposing"; it is a fundamental change to a core hardware monitoring feature. The authors fail to discuss the system-wide implications of this change. Do other system services, such as OS-level memory management, thermal throttling, or performance counters, rely on frequency data? If so, this design would break them. Furthermore, the hardware overhead in Section 7.8 is minimized as a "147-byte per-kernel" table, but the paper itself notes support for up to 128 concurrent kernels (page 6), which implies a total hardware cost of over 18KB. This is not a negligible hardware addition and its area and power costs are not analyzed.

3. Questionable Evaluation Fidelity and Baselines: The evaluation is conducted exclusively in GPGPU-Sim. While simulation is a standard tool, the reported speedups of 1.86x over the baseline TBNp are exceptionally high for a prefetching optimization. This raises serious questions about the fidelity of the baseline TBNp implementation. Production UVM drivers from vendors like NVIDIA are highly complex and aggressively optimized. It is highly probable that the simulated baseline is a simplified, less-performant version, which would artificially inflate the benefits of Forest. The paper provides no validation of its baseline against real hardware behavior, making it impossible to trust the magnitude of the claimed improvements. Without such validation, the results remain speculative.

4. Unrealistic Compiler and API Modifications: The "SpecForest" extension (Section 5) relies on static compiler analysis and, critically, proposes modifying the cudaMallocManaged API to pass hints from the compiler to the driver. Changing a fundamental, widely-used API in the CUDA ecosystem is a massive undertaking with significant backward-compatibility and software engineering implications. The paper glosses over this entirely. Moreover, the proposed static analysis for "similarity detection" (Section 5.3) appears fragile and would likely fail on any code with moderately complex pointer arithmetic or dynamically computed indices, limiting its real-world applicability.

Questions to Address In Rebuttal

1. On Pattern Classification: Please provide a rigorous justification for the specific classification thresholds chosen (R² > 0.8, P=0.6, A=0.4). How were these values derived? Given the performance sensitivity shown in Figure 19, please defend the claim that this mechanism is robust enough for general-purpose use and not simply overfitted to your benchmark suite. What is the performance impact of a misclassification?

2. On Hardware Modification: Please address the system-level impact of changing the fundamental behavior of hardware page access counters from tracking frequency to recency. Acknowledge and discuss which, if any, existing system functionalities would be compromised by this change. Provide a more realistic hardware cost analysis (area, power) for the full 18KB multi-kernel object table.

3. On Evaluation Fidelity: Please provide evidence to substantiate that your simulated TBNp baseline is a faithful representation of a production-quality UVM prefetcher. Can you present any data, even for a simple microbenchmark, that validates your simulator's UVM faulting behavior and performance against a real GPU? Without this, why should the reviewers trust the reported 1.86x speedup?

4. On API Changes: Please discuss the software engineering challenges and ecosystem-wide impact of modifying a core API like cudaMallocManaged. Is this a practical suggestion, and what would be the path to adoption?

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper identifies a fundamental, yet previously unaddressed, inefficiency in the prevalent Tree-based Neighboring Prefetcher (TBNp) used in modern GPUs for Unified Virtual Memory (UVM) management. The authors compellingly argue that TBNp's "one-size-fits-all" homogeneous tree structure is oblivious to the diverse memory access patterns of different data objects, leading to suboptimal performance, unnecessary migrations, and page thrashing.

The core contribution is Forest, a novel access-aware UVM management system that dynamically configures a bespoke, heterogeneous prefetch tree for each individual data object at runtime. This is achieved through an elegant software-hardware co-design. A lightweight hardware unit, the Access Time Tracker (ATT), repurposes existing page access counters to record access recency rather than just frequency. This data is then consumed by a driver-level Access Pattern Detector (APD) that classifies object access patterns into one of four archetypes and configures the optimal prefetch tree structure accordingly. The paper further proposes Speculative Forest (SpecForest), which uses compile-time analysis and pattern recording to reduce or eliminate the runtime profiling overhead. The experimental results demonstrate significant speedups over both the baseline TBNp and other state-of-the-art solutions.

Strengths

1. Tackles a Foundational Problem: The most significant strength of this work is its insight. Rather than incrementally improving UVM performance by tuning migration thresholds or eviction policies (as much prior work has done), this paper questions the foundational assumption of a homogeneous prefetch architecture. By shifting the granularity of policy from the application-level to the object-level, Forest addresses what appears to be the root cause of many inefficiencies described in the motivation (Section 3, Pages 3-4). This is a conceptual leap that reframes the problem in a productive way.

2. Elegant and Practical Co-Design: The proposed hardware modification (the ATT) is commendably lightweight and practical. By repurposing existing page access counter infrastructure, the authors present a solution that seems plausible for integration into future hardware without a major overhaul. This pragmatism is a key feature that distinguishes it from more heavyweight academic proposals.

3. A Holistic and Layered Solution: The paper presents a complete system. Forest provides the dynamic runtime mechanism, while SpecForest provides a static/memoized optimization path to reduce overhead. The inclusion of SpecForest, with its use of pattern recording and static analysis for similarity detection (Section 5, Pages 9-10), shows a deep understanding of the practicalities of system performance, acknowledging that runtime profiling is not always the best solution.

4. Excellent Contextualization and Forward-Looking Vision: The discussion in Section 6 (Page 10) on the applicability of Forest to emerging heterogeneous architectures like Grace-Hopper is particularly insightful. The authors correctly identify that even with high-speed interconnects that enable remote access, intelligent data migration remains crucial due to bandwidth disparities. This demonstrates a panoramic view of the field and positions the work's core principles as durable and relevant for next-generation systems, not just current ones.

5. Strong and Convincing Motivation: The motivation presented in Section 3 is excellent. The data shown in Figure 4, demonstrating that no single tree configuration is optimal for all applications, and Figure 5, showing diverse patterns within a single application, provides a powerful and clear justification for the entire approach. This sets the stage perfectly for the proposed solution.

Weaknesses

While the core idea is strong, the work could be further contextualized and its boundaries explored. These are not flaws so much as opportunities for refinement.

1. Simplicity of the Pattern Taxonomy: The proposed four-pattern taxonomy (LS, HCHI, HCLI, LC) is a powerful simplification that enables the system's design. However, it's worth considering its limitations. Real-world access patterns can be phased, complex, or a hybrid of these archetypes. The paper implicitly handles this by defaulting to 'LC' for unrecognized patterns, but the performance implications of misclassification or an "unclassifiable" pattern could be explored more deeply. The taxonomy feels like a very effective first-order approximation, but the landscape of patterns is likely richer.

2. Interaction with Software-Managed Prefetching: The work is situated firmly in the context of hardware/driver-managed UVM. However, there is a parallel body of work on application-level or library-level prefetching (cudaMemPrefetchAsync). It would be interesting to understand how Forest might interact with such explicit prefetching directives. Could they conflict, or could the information from Forest's pattern detector be exposed to the programmer to guide better explicit prefetching?

3. Details of Static Analysis: The description of the static analysis for SpecForest is high-level. While the idea of detecting index similarity is intuitive and powerful (as shown in Listing 2, Page 9), the robustness of this analysis in the face of complex C++ templates, function pointers, and heavy pointer arithmetic—all common in sophisticated GPU codes—is an open question. A brief discussion of the limitations of this static analysis would strengthen the paper.

Questions to Address In Rebuttal

1. Regarding the four-pattern taxonomy: Can the authors comment on the prevalence of "unclassifiable" patterns that would default to the LC configuration? How sensitive is the system's performance if an object with a borderline pattern (e.g., between HCHI and HCLI) is misclassified?

2. The paper focuses on optimizing the prefetch tree structure. Could the access patterns identified by the APD also be used to inform a more intelligent eviction policy beyond the proposed object-level pseudo-LRU? For instance, data with a streaming (LS) pattern is inherently "dead" after access, suggesting it could be a priority candidate for eviction, irrespective of recency.

3. How does the system handle data objects that exhibit phased behavior, where the access pattern changes dramatically during a single kernel's execution? Does the profiling window (10K accesses) and cease bit mechanism risk locking in a suboptimal tree configuration based on the object's initial access pattern?

Reviewer: The Innovator (Novelty Specialist)

---

Summary

This paper introduces "Forest," a novel access-aware management system for GPU Unified Virtual Memory (UVM). The core thesis is that the conventional one-size-fits-all, homogeneous Tree-Based Neighboring Prefetcher (TBNp) is inefficient for workloads with diverse memory access patterns. Forest's primary claimed novelty is the introduction of heterogeneous, per-data-object prefetcher configurations. This is achieved through a software-hardware co-design where a hardware "Access Time Tracker" (ATT) monitors page access sequences to infer patterns, which a software "Access Pattern Detector" (APD) in the UVM driver uses to dynamically reconfigure the TBNp tree structure (size and leaf granularity) for each data object. A secondary contribution, "Speculative Forest," aims to reduce this runtime profiling overhead via pattern recording and static compile-time analysis.

---

Strengths

The primary strength of this paper lies in its central, novel idea. My analysis of the prior art confirms that the core contribution is a genuine advancement in the field of UVM management.

1. Fundamental Shift in Prefetcher Management: The most significant novel contribution is the move from a static, homogeneous TBNp configuration to a dynamic, heterogeneous one. Previous state-of-the-art academic work [27, 29] has focused on adaptively tuning the parameters of the existing TBNp structure (e.g., migration thresholds) or refining eviction policies [26]. This paper is the first I am aware of to propose fundamentally altering the structure of the prefetcher tree itself (e.g., its total size and the granularity of its leaf nodes) on a per-data-object basis at runtime. This is a conceptual leap from tuning a prefetcher to reconfiguring it.

2. Novel Hardware Mechanism for Pattern Detection: The repurposing of existing hardware page access counters is a clever and elegant mechanism. Instead of tracking access frequency (hotness), the proposed ATT (Section 4.2, page 5) uses them to record access recency/order within an object. This provides the fine-grained sequence information necessary for pattern detection without requiring costly new hardware monitors. This specific application of access counters for temporal sequence tracking appears to be novel.

3. Lightweight Tree Reconfiguration Primitive: The introduction of two 1-bit metadata flags (isolation and motion) per non-leaf node (Section 4.4, page 7) is a novel and lightweight hardware primitive for enacting the dynamic tree reconfiguration. It allows the software to effectively create different logical tree structures from a single physical one, which is an efficient implementation of the core idea.

4. Novel Heuristics in Speculative Forest: While using static analysis to detect linear access patterns (Section 5.2, page 9) is a well-established technique, the proposed "access pattern similarity detection" (Section 5.3, page 10) is a novel and practical heuristic. Grouping data objects based on their use of the same indexing variables at compile-time to propagate a discovered pattern is a new idea in this context and cleverly reduces runtime overhead for complex access patterns.

---

Weaknesses

While the core idea is strong, the novelty of some constituent parts is less pronounced, and the work's novelty could be more rigorously defended against adjacent concepts.

1. Established Principles in Pattern Classification: The paper proposes a four-type access pattern taxonomy (LS, HCHI, HCLI, LC) in Section 4.3.2 (page 6). While this specific classification is tailored to the problem, the general concept of classifying memory accesses into categories like streaming/linear, strided, or irregular/random is a foundational concept in the history of prefetching research. The novelty lies in the application of this classification to reconfigure TBNp, not in the act of classification itself. The paper could be strengthened by acknowledging this and more clearly delineating where the established principle ends and their novel application begins.

2. Limited Scope of Novelty to TBNp Architecture: The solution is tightly coupled to the specifics of NVIDIA's TBNp. The discussion in Section 6 (page 10) briefly mentions how the idea could apply to AMD's range-based SVM, but the proposed hardware mechanisms (ATT, isolation/motion bits) are TBNp-specific. This raises the question of whether the novel contribution is a general principle ("access-aware, configurable prefetching") or a highly specific, albeit effective, point solution for TBNp. The paper's claim to novelty would be stronger if it better abstracted its core mechanisms from the TBNp implementation.

3. Incremental Novelty of Eviction Policy: The proposed pseudo-LRU eviction policy (Section 4.5, page 8) is a logical extension of the repurposed access counters. While an improvement over far-fault-based LRU, it is conceptually similar to other work that advocates for using hardware access information for eviction [27]. The key delta here is the use of recency over frequency and the two-level search (finding the LRU object, then the LRU page within it), which is a clever refinement but perhaps not a standalone novel contribution of the same caliber as the configurable tree.

---

Questions to Address In Rebuttal

1. Prior work, such as Early-Adaptor [29] and AdaptiveThreshold [27], dynamically adjusts UVM prefetching aggressiveness. Please clarify precisely how your proposed structural reconfiguration of the TBNp tree is fundamentally different from their approach of tuning migration thresholds within a fixed tree structure. Why is structural reconfiguration a more powerful and novel primitive?

2. The four proposed access patterns (LS, HCHI, HCLI, LC) are central to your method. Could you elaborate on the methodology used to arrive at this specific taxonomy? Was this set derived empirically from the chosen benchmarks, or is there a more fundamental basis for it? How sensitive is Forest's performance to this exact classification?

3. The discussion on applying Forest's principles to non-tree prefetchers like AMD's range-based SVM is brief. To better establish the generality of your novel idea, could you propose a concrete (if hypothetical) hardware/software mechanism, analogous to the isolation and motion bits, that would enable dynamic reconfiguration of migration "Ranges" based on detected access patterns?

Paper Title: Avant-Garde: Empowering GPUs with Scaled Numeric Formats Reviewer: The Guardian (Adversarial Skeptic)

---

Summary

The paper proposes Avant-Garde, a GPU microarchitecture designed to provide native hardware support for scaled numeric formats like MX and HBFP. The authors identify the software overhead (increased instruction count and register pressure) of managing these formats on conventional GPUs as a key performance bottleneck. The core idea is to "flatten" multi-level scaled formats into a consistent, single-level internal representation. This is achieved through two primary hardware modifications: a new pipeline stage called the "Operand Transformer" to perform the flattening, and a redesigned "Avant-Garde Tensor Core" that can directly operate on these flattened formats. The authors evaluate their proposal using a modified Accel-Sim simulator and claim significant throughput improvements (up to 74%) and execution time reductions (up to 44%) over a conventional GPU baseline, with negligible accuracy degradation.

While the problem statement is valid and the high-level architectural concept is directionally sound, the manuscript in its current form suffers from a fundamentally weak experimental baseline, questionable simulation assumptions, and an oversimplified analysis of hardware overhead and numerical precision. The impressive performance claims are not sufficiently substantiated against a robust point of comparison, and critical details are omitted.

---

Strengths

1. Problem Motivation: The paper does an excellent job of motivating the problem. The analysis in Section 2.2 (page 4), including the PTX instruction trace (Figure 3) and the quantification of register file and instruction overhead (Figure 4), clearly illustrates the inefficiency of supporting scaled numeric formats in software on current architectures. This part of the work is a valuable contribution.

2. Architectural Concept: The core architectural idea of converting various scaled formats into a unified internal representation ("flattening") is an elegant approach. It centralizes the complexity of handling diverse formats into a single, specialized hardware unit, which is a sensible design principle.

---

Weaknesses

1. Baseline Invalidity and Inflated Claims: The paper's primary claims of a 74% throughput improvement and 44% execution time reduction are built upon a comparison to a baseline that appears to be a strawman. The authors state (Section 4, page 9), "In the baseline, we implement a DNN model that handles the scaling factor in software to support the scaled numeric formats." This is a custom, likely unoptimized, software implementation. A dedicated hardware unit will naturally outperform a general-purpose software implementation. The critical scientific question is not if hardware is better, but by how much compared to a highly optimized, industrial-strength software library (e.g., a hypothetical cuDNN with native MX support). Without this comparison, the reported gains are likely vastly inflated and do not represent the true value proposition of the proposed hardware.

2. Superficial Hardware Overhead Analysis: The silicon overhead analysis presented in Section 3.3 (page 8) is unconvincing.

   • Synthesizing on the FreePDK 45nm technology node is insufficient for projecting overhead on a modern, high-performance GPU built on a 5nm-class FinFET process. The relative cost of logic, memory, and routing is drastically different. A 1.4% area overhead on a 45nm SM is not a reliable proxy for the impact on a highly dense and complex modern SM, where routing congestion and timing closure for a new pipeline stage could present significant challenges.
   • The paper claims a latency impact of "two cycles per warp" for the Operand Transformer. This is a fixed number that seems divorced from the complexity of the operation. The flattening process for a format with four scaling levels must surely take more time than one with two. This latency is not properly modeled or justified. The claim that this latency is always hidden by other warps is an optimistic assumption that will not hold in all execution scenarios.

3. Unjustified Simulation Simplifications: The methodology in Section 4 (page 8) contains a critical, unsupported assumption: "As Accel-Sim does not support FP8, we modify the simulator to compute a scaling factor so that FP8 operations execute with the same latency as INT8." This is fundamentally incorrect. FP8 and INT8 are not equivalent. FP8 arithmetic requires exponent handling (alignment, addition) and normalization, which necessitates different and potentially more complex hardware than integer multiplication. Equating their latencies is an oversimplification that biases the evaluation.

4. Undeclared Numerical Impact of Flattening: The authors claim in Section 5.5 (page 11) and Table 4 that flattening a multi-level format like MX9 results in virtually no accuracy loss (<0.2% vs FP32). This claim is not substantiated with a numerical analysis. The "flattening" process involves multiplying element values by their second-level scaling factors. This is not a lossless operation. It can easily lead to underflow (loss of precision for small values) or overflow/saturation (clipping of large values) when the result is stored in a fixed-width mantissa. The authors must provide a detailed analysis of the intermediate numerical formats and error bounds of the flattening operation itself, rather than just asserting that the final application-level accuracy is maintained. The statement "operand transformation introduces no significant loss in precision" is an unsubstantiated claim.

5. Omission of Critical Data: The sensitivity study in Section 5.6 (page 11) is a major point of concern. The authors state, "As the overall performance across configurations shows minimal variation, we omit a plot for this analysis." This is unacceptable in a rigorous scientific paper. This analysis is crucial for understanding the scalability and limitations of the proposed architecture. Hiding this data prevents reviewers from assessing the performance at corner cases (e.g., many scaling levels, very large block sizes) where the "less than 1% of total execution time" claim for flattening might break down.

---

Questions to Address In Rebuttal

1. Regarding the Baseline: Please justify why your custom software baseline is a fair comparison point. Can you provide any evidence or theoretical argument to suggest that an industry-optimized software library for MX formats would not significantly close the performance gap you report?

2. Regarding Hardware Cost: Can you provide a more robust analysis of the hardware overhead? Specifically, how do your overhead projections change when considering a modern process node (e.g., 7nm or 5nm)? What is the cycle latency of the Operand Transformer as a function of the number of scaling levels (L), and how was this latency determined?

3. Regarding Simulation Fidelity: Please defend the assumption that FP8 and INT8 operations have identical latency. What is the justification for this simplification, and how would the results change if a more realistic latency model for FP8 arithmetic were used?

4. Regarding Numerical Precision: Please provide a detailed numerical analysis of the flattening operation. What is the bit-width of the internal datapaths within the Operand Transformer? How do you handle potential overflow and underflow during the multiplication of elements by scaling factors to guarantee that precision is maintained, as you claim?

5. Regarding Omitted Results: Please provide the full data and plots for the sensitivity study described in Section 5.6. Specifically, show the impact on total execution time as a) the number of scaling levels is varied from 1 to 4, and b) the block size is varied from 32 to 512.

Paper Title: Avant-Garde: Empowering GPUs with Scaled Numeric Formats Reviewer Persona: The Synthesizer (Contextual Analyst)

---

Summary

This paper identifies a critical and growing bottleneck in modern GPU architectures: the inefficient, software-based handling of advanced scaled numeric formats like MX and HBFP. The authors compellingly argue that while these formats are crucial for improving arithmetic density, the software overhead in managing their hierarchical scaling factors (e.g., per-tensor, per-block, per-subset) negates many of their potential performance benefits, primarily through increased instruction count and register pressure.

The core contribution is Avant-Garde, a novel microarchitectural extension for GPUs designed to natively support these diverse formats. The central idea is elegant: a hardware module called the "Operand Transformer" intercepts operands in their scaled format and "flattens" them into a canonical, single-level internal representation. This flattened format, consisting of a single shared scaling factor and a block of elements, can then be processed efficiently by a redesigned Tensor Core. This approach effectively moves the costly format conversion out of the software domain and into a dedicated, low-latency pipeline stage, thereby unifying the computation pipeline for a wide array of present and future scaled numeric formats. The authors demonstrate significant throughput (up to 74% higher) and execution time (up to 44% lower) improvements with negligible accuracy loss.

Strengths

1. Excellent Problem Formulation and Motivation: The paper does a superb job of contextualizing its contribution. The analysis in Section 2, particularly the illustration of the PTX instruction stream (Figure 3, page 4) and the quantification of register and instruction overhead (Figure 4, page 4), provides a clear and convincing motivation. The authors are not solving a contrived problem; they are addressing a tangible and increasingly relevant challenge at the intersection of DNN model design and hardware architecture.

2. Elegant and Generalizable Core Concept: The central idea of "flattening" is a powerful architectural pattern. By creating a canonical internal representation, the design avoids the trap of building bespoke hardware for every new numeric format (FP8, MX4, MX6, etc.). Instead, it provides a general mechanism that can potentially accommodate future formats that fit the scaled numeric paradigm. This abstraction layer between the programmer-visible format and the internal execution format is a hallmark of strong architectural design.

3. Timeliness and High Potential Impact: This work is situated perfectly within the current landscape of AI hardware research. As the industry, led by efforts like the Open Compute Project (OCP) for Microscaling, moves towards standardizing sub-8-bit formats, the need for efficient hardware support becomes paramount. Avant-Garde provides a well-reasoned blueprint for how major GPU vendors could integrate such support. If adopted, this approach could significantly accelerate the adoption of more aggressive quantization schemes for both training and inference, unlocking further gains in model efficiency.

4. Holistic Architectural Vision: The proposal is not just a single-trick module; it is a coherent set of microarchitectural extensions. The combination of the Operand Transformer, the redesigned Tensor Core, and the corresponding API (Section 3.2, page 7) presents a complete solution. It considers the full data path from memory to execution and back, providing a practical and seemingly implementable design.

Weaknesses

While the core idea is strong, the paper could benefit from a deeper exploration of its implications and limitations:

1. The "Unflattening" Path for Training: The paper's primary focus is on the forward (flattening) path, which is dominant in inference. The reverse path—"unflattening" updated weights back into their original scaled format during training—is discussed more briefly (page 8). The authors state this process leverages CUDA cores and has minimal impact due to its infrequency. However, this complex data-dependent transformation (requiring finding new scaling factors, etc.) could become a non-trivial overhead in training-intensive workloads or novel training algorithms. A more quantitative analysis of this reverse path would strengthen the paper's claims for training efficiency.

2. Programmability and Extensibility of New Formats: The API presented in Figure 9 (page 7) seems to rely on predefined formats (scaled mx9;). While this is practical, it raises questions about the system's true flexibility. How would a researcher experimenting with a novel scaled format (e.g., a three-level hierarchy or non-power-of-two block sizes) utilize Avant-Garde? A more detailed discussion on the boundary between what the hardware can parametrically support and what would require new software libraries or microcode would better define the limits of the proposed architecture's "future-proofing."

3. Interaction with Orthogonal Optimizations: Modern GPUs incorporate many specialized features beyond numeric formats, most notably hardware support for sparsity. How does the flattened internal representation interact with structured or unstructured sparsity? Does it create new opportunities for compression, or does it potentially complicate the identification of zero-value blocks? Placing Avant-Garde in the context of other major architectural trends like sparsity and data compression would provide a more complete picture of its role in a future GPU.

Questions to Address In Rebuttal

1. Could the authors provide a more detailed analysis of the overhead associated with the "unflattening" process required for training? For instance, what percentage of total training time for one epoch of a model like BERT or GPT-2 would be spent in this CUDA-based weight reorganization? Is there a scenario where this could become a bottleneck?

2. Regarding the API's extensibility: Could you clarify the mechanism by which a user could define and use a novel scaled numeric format not pre-compiled into the driver? What are the specific parameters (e.g., number of levels, block sizes, bitwidths) that the Operand Transformer hardware can handle dynamically?

3. Could you speculate on the interplay between Avant-Garde's flattening mechanism and hardware support for sparsity? For example, would sparsity checks be more efficient on the original multi-level format or on the flattened, single-level internal representation?

Paper: Avant-Garde: Empowering GPUs with Scaled Numeric Formats Review Form: The Innovator (Novelty Specialist)

---

Summary

The paper proposes Avant-Garde, a GPU microarchitecture designed to natively support diverse scaled numeric formats, particularly those with multiple scaling levels like the Microscaling (MX) format. The authors identify that current GPUs rely on inefficient software-based methods to handle these formats, leading to significant instruction and register overhead. The central novel mechanism proposed is "operand flattening," where multi-level formats are converted in hardware by a new "Operand Transformer" pipeline stage into a canonical, single-level internal representation. This "flattened" format, which is functionally equivalent to a block floating-point (BFP) representation, is then processed by a modified Tensor Core. The authors claim this approach eliminates software overhead, achieving significant throughput and execution time improvements with modest hardware cost.

---

Strengths

The primary strength of this work lies in its novel microarchitectural approach to a well-known and increasingly important problem. While the challenge of supporting complex numeric formats is not new, the proposed solution is architecturally elegant.

1. Novel Architectural Pattern: The core idea of "flattening" multi-level formats into a canonical single-level internal representation is a clean and compelling architectural pattern. It effectively decouples the complexity of handling a diverse and evolving ecosystem of external numeric formats from the design of the core arithmetic units. This creates a stable internal interface that simplifies the computational core and allows for future format support with potentially minimal changes to the execution units.

2. Instantiation of a Hardware Conversion Stage: The novelty is not in the mathematical conversion itself, but in the specific proposal to instantiate this conversion as a dedicated hardware stage (the "Operand Transformer") within a general-purpose GPU pipeline. This is a concrete and well-defined microarchitectural contribution that directly addresses the software overhead documented in Section 2.2 (Page 4, Figure 4).

3. Justified Complexity: The authors present a clear trade-off analysis. The proposed hardware additions are reported to be modest (Section 3.3, Page 8), while the performance benefits are substantial. This indicates that the novel complexity introduced is well-justified by the gains it provides, a crucial aspect for any new architectural feature.

---

Weaknesses

My critique focuses on the precise boundaries of the novelty and the paper's positioning relative to existing concepts.

1. The "Flattening" Concept is a Re-framing of a Known Conversion: The underlying concept of "flattening" is, in essence, a pre-computation of scaling factors to convert a hierarchical format (like MX) into a standard Block Floating-Point (BFP) format. The paper's novelty rests entirely on the proposal to instantiate this conversion as a dedicated hardware pipeline stage, not on the invention of the conversion process itself. The manuscript could be more explicit in distinguishing its microarchitectural novelty from the underlying mathematical operation, which is not new.

2. Insufficient Comparison with Prior Accelerator Designs: The Related Work section (Section 6, Pages 11-12) discusses other accelerators for scaled formats (e.g., MSFP-based [41], MX-based [42]), but it does not sufficiently analyze their internal datapath designs. The critical question for evaluating novelty is: how did prior dedicated accelerators handle multi-level scaling? Did they also use an explicit conversion to an internal BFP-like format, or did they use a more complex, on-the-fly arithmetic unit that directly managed the hierarchy? Without this direct comparison, the uniqueness of the "Operand Transformer" as an architectural strategy is not as sharply defined as it could be. For instance, accelerators from works like [24] or [39] might contain logic that is functionally equivalent to flattening, even if not described with that term.

3. Limited Novelty of the Computational Core: The design of the "Avant-Garde Tensor Core" (Section 3.2, Page 7), which consumes the flattened format, is functionally very similar to previously proposed BFP or HBFP arithmetic units (e.g., [9], [50]). These units are designed to perform dot products on blocks of values sharing a single exponent/scaling factor. The paper should more clearly state that the novelty is concentrated in the production of this BFP-like format by the Operand Transformer, rather than its consumption by the Tensor Core. The contribution is the bridge, not the destination.

---

Questions to Address In Rebuttal

1. Could the authors please clarify the distinction between the mathematical operation of flattening (which is a known conversion from a hierarchical format to a BFP format) and their specific microarchitectural contribution? Is the primary novel claim the proposal of a dedicated, explicit pipeline stage for this conversion within a GPU architecture?

2. Can the authors provide a more detailed comparison of their "Operand Transformer" approach to the internal datapath designs of prior art in dedicated MX-specific accelerators [24, 39, 42]? Specifically, do these accelerators also convert to a canonical internal representation, or do they employ a different strategy (e.g., on-the-fly scaling factor application)? A direct comparison of architectural strategies would significantly strengthen the paper's novelty claim.

3. The proposed architecture maintains the flattened format in the register file and memory for potential reuse. What are the microarchitectural complexities and overheads associated with managing both the original and flattened data representations in the memory subsystem, particularly in scenarios with frequent register spills/fills or complex data reuse patterns that might require coherence between the two forms?

Paper Title: The XOR Cache: A Catalyst for Compression Reviewer Persona: The Guardian (Adversarial Skeptic)

---

Summary

The paper introduces the "XOR Cache," a compressed Last-Level Cache (LLC) architecture that aims to exploit data redundancy stemming from inclusive cache hierarchies. The core mechanism involves storing the bitwise XOR of two cache lines in a single LLC slot, provided at least one of the original lines is present in a higher-level (private) cache. This inter-line compression is intended to not only save space but also to act as a "catalyst" for existing intra-line compression schemes by reducing the entropy of the resulting XORed line. The authors present a modified coherence protocol to manage these compressed line pairs and provide an evaluation showing significant area/power savings with what they claim is a minor performance overhead.

While the premise of leveraging inclusion redundancy is conceptually interesting, the work suffers from critical weaknesses in its evaluation of performance overhead, the justification of its complex coherence protocol, and its claims of superiority over simpler, more established baselines. The presented benefits appear to come at the cost of unquantified complexity and potentially severe performance penalties in scenarios not fully captured by the evaluation.

Strengths

1. Novel Premise: The core concept of transforming the inclusion property from a source of redundancy into an opportunity for compression is clever and thought-provoking.
2. Synergistic Potential: The paper provides a compelling, albeit idealized, motivation in Section 1.2 (Figure 2) that XORing similar lines can significantly boost the effectiveness of existing intra-line compression schemes.
3. Comprehensive Baselines: The evaluation compares the proposed design against a reasonable set of state-of-the-art compression schemes (BAI, BPC, Thesaurus) and, importantly, an Exclusive LLC baseline, which represents a direct and simpler alternative for mitigating inclusion-based redundancy.

Weaknesses

1. Gross Underestimation of Coherence and Decompression Overhead: The central claim of a mere 2.06% average performance overhead is highly suspect and appears inconsistent with the paper's own description of the protocol.

   • Complex Recovery Paths: The decompression process for an LLC hit on an XORed line is not a simple, single-cycle operation. As described in Section 4.3 and Figure 7, cases like "remote recovery" require a multi-hop sequence: request to LLC -> LLC forwards request + XORed data to a partner sharer -> partner sharer performs XOR and forwards data to the original requestor. This is an extremely long latency path for what should be an LLC hit.
   • Expensive UnXORing Operations: The unXORing mechanism, required for writes (getM), last-sharer evictions (putS), and data evictions (Section 4.4), is fundamentally expensive. It necessitates issuing a "special write-back request" to a private cache to retrieve the partner line, waiting for the data, performing the XOR, and only then proceeding. Stalling an incoming write request while this multi-message round trip completes will introduce significant latency.
   • Contradictory Latency Claims: The paper quantifies the message overhead at 18.2% and network traffic increase at 23.4% (Section 6.4.2), yet dismisses the performance impact. This is a major contradiction. Such a significant increase in traffic and dependency-chains in the coherence protocol cannot plausibly result in a near-negligible 2.06% slowdown. The authors' assertion that future network bandwidth scaling will absorb this cost is speculative and constitutes hand-waving.

2. Insufficient Justification for Protocol Correctness (Deadlock Freedom): The argument for deadlock freedom in Section 4.5 is unconvincing. The authors state they use Murphi for single-address verification and then "analytically evaluate" for multiple addresses. Analytical proofs for complex, unblocking coherence protocols are notoriously difficult and prone to missing subtle corner cases involving request/response ordering and resource dependencies. Given that the protocol introduces new inter-line dependencies (a request for line B can trigger a write-back request for line A), a more rigorous formal proof or a more comprehensive model-checking setup is required to substantiate the claim of deadlock freedom without extra virtual networks.

3. Unconvincing Advantage Over a Simpler, Stronger Baseline: The Exclusive LLC is a critical baseline because it also eliminates redundancy due to inclusion, but does so with a much simpler coherence implementation. According to the authors' own data in Figure 14, XOR Cache+BDI is only 1.30x smaller than an Exclusive LLC+BDI. Is the immense complexity of the XOR Cache coherence protocol—with its multi-hop recovery paths, special write-back requests, and inter-line dependencies—a justifiable trade-off for this marginal density improvement? The paper fails to make this case convincingly. A simpler path to higher density would be to apply a stronger intra-line compression scheme to the Exclusive LLC.

4. Ambiguous Critical Path Description: There is a critical ambiguity in the read path description. Section 5.2.1 states that for a read on an XORed line, the request "also needs to access the XORed partner's coherence metadata." The text states: "First, we follow XORPtr to access the tag entry of the XOR partner. Then, a second lookup in the directory is performed." This describes a serial lookup process. This serialized dependency on the critical path of an LLC hit is a major source of latency that does not appear to be adequately modeled or justified in the performance results.

Questions to Address In Rebuttal

1. Please provide a detailed cycle-level breakdown of the three data forwarding cases (local, direct, remote recovery) for an LLC read hit on an XORed line. How do these latencies compare to a standard LLC hit, and how can the average performance overhead be only 2.06% when "remote recovery" is plausibly tens or hundreds of cycles long?

2. Similarly, what is the latency penalty for a getM request to an XORed line that triggers the unXORing process? Please provide a cycle-level breakdown of this critical operation.

3. The paper presents aggregated performance overheads. Please provide per-benchmark performance results. It is crucial to see if the 2.06% average hides cases of catastrophic slowdowns on workloads that heavily trigger the complex recovery and unXORing paths.

4. Please provide a more rigorous argument for deadlock freedom in the multi-address case. The current "analytical evaluation" is insufficient. What specific race conditions were considered, and how does the protocol prevent request-dependency cycles (e.g., Core1 waiting on Core2 for an unXOR dependency, while Core2 is waiting on Core1 for an unrelated resource)?

5. Please clarify the read access path described in Section 5.2.1. Are the lookups for the requested line's tag and the partner's directory entry performed in parallel or series? If in series, how is this significant latency accounted for in the evaluation?

6. Justify the decision to introduce the XOR Cache's complexity over simply using an Exclusive LLC, which also eliminates inclusion redundancy. A 30% area improvement over this baseline may not be worth the verification effort and potential performance risks of the proposed coherence protocol.

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper introduces the XOR Cache, a novel LLC architecture designed to reduce the area and power footprint of the last-level cache. The core idea is to exploit the data redundancy inherent in modern inclusive cache hierarchies. The mechanism is twofold:

1. Inter-line Compression: It stores the bitwise XOR of two cache lines (A⊕B) in a single LLC slot. This is only possible when at least one of the original lines (e.g., A) is guaranteed to exist in a higher-level private cache, which serves as the "key" for decompression. This effectively turns the "wasted" capacity of inclusive hierarchies into a compression opportunity.
2. Intra-line Compression Catalyst: By carefully selecting similar lines (A ≈ B) to XOR, the resulting line (A⊕B) has significantly lower entropy (i.e., is full of zeros). This low-entropy data is far more compressible by existing intra-line compression algorithms (like BDI, BPC), thus acting as a "catalyst" that boosts their effectiveness.

The authors present a detailed coherence protocol to manage these compressed pairs, analyze the design of the XOR pairing policy, and provide a comprehensive evaluation showing significant improvements in LLC area (1.93x) and power (1.92x) for a modest performance overhead (~2.06%), leading to a 26.3% reduction in the energy-delay product.

Strengths

The primary strength of this work lies in its elegant and insightful re-framing of a long-standing architectural feature.

1. Novel Conceptual Contribution: The idea of leveraging inclusion-based redundancy as a primitive for compression is the paper's most significant contribution. For years, the data duplication in inclusive caches has been viewed primarily as a drawback (wasted space) or a simple enabler for coherence snooping. This work cleverly transforms it into a core asset for compression, providing one half of the XOR pair "for free." This is a fundamental shift in perspective.

2. Powerful Synergy: The "catalyst" concept is compelling. Instead of proposing yet another standalone compression algorithm to compete with the state-of-the-art, the XOR Cache is designed to be a synergistic substrate that improves them. This makes the work broadly applicable and positions it as a foundational enhancement rather than a mutually exclusive alternative. The potential to nearly double the effectiveness of existing schemes, as suggested by the idealBank results in Figure 2 (Page 3), is profound.

3. Architectural Completeness: The authors go beyond the high-level idea and tackle the difficult implementation details, particularly the necessary cache coherence protocol modifications (Section 4, Page 5). The detailed discussion of compression, decompression, and "unXORing" flows, along with the analysis of deadlock freedom, demonstrates a mature and thorough architectural consideration that is often missing in more conceptual papers.

4. Strong Connection to Physical Design: The choice of XOR as the operator is not arbitrary. It connects directly to a body of work on in-SRAM computing and logic, where bitwise operations can be performed with extreme efficiency [2, 51]. This grounds the proposal in physical reality and strengthens the claims of low overhead for the compression/decompression logic itself.

Weaknesses

The weaknesses are less about fundamental flaws in the idea and more about its dependencies and the complexities it introduces.

1. Dependence on Cache Hierarchy Policy: The entire mechanism is predicated on an inclusive or at least partially-inclusive (NINE) hierarchy where the "key" line can be located in an upper cache. While inclusive hierarchies are common, there is also a significant design space for strictly exclusive or NINE hierarchies [6, 55]. The paper doesn't fully explore how the XOR Cache's benefits would degrade or how the protocol would need to adapt in a non-inclusive environment where the presence of the key is not guaranteed. This is a crucial piece of context in the broader landscape of cache design.

2. Complexity of Coherence and Recovery: The proposed coherence protocol is necessarily more complex than a baseline MSI protocol. The "remote recovery" path described in Section 4.3 and Figure 7 (Page 6) appears to introduce significant latency: a request for line B could trigger a multi-hop sequence involving forwarding the request and the XORed data to a sharer of line A, which then performs the computation and forwards the result. While the average performance impact is shown to be low, the worst-case latency and its potential to create critical path stalls in certain applications is a concern. The 18.2% increase in coherence messages is also a non-trivial system-level cost.

3. The Efficacy of the Pairing Policy: The synergistic "catalyst" effect is entirely dependent on the ability of the map table (Section 5.1.3, Page 7) to find similar lines to pair. This approach places the XOR Cache in the same family as other hash-based schemes like Thesaurus [24], making it susceptible to hash collisions and the limitations of the chosen map function. While the results with 7-bit SBL are good, the gap between this practical implementation and the idealBank potential shown in Figure 2 remains large, suggesting the pairing heuristic is a critical and sensitive component of the design.

Questions to Address In Rebuttal

1. Beyond Inclusion: Could the authors elaborate on the viability of the XOR Cache concept in a non-inclusive (NINE) or strictly exclusive cache hierarchy? How would the "minimum sharer invariant" be maintained if inclusion is not guaranteed? Would it require a separate mechanism to track private cache contents, and what would the overhead of such a system be?

2. Worst-Case Performance Impact: The average performance overhead is presented as a modest 2.06%. However, could the authors provide more insight into the tail latency or worst-case slowdown? Specifically, how frequent is the multi-hop "remote recovery" path, and are there specific workload types (e.g., with irregular sharing patterns) that are disproportionately impacted by this longer-latency decompression path?

3. Scalability of Coherence Traffic: The paper notes a 23.4% increase in network traffic due to forwarding messages (Section 6.4.2, Page 11). While this may be acceptable in a 4 or 8-core system, how do the authors see this scaling in future many-core (64+ cores) or chiplet-based architectures where interconnect bandwidth is a primary performance limiter and energy consumer? Does this overhead threaten to erode the power savings from a smaller LLC at scale?

Review Form: The Innovator

Summary

The paper introduces the "XOR Cache," a last-level cache (LLC) architecture designed to enhance compression by exploiting data redundancy inherent in modern inclusive and non-inclusive, non-exclusive (NINE) cache hierarchies. The core technical proposal is a form of inter-line compression where two cache lines (A and B) are stored in a single LLC slot as their bitwise XOR value (A⊕B). The mechanism relies on the fact that for an inclusive LLC, if line A is also present in a higher-level private cache (e.g., L1), it can be used as a key to reconstruct line B from the stored A⊕B on an LLC access. The authors claim this approach has a dual benefit: 1) it achieves a near 2:1 inter-line compression ratio, and 2) it acts as a "catalyst" for traditional intra-line compression schemes, as XORing two similar lines (A ≈ B) results in a low-entropy line (A⊕B) that is more compressible. The authors propose a corresponding coherence protocol and architectural extensions to support this mechanism.

Strengths

The primary strength of this work lies in its novel conceptual framing. The key innovation is not merely the use of XOR for compression—a well-established technique—but its specific application to exploit a feature of modern cache hierarchies that is almost universally regarded as a liability: data redundancy due to inclusion.

1. Novel Repurposing of a System Drawback: Prior work has predominantly focused on mitigating the negative effects of inclusion (e.g., wasted capacity, back-invalidations) by moving towards NINE or exclusive hierarchies [15, 26, 55], or by designing clever replacement policies [48]. This paper presents a paradigm shift by treating the duplicated data in private caches not as waste, but as a necessary key for decompression. This reframing of a problem into a solution is a significant and elegant conceptual contribution.

2. Synergistic Compression Model ("Catalyst"): The paper's second novel claim—that this inter-line XORing can catalyze intra-line compression—is well-argued. While pre-processing data to improve its compressibility is a known concept (e.g., the transformations in BPC [30]), the specific mechanism of using value-similarity-aware inter-line compression to directly boost an orthogonal intra-line compression scheme appears to be a new and powerful combination.

3. Distinction from Prior Art: The authors have correctly identified the closest prior art and articulated the novelty of their approach. For instance, while in-SRAM XOR compression has been proposed (Wang et al. [51]), the authors rightly note that such work does not target or leverage redundancy from inclusion across the hierarchy. Similarly, this work is distinct from deduplication schemes [49] which target identical, not merely similar, lines, and from approximate caches [46, 47] as it provides lossless reconstruction. The novelty is in the system-level integration and the cross-level dependency, which is a unique mechanism.

Weaknesses

While the core idea is novel, its contribution and applicability are tightly bound to specific architectural assumptions. The evaluation of novelty must consider the context in which it operates.

1. Narrowing Applicability of the Core Premise: The entire innovative mechanism is predicated on the existence of data redundancy between the LLC and private caches. While currently common, there is a clear industry and academic trend towards NINE and strictly exclusive LLCs to maximize effective capacity. The novel contribution of this work, therefore, exists within a potentially shrinking design space. In a strictly exclusive hierarchy, the core mechanism is fundamentally broken, and the entire contribution vanishes. The novelty is therefore conditional, not universal.

2. Complexity vs. Incremental Novelty: The core operation, XORing two blocks, is not new. The novelty is exclusively in its system-level application, which requires a substantial increase in complexity: a redesigned coherence protocol with new forwarding paths (Section 4), deadlock considerations (Section 4.5), and new hardware structures like the map table (Section 5.1). While the performance results seem to justify this complexity, one must question if a less complex, and therefore less novel, approach could achieve similar gains. For example, would simply investing the same transistor budget into a more powerful standalone intra-line compression scheme on a conventional NINE cache yield comparable results without the coherence overhead? The paper does not sufficiently argue that this complex, cross-level approach is fundamentally superior to enhancing existing single-level techniques.

Questions to Address In Rebuttal

1. On the Delta vs. Self-Contained XOR Compression: The authors differentiate their work from Wang et al. [51] by highlighting the focus on inclusion redundancy. This is the critical delta. Could the authors elaborate on the fundamental trade-offs of their cross-level-dependent recovery mechanism versus a hypothetical intra-LLC scheme where both lines (or a base and XORed delta) are recoverable from within the LLC itself? What are the specific performance and complexity advantages that make the reliance on private caches a superior novel approach?

2. On the Longevity of the Novel Contribution: The novelty of this work is contingent on inclusive/NINE hierarchies. Could the authors comment on the applicability of their core idea to future systems that may aggressively pursue strict exclusivity to maximize capacity? Is there any facet of the "XOR Cache" concept that could be adapted to provide a novel benefit in such a context, or is the contribution strictly limited to non-exclusive hierarchies?

3. On the "Catalyst" Effect as a Standalone Concept: The synergy between inter- and intra-line compression is a key claim. How much of the benefit of the "catalyst" effect is due to the specific XOR mechanism versus the general principle of pairing similar lines? Could a simpler pre-processing hash or signature, used to identify similar lines for a state-of-the-art intra-line compressor, achieve a significant fraction of the "catalyst" benefit without the need for the full XOR Cache architecture and coherence protocol? This would help isolate the novelty of the complete system from the novelty of its constituent ideas.

Paper Title: Precise exceptions in relaxed architectures Reviewer: The Guardian

---

Summary

This paper investigates the interaction between exceptions and relaxed memory models, focusing on the Arm-A architecture as a representative case. The authors identify the inadequacy of the classic, sequentially-consistent definition of "precise exceptions" in the context of modern multicore processors. They explore various relaxed behaviors across exception boundaries using a suite of litmus tests, conduct preliminary hardware testing, and propose an extension to an existing axiomatic model to account for some of these behaviors. The paper concludes by sketching a model for software-generated interrupts (SGIs) and their role in synchronization.

While the paper addresses an undeniably important and under-explored area, its contributions are undermined by a limited scope, insufficient empirical validation, and a reliance on informal specifications. The proposed models, particularly for the Generic Interrupt Controller (GIC), are presented as "drafts" and "sketches," raising questions about the completeness and maturity of the work for a top-tier venue like ISCA.

Strengths

1. Problem Identification: The paper correctly identifies a critical and long-overlooked gap in architectural specifications: the definition of exception precision in the presence of relaxed memory behaviors. This is a fundamentally important problem for systems software developers and architects.
2. Executable Model Integration: The use of the Isla tool to create an executable-as-a-test-oracle for the axiomatic model (§5.1, pg 8) is a methodologically sound approach. Providing an executable artifact for exploring behaviors is a valuable contribution.
3. Litmus Test Curation: The development of a library of litmus tests (§3.2, pg 4) to probe specific out-of-order behaviors across exception boundaries provides a concrete basis for discussion and future testing efforts.

Weaknesses

1. Severely Limited and Incomplete Models: The paper’s claims of providing semantic models are significantly overstated.

   • The model for software-generated interrupts is explicitly called a "draft axiomatic extension" (§7.5, pg 12). The authors acknowledge that a large quantity of the GIC architecture, a component with a 950-page specification, would need to be formalized (§1.2, pg 2; §7.5, pg 12). Presenting a "sketch" of such a complex component is insufficient; this is exploratory work, not a complete model.
   • The paper explicitly scopes out imprecise exceptions and the behavior of 'constrained unpredictable' states (§1.2, pg 2). This is a major omission. A robust treatment of exceptions cannot simply ignore the most difficult cases, as they are often where the architectural guarantees are weakest and most critical. The title promises a treatment of "precise exceptions," but a full understanding requires contrasting them with the alternative.

2. Insufficient Empirical Validation: The hardware testing (§3.6, pg 6) is not comprehensive enough to substantiate the paper's claims about architectural behavior.

   • The test suite comprises only 61 hand-written tests. For the vast state space of relaxed memory and exception interactions, this is a very small sample. The authors themselves note that the suite "is relatively small" and "ideally could be auto-generated" (§1.2, pg 2), which concedes a major methodological weakness.
   • The results table (Fig 9, pg 6) is replete with U outcomes (allowed but unobserved). An absence of observed behavior on a limited set of microarchitectures is not strong evidence that the behavior is disallowed across the entire architecture or that the model is correct. This is particularly weak for forbidding behaviors.
   • The selection of hardware, while varied, is not exhaustive. Key server-class microarchitectures that might exhibit different relaxed behaviors are notably absent.

3. Reliance on Informal "Architectural Intent": The foundation of the axiomatic model's correctness appears to rest on "detailed discussions with Arm senior staff" (§1.1, pg 1) and the authors' interpretation of "architectural intent" (§5.1, pg 8). A formal model cannot be grounded in informal conversations. This is not a verifiable or repeatable scientific process. Without a formal, machine-readable specification from the vendor to check against, the model is merely a hypothesis, not a validated artifact. The rephrasing of the Arm prose specification in Figure 2 (pg 4) is the authors' own, not an official definition.

4. Failure to Solve the Core Problem: The paper frames the core challenge as defining precision in a relaxed setting (§6, pg 8). However, it does not actually provide a new, formal definition. Instead, it "identif[ies] and discuss[es] the substantial open problem" (§1.1, pg 2). The paper enumerates phenomena and challenges but stops short of delivering a conclusive, generalizable definition of precision that could be applied across architectures. It highlights the problem effectively but does not solve it.

Questions to Address In Rebuttal

1. The model for SGIs and the GIC is described as a "draft" (§7.5, pg 12) and relies on non-existent or non-public ASL. How can the community build upon or verify a contribution that is admittedly incomplete and based on un-specified machinery? Is it appropriate to present such preliminary work as a primary contribution at this venue?

2. Your validation strategy relies on consistency between your model, a small set of hardware tests, and "architectural intent" derived from informal discussions (§5.1, pg 8). How do you formally guarantee that your model is not overly permissive (i.e., allows behaviors forbidden by the true architecture) or overly restrictive (i.e., forbids behaviors that some implementations might allow)? What is your strategy for validation beyond this informal and limited-scope approach?

3. Given the high number of 'allowed but unobserved' (U) outcomes in your hardware testing (Fig 9, pg 6), how can you confidently forbid certain behaviors in your model (e.g., those in Fig 5, pg 5)? Absence of evidence is not evidence of absence. Is it not possible that these forbidden behaviors could manifest on other microarchitectures or under different testing conditions?

4. The paper's title concerns "Precise exceptions," yet the work explicitly scopes out imprecise exceptions, which form the necessary contrast for a complete definition. How does this significant omission affect the utility and generalizability of your findings and model? Can one truly clarify precision without also formally bounding imprecision?

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper tackles a long-standing and fundamental problem at the intersection of computer architecture, formal methods, and systems programming: what does it mean for an exception to be "precise" in a world of relaxed memory concurrency? The authors correctly identify that the classical definition, rooted in a sequential execution model, is woefully inadequate for modern out-of-order, multi-core processors like Arm-A. The core contribution is the development of a formal axiomatic model that clarifies the concurrency semantics across exception boundaries. This model is not merely theoretical; it is grounded in hardware testing, deep architectural discussions, and implemented in the Isla tool for validation. Crucially, the work extends this analysis to the complex, real-world use case of software-generated interrupts (SGIs), demonstrating the direct impact of these semantics on critical systems software like the Linux kernel's RCU mechanism.

Strengths

1. Addresses a Foundational Gap: This work is highly significant because it addresses a conceptual blind spot that has existed for decades. The community has largely modeled user-space relaxed memory behavior, but the systems-level interactions involving exceptions and interrupts have remained a "grey area" governed by folklore and cautious over-approximation. By providing a formal basis for reasoning about these interactions, this paper provides an essential service to architects, compiler writers, and OS developers.

2. Excellent Methodological Synthesis: The paper is a model of how to conduct research in this domain. It skillfully combines several research traditions:

   • Empirical investigation through hardware litmus testing (§3.6, page 6).
   • Deep architectural analysis based on vendor documentation and direct discussion with architects.
   • Formal modeling using the established axiomatic 'cat' framework (§5, page 7).
   • Tool-based validation using an executable-as-test-oracle (Isla) (§5.1, page 8). This multi-pronged approach builds high confidence in the results and connects the abstract model to concrete reality.

3. Strong Connection to Real-World Systems: The paper's final act—the analysis of SGIs for synchronization patterns like RCU (§7, page 9-11)—is its masterstroke. This section elevates the work from a purely architectural or formal methods exercise into something of immediate relevance to the systems community. It explains the why behind the need for this formal clarity and demonstrates that subtle architectural ordering rules have a direct and profound impact on the correctness and performance of widely-used software.

4. Intellectual Honesty and Vision: The authors do an excellent job of delineating the scope of their contribution while also clearly articulating the remaining open challenges. The discussion in §6 ("Challenges in defining precision", page 8) is particularly insightful, framing their work as a crucial step towards solving a much larger problem, rather than claiming to have solved it entirely. This positions the paper as a foundational piece upon which a great deal of future work can be built.

Weaknesses

While this is an excellent paper, its primary weakness is a natural consequence of its depth and ambition: complexity and accessibility.

1. High Barrier to Entry: The subject matter is inherently complex, and a reader not already steeped in the literature of relaxed memory models and axiomatic semantics will find it challenging. While the authors do a commendable job, a short, high-level "Programmer's Takeaway" section summarizing the key invariants that a systems programmer can now rely on (or must be wary of) would significantly broaden the paper's impact.

2. Arm-A Focus: The deep dive into Arm-A is a major strength, but as a synthesizer, I am left wondering about the generality of the phenomena described. While the authors expect challenges to appear in other architectures, a brief discussion of how the core concepts (e.g., context synchronization, speculative entry) might manifest in RISC-V (with its evolving specifications) or POWER would help contextualize the Arm-A-specific details and frame a more general research agenda.

3. Preliminary SGI Model: The authors are upfront that their axiomatic extension for SGIs is a "draft" (§7.5, page 12). While the analysis leading up to it is sound, this leaves a key part of the contribution feeling less complete than the rest of the work. The full formalization of the GIC is rightly identified as a major undertaking, but the current model's limitations could be more sharply defined.

Questions to Address In Rebuttal

1. Generalizability: How would your conceptual framework for relaxed exceptions apply to other major architectures? For instance, the RISC-V specification includes extensions for supervisor-level behavior ("S"-extension) but is less prescriptive than Arm's. Do you foresee similar or fundamentally different challenges in modeling precision there?

2. Actionable Guidance for Programmers: Based on your findings, what is the single most surprising or important behavior that a kernel developer writing an exception handler on Arm-A should be aware of? For example, is it the fact that memory accesses can be reordered across the SVC/ERET boundary, or something more subtle related to system register dependencies?

3. The Path to a Full Definition of Precision: Your discussion in §6 is excellent. Does your model offer a partial, constructive definition of precision for a subset of cases (e.g., for resumable faults), or does its primary utility lie in formally demonstrating the inadequacy of the old definition? What do you see as the most critical next step—is it modeling the UNKNOWN states, or something else?

Reviewer: The Innovator (Novelty Specialist)

Summary

This paper presents a new, formal framework for defining and reasoning about precise exceptions in the context of relaxed-memory computer architectures. The core novel claim is the creation of the first-ever formal, axiomatic model that precisely specifies the observable behaviors at exception boundaries on a modern, weakly-ordered processor (using ARMv8-A as the primary example). This is achieved by developing a new set of axioms within the isla specification tool that constrain the allowed interactions between the instruction stream, the memory system, and the exception handling mechanism. The work claims this provides a new, rigorous foundation for understanding a part of the architectural specification that has, until now, been defined only by informal prose.

Strengths

From a novelty standpoint, this paper is a significant breakthrough. It does not just incrementally improve an existing idea; it creates a new one by applying formal methods to a domain where they had not been successfully applied before.

• A Fundamentally New Type of Architectural Model: The most significant "delta" of this work is that it creates a new category of architectural artifact. While formal models for memory consistency and instruction semantics have existed for years, this is the first formal, axiomatic model of the exception mechanism for a modern, relaxed-memory architecture. It takes a concept that was previously "in the ether"—defined by prose, convention, and folklore—and crystallizes it into a mathematically precise and machine-checkable definition. This is a foundational and highly novel contribution. 🧠
• Novel Insights from a New Perspective: By applying the lens of formal methods to this problem, the authors have uncovered new and non-obvious behaviors and ambiguities in the existing ARMv8-A specification (Section 5, Page 10). The very act of formalizing the prose revealed corner cases and interactions that were not apparent from a traditional, informal reading. The discovery and formalization of these subtleties is a novel and important scientific result.
• A New Methodology for Architectural Specification: This work doesn't just produce a model; it implicitly proposes a new, more rigorous way to specify computer architectures. It makes a powerful case that informal prose is no longer sufficient for defining the complex interactions in modern processors. The methodology of using a tool like isla to create an executable, testable specification for the exception model is a novel approach that could, and should, be adopted by hardware vendors in the future.

Weaknesses

The novelty of the work is so profound that its primary weakness is that it stands alone. It is a new foundation, but the structure that will be built upon it is not yet clear.

• The "So What?" Question: The paper does an excellent job of proving that a formal model can be built and what that model looks like. However, it spends less time demonstrating the novel capabilities that this new model enables. The evaluation is focused on demonstrating the model itself, not on using the model to do something that was previously impossible (e.g., formally verifying a piece of a hypervisor's exception handling code). The novelty is in the creation of the tool, not yet in the results of its application.
• A Bespoke Creation: The formal model is a massive, bespoke intellectual effort. It is not yet clear how the process of creating this model could be generalized or automated. Is this a one-time heroic effort, or does it represent a new, repeatable methodology that could be applied to other architectures (like RISC-V or x86) with a reasonable amount of effort? The novelty of the methodology, as a repeatable process, is not yet proven.
• Conceptual, Not Algorithmic, Novelty: It is important to note that the novelty here is in the application and the conceptual framework, not in the invention of new core formal methods algorithms. The work leverages an existing tool (isla) and standard techniques from the formal methods community. The innovation is in the bridge between two fields, not in a breakthrough within the field of formal methods itself.

Questions to Address In Rebuttal

1. Your work is the first to create a formal model of this kind. What is the most significant, previously unknown or ambiguous architectural behavior that your novel model helped to clarify or discover?
2. The creation of this model was clearly a massive undertaking. What is the key novel insight you have gained that would make the process of formally modeling the exception architecture of a different processor (e.g., RISC-V) an order of magnitude easier?
3. Now that you have this novel formal model, what is the single most important, previously unsolvable verification problem that you believe can now be solved with it?
4. The novelty of this work is in applying formal methods to a new architectural domain. What do you see as the next "dark corner" of the architectural specification that is currently defined only by prose and is in desperate need of a similar formal modeling effort?

Persona 1: The Guardian (Adversarial Skeptic)

Review Form

Summary

This paper introduces IPEX, an extension for hardware prefetchers designed for energy harvesting systems (EHSs) that experience frequent power failures. The core idea is to make prefetching "intermittence-aware." By predicting an impending power outage, IPEX throttles the aggressiveness of the prefetcher to avoid fetching data that will never be used before the system loses power, thereby saving energy. The authors claim this approach reduces energy consumption and improves forward progress compared to conventional prefetching in an intermittent execution environment.

Strengths

The paper correctly identifies a clear and logical problem.

• Valid Problem Identification: The central premise is sound: aggressive, conventional prefetching designed for always-on systems is likely to be energy-inefficient in a system that loses power frequently, as many prefetched lines will be useless (Section 1, Page 1). This is a valid and important problem to address for this class of device.

Weaknesses

The paper's conclusions are built upon a foundation of unrealistic assumptions, a flawed evaluation methodology, and an incomplete analysis of critical system overheads.

• Fundamentally Unrealistic Power Failure Prediction: The entire mechanism of IPEX hinges on the ability to "properly" predict an upcoming power failure (Section 1, Page 1). The paper relies on a simple voltage-threshold-based predictor and hand-waves away the immense difficulty of this problem. In real EHSs, the power input is noisy and unpredictable. A simple voltage drop is not a reliable indicator of an imminent, unrecoverable outage; it could be a transient dip. This means the predictor will have non-zero false positives (throttling prefetching when it shouldn't, hurting performance) and false negatives (failing to throttle when it should, wasting energy). The evaluation completely ignores the impact of prediction inaccuracy, assuming a perfect oracle. This is a fatal flaw that invalidates the real-world applicability of the results.
• Overhead of IPEX Logic is Ignored: The paper claims energy savings but fails to account for the energy consumed by the IPEX hardware itself. The logic to track prefetch metadata, monitor the voltage predictor, and dynamically adjust the prefetch degree (Section 4, Page 4) is not "free." It consumes static and dynamic power. Without a detailed, post-layout power analysis of the IPEX module itself, the claimed net energy savings are unsubstantiated. The savings could be entirely consumed by the overhead of the controller.
• Incomplete and Unfair Baseline Comparison: The evaluation compares IPEX integrated with a stride prefetcher against the same stride prefetcher without IPEX (Section 5, Page 6). This is an insufficient baseline. A more rigorous evaluation would compare IPEX against other, more conventional energy-saving techniques. For example, a simple approach would be to statically configure the stride prefetcher to be less aggressive (e.g., reduce its degree or distance). It is not proven that the complexity of IPEX's dynamic throttling is superior to a simpler, statically-tuned conservative prefetching policy.
• Contradictory Performance Claims: The paper claims that by saving energy, IPEX allows the system to "make further execution progress" (Abstract, Page 1). This is a logical leap. The act of throttling the prefetcher inherently increases the cache miss rate when the power doesn't fail, which reduces performance. The paper is conflating two scenarios: it improves energy efficiency when a power failure is imminent, but it necessarily degrades performance when a failure is incorrectly predicted. The reported average performance improvement (8.96%) is an artifact of an evaluation that likely does not properly penalize the system for false positives from the power failure predictor.

Questions to Address In Rebuttal

1. Please provide a sensitivity analysis showing how the energy and performance benefits of IPEX degrade as the accuracy of the power failure predictor decreases. What is the break-even point in terms of predictor accuracy where IPEX provides no net benefit?
2. Provide a detailed, post-layout power and area analysis of the IPEX hardware module itself. You must explicitly report this overhead to prove that the mechanism provides a net system-wide energy saving.
3. How does IPEX compare to a baseline system that uses a simple, statically configured, less-aggressive prefetcher (e.g., a stride prefetcher with a maximum degree of 1)? You must show that the dynamic complexity of IPEX is superior to a simple, conservative policy.
4. Can you provide a detailed, cycle-by-cycle analysis of a scenario where the power predictor generates a false positive? You must quantify the performance loss (i.e., the increase in stall cycles due to additional cache misses) incurred by unnecessarily throttling the prefetcher.

Persona 2: The Synthesizer (Contextual Analyst)

Review Form

Summary

This paper, "Rethinking Prefetching for Intermittent Computing," introduces IPEX, an intelligent hardware extension designed to make conventional prefetchers suitable for the unique environment of battery-less, energy harvesting systems (EHSs). The core contribution is to make the prefetcher "intermittence-aware." By monitoring the system's energy state and predicting impending power outages, IPEX dynamically throttles the aggressiveness of the prefetcher. This prevents the system from wasting precious harvested energy fetching data that will be lost on power failure anyway, allowing that energy to be used for useful computation instead. This work effectively bridges the gap between classic high-performance computing techniques and the emerging domain of energy-constrained, intermittent computing.

Strengths

This paper presents a simple, elegant, and powerful idea that addresses a critical and forward-looking problem. Its strength lies in its clear understanding of the fundamental constraints of its target domain and its pragmatic approach to solving them.

• A Necessary Rethinking of a Classic Technique: The most significant contribution of this work is its recognition that a classic, well-understood performance optimization—prefetching—needs to be fundamentally re-thought for a new class of computing systems (Section 1, Page 1). The paper correctly identifies that the design assumptions of the "always-on" world do not apply to intermittent systems, and it provides a clear and logical framework for adapting to this new reality. This is a crucial step in making high-performance computing techniques viable in ultra-low-power environments. 💡
• Elegant Synthesis of Multiple System Signals: IPEX is a beautiful example of a holistic system design. It doesn't just look at the memory access stream; it synthesizes information from multiple parts of the system—the memory system (prefetch requests), the power management unit (voltage levels), and the processor core (execution progress)—to make an intelligent, system-level decision. This cross-layer approach, where the memory system is aware of the power system, is a hallmark of a well-designed, domain-specific architecture.
• Enabling a More Capable Class of Devices: The practical impact of this work could be significant. By improving the energy efficiency of EHSs, IPEX allows them to do more useful work with the same amount of harvested energy. This could enable a new generation of more capable, intelligent, and autonomous sensor devices for applications in the Internet of Things (IoT), medical implants, and environmental monitoring, where battery replacement is not an option. It helps push intermittent computing from a research curiosity toward a practical reality.

Weaknesses

While the core idea is strong, the paper could be strengthened by broadening its scope to the larger software and system ecosystem in which IPEX would operate.

• The Checkpointing Interaction: The paper focuses exclusively on the interaction between prefetching and power failures. However, all intermittent systems rely on a checkpointing mechanism to save and restore state. There is a deep and unexplored interaction here: a more energy-efficient system might be able to take more frequent checkpoints, which in turn could influence prefetching decisions. A discussion of how IPEX could be co-designed with the system's checkpointing policy would be a fascinating addition.
• The Role of the Compiler: The current IPEX design is purely hardware-driven. However, a compiler could provide valuable hints to the prefetcher. For example, a compiler could identify critical data structures that are essential for forward progress and instruct the hardware to prioritize prefetching them, even when power is low. A discussion of the potential for a hardware/software co-designed, intermittence-aware prefetching system would be very interesting.
• Beyond Simple Prefetchers: The paper evaluates IPEX with a simple stride prefetcher. However, the world of prefetching is incredibly rich, with more advanced techniques like stream, delta, and instruction-based prefetchers. An exploration of how the IPEX control philosophy could be adapted to these more complex and powerful prefetching engines would broaden the applicability of the work.

Questions to Address In Rebuttal

1. Your work makes the prefetcher aware of the power system. How do you see this idea being co-designed with the system's checkpointing mechanism? Could the prefetcher, for example, be instructed to prioritize fetching data that is about to be checkpointed?
2. What role could a compiler play in an IPEX-like system? Could compiler-inserted "hints" help the hardware make more intelligent decisions about which data is most critical to prefetch before a power failure? 🤔
3. How would you adapt the IPEX control policy to a more complex, stateful prefetcher, such as a stream prefetcher? Would the decision to throttle be a simple on/off switch, or would it involve a more nuanced adjustment of the prefetcher's internal parameters?
4. Looking at the bigger picture, this work adapts a performance technique for energy efficiency. What other "classic" performance optimizations from the high-performance computing world (e.g., branch prediction, speculative execution, caching policies) do you think need to be fundamentally re-thought to be effective in the intermittent computing domain?

Persona 3: The Innovator (Novelty Specialist)

Review Form

Summary

This paper proposes IPEX, an "Intermittence-aware Prefetching Extension" for energy harvesting systems. The core novel claim is a new hardware control mechanism that dynamically modulates the aggressiveness of a conventional hardware prefetcher based on the likelihood of an impending power failure. This is achieved by coupling the prefetcher's control logic with a power failure predictor. When a power outage is predicted to be imminent, IPEX throttles the prefetcher's degree to prevent it from fetching data that would be lost, thereby saving energy. The synthesis of a power predictor with a prefetch controller to create a new, "intermittence-aware" hardware module is presented as the primary novel contribution.

Strengths

From a novelty standpoint, this paper's strength lies in its creation of a new, domain-specific control policy for a well-known hardware component, effectively creating a new class of prefetcher.

• A Novel Prefetcher Control Policy: The most significant "delta" in this work is the introduction of a new axis for prefetcher control: intermittence awareness. While prior art has explored adaptive prefetchers that modulate their behavior based on accuracy or memory bandwidth, this is the first work to propose a prefetcher that adapts its behavior based on the energy state of the system and the temporal horizon of its own utility (i.e., the time until the next power failure). This is a fundamentally new and non-obvious control heuristic for prefetching hardware. 💡
• A New Synthesis of System Components: The IPEX architecture itself is a novel synthesis of previously disconnected system components. It creates a direct data path and control relationship between the power management unit (the voltage sensor) and the memory subsystem (the prefetcher). This tight coupling of power and memory management at the microarchitectural level is a novel design pattern that has not been explored in prior work on conventional or low-power systems.

Weaknesses

While the core concept is novel, it is important to contextualize its novelty. The work cleverly combines existing components but does not invent fundamentally new predictive or prefetching mechanisms.

• Core Components Are Not New: The novelty of IPEX is in the synthesis, not in the constituent parts.
  • Prefetcher: The paper uses a standard stride prefetcher as its base. It does not propose a new algorithm for identifying memory access patterns.
  • Power Failure Predictor: The paper uses a simple voltage-threshold-based power failure predictor. While effective for the paper's purpose, it is an adaptation of known techniques in the energy harvesting systems literature, not a new prediction algorithm.
• "Throttling" is a Standard Control Action: The mechanism of reducing the degree or aggressiveness of a prefetcher is a standard knob in adaptive prefetching designs. The novelty is not the act of throttling, but the trigger for that throttling (i.e., a predicted power failure).
• Performance Gains Are Not a Novel Insight: The paper's reported performance and energy improvements are a direct and expected consequence of a more intelligent control policy. It is not a novel discovery that avoiding useless work saves energy. The novelty is in the creation of the mechanism that can identify and prevent that useless work, not in the result itself.

Questions to Address In Rebuttal

1. The core of your novelty is the "intermittence-aware" control policy. Can you contrast your approach with prior work on power-aware or energy-aware prefetching in the mobile/low-power computing domain? What is the key "delta" that makes your intermittence-aware approach fundamentally different?
2. The IPEX module creates a new, direct interface between the power predictor and the prefetcher. Is this a point solution, or do you see this as a novel, generalizable microarchitectural pattern? What other components in the processor could benefit from being directly controlled by a power failure predictor?
3. If a competitor were to propose a system that used a sophisticated machine-learning-based power failure predictor, but a very simple prefetch throttling mechanism, where would the novelty of your system lie? Does the novelty rest more on the concept of the coupling, or on the specific implementation of the control logic?
4. What is the most non-obvious or surprising interaction you discovered between the behavior of the prefetcher and the power state of the system that led you to the specific design of the IPEX control policy?

Persona 1: The Guardian (Adversarial Skeptic)

Review Form

Summary

This paper proposes Qtenon, a tightly coupled hardware/software architecture for hybrid quantum-classical computing. The authors identify the high latency of communication between the classical host and the quantum processor (QPU) as a major bottleneck. To address this, they propose a system with a unified memory hierarchy, an efficient multi-stage quantum controller, and a corresponding compiler stack based on a custom IR. The authors claim this tightly integrated approach provides significant speedups (up to 21.6x) for hybrid algorithms like VQE and QAOA compared to current, loosely-coupled designs.

Strengths

The paper correctly identifies a well-known and critical problem in the field of quantum computing.

• Valid Problem Identification: The central premise is sound: the communication latency between the classical and quantum components in current systems is a first-order performance bottleneck that severely limits the practicality of many promising hybrid algorithms (Section 1, Page 1).

Weaknesses

The paper's conclusions are built upon a foundation of an oversimplified problem representation, a flawed and inequitable baseline, and a failure to address the most difficult physical and engineering challenges inherent in its proposal.

• Fundamentally Unrealistic Baseline Comparison: The headline performance claims are invalid because the baseline "decoupled system" is a strawman. It models the quantum-classical interface as a simple network socket connection (Section 5.1, Page 8), which represents the most naive possible implementation. State-of-the-art quantum control systems from both industry and academia employ far more sophisticated real-time control hardware and optimized communication protocols. By comparing against a simplistic baseline, the reported speedups are grossly exaggerated and do not reflect the true benefit of the Qtenon architecture over a realistic, well-engineered alternative.
• Critical Overheads are Ignored: The paper's core proposal, the "unified memory hierarchy," is presented without a rigorous analysis of its physical implementation and associated overheads. The QPU operates at cryogenic temperatures (millikelvin), while the classical host operates at room temperature. The paper completely ignores the immense physical and engineering challenges of creating a low-latency, high-bandwidth memory interface that bridges this thermal gap. The latency and power consumption of the specialized I/O links required for this are not modeled, and could easily consume a significant portion of the claimed performance gains.
• Scalability Claims are Unsubstantiated: The evaluation is performed on small-scale quantum algorithms with a limited number of qubits (up to 30) and parameters (Table 1, Page 9). There is no evidence to suggest that the proposed architecture, particularly the centralized quantum controller and memory arbiter, can scale to the thousands or millions of qubits required for fault-tolerant quantum computing. The centralized design is a classic bottleneck, and the paper fails to provide a convincing argument for how it will avoid becoming a performance limiter at larger scales.
• Compiler Optimizations Are Not Rigorously Evaluated: The paper proposes several compiler optimizations, such as "fine-grained instruction scheduling" and "just-in-time (JIT) compilation" (Section 4, Page 7). However, the performance impact of these optimizations is not isolated in the evaluation. It is unclear how much of the speedup comes from the hardware architecture versus these software techniques. A fair evaluation would compare the full Qtenon system against a baseline that also employs the same advanced compiler optimizations.

Questions to Address In Rebuttal

1. To provide a fair comparison, please evaluate Qtenon against a more realistic baseline that models a state-of-the-art, real-time quantum control system, not a simple socket-based interface.
2. Please provide a detailed physical model and analysis of your proposed unified memory interface. What is the projected latency, bandwidth, and power consumption of the cryogenic-to-room-temperature link, and how does this impact your end-to-end performance claims?
3. Provide a scalability analysis of your centralized quantum controller. At what number of qubits do you project that the controller will become the primary performance bottleneck, and what is your strategy for scaling beyond that limit?
4. Please provide an ablation study that isolates the performance contribution of your proposed hardware architecture (unified memory, controller) from your proposed compiler optimizations. This is necessary to prove the specific value of the hardware design itself.

Persona 2: The Synthesizer (Contextual Analyst)

Review Form

Summary

This paper introduces Qtenon, a tightly coupled, co-designed hardware and software system for accelerating hybrid quantum-classical algorithms. The work's central contribution is a new architectural paradigm that moves beyond the current, loosely-coupled model of a classical host computer controlling a remote quantum accelerator. Qtenon proposes a unified memory hierarchy that allows the classical and quantum processors to communicate directly through shared memory, a multi-stage, pipelined quantum controller to efficiently manage the QPU, and a sophisticated, LLVM-based compiler stack to orchestrate the entire system. By dramatically reducing the quantum-classical communication latency, Qtenon aims to make complex, iterative hybrid algorithms practical for the first time.

Strengths

This paper presents a forward-looking and deeply important vision for the future of quantum computing systems. Its strength lies in its application of well-understood principles from classical computer architecture to solve a critical, emerging problem in the quantum domain.

• A Necessary Architectural Evolution: The most significant contribution of this work is that it provides a concrete and well-reasoned architectural blueprint for the next generation of quantum computers. The current "remote accelerator" model is a necessary first step, but it is not a viable long-term solution. Qtenon correctly identifies that a tighter integration is essential and thoughtfully applies decades of lessons from classical heterogeneous computing (e.g., the evolution of CPU-GPU interfaces) to the quantum world. This is not an incremental improvement; it is a fundamental and necessary evolution of the quantum computing system stack. 🚀
• Elegant Synthesis of Hardware and Software: Qtenon is a beautiful example of a true hardware/software co-design. It recognizes that the latency problem cannot be solved by hardware alone. The proposed compiler stack, with its custom QIR intermediate representation and its ability to perform fine-grained scheduling and JIT compilation (Section 4, Page 7), is just as important as the unified memory and the advanced controller. This holistic, full-stack approach is a hallmark of a mature and well-considered system design.
• Enabling the Future of Quantum Algorithms: The practical impact of this work could be immense. The massive latency of current systems makes many of the most promising hybrid algorithms, particularly those that require rapid feedback between the classical and quantum components (like certain quantum machine learning or optimization algorithms), completely impractical. By reducing this latency by over an order of magnitude (Figure 9, Page 11), Qtenon could enable a whole new class of algorithms to be explored, potentially accelerating the timeline for achieving a practical quantum advantage.

Weaknesses

While the high-level vision is compelling, the paper could be strengthened by exploring the deeper engineering and ecosystem challenges that would need to be overcome for this vision to become a reality.

• The Cryogenic Memory Challenge: The paper's concept of a "unified memory" (Section 3.1, Page 3) is powerful, but it abstracts away the immense physical challenge of building a high-performance memory system that spans the cryogenic-to-room-temperature boundary. A discussion of the potential physical technologies that could enable this (e.g., superconducting SFQ logic, high-bandwidth optical links) would ground the architectural concept in physical reality.
• The Role of Error Correction: The paper focuses on accelerating NISQ-era algorithms. However, the long-term future of quantum computing is in fault-tolerant systems, which will require massive amounts of real-time classical processing for decoding quantum error correction codes. A discussion of how the Qtenon architecture could be adapted or scaled to meet the even more demanding latency requirements of fault-tolerant decoding would be a fascinating extension.
• Standardization and the Ecosystem: For a tightly coupled system like Qtenon to succeed, there needs to be standardization at the hardware-software interface. The paper proposes a custom IR, but it would be beneficial to discuss how this work aligns with emerging industry and academic efforts to standardize quantum intermediate representations (like QIR). A discussion of the path from this research prototype to an open, standardized ecosystem would be valuable.

Questions to Address In Rebuttal

1. Your work brilliantly applies lessons from classical heterogeneous computing to the quantum domain. Looking forward, what is the next major lesson from the history of classical architecture that you believe should be applied to the design of quantum computers?
2. The unified memory is a powerful abstraction. If you were to start building a physical prototype tomorrow, what specific technologies would you investigate to bridge the cryogenic-to-room-temperature gap, and what do you see as the biggest engineering challenge?
3. How do you see the Qtenon architecture evolving to meet the demands of real-time decoding for fault-tolerant quantum error correction, which requires even lower latencies than the hybrid algorithms you evaluated? 🤔
4. The success of your system depends on a rich software ecosystem. How does your proposed compiler stack align with broader community efforts like the QIR alliance, and what is the path to creating a standardized interface that would allow different quantum programming languages to target a Qtenon-like architecture?

Persona 3: The Innovator (Novelty Specialist)

Review Form

Summary

This paper introduces Qtenon, a new, tightly coupled system architecture for hybrid quantum-classical computing. The core novel claim is the architecture itself: a synthesis of hardware and software components designed to fundamentally reduce the communication latency between a classical host and a quantum processor (QPU). The primary novel hardware components are 1) a unified memory hierarchy that serves as a low-latency communication buffer between the host and the QPU (Section 3.1, Page 3), and 2) a multi-stage, pipelined quantum controller designed for efficient, just-in-time instruction generation (Section 3.2, Page 4). The novel software component is a custom compiler stack that uses a new Quantum Intermediate Representation (QIR) to enable fine-grained instruction scheduling and runtime code generation (Section 4, Page 7).

Strengths

From a novelty standpoint, this paper is a significant contribution because it proposes a fundamentally new system-level architecture for a problem that has previously been addressed with ad-hoc, non-integrated solutions.

• A Novel System-Level Architecture: The most significant "delta" in this work is the shift from a "networked" or "distributed" model of quantum-classical computing to an integrated, shared-memory model. While shared memory is a classic concept, its application to bridge the quantum-classical divide, complete with a detailed proposal for the memory hierarchy, the controller pipeline, and the software stack, is a fundamentally new architectural paradigm for this domain. It is the first paper to present a complete, cohesive blueprint for a truly integrated hybrid system. 🧠
• A New Quantum Controller Design: The proposed multi-stage, pipelined quantum controller is a novel piece of microarchitecture. Prior work has focused on the lower-level pulse generation aspects of quantum control. The Qtenon controller, with its explicit stages for instruction fetching, decoding, scheduling, and JIT compilation (Figure 4, Page 6), is a new, more sophisticated design that elevates the controller from a simple sequencer to a true co-processor.
• Novel Synthesis of Compiler and Hardware: The co-design of the compiler and the hardware is a key novelty. The introduction of a new IR (QIR) and the use of JIT compilation in the controller's pipeline are not just software add-ons; they are integral to the hardware's design and enable its low-latency operation. This tight, synergistic coupling of a JIT compiler with a hardware controller pipeline is a novel approach in the quantum domain.

Weaknesses

While the overall architecture is novel, it is important to contextualize its novelty. The work cleverly applies well-known principles from classical computing to a new domain, but it does not invent fundamentally new low-level primitives.

• Component Concepts are Inspired by Prior Art: The novelty is primarily in the synthesis and application, not in the invention of the base concepts.
  • Unified Memory: The idea of a shared memory space for a host and an accelerator is a well-established concept in classical heterogeneous computing (e.g., CUDA's Unified Memory). The novelty here is its application to the unique quantum-classical context.
  • Pipelining: Pipelining is one of the most fundamental concepts in computer architecture. The novelty is the specific pipeline designed for the quantum control task, not the idea of pipelining itself.
  • LLVM/IR-based Compilation: Using a formal IR and the LLVM toolchain for hardware compilation is a standard practice in the HLS and accelerator design communities. The novelty is the creation of a new IR (QIR) with the specific semantics required for quantum computation.
• The "First" Claim is Specific: The claim to be the "first" tightly coupled architecture is a strong one, but it is specific to the publicly-documented academic literature. The novelty lies in being the first to describe and evaluate such an architecture in detail, not necessarily in being the first to conceive of the idea.

Questions to Address In Rebuttal

1. The core of your novelty is the unified memory architecture. Can you contrast your approach with prior work on shared virtual memory for classical heterogeneous systems (e.g., CPU-GPU)? What is the key "delta" or non-obvious challenge that makes unified memory for a quantum system a fundamentally new problem?
2. The multi-stage controller pipeline is a key component. What is the most novel aspect of this pipeline compared to the real-time micro-sequencers and controllers used in other domains that require low-latency feedback, such as high-frequency trading or particle accelerator control systems?
3. The QIR is central to your software stack. How does your proposed IR differ fundamentally from other emerging quantum intermediate representations, and what novel capabilities does your IR enable that others do not?
4. If a competitor were to achieve a similar latency reduction using a different approach (e.g., a highly-optimized, dedicated network-on-chip instead of unified memory), would the novelty of your work be diminished? What is the fundamental, enduring novelty of the "shared memory" approach that makes it superior to a highly-optimized message-passing approach?

Persona 1: The Guardian (Adversarial Skeptic)

Review Form

Summary

This paper introduces HiPER, a processing array architecture designed to accelerate Learning-Based Model Predictive Control (LMPC) for robotics. The authors identify that LMPC workloads exhibit dynamic, fine-grained parallelism that is poorly suited to conventional CPU-GPU systems. To address this, HiPER proposes a "hierarchically-composed" processing array, a fractal interconnect topology, and a pointer queue hierarchy for program execution. The authors claim this design adapts efficiently to the LMPC workload, demonstrating a 10.75x improvement over an NVIDIA Jetson AGX Orin in a 16nm CMOS synthesis.

Strengths

The paper correctly identifies a clear and relevant problem domain.

• Valid Problem Identification: The central premise is sound: LMPC presents a challenging workload with a mix of parallelizable (ML inference) and sequential (MPC optimization) phases, and the communication and control overhead between a CPU and GPU can be a significant bottleneck for this type of real-time application (Section 1, Page 1).

Weaknesses

The paper's conclusions are fundamentally undermined by a flawed and inequitable baseline comparison, an over-simplification of the workload, and an incomplete analysis of critical system overheads.

• Fundamentally Unsound Baseline Comparison: The headline 10.75x speedup claim is invalid because the comparison between a custom, application-specific ASIC (HiPER) and a general-purpose, off-the-shelf mobile SoC (NVIDIA Jetson AGX Orin) is a classic apples-to-oranges fallacy. An ASIC will almost always outperform a general-purpose processor on its target workload. A rigorous and fair comparison would require evaluating HiPER against another custom accelerator designed for a similar class of problems or, at a minimum, against a highly optimized implementation on the GPU baseline that uses advanced features like CUDA graphs to minimize launch overhead. The reported speedup is an artifact of specialization, not a demonstrated architectural superiority over a comparable solution.
• Workload Representation is Oversimplified: The evaluation is based on a single LMPC workload for a specific drone navigation task (Section 5, Page 7). This is an insufficient sample to prove that the architecture is generally effective for the broad class of LMPC algorithms. Real-world robotics applications involve far more complex sensor processing, state estimation, and planning tasks that are not represented in this simple workload. The HiPER architecture, with its fine-grained PEs, is likely to be highly inefficient for the coarse-grained, memory-intensive tasks that were conveniently excluded from the evaluation.
• Critical Overheads are Ignored: The paper's performance analysis focuses on the execution time within the PE array but fails to properly account for critical system overheads. The pointer queue hierarchy is presented as a solution for control flow, but the latency and energy cost of traversing this multi-level queue structure to dispatch work are not adequately modeled. Furthermore, the paper provides no analysis of the off-chip memory bandwidth requirements. It is unclear if the system would be compute-bound, as the paper assumes, or if it would be severely bottlenecked by DRAM access in a real implementation with large ML models and complex environments.
• Fractal Interconnect Justification is Weak: The paper proposes a "fractal interconnect" but provides insufficient evidence that this specific, complex topology is meaningfully superior to a standard, well-understood 2D mesh or a flattened butterfly network. The claim that it "efficiently supports the workload's traffic characteristics" (Abstract, Page 1) is not backed by a detailed traffic analysis or a comparison to alternative topologies under the same workload. The choice appears to be a novelty for its own sake rather than a rigorously justified engineering decision.

Questions to Address In Rebuttal

1. To provide a fair comparison, how does HiPER perform against a state-of-the-art, highly-optimized implementation on the Jetson AGX Orin baseline that uses techniques like CUDA graphs and expert-tuned kernels to minimize the software overhead you identify as the primary bottleneck?
2. Provide a detailed analysis of the performance and energy overhead of the pointer queue hierarchy. What is the end-to-end latency, from the host writing a command to a PE beginning execution, and how does this scale as the hierarchy deepens?
3. Please provide a sensitivity analysis showing how HiPER's performance degrades as the complexity of the non-LMPC parts of a realistic robotics pipeline (e.g., sensor fusion, SLAM) are included in the workload.
4. To justify your choice of a fractal interconnect, please provide a direct, quantitative comparison of its performance (latency, throughput) and cost (area, power) against a conventional 2D mesh and a flattened butterfly network for the exact same LMPC workload.

Persona 2: The Synthesizer (Contextual Analyst)

Review Form

Summary

This paper introduces HiPER, a novel accelerator architecture explicitly co-designed for the unique computational demands of Learning-Based Model Predictive Control (LMPC) in robotics. The core contribution is a "hierarchically-composed" architecture that mirrors the hierarchical nature of the LMPC algorithm itself. This is achieved through a reconfigurable array of processing elements (PEs), a novel fractal interconnect that provides both local and global communication paths, and a pointer-based queueing system for low-overhead task dispatch. By creating a hardware architecture that is a direct physical manifestation of the target algorithm's structure, HiPER aims to eliminate the software and communication overheads that plague traditional CPU-GPU solutions, enabling a new level of performance and efficiency for autonomous robot control.

Strengths

This paper is a significant and forward-looking contribution that sits at the cutting edge of robotics, machine learning, and computer architecture. Its primary strength is its deep, system-level understanding of the target workload and its creation of a truly domain-specific, co-designed solution.

• A Brilliant Example of Algorithm/Hardware Co-design: The most significant contribution of this work is its textbook execution of algorithm/hardware co-design. The authors have not simply accelerated a single kernel; they have analyzed a complete, complex application (LMPC) and designed a hardware architecture that is a direct, physical embodiment of the algorithm's structure (Section 2, Page 2; Section 3, Page 3). The hierarchical composition of the processing array, the fractal nature of the interconnect, and the pointer-based control flow are all direct responses to the specific needs of the LMPC workload. This is a masterclass in domain-specific architecture. 🤖
• Enabling the Future of Robotics: The practical impact of this work could be immense. LMPC is a powerful technique that is central to the next generation of intelligent, autonomous robots. However, its real-world deployment has been severely limited by its computational cost. By providing an order-of-magnitude improvement in performance and efficiency (Figure 10, Page 9), HiPER could make advanced LMPC practical for a wide range of real-time, power-constrained applications, from autonomous drones and self-driving cars to surgical robots. This work could be a key enabler for the future of robotics.
• Connecting Disparate Architectural Concepts: HiPER is a beautiful synthesis of ideas from different domains of computer architecture. It combines the fine-grained parallelism of systolic arrays (for local dataflow) with the coarse-grained control of a VLIW or dataflow processor (via the pointer queue hierarchy), and it connects them with an interconnect inspired by fractal geometry. This creative combination of established principles to create a new, powerful whole is a hallmark of innovative architectural thinking.

Weaknesses

While the core design is powerful, the paper could be strengthened by broadening its focus to the software ecosystem and the long-term evolution of the architecture.

• The Programmability Challenge: The HiPER architecture is highly specialized. A key challenge, which is not fully explored, is how a developer would actually program it. The paper describes the pointer queue mechanism but does not detail the high-level programming model or the compiler toolchain that would be required to map a complex LMPC application onto the hardware. A discussion of the software and compiler challenges is critical for assessing the practical usability of the architecture.
• Beyond LMPC: The architecture is exquisitely tailored to LMPC. However, it is possible that the core principles of hierarchical composition and fractal interconnects could be beneficial for other important workloads with similar characteristics. A discussion of how the HiPER architecture could be generalized or adapted to other domains, such as graph analytics, scientific computing, or other forms of reinforcement learning, would broaden the impact of the work.
• The Memory System: The paper focuses on the processing array and interconnect but is relatively light on the details of the off-chip memory system. For more complex robotics applications with large ML models and high-dimensional state spaces, the interface to external DRAM will be a critical performance bottleneck. A more detailed analysis of the memory system's design and its interaction with the PE array would be a valuable addition.

Questions to Address In Rebuttal

1. Your work is a fantastic example of co-design. Looking forward, how do you envision the programming model for HiPER? What new language abstractions or compiler techniques would be needed to allow a robotics developer to productively map their algorithms onto your architecture?
2. The fractal interconnect is a beautiful match for the LMPC workload. What other application domains do you think could benefit from this type of hierarchically-aware interconnect topology? 🤔
3. How would the HiPER architecture need to evolve to efficiently handle the massive, irregular memory access patterns of other critical robotics tasks, such as running a large Vision Transformer for perception or performing a graph-based SLAM optimization?
4. If a new, non-hierarchical learning-based control algorithm were to emerge in the future, how adaptable is the HiPER architecture? Does its strength in LMPC come at the cost of being overly specialized?

Persona 3: The Innovator (Novelty Specialist)

Review Form

Summary

This paper introduces HiPER, a new accelerator architecture for Learning-Based Model Predictive Control (LMPC). The core novel claims are the synthesis of three primary architectural features: 1) A "hierarchically-composed" processing array that can be dynamically configured to match the structure of the LMPC algorithm (Section 3.2, Page 3); 2) A pointer queue hierarchy for low-overhead, distributed control and orchestration of the processing elements (Section 3.3, Page 4); and 3) A fractal interconnect topology that provides both high-bandwidth local connections and efficient long-range global connections (Section 3.4, Page 5). The synergistic combination of these three features to create a new, workload-adaptive architecture is presented as the primary novel contribution.

Strengths

From a novelty standpoint, this paper is a significant contribution because it proposes a fundamentally new, holistic system architecture built from a collection of individually novel and clever components.

• A Novel Architectural Paradigm: The most significant "delta" in this work is the concept of a "hierarchically-composed" architecture. While reconfigurable and array-based processors are known, HiPER is the first to propose an architecture where the composition and orchestration of the processing elements is itself hierarchical and is designed to be a direct, physical mirror of the target algorithm's structure. This is a new and powerful paradigm for domain-specific acceleration that moves beyond simple kernel offload to a more deeply co-designed system. 🧠
• A Novel Control and Orchestration Mechanism: The pointer queue hierarchy is a novel and elegant solution to the problem of low-overhead control in a massively parallel array. While dataflow and VLIW architectures have explored distributed control, HiPER's use of a multi-level hierarchy of pointer-based FIFOs is a new and specific mechanism that is a clean fit for the target LMPC workload. It represents a new point in the design space between centralized, high-overhead control (like a traditional CPU) and purely data-driven execution.
• A Novel Interconnect Topology: The application of a fractal H-tree topology as the basis for a reconfigurable NoC is a novel and non-obvious choice for an accelerator interconnect. While H-trees are used for clock distribution, their use as a general-purpose data network, particularly in a hierarchical and reconfigurable manner, has not been explored in prior art. It is a clever solution that directly addresses the mix of local and global communication required by the workload.

Weaknesses

While the core architecture is highly novel, it is important to contextualize its novelty. The work is a new synthesis of many ideas, and its direct applicability may be limited to its specific target domain.

• Component Concepts Have Precedents: While the synthesis is new, the underlying ideas have conceptual roots in prior work. Hierarchical processing can be seen as an evolution of coarse-grained reconfigurable arrays. Pointer-based control has echoes of dataflow and transport-triggered architectures. The novelty is not in the invention of these base concepts from first principles, but in their clever adaptation and synergistic combination to solve a new, domain-specific problem.
• High Degree of Specialization: The novelty of the architecture is inextricably linked to its high degree of specialization for the LMPC workload. The tight coupling of the architecture to the algorithm is its primary strength, but it also limits the scope of its novelty. It is a new architecture for LMPC, but it is not a fundamentally new general-purpose architectural paradigm.
• Performance Gains Are a Consequence of Novelty: The reported speedups are a direct and expected consequence of creating a novel, application-specific architecture. It is not a novel discovery that a custom ASIC is faster than a general-purpose GPU. The novelty is in the creation of the architecture that enables this speedup, not in the speedup itself.

Questions to Address In Rebuttal

1. The core of your novelty is the "hierarchically-composed" architecture. Can you contrast your approach with prior work on coarse-grained reconfigurable architectures (CGRAs)? What is the key "delta" that makes your dynamic composition mechanism a fundamentally new approach?
2. The pointer queue hierarchy is a novel control mechanism. How is this fundamentally different from the token-based firing rules in classical dataflow architectures? What new capability does the pointer-based approach enable?
3. The use of a fractal H-tree for a data network is a novel choice. What is the most non-obvious advantage of this topology for the LMPC workload that would not be provided by a more conventional flattened butterfly or a concentrated mesh network?
4. If a competitor were to design a different LMPC accelerator using a different (but still novel) set of architectural principles, on what fundamental, enduring novelty would HiPER compete? Does the novelty lie in the specific components (fractal interconnect, pointer queues), or in the more general philosophy of hierarchical co-design?

Persona 1: The Guardian (Adversarial Skeptic)

Review Form

Summary

This paper introduces Dadu-Corki, a supposed algorithm-architecture co-design for robotic manipulation. The authors propose a new algorithm, CORKI, which is a variant of the existing RoboFlamingo model, and a new hardware accelerator, DADU, designed to run it. The core idea is that CORKI is modified to better suit hardware acceleration by, for example, using a fixed number of reasoning steps. The authors claim this co-design approach yields significant performance and energy efficiency gains over baseline GPU implementations.

Strengths

The paper correctly identifies a relevant and challenging problem.

• Valid Problem Identification: The central premise is sound: vision-language models (VLMs) for robotic control are computationally expensive, and the latency of executing them on general-purpose hardware like GPUs is a major bottleneck for real-time manipulation (Section 1, Page 1).

Weaknesses

The paper's claims of a successful "co-design" are fundamentally undermined by a flawed evaluation, questionable algorithmic contributions, and an incomplete architectural analysis. The work is a classic example of designing a custom accelerator for a slightly modified algorithm and then claiming the result is a profound co-design.

• Fundamentally Unsound Baseline Comparison: The headline performance claims (e.g., 6.4x speedup, 13.9x energy savings) are invalid because they are the result of an apples-to-oranges comparison. An application-specific ASIC (DADU) is being compared to a general-purpose GPU (NVIDIA RTX 4090) (Section 5.3, Page 8). An ASIC will always be more efficient for its target workload. This is not a fair or meaningful comparison. A rigorous evaluation would require comparing DADU against other, comparable robotic accelerators or proving that the algorithmic changes alone account for the benefits.
• Algorithmic Contribution is Trivial and Unproven: The CORKI algorithm is presented as a key part of the co-design, but its novelty is minimal. It is a minor variant of the existing RoboFlamingo, with changes made for hardware convenience (e.g., fixed reasoning steps) rather than fundamental algorithmic improvement. Crucially, the paper provides insufficient evidence that CORKI is actually a better algorithm. The evaluation (Table 1, Page 10) shows that CORKI-3 has a longer average task sequence length than the baseline RoboFlamingo, suggesting it is a less efficient and possibly less intelligent policy. The claim of co-designing a better algorithm is unsubstantiated.
• Critical Overheads are Ignored: The paper's analysis focuses on the core VLM computation but appears to ignore or minimize other critical system overheads. There is no detailed analysis of the latency or energy cost of the off-chip memory accesses, which would be significant for a model of this size. Furthermore, the paper provides no analysis of the full system stack, including the low-level robot control and sensor processing that would run alongside the VLM. The claimed end-to-end performance benefits are based on an incomplete and oversimplified view of the full robotics pipeline.
• "Co-design" is Asserted, Not Demonstrated: The paper repeatedly uses the term "co-design," but there is no evidence of a genuine feedback loop between the algorithm and the architecture. It appears the authors simply took an existing algorithm, made minor modifications to make it more hardware-friendly, and then built a standard accelerator for it. A true co-design would involve using architectural insights to drive fundamental new algorithmic developments, or vice-versa. This is not demonstrated here.

Questions to Address In Rebuttal

1. To provide a fair comparison, how does the DADU accelerator perform against other, published ASIC accelerators for robotic or VLM workloads, when normalized for technology node and silicon area?
2. Your own results (Table 1, Page 10) show that your CORKI algorithm requires a longer sequence of steps to complete tasks than the RoboFlamingo baseline. How can you claim this is a superior algorithm when it appears to be less efficient at the task level?
3. Please provide a detailed breakdown of the off-chip DRAM traffic generated by your system. What percentage of the total execution time and energy is spent on DRAM access, and how does this compare to the on-chip computation?
4. Can you provide a concrete example of how a specific hardware design choice in DADU led to a fundamental, non-obvious change in the CORKI algorithm (or vice-versa)? This is necessary to prove that this is a true co-design, not just an algorithm port to a custom chip.

Persona 2: The Synthesizer (Contextual Analyst)

Review Form

Summary

This paper introduces Dadu-Corki, a holistic, co-designed system that pairs a new robotic manipulation algorithm (CORKI) with a new, purpose-built hardware accelerator (DADU). The central contribution is the tight, synergistic coupling of the algorithm and the hardware. The CORKI algorithm, a variant of the powerful RoboFlamingo vision-language model, is optimized for hardware-friendliness, while the DADU architecture is a domain-specific accelerator with features explicitly designed to accelerate CORKI's unique computational patterns, such as its mix of attention and cross-attention mechanisms. This work aims to provide a complete, full-stack solution that bridges the gap between high-level AI-based robotics and low-level silicon design, enabling a new level of performance and efficiency for embodied AI.

Strengths

This paper is a significant and forward-looking contribution that exemplifies the future of high-performance robotics. Its strength lies in its deep, full-stack understanding of the problem and its creation of a complete, end-to-end solution.

• A Brilliant Example of Algorithm/Hardware Co-design: The most significant contribution of this work is its textbook execution of algorithm/hardware co-design. The authors have not simply accelerated an existing piece of software; they have thoughtfully modified the CORKI algorithm to be more amenable to hardware acceleration and, in parallel, designed the DADU hardware to perfectly match the algorithm's needs (Section 2, Page 2). This virtuous cycle, where algorithmic insights inform hardware design and hardware constraints inform algorithmic choices, is the essence of true co-design and is a model for future work in the field. 🤖
• Enabling the Future of Embodied AI: The practical impact of this work could be immense. The dream of "Embodied AI"—robots that can understand and interact with the world with human-like intelligence—is currently limited by the immense computational cost of the underlying AI models. By providing a solution that is an order of magnitude more performant and efficient than current GPU-based systems (Figure 6, Page 9), Dadu-Corki could be a key enabler for making this dream a practical reality. It helps pave the way for real-time, power-efficient, and truly intelligent robotic systems.
• Connecting the Full System Stack: Dadu-Corki is a beautiful synthesis of ideas from across the entire computing stack. It combines insights from high-level AI and robotics (the VLM-based control policy), compiler and software engineering (the custom instruction set and mapping), and low-level digital logic design (the dataflow accelerator architecture). This ability to reason about and optimize the problem from the application all the way down to the silicon is a hallmark of innovative, impactful systems research.

Weaknesses

While the core design is powerful, the paper could be strengthened by broadening its focus to the programmability and long-term evolution of the architecture.

• The Programmability Challenge: The DADU accelerator is highly specialized for the CORKI algorithm. A key challenge, which is not fully explored, is how a robotics developer would program or adapt it for a different VLM or a new manipulation task. The paper mentions an algorithm framework (Section 4.3, Page 7), but a deeper discussion of the programming model and the compiler toolchain would be critical for assessing the long-term viability and flexibility of the architecture.
• Beyond Manipulation: The paper focuses on a specific class of robotic manipulation tasks. However, a real-world autonomous robot needs to do much more, including perception, localization (SLAM), and path planning. A discussion of how the DADU accelerator could be integrated into a larger, heterogeneous system-on-chip that also accelerates these other critical robotics tasks would provide a more complete picture of a full system solution.
• The Pace of VLM Research: The paper co-designs an accelerator for today's VLM architectures. However, the field of large AI models is evolving at a breathtaking pace. A discussion of how the DADU architecture could be adapted or future-proofed to handle the next generation of VLM models, which might have different structures (e.g., Mixture-of-Experts, new attention mechanisms), would be a valuable addition.

Questions to Address In Rebuttal

1. Your work is a fantastic example of co-design. Looking forward, how do you envision the programming model for DADU? What new language abstractions or compiler techniques would be needed to allow a robotics developer to easily map a new or different VLM onto your architecture?
2. How do you see the DADU accelerator being integrated into a full, heterogeneous SoC for a real-world robot? Would it be a co-processor on a larger chip that also includes accelerators for perception and planning?
3. The CORKI algorithm is based on the Flamingo architecture. How would your DADU architecture need to evolve to efficiently handle a future VLM that uses a fundamentally different structure, such as a Mixture-of-Experts (MoE) model? 🤔
4. This work pushes intelligence closer to the robot's "muscles." What do you think is the next major bottleneck in robotics that needs to be solved through a similar, full-stack, algorithm/hardware co-design approach?

Persona 3: The Innovator (Novelty Specialist)

Review Form

Summary

This paper introduces Dadu-Corki, a new system for robotic manipulation. The core novel claims are the synergistic co-design of two new components: 1) CORKI, a new robotic control algorithm that modifies the existing RoboFlamingo VLM architecture for hardware efficiency by, for example, using a fixed number of reasoning steps and a simplified action space (Section 2, Page 2). 2) DADU, a new, domain-specific dataflow accelerator architecture explicitly designed to accelerate the CORKI algorithm (Section 3, Page 4). The primary novel feature of the DADU architecture is its heterogeneous design, with specialized processing units and data paths for the different components of the CORKI model (e.g., the Perceiver, the Cross-Attention, and the Gated-Recurrent Unit).

Strengths

From a novelty standpoint, this paper's strength lies in its holistic, full-stack approach to innovation. It does not just propose a new algorithm or a new piece of hardware in isolation; it proposes a new, tightly-coupled system where both components are new and are designed to work together.

• A Novel System-Level Co-design: The most significant "delta" in this work is the methodology itself: the tight, synergistic co-design of a new algorithm (CORKI) and a new hardware architecture (DADU) in a closed loop. While algorithm/hardware co-design is a known concept, this paper is a rare example of a work that presents novel contributions at both the algorithm and the architecture level within a single, cohesive project. The innovation is not just in CORKI or DADU, but in the Dadu-Corki system as a whole. 💡
• A Novel Algorithm for Hardware Acceleration: While CORKI is an evolution of RoboFlamingo, its specific modifications—such as the fixed-iteration reasoning and the simplified action decoder—are novel algorithmic changes made explicitly for the purpose of enabling efficient hardware acceleration (Section 2.3, Page 4). This represents a new and important design point in the space of VLM-based control, prioritizing hardware-friendliness alongside task performance.
• A Novel, Heterogeneous Dataflow Architecture: The DADU accelerator is a novel architecture. While dataflow accelerators are known, DADU's specific, heterogeneous pipeline—with dedicated hardware units and memory subsystems for the different phases of the CORKI algorithm (Figure 3, Page 6)—is a new and domain-specific design. It is not a generic VLM accelerator; it is a CORKI accelerator, and this specialization is a key part of its novelty.

Weaknesses

While the overall system is novel, it is important to contextualize the novelty of the individual components, which are largely clever adaptations of existing ideas.

• Algorithmic Novelty is Evolutionary, Not Revolutionary: The CORKI algorithm is a clear and direct descendant of the Flamingo family of models. It does not propose a fundamentally new way of performing vision-language reasoning. Its novelty is in the specific, pragmatic trade-offs it makes to improve hardware efficiency, which is a significant engineering contribution but not a revolutionary algorithmic breakthrough.
• Architectural Primitives are Known: The DADU architecture, while novel in its overall composition, is built from well-understood architectural primitives. It is a dataflow accelerator that uses standard components like systolic arrays, memory controllers, and specialized functional units. The novelty is not in the invention of a new type of processing element or memory system, but in the specific arrangement and specialization of these known components to create a new, application-specific pipeline.
• Performance Gains are a Consequence of Novelty: The reported performance and efficiency gains are a direct and expected consequence of the novel, application-specific co-design. It is not a novel discovery that a custom ASIC is more efficient than a general-purpose GPU. The novelty is in the creation of the co-designed system that enables these gains, not in the gains themselves.

Questions to Address In Rebuttal

1. The core of your novelty is the co-design. Can you provide a specific example where a limitation in the DADU hardware design forced you to make a non-obvious, novel change to the CORKI algorithm that you would not have otherwise made?
2. The CORKI algorithm is an adaptation of RoboFlamingo. What is the most significant, novel insight you gained about the structure of VLMs for robotics by modifying the algorithm for hardware, an insight that would be valuable even for researchers who are not building custom hardware?
3. The DADU architecture is a heterogeneous dataflow design. Can you contrast your approach with prior work on other heterogeneous, domain-specific accelerators (e.g., for wireless baseband or signal processing)? What is the key "delta" that makes your architecture a fundamentally new approach?
4. If a new, superior VLM architecture for robotics were to be published tomorrow, which part of the Dadu-Corki system's novelty would be more enduring: the specific design of the DADU accelerator, or the more general methodology of algorithm-hardware co-design that you have demonstrated?

Persona 1: The Guardian (Adversarial Skeptic)

Review Form

Summary

This paper proposes RTSPMSpM, a method for accelerating Sparse Matrix-Matrix Multiplication (SpMSpM) by offloading the computation to the dedicated Ray Tracing hardware (RT Cores) found on modern GPUs. The core idea is to represent the sparse matrices as geometric scenes and then use the hardware-accelerated Bounding Volume Hierarchy (BVH) traversal to find the overlapping non-zero elements required for the multiplication. The authors claim this approach provides a 1.85x speedup over the state-of-the-art cuSPARSE library.

Strengths

The paper correctly identifies a potentially underutilized, powerful hardware resource on modern GPUs.

• Valid Problem Identification: The work is motivated by the desire to accelerate SpMSpM, which is a critical kernel in many scientific and machine learning applications. Seeking new hardware acceleration pathways for this problem is a valid research direction.

Weaknesses

The paper's conclusions are founded on a fundamentally flawed analogy, an incomplete performance analysis that ignores critical overheads, and an inequitable baseline comparison.

• Fundamentally Flawed Architectural Analogy: The entire premise rests on a superficial similarity between SpMSpM and ray tracing. SpMSpM is fundamentally an irregular memory access and integer arithmetic problem. Ray tracing hardware is a highly specialized, complex engine designed for floating-point geometric intersection tests and spatial data structure traversal. Using a complex geometric engine to perform what is essentially a glorified database join on integer coordinates is a gross architectural mismatch. The mechanism is fundamentally inefficient, using complex ray-box intersection tests to perform what should be simple integer comparisons.
• Critical BVH Construction Overhead is Ignored: The proposed method requires a costly pre-processing step to convert the sparse matrices into a BVH data structure (Section 3.2, Page 4). This BVH construction is not a "free" operation; it is a computationally intensive task that consumes significant time and memory. The paper's evaluation (Section 4.4, Page 6) appears to completely exclude the time taken for this construction from its end-to-end performance measurements. By only reporting the time for the traversal and merge steps, the reported speedups are artificially inflated and deeply misleading. The BVH construction time could easily negate any performance benefit from the hardware acceleration.
• Unfair Baseline Comparison: The 1.85x speedup claim is based on a comparison against a general-purpose library (cuSPARSE) running on standard CUDA cores. The RT Core is a massive, power-hungry, fixed-function ASIC. A rigorous and fair comparison would require evaluating RTSPMSpM against a baseline where the standard CUDA cores are given an equivalent silicon area and power budget to the RT Core. It is highly probable that a larger array of CUDA cores or a more specialized SpMSpM unit built with the same resources would outperform this inefficient repurposing of a ray tracer.
• Scalability is Unproven and Likely Poor: The evaluation is performed on a selection of matrices from the SuiteSparse collection (Section 4.1, Page 5). There is no analysis of how the performance, and particularly the BVH construction time, scales as the matrix dimensions and number of non-zero elements grow to the massive sizes seen in modern large-scale graph analytics or scientific simulations. BVH construction for large, unstructured point clouds (analogous to a sparse matrix) is a notoriously difficult and non-linear problem. The claim of general applicability is unsubstantiated.

Questions to Address In Rebuttal

1. To provide a fair and complete analysis, please report the end-to-end execution time of your method, including the BVH construction phase. How does the total time, including pre-processing, compare to the cuSPARSE baseline?
2. Please provide a sensitivity analysis showing how the BVH construction time scales with the matrix size (N) and the number of non-zero elements (NNZ). At what point does this pre-processing overhead overwhelm the benefits of the hardware traversal?
3. To provide a fair comparison, how does your approach compare to a baseline where the SpMSpM kernel is run on a hypothetical GPU with no RT Core, but where the CUDA cores are given an additional silicon area and power budget equivalent to that of the RT Core?
4. Can you justify the fundamental architectural choice of using a floating-point geometric intersection engine for an integer-based sparse linear algebra problem? What is the raw, cycle-for-cycle efficiency (in terms of useful multiply-accumulates per second per mm²) of your approach compared to a standard CUDA core implementation?

Persona 2: The Synthesizer (Contextual Analyst)

Review Form

Summary

This paper, "RTSPMSpM," introduces a novel and creative approach to accelerating Sparse Matrix-Matrix Multiplication (SpMSpM) by harnessing the dedicated Ray Tracing (RT) hardware that is now a standard component of modern GPUs. The core contribution is a new algorithm that cleverly maps the abstract problem of SpMSpM onto the geometric domain of ray tracing. By representing the non-zero elements of the input matrices as geometric objects (Axis-Aligned Bounding Boxes) and building a Bounding Volume Hierarchy (BVH) over them, the authors are able to use the massively parallel, hardware-accelerated intersection testing capabilities of RT Cores to rapidly identify the overlapping terms required for the matrix product. This work opens up a new and unexpected pathway for accelerating sparse linear algebra by repurposing a powerful, ubiquitous, and often underutilized hardware resource.

Strengths

This paper's primary strength is its brilliant "out-of-the-box" thinking, which connects two seemingly unrelated computational domains to create a new and unexpected synergy. This is a work of great creativity and insight.

• A Brilliant Repurposing of Specialized Hardware: The most significant contribution of this work is its clever and non-obvious idea to use a ray tracing engine for sparse linear algebra (Section 1, Page 2). Modern GPUs are becoming collections of specialized accelerators, and a key architectural challenge is avoiding "dark silicon." This paper provides a compelling solution by finding a "secondary purpose" for the RT Core, effectively democratizing a specialized graphics unit for a general-purpose scientific computing task. This is a powerful and important direction for heterogeneous computing. 💡
• Connecting Disparate Computational Worlds: This work serves as an intellectual bridge between the worlds of computer graphics and high-performance scientific computing. It recognizes a deep algorithmic similarity between finding ray-object intersections in a 3D scene and finding row-column overlaps in a matrix multiplication. By showing how the SpMSpM problem can be formally reframed in the language of geometry (Section 2, Page 2), the paper provides a crucial Rosetta Stone that allows the scientific computing community to tap into the immense, highly-optimized hardware resources of the graphics world.
• Enabling a New Class of General-Purpose Ray Tracing: This paper is a pioneering example of what could be called "General-Purpose Computing on Ray Tracing Units" (GPRT), a spiritual successor to GPGPU. It opens up a new field of research focused on finding other non-graphics algorithms that are fundamentally about search, intersection, or traversal, and mapping them onto RT hardware. This could have a significant impact on fields ranging from computational biology to database acceleration.

Weaknesses

While the core idea is brilliant, the paper could be strengthened by broadening its focus from a specific kernel to the larger system and software ecosystem.

• The BVH Construction Bottleneck: The paper focuses on the acceleration of the core intersection phase, but the necessary pre-processing step of building the BVH is a significant undertaking. A more detailed discussion of how to optimize and accelerate this BVH construction phase, perhaps by using the CUDA cores in a pipelined fashion, would provide a more complete picture of a full system solution.
• The Software Abstraction Layer: For this technique to be widely adopted, it needs to be integrated into high-level scientific computing libraries and frameworks (e.g., SciPy, MATLAB, Eigen). A discussion of the software and compiler challenges in building this abstraction layer—which would need to hide the complexity of the geometric mapping from the end user—would be a valuable addition. What does the API for a "ray-tracing-accelerated BLAS" look like?
• Beyond SpMSpM: The paper successfully demonstrates the mapping for SpMSpM. The natural next question is: what other sparse algebra or graph algorithms could be accelerated with this approach? A discussion of how this geometric mapping could be generalized to other important kernels, such as Sparse Matrix-Vector Multiplication (SpMV) or graph traversal, would elevate the work from a clever trick for one problem to a truly general-purpose platform.

Questions to Address In Rebuttal

1. Your work brilliantly repurposes the RT Core. Looking forward, what other major computational problems, from any field of science or engineering, do you think are fundamentally "intersection problems" that could be mapped onto ray tracing hardware?
2. The BVH construction is a key overhead. How do you envision this being managed in a real-world scientific computing workflow? Should the BVH be constructed once and reused, and what are the trade-offs of this approach?
3. For this to have broad impact, it needs a clean software interface. What do you see as the biggest challenge in building a compiler or library that could automatically and transparently offload sparse algebra operations to the RT Core without the user needing to know anything about geometry or ray tracing? 🤔
4. If you were designing the next generation of GPUs, knowing about this potential use case, would you still design a separate, highly-specialized RT Core? Or would you create a more general-purpose, programmable "Traversal Engine" that could be used for both graphics and these new GPRT workloads?

Persona 3: The Innovator (Novelty Specialist)

Review Form

Summary

This paper introduces RTSPMSpM, a new system for accelerating Sparse Matrix-Matrix Multiplication (SpMSpM). The core novel claim is the algorithmic mapping of the SpMSpM problem onto the hardware-accelerated ray tracing pipeline of modern GPUs. This is achieved through a series of novel techniques: 1) A new problem formulation that represents the non-zero elements of the input matrices as Axis-Aligned Bounding Boxes (AABBs) in a 2D space (Section 2.2, Page 3). 2) The application of the standard Bounding Volume Hierarchy (BVH) data structure to this novel geometric representation of a sparse matrix (Section 3.2, Page 4). 3) The use of hardware-accelerated ray-intersection tests to perform the row-column overlap computation. The synthesis of these steps to repurpose a graphics accelerator for a sparse algebra task is presented as the primary novel contribution.

Strengths

From a novelty standpoint, this paper is a significant contribution because it proposes a fundamentally new and non-obvious cross-domain algorithm. It does not merely optimize an existing method; it creates a new one by drawing an analogy between two previously disconnected fields.

• A Novel Architectural Analogy: The most significant "delta" in this work is the conceptual leap required to see the algorithmic similarity between sparse matrix multiplication and ray tracing. While both can be viewed as search problems, the specific, concrete mapping of matrix indices to geometric coordinates and the use of intersection tests to represent the multiplication is a genuine and highly innovative insight. It is a new way of thinking about the problem that has not been explored in prior art. 🧠
• A New Mechanism for General-Purpose Acceleration: This work is a pioneering example of a new class of general-purpose computation: "GPRT" (General-Purpose computing on Ray Tracing units). While GPGPU is a well-established field, prior work has focused on the programmable CUDA cores. This is the first paper to propose a detailed, end-to-end mapping of a major scientific computing kernel onto the fixed-function ray tracing pipeline. This opens up a new and unexplored avenue for hardware acceleration research.
• Novel Problem Representation: The specific techniques used to enable the mapping are themselves novel. The representation of a sparse matrix as a collection of AABBs, where the geometry of the boxes encodes the row and column indices of the non-zero elements, is a new and clever data representation that is the key enabler for the entire system.

Weaknesses

While the core idea is highly novel, it is important to contextualize its novelty. The work proposes a new use for existing hardware and algorithms, but does not invent new fundamental hardware primitives or data structures.

• Underlying Primitives are Not New: The novelty is purely in the abstraction and the mapping. The work uses standard ray tracing concepts and a standard data structure (the BVH). It does not propose a new type of traversal algorithm or a new hardware unit. The innovation is in the creative application of these existing tools to a new problem domain.
• Novelty is in the "Hack": The solution is, in essence, a very clever "hack." It tricks a piece of hardware designed for one purpose into doing something entirely different. The novelty lies in the ingenuity of this trick, but it also means the mechanism is not a "natural" or fundamentally optimal way to perform the computation. The novelty is in making the imperfect analogy work, not in creating a perfect solution from first principles.
• Performance is a Consequence, Not an Invention: The reported speedup is a direct and expected consequence of successfully offloading a parallelizable workload to a powerful, dedicated hardware unit. The novelty is in enabling this offload via the new algorithm, not in the discovery that hardware acceleration makes things faster.

Questions to Address In Rebuttal

1. Your work proposes a novel mapping of a linear algebra problem onto a graphics pipeline. Can you discuss any prior art in the broader history of GPGPU where a similarly non-obvious, domain-crossing mapping was used to accelerate a non-graphics workload?
2. The representation of the matrix as a collection of AABBs is a key enabling technique. Is this a point solution, or do you see this as a new, generalizable method for representing other abstract discrete structures (e.g., graphs, sets) in a geometric form to leverage graphics hardware?
3. The core of your novelty is the SpMSpM-to-ray-tracing algorithm. If a future GPU were to include a more general-purpose, programmable "Traversal Engine" instead of a fixed-function RT Core, how much of the novelty of your work would remain? Does the contribution lie primarily in overcoming the limitations of today's hardware?
4. What is the most surprising or unexpected challenge you faced when trying to map the precise, discrete world of integer linear algebra onto the continuous, floating-point world of geometric ray tracing?

Persona 1: The Guardian (Adversarial Skeptic)

Review Form

Summary

This paper proposes ANVIL, an in-storage accelerator designed to speed up queries on Name-Value Pair (NVP) data stores. The system consists of a new API, a format-agnostic hardware accelerator called the Name-Value Processing Unit (NVPU) integrated into an SSD, and a software framework for registering NVP formats. By performing queries directly on the flash storage, ANVIL aims to eliminate the data movement overhead between the SSD and the host CPU, claiming significant performance and energy improvements.

Strengths

The paper correctly identifies a well-known and important performance bottleneck.

• Valid Problem Identification: The central premise is sound: for many data-intensive applications, the cost of moving data from storage to the host processor over the PCIe bus is a dominant performance and energy bottleneck (Section 1, Page 1). Targeting this bottleneck is a valid research direction.

Weaknesses

The paper's conclusions are built upon a foundation of an inequitable baseline, an oversimplified analysis of system overheads, and unsubstantiated claims of generality and programmability.

• Fundamentally Unsound Baseline Comparison: The headline performance claims (e.g., 269x speedup) are invalid and deeply misleading because the comparison is an apples-to-oranges fallacy. A custom, application-specific hardware accelerator (the NVPU) is compared against a general-purpose CPU running a single-threaded C++ implementation (Section 5.2, Page 11). An ASIC will always outperform a single CPU core on its target task. A rigorous evaluation would compare ANVIL against a highly-optimized, multi-threaded CPU baseline that uses modern libraries (e.g., Intel TBB, optimized hashing libraries) or, even more importantly, against a GPU-based solution, which is the standard for high-throughput data processing. The reported speedups are an artifact of a weak baseline, not a demonstrated architectural superiority.
• Critical Overheads are Ignored and Minimized: The paper fails to properly account for the significant overheads of its own system.
  1. Programming Overhead: The NVPU must be configured for each new query and NVP format. The paper provides no analysis of the latency or energy cost of programming the hardware, which could be substantial, especially for queries that operate on small amounts of data.
  2. API and Driver Overhead: The ANVIL API introduces a new software layer. The cost of traversing this stack—from user space, through the OS kernel and driver, to the SSD—is not quantified. This overhead could easily dominate the execution time for low-latency queries.
  3. "Bit-Funneling" Complexity: The "bit-funneling" mechanism (Section 4.3, Page 9) is presented as a general solution for handling diverse data formats, but its implementation details are sparse. The logic required to parse the format description, dynamically configure the data paths, and extract specific bit fields is highly complex. The area and power cost of this fully-general programmable logic is likely much higher than the paper's estimates, which appear to be based on a simple datapath.
• Claim of Generality is Unproven: The paper claims ANVIL can accelerate "most formats of NVPs" (Abstract, Page 1). However, the evaluation is limited to a few simple, flat data structures like arrays and hash maps (Section 5.1, Page 11). There is no evidence that the architecture can efficiently handle more complex, real-world formats, such as nested structures (e.g., JSON), graph data stores, or formats requiring multi-step, dependent queries. The claim of general applicability is an unsubstantiated leap from a few simple examples.
• Wear-Out and Reliability Concerns: The proposed mechanism encourages many small, random reads from the flash memory. This access pattern is known to be the worst-case scenario for NAND flash endurance and can significantly increase read disturb errors and reduce the lifetime of the SSD. The paper completely ignores this critical reliability issue.

Questions to Address In Rebuttal

1. To provide a fair comparison, please evaluate ANVIL against a state-of-the-art, multi-threaded CPU implementation and a GPU-based implementation of the same query workloads.
2. Provide a detailed, cycle-accurate performance breakdown of a full query, including the software overhead of the API call, the time to program the NVPU, the query execution time, and the time to return the result.
3. To substantiate your claim of generality, please demonstrate ANVIL's effectiveness on a complex, nested NVP format, such as parsing a large JSON file or performing a graph traversal query.
4. Please provide an analysis of the impact of ANVIL's access patterns on the endurance and long-term reliability of the underlying NAND flash. What is the expected reduction in the SSD's lifetime compared to a conventional workload?

Persona 2: The Synthesizer (Contextual Analyst)

Review Form

Summary

This paper introduces ANVIL, an end-to-end system for accelerating queries on Name-Value Pair (NVP) data stores through in-storage processing. The core contribution is a complete, vertically-integrated solution that includes a new, generalized NVP data abstraction, a high-level API for offloading queries, and a programmable hardware accelerator, the Name-Value Processing Unit (NVPU), embedded within an SSD. By moving the computation to the data, ANVIL aims to circumvent the PCIe bus, which is a major bottleneck in modern data-intensive applications. This work represents a significant step towards creating truly general-purpose, programmable computational storage devices.

Strengths

This paper is a significant and forward-looking contribution that elegantly synthesizes ideas from database theory, computer architecture, and storage systems to create a powerful and practical solution to a fundamental problem.

• A Pragmatic and General Approach to Computational Storage: The most significant contribution of this work is its creation of a general and extensible framework for computational storage. While prior work has focused on offloading specific, fixed functions (like compression or encryption), ANVIL is the first to propose a truly programmable engine built around a broad and fundamental data abstraction—the Name-Value Pair (Section 2, Page 2). This is a massive leap forward. It moves computational storage from a collection of niche point-solutions to a general-purpose platform, which is a critical step for widespread adoption. 🚀
• Elegant Synthesis of the Full System Stack: ANVIL is a beautiful example of a true full-stack co-design. It doesn't just propose a piece of hardware; it presents a complete, end-to-end solution that includes the programmer-facing API, the software driver, the hardware microarchitecture, and the physical storage interface (Figure 1, Page 2). This holistic vision, which considers the problem from the application all the way down to the silicon, is a hallmark of mature and impactful systems research.
• Connecting to the Core of Modern Applications: The work is brilliantly motivated by its focus on NVPs. As the paper correctly identifies, NVPs are a ubiquitous data abstraction, forming the foundation of everything from simple arrays and dictionaries to massive key-value stores and AI feature tables (Section 2.1, Page 3). By targeting this fundamental building block, the ANVIL framework has the potential for incredibly broad impact, accelerating a huge swath of modern data-intensive applications.

Weaknesses

While the core vision is powerful, the paper could be strengthened by broadening its focus to the software ecosystem and the long-term evolution of the storage landscape.

• The "Killer App" and Software Ecosystem: The paper proves the potential of ANVIL with a set of microbenchmarks and specific applications. The next critical step for impact is integration into a major, real-world software system. For example, how could ANVIL be used as a transparent acceleration backend for a popular key-value store like RocksDB or a data analytics framework like Apache Spark? A discussion of the path to integration with a major open-source project would provide a clearer roadmap to real-world impact.
• The Future of Storage Interconnects: ANVIL's primary benefit comes from avoiding the PCIe bus. However, new interconnect technologies like CXL (Compute Express Link) are emerging that promise to create a much more tightly-coupled, lower-latency connection between processors and devices. A discussion of how the value proposition of ANVIL changes in a world with CXL would be a fascinating and forward-looking addition.
• Beyond Simple Queries: The paper focuses on relatively simple "scan-and-filter" style queries. A discussion of how the ANVIL architecture could be extended to support more complex, multi-stage queries (e.g., queries that require joins or aggregations) would be a valuable exploration of the architecture's future potential.

Questions to Address In Rebuttal

1. Your work provides a powerful new hardware capability. Looking forward, what do you see as the biggest challenge in integrating ANVIL into a major data-processing framework like Apache Spark to make its benefits transparently available to a broad base of users?
2. How does the emergence of CXL, which promises a high-bandwidth, cache-coherent interconnect between the CPU and devices, change the design trade-offs for in-storage processing? Does it reduce the need for a system like ANVIL, or does it create new opportunities? 🤔
3. How would you extend the ANVIL hardware and API to support more complex, multi-stage database queries, such as a "scan-filter-aggregate" pipeline, all within the SSD?
4. The NVP abstraction is very powerful. What other common data structures, beyond NVPs, do you think are a good target for a similar, generalized in-storage acceleration framework?

Persona 3: The Innovator (Novelty Specialist)

Review Form

Summary

This paper introduces ANVIL, an end-to-end system for in-storage acceleration of queries on Name-Value Pair (NVP) data stores. The core novel claims are the synthesis of three components into a single, cohesive framework: 1) A new, generalized NVP Abstraction that formally describes a wide variety of NVP formats in a machine-readable way (Section 3, Page 5). 2) A new, programmable hardware accelerator, the Name-Value Processing Unit (NVPU), designed to be integrated into an SSD controller and to interpret the NVP Abstraction to perform queries (Section 4, Page 7). 3) A new API and software stack that allows a programmer to register NVP formats and offload queries to the NVPU. The creation of this complete, general-purpose, and programmable in-storage processing framework is presented as the primary novel contribution.

Strengths

From a novelty standpoint, this paper is a significant contribution because it proposes a fundamentally new, general-purpose framework for a problem that has previously only been addressed by specific, fixed-function point-solutions.

• A Novel General-Purpose Framework for Computational Storage: The most significant "delta" in this work is its generality. While the concept of computational storage is known, prior art has been dominated by fixed-function accelerators for specific tasks (e.g., compression, encryption, database filtering for a single, predefined schema). ANVIL is the first work to propose a truly programmable and format-agnostic computational storage device. The combination of the NVP Abstraction and the programmable NVPU creates a new, flexible paradigm that has not been explored in prior work. It moves computational storage from a "feature" to a "platform." 🧠
• A Novel Hardware/Software Interface: The NVP Abstraction, which allows a programmer to describe the bit-level layout of their data structure to the hardware, is a novel hardware/software interface. This is a significant departure from traditional fixed-ISA designs. It is a new, data-centric approach to programmability that is a perfect fit for the domain of in-storage processing.
• A Novel Microarchitecture for Data-Intensive Search: The NVPU architecture itself, with its combination of a programmable "bit-funneling" unit and fixed-function comparators, is a novel microarchitectural design point specifically tailored for high-throughput, on-the-fly data parsing and filtering (Section 4.3, Page 9). It is not a general-purpose processor, but a new type of data-flow engine designed for the specific task of searching semi-structured data.

Weaknesses

While the overall framework is highly novel, it is important to contextualize its novelty. The work cleverly synthesizes many ideas, but the underlying technologies are adaptations of existing concepts.

• Component Concepts are Inspired by Prior Art: The novelty is primarily in the synthesis and generalization, not in the invention of the base concepts from first principles.
  • In-Storage Processing: The core idea is part of the well-established field of computational storage.
  • Programmable Data Parsers: The NVPU's "bit-funneling" unit is conceptually similar to the programmable packet-parsing engines found in modern network interface cards (NICs) and network switches. The novelty is the application of this concept to the storage domain.
  • API Offloading: The model of using a driver and API to offload tasks to a hardware accelerator is the standard model for all hardware acceleration (e.g., CUDA, DPDK).
• The "First" Claim is Specific: The claim to be the "first" general-purpose framework for NVPs is a strong one, but it is specific. It does not invent the idea of in-storage processing, but it is the first to propose a credible path to making it truly flexible and widely applicable. The novelty is in the leap from fixed-function to programmable.

Questions to Address In Rebuttal

1. The core of your novelty is the general-purpose, programmable nature of ANVIL. How does your NVP Abstraction and NVPU design differ fundamentally from the programmable parsing and filtering engines used in modern SmartNICs (e.g., Mellanox BlueField, Intel IPUs)? What is the key "delta" that makes your work a novel contribution in this context?
2. The "bit-funneling" concept is central to the NVPU. Can you discuss any prior art in other domains (e.g., compilers, network protocol processing) that has used a similar, descriptor-driven approach to data extraction and parsing in hardware?
3. If a competitor were to propose an alternative in-storage solution based on a small, general-purpose RISC-V core instead of your specialized NVPU, what is the fundamental, enduring novelty of the ANVIL microarchitecture that would make it superior?
4. What is the most non-obvious or surprising data format that your generalized NVP Abstraction can describe, which would have been difficult or impossible to accelerate with prior, fixed-function computational storage devices?

Persona 1: The Guardian (Adversarial Skeptic)

Review Form

Summary

This paper introduces Universal Predicate Pushdown (UPP), a framework for offloading database filter operations to an in-storage FPGA. The core idea is to compile user-defined functions (UDFs) and filter predicates, written in high-level languages like C++, into WebAssembly (WASM). This WASM module is then translated into a hardware circuit description (Verilog) that is programmed onto an FPGA inside a "smart" storage device. By compiling arbitrary predicates to hardware, the authors claim to provide a general-purpose in-storage processing solution, reporting significant speedups over a CPU-based baseline.

Strengths

The paper correctly identifies a well-known and important performance bottleneck.

• Valid Problem Identification: The central premise is sound: for data-intensive analytics, the cost of moving data from storage to the host CPU is a first-order bottleneck (Section 1, Page 1). The goal of pushing computation (in this case, predicate evaluation) to the data is a valid and important research direction.

Weaknesses

The paper's conclusions are built upon a fundamentally flawed and inequitable baseline, a questionable compilation pipeline, and a complete failure to address the most critical challenges of real-world in-storage processing.

• Fundamentally Unsound Baseline Comparison: The headline performance claims are invalid and grossly misleading. The paper compares a custom, application-specific FPGA accelerator against a general-purpose CPU running a database (e.g., MonetDB) (Figure 6, Page 9). An FPGA will almost always outperform a CPU on a highly-parallelizable, data-streaming task like filtering. This is a classic apples-to-oranges fallacy. A rigorous and fair comparison would require evaluating UPP against a state-of-the-art, multi-threaded CPU implementation that uses SIMD instructions, or against a GPU-based filtering solution. The reported speedups are an artifact of an unfair comparison, not a demonstrated architectural superiority.
• Compilation and Reconfiguration Overheads are Ignored: The entire UPP flow depends on a complex, multi-stage compilation process: C++ to WASM, WASM to Verilog, and finally, synthesis and place-and-route to generate an FPGA bitstream (Figure 2, Page 4). This end-to-end compilation, especially the hardware synthesis step, is an extremely slow process that can take minutes or even hours for non-trivial predicates. The paper completely ignores this overhead in its performance analysis. The claim of supporting "ad-hoc queries" is absurd when the reconfiguration time for each new query predicate would be orders of magnitude larger than the data transfer time it is trying to save.
• "Universality" is a Gross Overstatement: The paper claims to support "universal" predicate pushdown, but the evaluation is limited to a handful of simple filter predicates from the TPC-H benchmark (Table 1, Page 8). There is no evidence that the framework can handle the full complexity of modern SQL, including complex UDFs with state, non-trivial control flow, or dependencies on external libraries. The WASM-to-Verilog translator is likely to be extremely brittle and support only a small, simple subset of the WASM specification. The claim of universality is unsubstantiated.
• Practical System-Level Issues are Ignored: The paper provides a cartoonish, high-level view of a smart storage device and ignores numerous real-world engineering challenges. How is the FPGA managed, programmed, and secured? How does the UPP framework handle complex data types, variable-length strings, and different file formats (e.g., Parquet, ORC)? How are errors and exceptions from the hardware-accelerated UDF propagated back to the host database? The work presents a toy system, not a practical one.

Questions to Address In Rebuttal

1. To provide a fair comparison, please evaluate UPP against a state-of-the-art, multi-threaded, SIMD-optimized CPU implementation and a GPU-based implementation of the same TPC-H filter predicates.
2. Please report the end-to-end time for a new, ad-hoc query, including the full C++-to-WASM-to-bitstream compilation and FPGA reconfiguration time. How does this "query compilation time" compare to the query execution time?
3. To substantiate your claim of "universality," please demonstrate that your framework can successfully compile and accelerate a complex, stateful User-Defined Function from a real-world analytics workload, not just a simple, stateless TPC-H predicate.
4. Please provide a detailed description of the runtime system required to manage the UPP FPGA. How does the host OS discover, program, and manage the lifecycle of the hardware-accelerated predicates, and how are runtime errors handled?

Persona 2: The Synthesizer (Contextual Analyst)

Review Form

Summary

This paper introduces Universal Predicate Pushdown (UPP), a novel, end-to-end framework for enabling general-purpose, in-storage processing. The core contribution is a complete, vertically-integrated toolchain that allows developers to write arbitrary filter predicates in a high-level language (like C++), compiles them to a universal and portable intermediate representation (WebAssembly), and then automatically synthesizes a custom hardware accelerator for that predicate on an FPGA embedded in a smart storage device. By providing a seamless path from a high-level software abstraction to a low-level hardware implementation, UPP aims to make computational storage truly programmable and accessible, breaking the bottleneck of the host-storage interconnect for a wide range of database and analytics workloads.

Strengths

This paper is a significant and forward-looking contribution that elegantly synthesizes ideas from database systems, compiler theory, and reconfigurable hardware to create a powerful and practical vision for the future of computational storage.

• A Pragmatic and General Approach to Programmable Storage: The most significant contribution of this work is its creation of a complete and plausible end-to-end toolchain for programmable storage. While the idea of computational storage has been around for decades, its adoption has been crippled by the lack of a viable programming model. UPP provides a brilliant solution to this problem by leveraging WebAssembly (WASM) as a universal, hardware-agnostic intermediate representation (Section 3, Page 4). This is a massive conceptual leap forward. It connects the world of smart storage to the vast and mature ecosystem of modern software compilers (e.g., LLVM), which is a critical step for making computational storage truly usable by everyday developers. 🚀
• Elegant Synthesis of the Full System Stack: UPP is a beautiful example of a true full-stack co-design. It doesn't just propose a piece of hardware; it presents a complete, coherent vision that includes the programmer-facing language, the compiler and IR, the hardware synthesis flow, and the runtime system. This holistic approach, which considers the problem from the application all the way down to the FPGA bitstream, is a hallmark of mature and impactful systems research.
• Enabling the Future of Data-Centric Computing: The work is brilliantly motivated by the "data gravity" problem—the fact that it is becoming easier to move compute to the data than to move massive datasets to the compute. UPP provides a concrete and compelling architectural blueprint for this data-centric future. By making storage devices active, programmable participants in the computation, UPP helps to blur the lines between storage and compute, which is a key trend in the evolution of modern data centers and high-performance computing systems.

Weaknesses

While the core vision is powerful, the paper could be strengthened by broadening its focus to the long-term challenges of deploying and managing such a flexible system.

• The Reconfiguration Bottleneck: The paper focuses on the performance of the final, accelerated query but spends less time discussing the significant latency of the FPGA compilation and reconfiguration process. For a truly interactive, ad-hoc query environment, this "time-to-first-result" is critical. A discussion of how this compilation overhead could be mitigated—perhaps through a library of pre-compiled common functions or a JIT-like compilation flow—would provide a more complete picture of a production-ready system.
• Security and Resource Management: A fully programmable storage device introduces significant new security and resource management challenges. How do you prevent a malicious or buggy user-defined predicate from compromising the integrity of the storage device? How do you manage and schedule the limited FPGA resources in a multi-tenant environment where many different users are trying to offload predicates? A discussion of the required OS and runtime support for managing this new class of device would be a valuable addition.
• Beyond Filtering: The paper focuses on the canonical computational storage workload: predicate pushdown (filtering). However, a programmable engine like UPP could potentially accelerate other important in-storage tasks, such as data transformation, compression, or even simple aggregations. An exploration of how the UPP framework could be extended to support a broader class of in-situ data processing would be a fascinating direction for future work.

Questions to Address In Rebuttal

1. Your use of WASM as a hardware-agnostic IR is a brilliant idea. Looking forward, how do you see this approach evolving? Could a future version of UPP use a JIT (Just-In-Time) compilation flow to dramatically reduce the reconfiguration latency for new queries?
2. A programmable storage device is a powerful tool, but also a potential security risk. What new hardware and software mechanisms do you think are needed to create a secure, multi-tenant execution environment on the UPP FPGA? 🤔
3. How could the UPP framework be extended to support more complex, "near-data" computations beyond simple filtering, such as data aggregation or format transcoding, all within the storage device?
4. What do you see as the biggest non-technical barrier to the widespread adoption of a programmable storage framework like UPP? Is it the need for new programming models, the lack of standardized APIs, or something else entirely?

Persona 3: The Innovator (Novelty Specialist)

Summary

This paper introduces Universal Predicate Pushdown (UPP), an end-to-end framework for in-storage processing. The core novel claim is the synthesis of a complete, language-agnostic toolchain that automatically translates high-level filter predicates into custom hardware accelerators on a smart storage device. This is achieved through a novel two-stage compilation process: 1) The use of WebAssembly (WASM) as a universal, hardware-agnostic intermediate representation (IR) for the predicate logic (Section 3, Page 4), a first for the computational storage domain. 2) A new WASM-to-Verilog compiler that automatically synthesizes a hardware datapath from the WASM representation (Section 4, Page 6). The creation of this seamless, fully-automated software-to-hardware pipeline for a storage device is presented as the primary novel contribution.

Strengths

From a novelty standpoint, this paper is a significant contribution because it proposes a fundamentally new, practical, and general-purpose programming model for a problem that has previously been addressed only with specific, fixed-function, or difficult-to-program solutions.

• A Novel Programming Abstraction for Computational Storage: The most significant "delta" in this work is its use of WebAssembly as the hardware/software interface. While computational storage is a known concept, prior work has required developers to use low-level, non-portable hardware description languages (like Verilog or HLS C++) to program the device. UPP is the first work to propose a high-level, language-agnostic, and portable programming model for a smart storage device. This is a massive leap forward in usability and is a truly novel approach that could finally make programmable storage practical. 🧠
• A New Compiler-for-Hardware-Synthesis Flow: The creation of a compiler that directly translates a software IR (WASM) into a hardware description (Verilog) for this domain is a novel and significant engineering contribution. While HLS tools exist, they are complex and require specialized programming expertise. The UPP compiler, which automatically generates a datapath from a restricted but universal software format, represents a new and more accessible path from software to custom hardware.
• A New Level of Generality: The synthesis of these ideas creates a new level of generality. Prior work focused on accelerating specific, hard-coded predicates. UPP is the first framework to propose a credible path to accelerating arbitrary, user-defined predicates, thanks to its novel compilation flow. The novelty is in the leap from a fixed-function device to a truly universal and programmable one.

Weaknesses

While the overall framework is highly novel, it is important to contextualize its novelty. The work cleverly synthesizes many ideas from the compiler and hardware communities, but the underlying technologies are adaptations of existing concepts.

• Component Concepts are Inspired by Prior Art: The novelty is primarily in the synthesis and the application to a new domain.
  • WASM: WebAssembly is a well-established, standardized IR from the software world. The novelty is being the first to recognize its potential as an IR for hardware synthesis in the storage domain.
  • High-Level Synthesis (HLS): The WASM-to-Verilog compiler is, in essence, a form of HLS. The novelty is in the choice of the source language (WASM) and the specific, data-flow architecture it targets, not in the general idea of compiling a high-level language to hardware.
  • FPGA-based Accelerators: Using FPGAs as reconfigurable accelerators in storage devices is a known concept.
• The "First" Claim is Specific: The claim to be the "first" universal framework is a strong one, but it is specific. It does not invent the idea of in-storage processing or HLS, but it is the first to combine them in this specific, powerful, and accessible way.

Questions to Address In Rebuttal

1. The core of your novelty is the use of WASM as an IR for hardware synthesis. Can you contrast your approach with prior work in the HLS community that has used other software-based IRs (like LLVM-IR) as a starting point for hardware generation? What is the key "delta" that makes WASM a fundamentally better or more novel choice for this specific domain?
2. The WASM-to-Verilog compiler is a key enabling technology. What is the most non-obvious or surprising challenge you faced when trying to map the semantics of a software-oriented ISA like WASM onto the parallel, spatial semantics of a hardware datapath?
3. If a competitor were to propose an alternative in-storage solution based on embedding a small, general-purpose RISC-V core that could interpret the WASM code directly (a "JIT-to-FPGA" vs. a "JIT-to-CPU" approach), what is the fundamental, enduring novelty of your ahead-of-time, full-synthesis approach that would make it superior?
4. What is the most complex or unexpected software predicate that your novel framework can compile and accelerate, which would have been difficult or impossible to implement using prior, more rigid HLS-based or fixed-function approaches to computational storage?

Persona 1: The Guardian (Adversarial Skeptic)

Summary

This paper proposes RAGX, an in-storage accelerator for Retrieval-Augmented Generation (RAG). The authors identify the Search & Retrieval phase of RAG as a key bottleneck when the knowledge base is stored on SSDs. To address this, RAGX offloads the vector search (specifically, the HNSW graph traversal) and document scoring/retrieval to a custom accelerator integrated into an SSD. The system is designed for a multi-tenant "as-a-Service" environment, featuring a custom scheduler and memory manager. The authors claim this approach provides significant end-to-end latency reduction and improves system throughput.

Strengths

The paper correctly identifies a clear and relevant performance bottleneck.

• Valid Problem Identification: The central premise is sound: as the knowledge bases for RAG systems grow too large for DRAM, the latency of retrieving context from persistent storage (SSDs) will become a first-order performance bottleneck (Section 2, Page 2). Targeting this bottleneck is a valid and important research direction.

Weaknesses

The paper's conclusions are fundamentally undermined by a flawed and inequitable baseline, a failure to address critical system overheads, and an oversimplified representation of the problem.

• Fundamentally Unsound Baseline Comparison: The headline performance claims are invalid because the comparison is an apples-to-oranges fallacy. A custom, application-specific ASIC (RAGX) is being compared to a general-purpose CPU and DRAM system (Section 5, Page 7). An ASIC will always be more power- and performance-efficient for its target workload. A rigorous and fair comparison would require evaluating RAGX against a state-of-the-art CPU baseline that is highly optimized for vector search (e.g., using SIMD, advanced memory prefetching, and multiple threads) or against another specialized vector accelerator. The reported speedups are an artifact of specialization, not a demonstrated architectural superiority over a comparable solution.
• Critical Overheads are Ignored: The paper's analysis focuses on the core graph traversal and scoring but appears to completely ignore or minimize several critical overheads.
  1. Programming and Scheduling Overhead: In a multi-tenant environment, the RAGX hardware must be constantly reconfigured and scheduled to handle requests from different users with different models and databases. The paper's scheduler is high-level (Section 4.3, Page 6), and there is no analysis of the latency or energy cost of this dynamic reconfiguration and context switching, which could be substantial.
  2. Off-chip Communication within the SSD: The RAGX accelerator needs to communicate with the SSD's Flash Translation Layer (FTL) and the NAND flash chips themselves. This internal SSD network is a shared resource and a potential bottleneck. The paper provides no analysis of this internal communication overhead.
• Graph Traversal is an Unsuitable Task for In-Storage Processing: The core task being accelerated is HNSW graph traversal. This is a memory-latency-bound problem with a highly irregular, pointer-chasing access pattern. This is precisely the worst-case workload for NAND flash, which is optimized for large, sequential block access. Offloading a latency-sensitive, random-access workload to a high-latency storage device is a fundamental architectural mismatch. The paper fails to provide a convincing argument for why this is a sensible design choice.
• "As-a-Service" Claims are Unsubstantiated: The paper claims to be designed for a multi-tenant "as-a-Service" environment, but the evaluation uses a simple, homogeneous workload of concurrent queries to the same database (Section 5.2, Page 8). There is no analysis of a more realistic, heterogeneous workload with different vector models, different databases, and a mix of high- and low-priority queries. The claim of being a robust multi-tenant solution is not proven.

Questions to Address In Rebuttal

1. To provide a fair comparison, how does RAGX perform against a state-of-the-art, multi-threaded CPU baseline that uses a highly-optimized vector search library like FAISS with full SIMD exploitation?
2. Please provide a detailed analysis of the overhead of your multi-tenant scheduler. What is the end-to-end latency for a single, small query in a heavily-loaded system, including all scheduling and hardware reconfiguration delays?
3. Can you justify the fundamental decision to offload a latency-sensitive, random-access graph traversal workload to a high-latency NAND flash-based device? Please provide a detailed breakdown of the internal SSD access latencies and show that they do not dominate the end-to-end execution time.
4. To substantiate your "as-a-Service" claims, please evaluate your system under a heterogeneous workload that includes multiple, concurrent queries using different embedding models and targeting different vector databases stored on the same device.

Persona 2: The Synthesizer (Contextual Analyst)

Review Form

Summary

This paper introduces RAGX, a complete, vertically-integrated system for accelerating Retrieval-Augmented Generation (RAG) through in-storage processing. The core contribution is a holistic, full-stack design that moves the computationally intensive "Search & Retrieval" phase of the RAG pipeline from the host CPU directly into a smart storage device. This is achieved through a co-design of a specialized hardware accelerator for vector search, an intelligent multi-tenant scheduler, and an optimized memory and data layout system. By co-locating the vector search computation with the massive knowledge bases stored on SSDs, RAGX aims to eliminate the host-storage communication bottleneck, enabling a new level of performance, efficiency, and scale for RAG-as-a-Service applications.

Strengths

This paper is a significant and timely contribution that sits at the cutting edge of AI, systems, and storage. Its strength lies in its deep, full-stack understanding of an important and emerging workload and its creation of a complete, end-to-end solution.

• A Brilliant Example of Full-Stack, Workload-Driven Design: The most significant contribution of this work is its textbook execution of full-stack, workload-driven design. The authors have not just accelerated a single kernel; they have analyzed a complete, real-world application (RAG-as-a-Service), identified its key bottleneck, and designed a comprehensive hardware and software solution to address it (Section 2, Page 2). The tight integration of the hardware accelerator, the scheduler, and the memory system (Section 4, Page 5) is a hallmark of a mature and well-considered system design. 🚀
• Enabling the Future of Enterprise AI: The practical impact of this work could be immense. RAG is rapidly becoming the dominant paradigm for deploying Large Language Models in the enterprise, as it allows them to be safely and securely augmented with proprietary, domain-specific knowledge. However, the cost and latency of performing vector search over massive, terabyte-scale knowledge bases is a major barrier to adoption. By providing a solution that is an order of magnitude more performant and efficient (Figure 5, Page 8), RAGX could be a key enabler for the widespread, cost-effective deployment of enterprise RAG.
• Connecting Storage and AI at a Fundamental Level: This work is a pioneering example of the deep, synergistic integration of storage and AI. It moves beyond the simple model of storage as a passive repository of data and re-imagines it as an active, intelligent component of the AI pipeline. This concept of a "smart knowledge base" that can perform its own search and retrieval is a powerful and important architectural evolution that aligns perfectly with the broader trend of data-centric computing.

Weaknesses

While the core vision is powerful, the paper could be strengthened by broadening its focus to the programmability of the system and its interaction with the rapidly evolving AI landscape.

• The Programmability Challenge: The RAGX accelerator is highly specialized for the HNSW vector search algorithm. A key challenge, which is not fully explored, is how the architecture would be adapted to support other, emerging vector search algorithms (e.g., those based on different graph structures or quantization techniques). A discussion of the programmability of the accelerator and the toolchain required to map new algorithms onto it would be a valuable addition.
• Beyond Vector Search: The paper focuses on accelerating the vector search component of RAG. However, a full RAG pipeline involves other important steps, such as document decompression, parsing, and chunking. A discussion of how the RAGX architecture could be extended into a more general-purpose "in-storage RAG pipeline," capable of accelerating these other tasks as well, would be a fascinating direction for future work.
• The Pace of AI Model Research: The paper evaluates its system with a set of current embedding models. However, the field of representation learning is evolving at a breathtaking pace. A discussion of how the RAGX system would need to adapt to a future where embedding models become much larger, or where the nature of the "retrieval" task itself changes (e.g., retrieving structured data or code instead of just text), would be valuable.

Questions to Address In Rebuttal

1. Your work is a fantastic example of co-design for a specific algorithm. Looking forward, how would you make the RAGX accelerator more programmable to support future, as-yet-unknown vector search algorithms?
2. How do you see the RAGX concept evolving from a vector search accelerator into a more complete, in-storage "RAG pipeline" that could also offload tasks like document parsing and chunking?
3. The RAG paradigm is currently dominated by dense vector retrieval. How would your architecture need to change to efficiently support a future where retrieval is based on different modalities (e.g., sparse vectors, images, or even small neural networks)? 🤔
4. This work pushes a key part of the AI stack into the storage device. What do you think is the next major component of the modern data center stack that is ripe for a similar, full-stack, in-storage acceleration approach?

Persona 3: The Innovator (Novelty Specialist)

Summary

This paper introduces RAGX, a new, in-storage acceleration system for Retrieval-Augmented Generation (RAG). The core novel claim is the holistic, full-stack co-design of a hardware/software system specifically for the "Search & Retrieval" phase of RAG-as-a-Service. The novel components are: 1) The RAGX accelerator, a new, domain-specific hardware architecture designed to accelerate HNSW graph traversal and document scoring directly within an SSD (Section 4, Page 5). 2) A multi-tenant scheduling and memory management system designed to support concurrent queries in a service environment (Section 4.3, Page 6). 3. The end-to-end synthesis of these components into the first published in-storage accelerator for the RAG workload.

Strengths

From a novelty standpoint, this paper is a significant contribution because it proposes a complete, new system architecture for a modern and important emerging workload. It does not just optimize a small piece of the problem; it presents a novel, end-to-end solution.

• A Novel System-Level Architecture for a New Workload: The most significant "delta" in this work is that it is the first paper to identify, characterize, and design a specialized hardware accelerator for the RAG workload. While in-storage processing and vector acceleration are known concepts in isolation, this work is the first to synthesize them into a complete, cohesive system explicitly designed for the unique demands of RAG-as-a-Service. The RAGX architecture, with its tight coupling of the HNSW traversal engine, the scoring unit, and the multi-tenant scheduler, is a fundamentally new architectural design point. 🧠
• A Novel In-Storage Graph Traversal Engine: The core of the RAGX accelerator is a hardware engine for traversing the HNSW graph. While graph accelerators are known, the RAGX engine is a new and specific design that is highly specialized for the memory access patterns and computational needs of HNSW (Section 4.1, Page 5). This is not a general-purpose graph processor; it is a novel, domain-specific traversal engine, and its design is a key contribution.
• A Novel Approach to Multi-Tenancy in Computational Storage: The paper's focus on a "as-a-Service" model and its inclusion of a hardware-level multi-tenant scheduler is a novel and important contribution to the field of computational storage. Prior work has largely focused on single-user, single-application scenarios. The RAGX scheduler, which manages concurrent contexts and prioritizes requests, is a new and necessary component for making in-storage processing practical in a real-world, shared environment (Section 4.3, Page 6).

Weaknesses

While the overall system is highly novel, it is important to contextualize its novelty. The work cleverly synthesizes many ideas from different domains, but the underlying technologies are adaptations of existing concepts.

• Component Concepts are Inspired by Prior Art: The novelty is primarily in the synthesis and the application to a new domain, not in the invention of the base concepts from first principles.
  • In-Storage Processing: The core idea is part of the well-established field of computational storage.
  • Vector Search Acceleration: The use of hardware to accelerate Approximate Nearest Neighbor (ANN) search is a known area of research.
  • Graph Traversal: The core operation is a graph traversal, and specialized hardware for graph processing is an active research field.
• The "First" Claim is Specific: The claim to be the "first" in-storage accelerator for RAG is a strong one, but it is specific to this new and emerging workload. The novelty lies in being the first to identify RAG as a key driver for computational storage and to design a complete system for it.

Questions to Address In Rebuttal

1. The core of your novelty is the full-stack design for the RAG workload. Can you contrast your approach with prior work on general-purpose, in-storage graph accelerators? What is the key "delta" in your architecture that makes it uniquely suited for HNSW traversal, a feature that a general-purpose graph engine would lack?
2. The multi-tenant scheduler is a key component for the "as-a-Service" model. How is this scheduler fundamentally different from the QoS and scheduling mechanisms found in modern, high-end enterprise SSD controllers? What novel capabilities does your scheduler have that are specific to the RAG workload?
3. If a new, superior ANN algorithm were to replace HNSW in the future, which part of the RAGX system's novelty would be more enduring: the specific design of the HNSW traversal engine, or the more general, full-stack methodology of co-designing an in-storage accelerator for a specific service-oriented workload?
4. What is the most non-obvious or surprising architectural trade-off you had to make when designing a system that is optimized for both the irregular, latency-sensitive graph traversal of HNSW and the regular, throughput-oriented streaming of the final document retrieval?

Review Form: The Guardian

---

Summary

The authors present Oaken, a solution for accelerating LLM inference serving by quantizing the Key-Value (KV) cache. The core contribution is a hybrid online-offline quantization algorithm. This approach involves profiling KV cache distributions offline to establish static outlier thresholds. These thresholds are then used online to partition KV values into three groups (inner, middle, outer). The authors propose a "group-shift" technique to narrow the dynamic range of outlier groups for better low-bit quantization and a "fused dense-and-sparse" encoding scheme to reduce the storage overhead of sparse outliers. The algorithm is implemented with custom hardware modules intended for integration into LLM accelerators. The evaluation, performed using a custom simulator, claims significant throughput improvements over baselines like NVIDIA A100 with vLLM and other quantization methods, with minimal accuracy degradation.

---

Strengths

1. Algorithmic Novelty: The concept of combining offline profiling with online quantization to reduce runtime overhead is sound. The proposed group-shift and fused dense-sparse encoding techniques are clever algorithmic contributions aimed at maximizing the compression ratio while managing quantization loss.
2. Problem Relevance: The paper correctly identifies the KV cache as a primary bottleneck in batched LLM inference, a critical and well-recognized problem in the field. Addressing both memory capacity and bandwidth constraints is the correct focus.
3. Comprehensive Hardware Design: The authors go beyond a purely algorithmic proposal by detailing a hardware implementation (Section 5), including quantization/dequantization engines and a specialized MMU. This demonstrates a thorough, system-level consideration of the problem.

---

Weaknesses

My review identifies several critical weaknesses that challenge the validity and robustness of the paper's central claims.

1. Unsupported Foundational Assumption: The entire premise of the "offline" profiling strategy rests on the assumption that KV cache value distributions are largely independent of the input data distribution (Observation 2, Section 4.1, page 6). The authors provide evidence for this using four NLP datasets (Wikitext2, PIQA, Hellaswag, Winogrande) which are stylistically similar. This is insufficient proof. The methodology is critically flawed if these static thresholds do not generalize to out-of-distribution prompts, such as those involving code generation, structured data, or multilingual text. The paper presents no evidence to support this crucial generalization, rendering the "offline" component potentially brittle.

2. Misleading and Inconsistent Accuracy Claims: The abstract claims an impressively "minimal accuracy loss of only 0.54% on average, compared to state-of-the-art KV cache quantization techniques." However, the data in Table 2 (page 11) does not substantiate this claim clearly and contradicts it in some cases.

   • Comparison to FP16 Baseline: The actual accuracy loss compared to the FP16 Original baseline is often significantly higher. For example, on Llama2-7B with Hellaswag, the accuracy drops from 75.98% to 73.72%, a loss of 2.26%. On Winogrande, the drop is from 69.13% to 67.64%, a loss of 1.49%. It is unclear how the 0.54% figure was calculated; it seems to selectively ignore the worst-case results.
   • Comparison to SOTA: The claim of superiority over other quantization methods is also inconsistent. For Llama2-7B on Wikitext2, Oaken's perplexity (5.53) is worse than KVQuant's (5.49). On PIQA, Oaken's accuracy (78.29%) is lower than KVQuant's (78.35%). While Oaken may perform better on other metrics, the claim of consistent superiority is an overstatement not fully supported by the authors' own data.

3. Use of Unjustified Global Hyperparameters: The authors fix the quantization group ratios to 4% outer, 90% middle, and 6% inner for all experiments across all models and datasets (Section 6.1, page 10). The justification provided in Figure 12(a) (page 12) is based solely on a single model (Llama2-7B) on a single dataset (Wikitext2). This is a classic case of over-fitting a hyperparameter to a narrow experimental setup and then generalizing without evidence. There is no reason to believe this ratio is optimal, or even good, for other models with different architectures (e.g., Mixtral-8x7B) or sizes. This undermines the robustness of the reported results.

4. Inequitable Baseline Comparisons: The performance evaluation methodology appears biased. In Section 6.1, the authors state they "disable" weight and activation quantization features for baselines like QServe and Tender for a "fair comparison." This is not a fair comparison. These are integrated systems, and their performance may rely on the interplay of all quantization features. By selectively disabling core functionalities, the authors are not comparing Oaken to the true state-of-the-art but to a crippled version of it. A proper evaluation must compare against these systems in their fully-enabled, intended configurations.

5. Conflation of Algorithmic and System-Level Gains: The paper presents Oaken-LPDDR as the top-performing configuration for large-scale serving. However, a significant portion of its ability to handle larger batches comes from the system-level choice of high-capacity LPDDR memory, not solely from the quantization algorithm. The paper needs to more rigorously disentangle the performance gains directly attributable to the compression from the Oaken algorithm versus the gains from the underlying memory hardware choice. The algorithm enables the use of LPDDR, but it does not, by itself, grant the capacity.

---

Questions to Address In Rebuttal

1. Please provide a precise, step-by-step calculation demonstrating how the "0.54% average accuracy loss" figure was derived from the results in Table 2. Explicitly state which baseline was used for this comparison and justify this choice.
2. The core assumption of data-independent KV cache distribution is foundational. Please provide experimental evidence showing that the offline-profiled thresholds are robust for out-of-distribution tasks, such as code generation or mathematical reasoning.
3. Please justify the use of a single, global 4%/90%/6% group ratio across all models. Provide a sensitivity analysis for this hyperparameter on at least one other model, such as Mixtral-8x7B or OPT-30B, to demonstrate that the chosen configuration is not simply cherry-picked.
4. Can the authors justify their decision to disable core features of the baseline systems (QServe, Tender)? Please either provide results comparing Oaken to these systems in their fully-optimized, published configurations or provide a stronger argument for why the current comparison is fair and representative.
5. Regarding the results in Figure 13 (page 12), how much of Oaken-LPDDR's ability to handle longer sequences (16K+) is due to the capacity of LPDDR versus the compression from the Oaken algorithm? Can you quantify the contribution of each?

Review Form: The Synthesizer

---

Summary

This paper introduces Oaken, a hardware-software co-designed solution to address the critical key-value (KV) cache bottleneck in large language model (LLM) inference serving. The core contribution is a novel online-offline hybrid quantization algorithm. This approach performs a one-time, offline profiling of a given model to determine robust thresholds for identifying outlier values in the KV cache. These pre-computed thresholds are then used during live inference to perform a lightweight, dynamic online quantization, separating values into inlier and outlier groups without the need for expensive online sorting or analysis.

To translate this algorithmic gain into performance, Oaken proposes custom hardware modules—including quantization/dequantization engines and a specialized memory management unit (MMU)—designed to be integrated into an LLM accelerator. The authors implement and evaluate this system, demonstrating significant throughput improvements (up to 1.58× over state-of-the-art methods on an A100 GPU) with minimal accuracy degradation. The work situates itself as a practical and effective solution to the conflicting demands for high memory bandwidth and high memory capacity in modern LLM serving.

---

Strengths

1. Elegant Core Idea: The central concept of an online-offline hybrid approach is a particularly insightful contribution to the field of KV cache quantization. The current landscape is largely divided between:

   • Fully Online Methods (e.g., KVQuant [22], KIVI [43]), which offer high accuracy by dynamically identifying outliers for every request but pay a steep performance penalty for sorting and mixed-precision handling.
   • Static/Reordering Methods (e.g., QServe [41], Atom [86]), which have lower runtime overhead but may sacrifice accuracy by relying on less adaptive, coarse-grained heuristics like channel reordering.

   Oaken carves out a clever and pragmatic middle ground. The insight that a model's activation distributions have stable, intrinsic properties that can be captured once offline (as shown in Section 4.1, page 6) is powerful. This allows the system to reap the benefits of outlier-aware quantization without bearing the crippling cost of dynamic discovery, effectively finding a sweet spot on the accuracy-performance curve.

2. Holistic System Co-Design: A major strength of this paper is its commitment to a full-stack solution. The authors do not merely propose an algorithm and speculate on its performance benefits. Instead, they meticulously design the hardware necessary to make the algorithm effective. The design of the custom quantization engine, the fused dense-and-sparse encoding scheme (Section 4.5, page 8), and the dual-table MMU for handling the resulting data structures (Section 5.2, page 9) demonstrates a deep understanding of the practical challenges. This co-design makes the impressive performance results far more credible than those in algorithm-only papers, as it directly accounts for implementation overheads.

3. Excellent Contextualization and Problem Framing: The paper does an outstanding job of positioning its work within the broader research landscape. The introductory discussion and, in particular, Figure 1 (page 2), provide a clear and compelling map of the solution space for LLM inference. By plotting existing works on a "bandwidth-capacity trade-off" axis, the authors immediately establish the context and significance of their contribution. This framing helps the reader understand that Oaken is not just another quantization paper, but a systemic attempt to push the Pareto frontier of serving efficiency. The analysis in Section 3, which motivates the work by examining the limitations of HBM vs. LPDDR memory, further grounds the research in tangible, real-world system design constraints.

---

Weaknesses

While the paper is strong, its central premise rests on an assumption whose boundaries could be explored more deeply.

1. Robustness of Offline Profiling: The entire approach is predicated on "Observation 2" (Section 4.1, page 6)—that the range of KV cache values is consistent across different input datasets. The authors validate this on several standard academic benchmarks. However, in real-world deployment, LLM services encounter a vast and unpredictable range of inputs, including adversarial prompts, out-of-distribution topics (e.g., code generation, non-English languages), and different fine-tuning domains. The paper would be strengthened by a discussion on the sensitivity of the offline-generated thresholds. How gracefully does the system degrade if it encounters inputs that generate activation distributions that deviate significantly from the profiling set? A sensitivity analysis could better define the operational envelope of Oaken.

2. Disentangling Algorithmic vs. Hardware Gains: The co-design is a strength, but it also makes it slightly difficult to isolate the benefit of the Oaken algorithm itself from the custom accelerator it runs on. The baselines are either GPU-based systems (vLLM, QServe) or another accelerator (Tender). While the latency breakdown in Figure 12b (page 12) is useful, a direct throughput comparison of an "Oaken-on-GPU" software kernel against other GPU baselines in Figure 11 would be highly illuminating. This would allow the community to understand how much of Oaken's advantage stems from its more efficient algorithm versus the inherent benefits of a specialized ASIC implementation, providing a clearer picture of its applicability to existing commodity hardware.

---

Questions to Address In Rebuttal

1. Could the authors comment on the robustness of the offline profiling step? Have they investigated how sensitive the chosen thresholds are to more dramatic domain shifts in input prompts (e.g., from prose to source code, or across different languages)? What would be the recommended procedure for a production system—is one-time profiling on a general-purpose dataset sufficient, or would you recommend re-profiling for specific, fine-tuned models or domains?

2. The hardware-software co-design is a key feature. To help the community better appreciate the algorithmic contribution, would it be possible to provide throughput data for a CUDA-kernel implementation of the Oaken quantization algorithm running on the A100 GPU? Placing this as a baseline in Figure 11 would clarify how much of the performance gain over methods like KVQuant comes from the superior algorithm (avoiding online sorting) versus the custom hardware.

3. The paper proposes Oaken as a set of modules that can be integrated with "any LLM accelerators." Could the authors briefly elaborate on the practical considerations for such an integration? For example, what are the primary interface requirements between the Oaken DMA unit and a host accelerator's processing cores and memory subsystem?

Reviewer: The Innovator (Novelty Specialist)

Summary

The paper presents Oaken, an algorithm-hardware co-design for accelerating batched LLM inference by quantizing the Key-Value (KV) cache. The authors identify the high cost of online outlier detection in existing mixed-precision KV cache quantization schemes as a key performance bottleneck.

The central claim to novelty lies in a hybrid online-offline approach. The authors propose to: 1. Offline Profile for Thresholds: Determine static thresholds that partition KV cache values into three groups (inner, middle, outer) based on pre-computed statistics from sample prompts. 2. Online Scale Calculation: During inference, dynamically calculate the quantization scale (min/max) for each of the three groups online, but only for the newly generated token. 3. Group-Shift Quantization: For the outer and middle groups (outliers), shift the values by the offline-determined thresholds before quantization to narrow their dynamic range. 4. Fused Dense-and-Sparse Encoding: Propose a hardware-aware memory layout where a portion of the sparse outlier data is stored in the zeroed-out entries of the dense inlier matrix to reduce metadata overhead.

These algorithmic choices are coupled with custom hardware units (quantization/dequantization engines and an MMU) designed to accelerate this specific workflow. The authors claim this co-design achieves superior throughput compared to existing GPU-based solutions with minimal accuracy loss.

Strengths

The primary strength of this work is the clever synthesis of existing quantization concepts into a novel, high-performance system designed to solve a well-defined problem. The novelty is not in a single disruptive idea, but in the specific combination and refinement of several techniques.

1. The Online-Offline Hybrid Scheme: The core idea of using offline profiling to determine static group thresholds while calculating quantization scales online is a novel and pragmatic solution to the performance-accuracy trade-off. It correctly identifies that online sorting (e.g., topK in KVQuant [22]) is prohibitively expensive, while a purely static, offline approach may lack adaptivity. This hybrid method appears to be a new point in the design space.

2. Group-Shift Quantization: The technique of subtracting offline-derived thresholds from outlier values before quantization (Section 4.4, page 7) is a non-trivial algorithmic contribution. It directly addresses the challenge of quantizing wide-range values to a low bitwidth without resorting to higher-precision formats (e.g., FP16), thereby reducing the overhead associated with mixed-precision data handling.

3. Fused Dense-and-Sparse Encoding: This memory layout optimization (Section 4.5, page 8) is a clever, hardware-aware contribution. While storing outliers sparsely is not new (e.g., SqueezeLLM [30]), repurposing the zeroed entries in the dense matrix to store part of the outlier data is a specific and novel technique for reducing the significant metadata overhead of sparse formats.

Weaknesses

From a novelty perspective, the primary weakness is that the constituent components of the proposed solution are variations on well-established themes in quantization and systems design. The work is more of an expert-level integration and optimization than a fundamental conceptual breakthrough.

1. Incremental Algorithmic Concepts:

   • The concept of identifying and separately handling outliers is the foundation of most modern high-accuracy quantization schemes (e.g., LLM.int8() [15], SmoothQuant [71], Olive [19]).
   • The extension from a binary inlier/outlier split to a ternary inner/middle/outer grouping (Section 4.3, page 6) is an incremental, rather than fundamental, advancement. The motivation to preserve small-magnitude values is well-documented in prior art [2, 13, 27, 34], which the authors cite.
   • The "group-shift" technique, while effective, is conceptually analogous to affine quantization, where a zero-point is used to shift the data range. Here, the shift value is cleverly tied to the outlier threshold, but the underlying principle of data shifting pre-quantization is not new.

2. Coupled Hardware Novelty: The hardware modules described in Section 5 (page 8) are themselves direct, albeit efficient, implementations of the proposed algorithm. Their novelty is therefore coupled to, and not independent of, the algorithmic novelty. The idea of building custom hardware for quantization/dequantization is not new; the contribution is that this hardware is purpose-built for the specific Oaken algorithm. As such, the hardware's novelty cannot be evaluated in isolation.

Questions to Address In Rebuttal

1. Delineation from Prior Hybrid Approaches: Could the authors more precisely delineate the novelty of their online-offline hybrid scheme against prior works that use offline calibration data to simplify online computation? For instance, SmoothQuant [71] uses an offline calibration set to determine scaling factors that are applied statically during inference. While the mechanism is different, the principle of using offline analysis to avoid expensive online calculations is shared. Please clarify what makes the "offline thresholds, online scaling" combination fundamentally new.

2. Necessity of the Three-Group Design: The paper justifies the three-group (inner/middle/outer) design to handle both large- and small-magnitude outliers. However, Table 3 (page 12) explores variations with four and five groups but does not include a direct comparison to a simpler two-group (inlier/outlier) scheme that uses the same group-shift and fused encoding techniques. Is the complexity of the third group (and its associated thresholds) essential for the reported performance, or would a two-group system achieve comparable results with less overhead? This is critical to justifying the novel complexity introduced.

3. Generality of Fused Encoding: The fused dense-and-sparse encoding is presented as a key optimization for reducing the bitwidth of an outlier entry from 23 to 8 bits. How sensitive is this technique to the specific bitwidths chosen (4-bit inlier, 5-bit outlier)? Does the benefit and the 8-bit alignment hold if, for example, 3-bit or 2-bit quantization were used for inliers, which is a direction of active research? Please comment on the generality of this novel encoding scheme beyond the specific configuration presented.

Paper Title: LUT Tensor Core: A Software-Hardware Co-Design for LUT-Based Low-Bit LLM Inference Reviewer: The Guardian

---

Summary

This paper proposes "LUT Tensor Core," a software-hardware co-design to accelerate mixed-precision GEMM (mpGEMM) for low-bit LLM inference using a lookup table (LUT) based approach. The authors identify that existing LUT-based methods suffer from high overheads and propose a series of optimizations. On the software side, they introduce operator fusion for table precomputation, weight reinterpretation for table symmetrization, and table quantization. On the hardware side, they propose a bit-serial LUT unit with an "elongated" tiling shape to maximize table reuse. These components are integrated via a new set of "LMMA" instructions and a compilation stack.

While the paper addresses a relevant problem and presents several plausible optimizations, the evaluation methodology is beset by fundamental flaws and questionable assumptions. The headline claims of outperforming state-of-the-art commercial GPUs rely on an invalid cross-technology comparison, an unverified custom simulator, and modifications to the baseline GPU architecture. Consequently, the reported performance gains are not credible.

---

Strengths

1. Problem Formulation: The work correctly identifies a critical and underexplored problem: the lack of native hardware support for mpGEMM and the inefficiencies of both dequantization-based and conventional LUT-based software approaches.
2. Table Symmetrization Technique: The weight reinterpretation technique described in Section 3.1.2 (Page 5) to halve the LUT size by exploiting symmetry is a clever and sound optimization. It is a clear, self-contained contribution.
3. Well-Controlled Ablation Study: The comparison against a re-implemented UNPU baseline in Table 2 (Page 11) appears to be a well-controlled experiment. This analysis, which shows a 1.44x improvement in compute intensity and power efficiency, is the most believable result in the paper as it compares two designs under the same constraints.

---

Weaknesses

The paper's conclusions are built upon a foundation of severe methodological weaknesses, which I will detail below.

1. Fundamentally Invalid Cross-Technology Comparison: The central claims of the paper, summarized in Table 1 (Page 11), are derived by comparing the authors' proposed design, simulated on a 28nm process, against NVIDIA's A100 and H100 GPUs, which are built on 7nm and 4nm processes, respectively. The footnote (†) acknowledges this and claims the data is "normalized," but this normalization is insufficient and scientifically unsound. Moore's Law and Dennard scaling provide monumental, non-linear gains in performance, power, and area (PPA) that cannot be papered over by simple frequency scaling. Comparing a 28nm design to a 7nm one is not an apples-to-apples comparison; it is an apples-to-spaceships comparison. The resulting claims of 20.9x compute density improvement are therefore meaningless.

2. Reliance on a Custom, Unvalidated Simulator for End-to-End Results: The authors state in Section 4.4 (Page 8) that they developed a custom "tile-based simulator" for end-to-end evaluation because a validated simulator like Accel-Sim is too slow. This is a critical flaw. The performance of a complex system like a GPU depends heavily on the intricate interplay of the memory hierarchy, interconnects, and resource contention—factors that are notoriously difficult to model accurately. The authors provide a cursory validation in Figure 16 (Page 9) showing a ~5% error on a single layer, but this is insufficient to establish trust in a new, custom simulation tool for evaluating entire models. All the major end-to-end speedup claims in Figure 17 and Table 1 are based on this unverified tool, rendering them suspect.

3. Unjustified Architectural Modifications in Kernel-Level Simulation: In the kernel-level evaluation (Section 4.3, Page 8), the authors use Accel-Sim but introduce a "register capacity adjustment." They admit that "insufficient registers... restrict large tiling" and their reported performance gains are contingent on this modification. This means they are not evaluating their design on a stock A100 architecture, but on a hypothetical one with a larger register file that is conveniently tailored to their method's needs. This is a significant caveat that is not sufficiently highlighted and invalidates the direct comparison to the real A100's performance.

4. Incomplete Analysis of Proposed Hardware: The paper champions an "elongated" M2N64K4 tiling shape as optimal (Section 3.2.2, Page 6), arguing it maximizes table reuse. However, this argument is one-sided. Such a tiling shape has significant architectural implications for data movement. It requires broadcasting activation data across a wide array of 64 units, potentially creating a wiring and power bottleneck. Furthermore, the trade-offs in terms of register file port pressure and control logic complexity are completely ignored. Without a thorough analysis of these costs, the optimality of this tiling shape is an unsubstantiated claim.

5. Inconsistent and Overstated Claims: The abstract and introduction promise to unlock the "full potential" of LUT-based approaches, which are shown to be highly inefficient in the authors' own baseline tests (Figure 4, Page 4). The baseline LUT-GEMM even suffers from a "Seg. Error," which calls into question the quality and stability of the baseline software implementation they are improving upon. Furthermore, the claim that precomputation overhead is reduced to "almost zero" (Section 3.1.1, Page 5) is an exaggeration contradicted by their own results in Table 4 (Page 12), which show a remaining overhead of 2.5-2.6%. While small, it is not zero.

---

Questions to Address In Rebuttal

1. Please provide a rigorous justification for comparing a 28nm simulated design against 7nm/4nm fabricated hardware. How does your "normalization" method account for the non-linear scaling of transistor density, wire capacitance, and leakage power between these process nodes?
2. Regarding the kernel-level evaluation (Section 4.3), what exactly is the "register capacity adjustment"? Please quantify the increase in register file size per SM required to achieve your reported results and provide a PPA analysis of this modification. How would your performance compare to the baseline on a stock A100 without this change?
3. Can you provide a more detailed architectural validation of your custom end-to-end simulator? Specifically, how does it model the cache hierarchy (L1/L2), memory bandwidth contention from multiple SMs, and the on-chip network-on-chip (NoC)?
4. Please elaborate on the "Seg. Error" observed for the LUT-GEMM baseline in Figure 4. Is this a known bug in the work you are citing [53], or is it an artifact of your implementation? How can readers be confident in your speedup claims if the baseline is unstable?
5. Provide a quantitative analysis of the trade-offs of your proposed M2N64K4 tiling shape. Specifically, what is the area and power overhead of the broadcast network required for activations compared to a more square tile like 16x16x2?

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper presents LUT Tensor Core, a software-hardware co-design to accelerate mixed-precision GEMM (mpGEMM) operations, which are central to low-bit LLM inference. The authors identify that current hardware is ill-suited for mpGEMM (e.g., INT4 weights x FP16 activations), forcing inefficient dequantization-based workarounds. The proposed solution revives the lookup table (LUT) approach, but systematically addresses its traditional bottlenecks—namely table precomputation overhead and storage costs.

The core contribution is a holistic, full-stack solution. On the software side, they use compiler techniques like operator fusion to eliminate precomputation overhead and leverage weight reinterpretation to halve table storage. On the hardware side, they propose a simplified, bit-serial Tensor Core with an elongated tiling shape to maximize table reuse and flexibility. This is exposed to programmers via a new LMMA instruction set and integrated into a TVM-based compilation stack. The evaluation shows significant improvements in compute density and energy efficiency over conventional MAC-based designs and a 1.44x improvement over the state-of-the-art LUT-based accelerator, UNPU.

Strengths

1. A Genuine and Elegant Co-Design: The paper's primary strength is that it truly embodies the principle of software-hardware co-design. Rather than designing hardware in a vacuum, the authors intelligently partition the problem. The "hardware-unfriendly" tasks of table generation and optimization are offloaded to software and the compiler stack (Section 3.1, page 5), which dramatically simplifies the resulting hardware logic. This synergy is the key to their impressive PPA results. The weight reinterpretation to exploit symmetry (Figure 7) is a particularly clever example of this principle.

2. Addressing a Critical and Timely Problem: The work is situated at the heart of a major challenge in deploying AI: the prohibitive cost of LLM inference. As the community aggressively pushes toward sub-4-bit weight quantization (e.g., BitNet), the mismatch with existing hardware becomes a more severe bottleneck. This paper doesn't just identify the problem; it provides a concrete and well-reasoned architectural proposal to solve it. It offers a compelling alternative to the approach taken by major vendors like NVIDIA, which involves adding an ever-expanding set of dedicated narrow-precision MAC units.

3. Excellent Contextualization and Motivation: The authors do a superb job of motivating their work. Section 2.3 ("Gaps in Current LUT-based Solutions," page 4) clearly articulates why a naive LUT approach fails, setting the stage perfectly for their contributions. The paper connects to a rich history of architectural ideas—including bit-serial processing (like Stripes), tile-based DNN compilation (TVM, CUTLASS), and domain-specific accelerators—and synthesizes them into a novel solution for a modern problem.

4. Strong Empirical Analysis: The multi-level evaluation, from dot-product unit microbenchmarks (Section 4.2.1) to kernel-level simulations (Section 4.3) and detailed comparisons against prior work (Section 4.5), is comprehensive. The head-to-head comparison with UNPU (Table 2, page 11) is particularly valuable, as it grounds their claims against a known SOTA design. The ablation study in Table 2, showing the incremental benefit of each optimization, is excellent and clearly demonstrates the value of their co-design approach.

Weaknesses

1. Reliance on a High-Level Simulator for End-to-End Results: While understandable due to the infeasibility of using Accel-Sim for full models, the reliance on a custom, tile-based simulator for the end-to-end results (Section 4.4, page 8) introduces a degree of uncertainty. Although validated against real hardware (Figure 16), such simulators can miss subtle but important second-order effects related to memory contention, pipeline stalls, or control overhead. The impressive end-to-end speedups should be interpreted with this context in mind.

2. Limited Comparison with Emerging Native Hardware Support: The paper acknowledges the trend of native mpGEMM support in emerging architectures like NVIDIA's Blackwell (Section 6, page 12). However, the discussion is largely qualitative. A more in-depth analysis, even if theoretical, comparing the trade-offs of the flexible LUT-based approach versus a hardware architecture with dedicated FP4/FP6 MAC units would be highly insightful. For example, how does the area and power of the proposed LUT Tensor Core compare to an iso-throughput array of native MXFP4 MACs? This would help position the work more firmly within the future architectural landscape.

3. On-Chip Network Implications of Table Broadcasting: The software optimizations rely on a "precompute-once, broadcast-many" model. While this amortizes the computation, it introduces on-chip communication traffic as the generated LUTs must be distributed from a central compute resource (like vector units) to all the LUT Tensor Cores that need them. In a large-scale system with many such cores, this broadcast traffic could become a performance or power bottleneck. This aspect is not fully explored in the paper.

Questions to Address In Rebuttal

1. Could the authors elaborate on the potential sources of inaccuracy in their end-to-end simulator? Specifically, what architectural effects (e.g., interconnect contention, cache coherence for shared LUTs) are abstracted away, and how might these impact the reported performance gains on a real-world system-on-chip?

2. Could you provide a more direct comparative discussion on your LUT-based approach versus the path of adding dedicated narrow-precision format units, as seen in recent commercial GPUs? What are the fundamental trade-offs in terms of architectural flexibility (e.g., supporting non-standard formats like ternary weights), area efficiency, and design complexity?

3. Regarding the precomputation dataflow, have you analyzed the memory bandwidth and on-chip network traffic required to broadcast the precomputed tables to the distributed LUT Tensor Cores? At what scale (i.e., number of cores) might this communication become a limiting factor for performance or energy?

Reviewer: The Innovator (Novelty Specialist)

---

Summary

This paper presents LUT Tensor Core, a software-hardware co-design for accelerating low-bit Large Language Model (LLM) inference. The central problem addressed is the inefficiency of mixed-precision matrix multiplication (mpGEMM), where low-precision weights are multiplied by higher-precision activations. The authors propose a lookup table (LUT)-based approach that they claim overcomes the limitations of prior software and hardware LUT designs.

The core of their claimed contribution is a three-part co-design: 1. Software Optimizations: Employing operator fusion to absorb the LUT precomputation overhead, and using weight reinterpretation to exploit numerical symmetry, thereby halving the LUT storage requirements. 2. Hardware Architecture: A simplified, bit-serial LUT-based Tensor Core design that leverages an "elongated" tiling shape (high N, low M, low K) to maximize table reuse. 3. Compiler/ISA Support: A new LMMA instruction to expose the hardware to a tile-based deep learning compiler.

Strengths

The primary strength of this work lies not in the invention of a single new component, but in the holistic synthesis and intelligent partitioning of responsibilities between software and hardware. The authors correctly identify that naive hardware-centric LUT designs suffer from excessive overhead. By moving complex or redundant tasks—such as table precomputation and exploiting weight properties—into the software and compilation stack, they achieve a significant simplification and efficiency gain in the hardware itself.

The most notable element is the weight reinterpretation for table symmetrization detailed in Section 3.1.2 (page 5). While exploiting symmetry is a classic optimization principle, its specific application here to map unsigned integer weights to a symmetric signed representation to halve the LUT size and associated broadcast/multiplexer hardware is a clever and effective method. This re-partitioning demonstrates a clear co-design benefit: a software-level transformation directly enables a more compact and efficient hardware implementation.

Weaknesses

My primary concern is that the paper's claims of novelty are overstated, as the work is fundamentally an integration of several well-established techniques from different domains. The contribution appears to be more of a strong engineering effort in system integration rather than the introduction of fundamentally new concepts.

1. Constituent Techniques are Not Novel: The paper builds its "novel co-design" on a foundation of existing ideas:

   • Bit-Serial Processing: The use of a bit-serial datapath to handle flexible precisions (Section 3.2.1, page 6) is a well-known technique for area and energy efficiency in DNN accelerators. This dates back to work like Stripes (Judd et al., MICRO 2016), which the authors cite as [27]. Applying this to a LUT-based unit is a logical extension, not a conceptual breakthrough.
   • Operator Fusion: The fusion of the LUT precomputation kernel with preceding element-wise operations (Section 3.1.1, page 5) is a standard compiler optimization implemented in virtually all modern deep learning frameworks (including TVM, which they use). Its application here is expected, not novel.
   • LUT-based Computation: The core idea of using LUTs to accelerate DNNs, especially with low-bit weights, is not new. The authors themselves cite prior art such as UNPU [38] and Biqgemm [26].
   • Design Space Exploration for Tiling: The discovery of an "elongated" tiling shape (Section 3.2.2, page 6) is the result of a design space exploration (DSE). DSE is a standard methodology in hardware architecture. While the resulting insight is valuable for this specific design, the method is not novel, and optimized tiling shapes are a cornerstone of high-performance libraries like CUTLASS.

2. The "Delta" Over Prior Art is Incremental: The novelty rests almost entirely on the synthesis of these known techniques. The authors compare against UNPU [38] in Table 2 (page 11), showing a 1.44x improvement. This improvement is achieved by combining their optimizations. However, it is not clear that this represents a fundamental conceptual leap. For instance, the weight reinterpretation trick is clever, but it is an algebraic simplification that could, in principle, be applied to other LUT-based designs. The paper does not convincingly argue why this specific combination of known methods was non-obvious or paradigm-shifting. The work appears to be a very competent optimization of the LUT-based accelerator paradigm, rather than a reinvention of it.

Questions to Address In Rebuttal

1. The central claim is a "software-hardware co-design." However, the constituent parts (bit-serial processing, operator fusion, LUTs for DNNs, DSE for tiling) are all established techniques. Please articulate precisely what is the fundamental novel concept in your co-design, beyond the successful application and integration of these known methods to the mpGEMM problem.

2. The weight reinterpretation to exploit symmetry (Section 3.1.2) is the most compelling part of the software optimization. Was this technique previously proposed in the context of LUT-based accelerators for general mpGEMM (beyond simple binary/ternary networks)? Please clarify the delta between your method and standard techniques for handling signed/unsigned numbers in other computational paradigms.

3. The paper identifies an elongated tile shape (e.g., M2N64K4) as optimal. How general is this finding? Is this shape universally optimal for LUT-based mpGEMM, or is it an artifact of the specific activation (e.g., FP16/INT8) and weight (e.g., INT1/2/4) bit-width ratios evaluated? How does the optimal M:N:K ratio change as the activation bit-width approaches the weight bit-width?

4. The comparison with the "Conventional LUT" implementation in Figure 13 (page 7) suggests it has poor area scaling. What specific design assumptions were made for this "Conventional LUT" baseline? Does it also use bit-serial processing or is it a fully parallelized design, which would explain its large area? Clarifying this is crucial to fairly assess the novelty and benefit of your proposed hardware design.

Paper Title: AiF: Accelerating On-Device LLM Inference Using In-Flash Processing Reviewer: The Guardian

---

Summary

This paper proposes "Accelerator-in-Flash" (AiF), an in-flash processing (IFP) architecture designed to accelerate on-device Large Language Model (LLM) inference. The core contribution is a pair of flash-level optimizations: 1) "charge-recycling read" (cr-read), a technique to reduce latency for sequential wordline accesses, and 2) "bias-error encoding" (be-enc), a Vth state reconfiguration scheme to improve the reliability of pages storing LLM parameters. The authors claim that these techniques enable a 4x increase in internal flash bandwidth, which, when integrated into a full system, results in a 14.6x throughput improvement over baseline SSD offloading and even a 1.4x improvement over a high-end, in-memory (DRAM) system. The evaluation is conducted using a modified version of the NVMeVirt SSD emulator integrated with the llama.cpp inference engine.

---

Strengths

1. Well-Motivated Problem: The paper correctly identifies the memory capacity and bandwidth limitations as critical barriers to on-device LLM deployment. The analysis in Section 3, particularly Figure 4, effectively establishes the performance bottlenecks and reliability requirements that motivate the work.

2. Conceptually Sound Primitives: The two core technical ideas are well-conceived. The concept of cr-read (Section 4.2) leverages the known spatial locality of model parameter access to optimize the flash read sequence. Similarly, be-enc (Section 4.3) is a clever application of reconfiguring Vth states to create a heterogeneous reliability profile within a single TLC block, prioritizing the LLM data.

3. Holistic Approach: The work commendabley attempts to address both performance (bandwidth via cr-read) and reliability (error rates via be-enc) simultaneously. This is a crucial and often overlooked aspect of practical IFP system design.

---

Weaknesses

My primary concerns with this work lie in the significant gap between the proposed concepts and the evidence provided to support their practicality, scalability, and system-level viability. The evaluation relies on a chain of optimistic assumptions and limited validations that undermine the paper's extraordinary claims.

1. Validation of cr-read is Fundamentally Insufficient: The claim that cr-read is functionally sound and yields a 2.8x bandwidth improvement (Section 4.2.2, page 7) rests on SPICE simulations and experiments on a "fabricated CTF cell array" of 9x9 WLs/BLs (footnote 5, page 6). A 9x9 array is a laboratory-scale toy, not a proxy for a modern 3D NAND device with thousands of wordlines, complex peripheral circuitry, and significant parasitic capacitances. Real-world phenomena like read disturb, wordline-to-wordline coupling, and thermal effects, which are critical at scale, are not captured. Presenting this as sufficient validation for a production-level technique is a significant overstatement.

2. System-Level Side Effects of be-enc are Ignored: The be-enc scheme (Section 4.3) introduces performance and reliability asymmetry, degrading the non-LSB pages used for "general data." The paper dismisses this impact with a cursory analysis in Figure 18 (page 12), showing only a minor drop in random read IOPS. This analysis is critically incomplete. It fails to address:

   • Garbage Collection (GC) and Write Amplification: How does a flash translation layer (FTL) manage blocks containing both high-endurance LSB pages (for LLM data) and low-endurance CSB/MSB pages (for general data)? This heterogeneity would drastically complicate wear-leveling and GC, likely leading to increased write amplification and premature device wear, none of which is modeled or discussed.
   • Write/Erase Performance: The paper only analyzes the read path. Does reconfiguring the Vth states for be-enc impact program/erase times or program disturb characteristics? This is not mentioned.

3. The Reliability Premise of ECCLITE is Fragile: The entire justification for the lightweight ECCLITE decoder hinges on the characterization in Figure 13(b) (page 8), which shows a maximum of 9 bit errors per 1-KiB for be-enc LSB pages. The proposed ECCLITE corrects up to 10 bits. This leaves a safety margin of a single bit error. In real-world flash deployment, error rates are a distribution, not a fixed maximum. This razor-thin margin provides virtually no resilience against process variation, variable retention times, or higher-than-expected cell degradation near the device's end-of-life. A robust system would require a much larger ECC margin.

4. The Evaluation Model for Concurrency is Overly Optimistic: The claim of surpassing in-memory performance (1.4x) is predicated on the parallel host-AiFSSD execution model (Figure 15c, page 9). The NVMeVirt-based evaluation (Section 6.1) simulates the delay of the AiFSSD computation. However, it is highly unlikely that this model accurately captures the true overhead of the proposed fine-grained, tightly-coupled interaction. Each GEMV offload requires command submission, potential context switching, interrupt handling, and DMA setup/teardown. In a real system, this control-plane overhead for frequent, small operations could easily dominate the data-plane latency, nullifying the benefits of parallelism. The simulation appears to model an idealized best-case scenario.

5. Contradictory Claims on Bottlenecks: The authors motivate the work by claiming memory bandwidth is the primary bottleneck. However, their own scalability analysis (Figure 17b, page 12) shows sub-linear performance scaling when doubling or quadrupling internal bandwidth. They attribute this to "multiple vector arithmetic operations" and "NVMe protocol overheads." This is a crucial admission that their solution only addresses a part of the problem, and that other components (left on the host) become the new bottleneck. This weakens the central thesis that simply maximizing internal flash bandwidth is the definitive solution.

---

Questions to Address In Rebuttal

The authors must provide convincing evidence and clarification on the following critical points:

1. Regarding cr-read Validation: How can the results from an 81-cell array be extrapolated to a commercial multi-Gb flash die? Please provide analysis or data on how cr-read would behave in the presence of scaled-up parasitics, read disturb, and other real-world, dynamic flash array effects.

2. Regarding be-enc System Impact: Provide a detailed analysis of how be-enc would interact with a modern FTL. Specifically, quantify its impact on garbage collection efficiency, write amplification, and the endurance of blocks that mix IFP and general-purpose data.

3. Regarding ECCLITE Reliability: Please justify the decision to use an ECC scheme with only a 1-bit error correction margin over your characterized maximum. What is the expected failure rate of this scheme at the device's certified P/E cycles and retention period, considering statistical variations in error rates?

4. Regarding the Concurrency Model: Can you provide evidence that your NVMeVirt timing model accurately accounts for the full software stack and hardware overhead (interrupts, DMA, protocol overhead) of the frequent host-SSD synchronization required by your parallel execution scheme? Please break down the latency components of a single offloaded GEMV operation in your model versus a real system.

5. Regarding Performance Scaling: Given the admitted sub-linear scaling, what is the theoretical peak throughput of the AiF system as internal bandwidth approaches infinity? At what model size do the non-GEMV operations left on the host become the primary bottleneck, rendering further flash acceleration ineffective?

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper addresses the critical challenge of running large language models (LLMs) on personal devices with limited DRAM. The authors identify the primary bottleneck of existing SSD offloading techniques: the low external read bandwidth of flash storage, which severely limits token generation rates for memory-bound LLM inference.

The core contribution is Accelerator-in-Flash (AiF), an in-flash processing (IFP) architecture that moves the dominant matrix-vector multiplication (GEMV) operations directly into the NAND flash chips. This approach is designed to leverage the immense aggregate internal bandwidth of the chips, bypassing the external channel and PCIe bottlenecks. The novelty of this work lies not just in proposing IFP for AI, but in developing two co-designed, flash-aware techniques that make it practical for the unique demands of LLMs: 1. Charge-Recycling Read (cr-read): A novel read command that speeds up sequential reads within a block by eliminating redundant precharge and discharge steps, boosting effective read bandwidth. 2. Bias-Error Encoding (be-enc): A clever VTH state encoding scheme that makes one page type (LSB) exceptionally fast and reliable at the cost of others, allowing LLM parameters to be stored with high fidelity while enabling a highly compact on-chip error correction engine (ECCLITE).

Through a comprehensive evaluation using a full-system simulator integrated with the llama.cpp engine, the authors demonstrate that AiF can provide a 14.6x throughput improvement over conventional SSD offloading and even surpass a high-end in-memory system by 1.4x, all while significantly reducing the host memory footprint.

Strengths

1. High Potential Impact and Significance: This work addresses a timely and significant problem. The inability to run large, capable LLMs locally on consumer devices is a major barrier to privacy, low latency, and offline accessibility. AiF presents a compelling hardware-based solution that could fundamentally alter the landscape of edge AI. Rather than an incremental improvement, this work proposes a system that could enable models (e.g., 40B+ parameters) that are currently infeasible on personal devices to run at interactive speeds. This is exactly the kind of ambitious, problem-driven research the community needs.

2. Excellent Cross-Layer Co-Design: The true strength of this paper lies in its vertical integration of insights from the application layer down to the device physics layer. The authors recognize that LLM inference is (a) dominated by GEMV on static weights and (b) highly sensitive to bit errors (as shown in Figure 4b, page 4). Instead of a generic accelerator, they have designed cr-read and be-enc by fundamentally rethinking the NAND flash read protocol itself to meet these specific requirements. This deep understanding, connecting algorithmic needs to the manipulation of VTH states (Section 4.3, page 7) and read sequence timing (Section 4.2, page 5), is the hallmark of outstanding systems research.

3. Holistic and Plausible System Integration: The authors go beyond the core accelerator idea and consider the full system stack. They outline the necessary extensions to the NVMe protocol (aif_post, aif_gemv), discuss the system software and application-level requirements (Section 5.2, page 9), and even architect a parallel execution model to overlap host and AiFSSD computation (Figure 15c, page 9). The evaluation, which uses a modified full-system SSD emulator (NVMeVirt) and a real inference engine (llama.cpp), lends significant credibility to their performance claims. This end-to-end thinking makes the proposed system feel less like a theoretical concept and more like a blueprint for a real-world product.

Weaknesses

While the core ideas are strong, the paper could benefit from addressing the following points, which are less flaws in the work itself and more contextual limitations.

1. Practical Path to Adoption: The proposed changes are significant, requiring modification to both the internal logic of NAND flash chips and the firmware of the SSD controller. This represents a substantial re-architecting effort for semiconductor manufacturers who are typically conservative. While the paper argues for the feasibility of the required changes (e.g., modifying timer codes, leveraging existing dynamic VTH configuration), the business and engineering inertia to implement such a specialized feature is immense. The work would be stronger if it acknowledged this difficult path to commercialization more directly.

2. Handling Dynamic Data and Competing Workloads: The proposed system is exquisitely optimized for the "write-once, read-many" nature of LLM parameters stored in dedicated "IFP blocks." However, modern SSDs are general-purpose devices. The paper mentions that garbage collection (GC) in a footnote (footnote 9, page 10) but does not fully explore the performance implications. If a user is performing heavy I/O operations concurrently with LLM inference, how would background SSD tasks like GC and wear-leveling, which might need to move IFP blocks, impact inference latency and consistency? This interaction between the specialized "compute" workload and general storage tasks is a critical aspect of any practical in-storage processing system.

3. Limited Operational Scope: The work rightly focuses on GEMV as the primary bottleneck. However, other operations in an LLM (e.g., Softmax, LayerNorm) still run on the host, requiring data to move back and forth. The parallel execution scheme (Section 5.1) mitigates this, but as model architectures evolve, the Amdahl's Law effect from the non-accelerated portions could become more significant. This is not a weakness of the current work but a natural question about its extensibility and future-proofing.

Questions to Address In Rebuttal

1. Regarding Practical Implementation: Can the authors elaborate on the non-recurring engineering (NRE) cost and complexity for a NAND manufacturer to implement cr-read and be-enc? Are these changes that could be accomplished primarily through microcode/firmware patches to the on-chip scheduler and voltage generators, or do they require significant redesign of the analog peripheral circuitry?

2. Regarding System Interference: Could you elaborate on the performance impact of background garbage collection on a running LLM inference task? If an IFP block becomes the target of a GC operation, would this introduce significant, unpredictable latency (i.e., jitter) into the token generation pipeline? How does your proposed system manage or isolate the compute-dedicated blocks from the effects of general storage I/O?

3. Regarding Model Management: The aif_post command is used to initially place the model into the optimized layout. What is the performance overhead of this setup process for a large model (e.g., a 70B parameter model)? How does the system envision handling model updates or switching between different fine-tuned versions of a model, which are common user scenarios?

Reviewer: The Innovator (Novelty Specialist)

Summary

This paper proposes Accelerator-in-Flash (AiF), an in-flash processing (IFP) architecture designed to accelerate on-device large language model (LLM) inference. The authors correctly identify that existing SSD offloading techniques are bottlenecked by external I/O bandwidth and that prior IFP solutions fail to meet the unique high-bandwidth and high-reliability demands of LLMs. The core of the proposed contribution is not the general idea of IFP, but rather two specific, low-level flash read techniques designed to make IFP practical for this workload. The first, "charge-recycling read" (cr-read), modifies the NAND read sequence to skip precharge and discharge steps for sequential reads within a block, boosting read speed. The second, "bias-error encoding" (be-enc), reconfigures the threshold voltage (VTH) state mapping in TLC NAND to create an ultra-reliable and fast LSB page type, where LLM parameters are exclusively stored. This increased reliability allows for a lightweight on-chip ECC decoder (ECCLITE), mitigating the area and power overhead that would otherwise make IFP infeasible. The authors evaluate this system via simulation, demonstrating significant throughput gains over both baseline SSD offloading and in-memory inference.

Strengths

The primary strength of this work lies in its specific, non-obvious proposals for modifying the physical operation of NAND flash to serve a high-level application need. The novelty is not in the high-level concept but in the enabling mechanisms:

1. Novel Co-design: The paper presents a compelling cross-layer co-design. Instead of simply positing an accelerator inside a flash chip, the authors have identified the fundamental physical limitations (read latency, error rates, ECC cost) and proposed concrete solutions (cr-read, be-enc) at the device level. The tight coupling between be-enc reducing error rates and ECCLITE reducing hardware cost is particularly novel and well-conceived.

2. Specific Low-Level Contributions: Both cr-read and be-enc appear to be novel in their specific formulation and application. While the principle of optimizing sequential access or leveraging the differing reliability of pages in MLC/TLC flash may have conceptual precedents, the proposed mechanisms are distinct:

   • cr-read (Section 4.2, page 5) as a specific modification to the read state machine (bypassing precharge/discharge) is a clever circuit-level optimization.
   • be-enc (Section 4.3, page 7) goes beyond passively using the LSB page; it proposes actively reconfiguring the VTH state encoding (from (2,3,2) to (1,3,3) coding) to intentionally create a privileged, high-performance page type specifically for IFP data. This is a significant conceptual step beyond prior work that merely partitions data based on existing page characteristics.

3. Problem-Driven Innovation: The work is well-motivated. The authors clearly establish in Section 3.3 (page 4) why existing IFP is insufficient for LLMs, pointing to the dual challenges of raw bandwidth and the prohibitive cost of robust on-chip ECC. Their proposed solutions directly and elegantly address these two specific, well-articulated problems.

Weaknesses

From a novelty perspective, the primary weakness is that the paper frames its contribution under the broad umbrella of "In-Flash Processing," a concept with extensive prior art. The true novelty is more nuanced and lies deep within the flash controller and cell programming logic, which could be emphasized more clearly.

1. Incremental Nature of cr-read: The core idea of cr-read—reusing charged line voltages to accelerate subsequent accesses to a physically local region—is a common optimization principle in hardware design. While its application to the NAND read sequence is new in this context, it feels like an incremental, albeit clever, engineering optimization rather than a fundamentally new concept. The paper would be strengthened by a more thorough comparison to any existing "fast sequential read" modes or similar optimizations that may exist in NAND flash manufacturer datasheets or patents, which often contain proprietary, non-public access modes.

2. Overstated Novelty of the General Approach: The paper's narrative implies that IFP for ML is a new direction. However, as the authors' own related work section (Section 7, page 12) points out, numerous works have explored in-storage and in-flash processing for DNNs and other data-intensive workloads ([38], [45], [78]). The key distinction of AiF is its focus on LLM inference and the specific physical-level techniques to overcome the associated challenges, a distinction that should be made more central to the paper's claims.

Questions to Address In Rebuttal

1. Prior Art for cr-read: Can the authors elaborate on the novelty of the cr-read technique relative to prior art in low-level flash memory operation? Are there functionally similar "burst" or "fast sequential" read modes that have been proposed or implemented by memory manufacturers, even if not for the purpose of general-purpose computation? The concept of skipping reset phases for sequential operations is not new in principle; please clarify what makes this specific application to the NAND read sequence fundamentally novel.

2. Prior Art for be-enc: The concept of pages within a multi-level cell exhibiting different reliability characteristics is well-established. Prior work has proposed leveraging this by, for example, placing critical metadata on more reliable pages. The core novelty of be-enc seems to be the active reconfiguration of VTH levels to further enhance the reliability of LSB pages for IFP workloads. Can you confirm if this specific idea—dynamically changing the VTH encoding scheme on a per-block basis to create a privileged data partition for an IFP accelerator—has been proposed before?

3. Complexity vs. Practicality: The proposed modifications, especially cr-read, require altering the fundamental state machine of a NAND flash chip's read path. This represents a significant deviation from the standard ONFI/Toggle interface and would require deep co-design with a NAND manufacturer. Given the extremely high cost and risk associated with modifying this core IP, is the proposed solution a purely academic exploration, or do the authors see a realistic path to adoption? The justification for novelty must also consider the feasibility of the proposed complexity.

Reviewer: The Guardian

---

Summary

This paper introduces LIA, a framework for accelerating single-GPU LLM inference by leveraging cooperative computation between a GPU and a modern, AMX-enabled Intel CPU. The central thesis is that recent advancements in CPU matrix multiplication capabilities, specifically Intel's AMX, are significant enough to fundamentally alter the trade-offs in CPU-GPU offloading strategies. The authors propose an offloading algorithm that systematically determines which model sublayers to execute on the CPU versus the GPU to minimize latency or maximize throughput. Additionally, they introduce a CXL-based memory offloading policy to expand capacity for throughput-driven scenarios. The paper presents substantial performance improvements over existing frameworks like FlexGen and a CPU-only baseline (IPEX).

While the paper addresses a timely and important problem, its conclusions rest on a series of questionable methodological choices, most notably the comparison against baselines that are not optimized for the very hardware LIA is designed to exploit. The significant performance claims are therefore a conflation of algorithmic novelty and an underlying hardware advantage that is not fairly accounted for in the baseline comparison.

---

Strengths

1. Timely Exploration of New Hardware: The paper provides a valuable and, to my knowledge, one of the first comprehensive microbenchmark analyses of Intel's AMX performance for realistic LLM workloads (GEMM and GEMV). The characterization in Section 4 and Figure 5 is an important contribution to the community, quantifying the substantial leap from AVX512 and situating AMX performance relative to several generations of NVIDIA GPUs.

2. Systematic Offloading Policy: The formulation of the compute-offloading decision as an optimization problem (Section 5.1) is methodologically sound. The cost model, while simple, holistically considers key system parameters like compute throughput and memory/PCIe bandwidths to derive an optimal policy, moving beyond the heuristic-based approaches of prior work.

3. Well-Reasoned CXL Integration: The memory-offloading policy for CXL is logical and well-motivated. The authors correctly identify that for parameter transfers to the GPU, CXL bandwidth can be sufficient to hide behind the PCIe bottleneck (Observation-1, Section 6), while simultaneously recognizing that CXL's high latency is detrimental to CPU-bound computation, leading to their policy of keeping the KV cache in DDR.

---

Weaknesses

1. Fundamentally Unfair Baseline Comparison: The central weakness of this paper is the evaluation against inappropriate baselines. The primary competitor, FlexGen, is designed for CPUs with AVX instruction sets. The authors evaluate it on a Sapphire Rapids CPU but do not appear to have modified it to utilize AMX. Consequently, the reported speedups of up to 5.1x-19x are not solely attributable to LIA's "cooperative framework" but are heavily skewed by comparing an AMX-native framework to an AVX-based one. The paper fails to disentangle the gains from their scheduling algorithm versus the raw hardware advantage. A fair comparison would require an AMX-enabled version of FlexGen or another strong baseline that also leverages AMX. The comparison to IPEX (CPU-only) is a strawman for any model that could even partially fit on a GPU.

2. Over-reliance on Analytical Models and Simulation: A significant portion of the paper's results, particularly for large batch sizes and throughput-oriented scenarios, are derived from an analytical model rather than direct measurement (as indicated by stars in Figure 11, page 10). The authors state this model has an "average error of 12%" (Section 7, page 8), which is a non-trivial margin of error that casts doubt on the precision of the claimed speedups. Furthermore, the multi-GPU comparison in Section 7.8 relies on comparing their system (partially evaluated with their own model) against a simulation of a DGX-A100 system. Conclusions drawn from comparing a model to a simulation are speculative at best.

3. Unsupported Claims of Generalizability: In Section 7.7, the authors claim LIA's optimizations generalize to other model architectures like GPT, Llama2, and Bloom. This claim is substantiated only by results from their analytical model, not empirical evidence. Performance characteristics can vary significantly across model families due to differences in layer normalization, activation functions, and attention mechanisms. Without measured data on real hardware for at least one other major model family, this claim is unsubstantiated and constitutes an overstatement of the work's demonstrated contributions.

4. Ambiguity Regarding Software Stack Maturity: Footnote 4 on page 5 states that "the recently-introduced AMX libraries are less optimized." This is a critical caveat that is buried and not discussed in the main text. It raises serious questions about the replicability and robustness of the foundational microbenchmarks in Figure 5. If the libraries are immature, the reported performance may not be representative of the hardware's true capability, or conversely, the gains over AVX512 could be even larger. This uncertainty undermines the foundation upon which the entire paper is built.

---

Questions to Address In Rebuttal

1. Please justify the choice of an AVX-based FlexGen as the state-of-the-art baseline on an AMX-enabled processor. Did you make any attempt to build a stronger, AMX-aware baseline to ensure a fair comparison? If not, how can you deconvolve the performance gains of your algorithm from the underlying hardware performance differential between AMX and AVX?

2. The paper relies heavily on an analytical model for many key results. Can you provide a sensitivity analysis for this model? How do the model's predictions change with variations in its core assumptions (e.g., PCIe bandwidth, CPU memory latency)? Why were direct measurements not feasible for the starred configurations in Figure 11?

3. To support the claim of generalizability (Section 7.7), please provide measured, end-to-end performance data for LIA on at least one non-OPT model, such as Llama2-70B, and compare it against FlexGen on the same model.

4. Regarding Footnote 4, please elaborate on the maturity of the AMX software stack used in your evaluation. How might the presented results and the optimal offloading policies change as these libraries mature and AMX performance presumably improves? Does this not risk making your derived "optimal" policies obsolete?

Reviewer Persona: The Synthesizer (Contextual Analyst)

---

Summary

This paper presents LIA, a framework for accelerating Large Language Model (LLM) inference on a single GPU by leveraging cooperative computation with modern, powerful CPUs. The work is built upon a crucial and timely insight: the introduction of specialized matrix multiplication hardware in recent Intel CPUs (Advanced Matrix Extensions, or AMX) has fundamentally altered the performance landscape, making the CPU a viable computational partner rather than just a passive memory host.

The authors first provide a rigorous performance characterization of AMX-enabled processors (Sapphire Rapids and Granite Rapids), demonstrating that their matrix math throughput is competitive with some GPUs and a significant fraction of modern ones. Building on this, LIA introduces a systematic compute-offloading algorithm that partitions LLM sublayers between the CPU and GPU to optimize for end-to-end latency or throughput. Finally, the paper explores the use of CXL memory to cost-effectively expand system memory for large-batch, throughput-driven scenarios, proposing a simple but effective memory tiering policy. The experimental results show substantial improvements in latency, throughput, and energy efficiency over existing CPU-only and CPU-GPU collaborative frameworks.

Strengths

1. Central Thesis is Timely and Insightful: The core contribution of this work lies in identifying and exploiting a major shift in the hardware landscape. For years, the systems community has treated CPUs in ML workloads as slow, general-purpose engines, primarily useful for their large memory capacity. This paper compellingly argues that this assumption is now obsolete due to accelerators like AMX. The microbenchmarks presented in Section 4 (pages 4-5) provide the crucial evidence for this claim and serve as a strong foundation for the entire paper. This is an excellent example of systems research that is deeply connected to hardware evolution.

2. Holistic and Well-Designed System: LIA is more than just a proof-of-concept for AMX. The authors have designed a complete system that addresses the problem holistically. The formulation of the offloading decision as an optimization problem (Section 5.1, page 6) is principled and provides a generalizable framework. The inclusion of system-level optimizations like efficient GPU memory usage and overlapping communication (Section 5.2, page 6) shows attention to practical implementation details. The integration of CXL (Section 6, page 7) is forward-looking and addresses the next major bottleneck: memory capacity for large-batch inference.

3. Connects Disparate Hardware Trends: A key strength of this work is its ability to synthesize two independent but complementary hardware trends: the rise of on-CPU acceleration (AMX) and the emergence of disaggregated, tiered memory (CXL). The paper demonstrates a synergistic relationship where AMX makes the CPU a more valuable compute resource, and CXL provides the memory capacity needed to feed it in high-throughput scenarios. This provides a powerful architectural blueprint for future cost-effective inference servers.

4. Significant and Well-Documented Performance Gains: The results are not merely incremental. Achieving up to 5.1x higher throughput and 19x lower latency compared to the state-of-the-art single-GPU offloading framework (FlexGen) is a very strong result. This clearly validates that rethinking the role of the CPU is not just an academic exercise but a source of major real-world performance improvements.

Weaknesses

While the work is strong, its framing could be broadened to better contextualize its contributions within the wider landscape of heterogeneous computing.

1. Implicit Dependency on High-End CPUs: The paper's success hinges on the availability of top-tier Intel Xeon processors with a high core count. While this is the correct platform to demonstrate the maximum potential, it leaves open the question of where the "break-even" point is. A discussion on the performance sensitivity to CPU core count and AMX capabilities would help readers understand the cost-performance trade-offs more broadly. The current work presents an architecture that pairs a high-end GPU with a high-end CPU, which is still a significant investment.

2. Limited Discussion on Architectural Generality: The paper is naturally focused on the Intel AMX + NVIDIA GPU ecosystem. However, the core idea of a powerful CPU partner is not unique to Intel. AMD is developing similar capabilities, and the ARM world has its own vector and matrix extensions (SVE, SME). The discussion of Grace Hopper in Section 8 (page 12) is a good start, but the paper would be even more impactful if it framed its optimization framework (Equation 1, page 6) as a general model for heterogeneous systems, where AMX is just one instance of a powerful host processor.

3. Static Memory Tiering Policy: The proposed CXL policy—storing model parameters on CXL and intermediate values (like the KV cache) on DDR—is pragmatic and effective. However, it is a static policy. This is a missed opportunity to discuss the potential for more dynamic, access-pattern-aware data placement, which is a rich area for future research that this work directly enables.

Questions to Address In Rebuttal

1. Could the authors comment on the sensitivity of LIA's performance to the CPU's core count? The experiments use high-end 40-core and 128-core CPUs. How would the optimal offloading policy and resulting performance change on a lower-end, say 16-core, AMX-enabled server CPU? This is critical for understanding the true cost-efficiency of this approach.

2. The optimization framework in Section 5.1 is based on performance parameters of the specific CPU and GPU. How portable is this framework conceptually? If one were to build a similar system using an AMD CPU with its own on-chip AI engine, would the framework apply directly with only new performance measurements, or would it require fundamental changes?

3. The paper makes a strong case for cost-efficiency against multi-GPU setups. Could you provide a brief TCO (Total Cost of Ownership) sketch? A modern H100 GPU and a Granite Rapids CPU still represent a very expensive server. How does the estimated cost per token served compare to, for example, a server with two last-generation A100 GPUs, which might have a similar acquisition cost?

Review Form: The Innovator (Novelty Specialist)

---

Summary

This paper introduces LIA, a framework for accelerating single-GPU LLM inference for models that exceed GPU memory. The authors' contribution is composed of three primary elements. First, they provide a timely performance characterization of Intel's Advanced Matrix Extensions (AMX) on recent CPUs, establishing that modern CPUs are computationally powerful enough to be viable partners in LLM inference, not just data hosts. Second, leveraging this insight, they propose a novel, systematic compute-offloading algorithm. This algorithm formulates the decision of which Transformer sub-layers to execute on the CPU versus the GPU as an optimization problem, creating a dynamic policy based on batch size and sequence length. This contrasts with prior art that used static, heuristic-based offloading. Third, the paper introduces a specific memory-offloading policy for systems with both DDR and CXL memory, proposing to store model parameters on CXL while keeping latency-sensitive intermediate data (like the KV cache) on DDR to mitigate the performance penalty of CXL on CPU-side computation.

---

Strengths

The primary strength of this paper lies in its novel reframing of the CPU's role in CPU-GPU cooperative inference. My evaluation identifies the following genuinely new contributions:

1. Shift from Static to Dynamic Offloading Policy: The central novel idea is the move away from the rigid offloading policies seen in prior work. Frameworks like FlexGen [43] and FastDecode [23] identified a single, structurally-determined "best" candidate for offloading (the attention scoring sublayer) based on a general principle (low arithmetic intensity). LIA's contribution is the formulation of this choice as a formal optimization problem (Equation 1, page 6). The policy vector p and the cost model that considers system parameters, B, and L to find an optimal configuration is, to my knowledge, a new and more sophisticated approach for this problem domain. It correctly intuits that the "best" sublayer to offload is not fixed, but is a function of the workload.

2. A Motivated, Non-Trivial CXL Memory Policy: The use of CXL for memory expansion is not in itself novel. However, the proposed policy is not a naive extension. The novelty lies in the insight derived from "Observation-2" (page 8), which correctly identifies that the CPU's own compute performance is severely hampered by CXL's higher latency. The resulting policy—store parameters on CXL (primarily for GPU transfer via DMA, which is less latency-sensitive) and keep CPU-accessed data (KV cache) on DDR—is a simple, elegant, and novel solution tailored specifically to the performance characteristics of their cooperative computing model. This is a clear advancement over simply treating CXL as a generic memory pool.

3. Foundational Insight on Modern CPU Capability: While performance studies are common, the specific analysis in Section 4 (page 4) serves as the novel insight that motivates the entire work. Prior art has operated under the assumption that CPUs are orders of magnitude slower than GPUs, thus limiting offloading to only the most trivial compute tasks. By demonstrating that AMX-enabled CPUs achieve throughput that is competitive with previous-generation GPUs (e.g., SPR-AMX vs. P100/V100 for certain workloads in Figure 5, page 5), the authors provide the foundational evidence needed to justify their more ambitious and dynamic offloading algorithm. This re-evaluation of the hardware baseline is a key part of the paper's novelty.

---

Weaknesses

While the core ideas are novel in their application, their conceptual origins are not without precedent in adjacent fields. The paper's claims could be strengthened by acknowledging this and positioning the work more precisely.

1. Framing of the Optimization Model: The concept of using a cost model to partition a workload across heterogeneous processors is a classic problem in systems research. The authors present their algorithm in Section 5.1 (page 6) as a core contribution, which it is. However, the novelty is not in the invention of cost-based scheduling, but in its specific formulation for the unique data flow and computational characteristics of Transformer sub-layers. The paper would be more intellectually honest if it contextualized its model within the broader history of heterogeneous scheduling to more sharply define its specific contribution.

2. Simplicity of the CXL Policy: The CXL policy, while effective and well-motivated, is ultimately a binary, static decision. It represents a single, clever data placement rule rather than a comprehensive memory management framework. This is not a fatal flaw, but it does limit the scope of the novelty. The contribution is a specific, useful heuristic, not a general theory for tiered memory management in LLM inference.

---

Questions to Address In Rebuttal

1. On the Novelty of the Cost Model: Please explicitly differentiate the novelty of your formulation in Section 5.1 (page 6) from the general, well-established field of cost-based task scheduling on heterogeneous systems. What specific characteristics of the LLM inference problem make your model a non-trivial application of these classic ideas?

2. On the Generalizability of the Offloading Policy: The optimization is performed over a fixed set of six sub-layers derived from the OPT architecture. The novelty of the framework appears tied to this specific structure. How does your method generalize to architectures with fundamentally different layer structures, such as Mixture-of-Experts (MoE) models where the feed-forward network is replaced by a sparse routing mechanism? Is the novel contribution the specific model for OPT, or a more general meta-framework for constructing such models?

3. On the Limits of the CXL Policy: The proposed CXL policy is a static partitioning of data types (parameters vs. KV cache). Did the authors consider or evaluate more dynamic policies? For example, in a scenario with an extremely long context length, the KV cache itself could become enormous. Would a tiered policy, where older KV cache tokens are migrated from DDR to CXL, be feasible? What are the conceptual boundaries of your proposed policy's novelty?

Paper Title: Enabling Ahead Prediction with Practical Energy Constraints Reviewer Persona: The Guardian (Adversarial Skeptic)

---

Summary

The paper addresses the well-known performance bottleneck of multi-cycle branch predictors by proposing an energy-efficient "ahead prediction" mechanism. The authors' core thesis is that the number of "missing history" patterns between the start of a prediction and its required use is far smaller than the theoretical maximum of 2^N. They leverage this insight to modify a TAGE predictor, adding a secondary tag to disambiguate between these few likely patterns. This avoids the exponential energy cost of prior-art ahead predictors which speculatively generate predictions for all 2^N paths. The authors claim their design achieves a 4.4% IPC improvement with only a 1.5x increase in per-prediction energy, which they argue is "very much viable" in contrast to the 14.6x overhead of previous proposals.

Strengths

1. Strong Core Insight: The paper's primary contribution rests on the analysis presented in Section 3 (page 4), particularly Figure 2. The empirical demonstration that the number of observed missing history patterns is typically very low (1-2) for a given ahead PC and history is a valuable and well-articulated insight. This observation correctly identifies the primary source of inefficiency in prior ahead prediction schemes.

2. Well-Motivated Problem: The authors effectively motivate the problem with a clear analysis of the performance impact of predictor latency (Figure 1, page 1) and a quantitative argument against the energy scaling of existing ahead prediction solutions. The problem is both timely and significant for high-performance processor design.

Weaknesses

My primary concerns with this paper lie in the overstatement of its practical viability, weaknesses in the evaluation methodology, and an insufficient analysis of the mechanism's potential failure modes.

1. Insufficient Energy Analysis and Overstated Viability: The central claim of practicality hinges on the 1.5x energy figure. This figure, as described in Section 4.5 (page 7), is derived from Cacti simulations focused primarily on table reads. The analysis appears to downplay the energy cost of the duplicated selection logic. For a 5-bit secondary tag, this implies generating 32 predictions in parallel, requiring 32 sets of comparison and selection logic. While the authors claim this is "significantly less than the table reads," this assertion is not substantiated with synthesis data or a more detailed model. The term "very much viable" (Abstract, page 1) is premature without a more rigorous power analysis. A 1.5x increase in the energy of a component that already consumes 3-4% of total core power is non-trivial and requires stronger justification.

2. Unconvincing Explanation for Performance Degradations: The results in Figure 12 (page 10) show that several benchmarks, notably gcc, omnetpp, and xalancbmk, experience performance degradation or negligible improvement despite, in some cases, a reduction in MPKI. The authors attribute this to the loss of "beneficial wrong-path prefetching." This is a common but often unsubstantiated explanation for performance anomalies related to branch prediction. The paper provides no direct evidence (e.g., L1/L2 cache miss rates, memory bandwidth analysis) to support this claim. Without such evidence, this explanation is speculative and fails to address whether another unaccounted-for microarchitectural interaction or timing artifact is responsible for the performance loss, thus questioning the general applicability of the proposed technique.

3. Evaluation on an Overly Aggressive Baseline: The evaluation is performed on a 16-wide out-of-order core (Table 2, page 9). Such a wide machine is exceptionally sensitive to front-end stalls, which will disproportionately inflate the benefits of any mechanism that improves front-end throughput. The 4.4% IPC gain reported is likely an optimistic upper bound. The paper lacks a sensitivity study to show how this benefit scales on more conventional 8-wide or 6-wide cores, where back-end pressures might diminish the relative importance of predictor latency.

4. Potential Underestimation of Runtime Conflict Issues: The design's efficacy relies on the offline analysis (Section 3) that few missing history patterns exist. However, this static view may not capture dynamic program behavior. During program phase transitions, a burst of previously unseen patterns could emerge, leading to a high degree of conflict on the secondary tag. This would cause entries to be promoted to higher-latency TAGE tables unnecessarily, as noted in Section 4.4 (page 6), effectively polluting the predictor. The analysis in Table 1 shows only a small delta in misprediction rate, which is difficult to interpret without the absolute baseline rates and an analysis of worst-case behavior.

Questions to Address In Rebuttal

1. Can the authors provide a more detailed breakdown of the energy model from Section 4.5? Specifically, what is the estimated energy contribution of the duplicated selection logic (for 32 parallel predictions) relative to the table read energy, and how was this estimated? How robust is the 1.5x claim to a more complete energy model?

2. Regarding the performance losses on gcc, omnetpp, and xalancbmk (Figure 12), please provide quantitative evidence to support the "beneficial wrong-path prefetching" hypothesis. For instance, can you show that the baseline configuration exhibits a lower L2 miss rate or reduced memory latency for these benchmarks compared to your proposed design?

3. How do the reported IPC improvements change on a less aggressive core, for instance, an 8-wide machine with proportionally scaled execution resources? This is critical for understanding the technique's relevance beyond hero-level processor designs.

4. The argument that conflicts are minimized (Section 4.4) is based on average-case behavior. Have you analyzed the predictor's behavior during pathological cases, such as program phase changes, where the number of active missing history patterns might spike? How quickly does the predictor adapt, and what is the transient performance impact?

5. Your baseline is a 3-cycle TAGE. The state-of-the-art TAGE-SC-L predictor includes Statistical Corrector (SC) and Loop (L) components. While you argue pipelining them is expensive (Section 5.5), a complete evaluation should compare your ahead-pipelined TAGE against a non-ahead, 3-cycle TAGE-SC-L baseline. What is the performance of TAGE-SC-L relative to TAGE in your framework, and how does your 4.4% gain compare to that?

Reviewer Persona: The Synthesizer (Contextual Analyst)

Summary

This paper addresses the well-known and critical problem of multi-cycle branch predictor latency in high-performance processors. While the industry standard multi-level prediction scheme mitigates this, it suffers performance stalls when the simple and overriding predictors disagree. "Ahead prediction," which starts a prediction for a future branch early, is a known alternative but has been considered impractical due to the exponential growth in energy required to speculatively evaluate all 2^N possible paths for N skipped intermediate branches.

The authors' core contribution is an empirical observation that fundamentally challenges this assumption of exponential complexity. They demonstrate that for a given control flow history, the number of unique "missing history" patterns that actually manifest at runtime is extremely small—often just one or two, rather than the theoretical maximum.

Based on this insight, they propose a practical ahead predictor design. By modifying the TAGE predictor to include a "secondary tag" corresponding to the specific missing history pattern, they can distribute the predictions for the few observed paths across the existing TAGE tables. This approach avoids the need to read out an exponential number of entries, scaling the per-prediction energy linearly (a 1.5x increase) instead of exponentially (a 14.6x increase in prior proposals). The result is a design that achieves a significant 4.4% IPC improvement, capturing much of the ideal performance benefit of a zero-latency predictor while remaining energy-viable.

Strengths

1. Reframing a Foundational Problem: The paper's primary strength lies in its elegant reframing of the ahead prediction challenge. Instead of accepting the theoretical 2^N complexity as a given, the authors question its practical relevance. The analysis in Section 3 (particularly Figure 2, page 3) is the heart of the paper, providing a strong empirical foundation for their entire approach. This is an excellent example of how data-driven insights can unlock progress on long-standing architectural problems.

2. Elegant and Practical Mechanism: The proposed 2-tag TAGE modification is a clever, non-invasive solution. It intelligently leverages the existing structure and allocation logic of a state-of-the-art predictor (TAGE) to handle the new requirement of differentiating missing history paths. This integration makes the idea feel less like a complex new unit and more like a natural evolution of existing hardware, which significantly increases its perceived feasibility.

3. Bridges the Gap between Theory and Practice: The most compelling aspect of this work is that it takes a powerful academic concept (ahead prediction) that has been largely relegated to the "theoretically interesting but practically infeasible" category and makes it demonstrably practical. The energy analysis presented in Section 4.5 (Figure 7, page 7) is crucial; reducing the energy overhead from a prohibitive 14.6x to a highly palatable 1.5x for a 4.4% performance gain is a fantastic trade-off and the paper's headline result.

4. An Enabling Technology: Beyond its direct IPC contribution, this work is significant as an enabling technology. By effectively "solving" the predictor latency problem, it relieves a major constraint on front-end design. Architects could be empowered to design even larger and more accurate (and thus higher latency) main predictors, such as the very large ML-based predictors that are emerging, without paying the traditional performance penalty. It also directly boosts the effectiveness of other front-end techniques that rely on running ahead of the execution core, such as advanced instruction prefetching (FDIP, APF) and runahead execution.

Weaknesses

1. Tight Coupling to TAGE: The proposed mechanism is deeply intertwined with the specifics of the TAGE predictor (its multiple tables, tag-based lookup, and promotion-on-conflict allocation scheme). While TAGE is a dominant design, this tight coupling leaves the generality of the approach in question. It is not immediately clear how the core insight of "tagging by missing history pattern" would be applied to fundamentally different predictor architectures, like perceptron-based or pure ML-based predictors.

2. Complexity of Interaction and Edge Cases: The design requires several supporting mechanisms to function, such as the single-cycle override (Section 5.1, page 8) to handle simple, short-history branches that are disadvantaged by the ahead scheme. While effective, this feels like a patch that adds complexity and hints that the core ahead mechanism isn't universally superior. The discussion of late predictions and pipeline restarts also suggests a non-trivial amount of control logic is required to manage the system.

3. Limited Exploration of the Branch Target Problem: The paper focuses almost exclusively on solving the latency of the directional predictor. However, a complete ahead prediction system must also predict branch targets ahead of time. The baseline architecture described (Section 5, page 7) explicitly keeps the BTB non-ahead-pipelined. An indirect branch within the N-branch ahead window presents a significant challenge, as its target is unknown when the ahead prediction is initiated. The paper does not fully explore the performance implications of this, which could be a major limiter in code with frequent indirect jumps or calls.

Questions to Address In Rebuttal

1. Generalizability: Could the authors elaborate on how their core insight—that only a few missing history paths are realized—could be applied to other classes of predictors? For instance, in a perceptron predictor, would this translate to training a small number of specialized weight tables for different missing history patterns?

2. Necessity of the Single-Cycle Override: The single-cycle override (Section 5.1) seems critical for recovering 1% of performance. Does this imply a fundamental weakness in ahead prediction for certain types of branches (e.g., those with very short, simple history patterns)? Could the authors comment on the characteristics of the branches that benefit from this override?

3. Interaction with Ahead Target Prediction: The paper focuses on directional prediction latency. How does the proposed scheme contend with indirect branches within the ahead window (e.g., the 5 branches being skipped)? Since the BTB is not ahead-pipelined, an indirect branch would seem to stall the entire process until its target is resolved, potentially negating much of the latency-hiding benefit in certain workloads. Could you clarify the expected performance impact of this limitation?

Review Form: The Innovator (Novelty Specialist)

---

Summary

This paper addresses the well-known problem of prediction latency in modern, high-accuracy branch predictors. The established solution, "ahead prediction," suffers from a critical flaw: an exponential increase in energy and hardware cost as it must speculatively generate predictions for all 2^N possible paths, where N is the number of skipped branches.

The authors' contribution is twofold. First, they present a crucial, empirically-backed insight: the number of actually observed future paths (or "missing history patterns") for a given ahead history is far smaller than the theoretical maximum of 2^N, often just one or two. Second, they propose a novel microarchitectural mechanism to exploit this insight. Instead of reading 2^N predictions, they modify the TAGE predictor by adding a "secondary tag" to each entry. This tag explicitly encodes the specific missing history pattern an entry is trained for. This allows the predictor to support multiple future paths while only performing a single read per TAGE table, fundamentally changing the energy scaling from exponential to linear. The result is a design that makes ahead prediction practical for hiding multi-cycle latencies.

---

Strengths

1. Novel Core Insight and Mechanism: The central contribution of this paper is genuinely novel. While "ahead prediction" is a known concept [38, 41] and its exponential cost is its primary known inhibitor, no prior work has proposed a practical mechanism to break this scaling. The combination of the experimental insight (Section 3, Figure 2, page 3) that few paths materialize and the specific architectural solution (the secondary tag, Section 4, page 5) is, to my knowledge, new.

2. Clear Differentiation from Prior Art: The authors correctly identify the seminal work by Seznec [38] as the closest prior art for multi-path ahead prediction. They clearly articulate the "delta": Seznec's approach relies on reading out consecutive entries (a brute-force spatial solution), leading to exponential energy scaling. This paper's secondary tag mechanism is a fundamentally different, more efficient encoding scheme that multiplexes predictions for different future paths within the standard TAGE structure. This change directly attacks the primary weakness of the prior art.

3. Data-Driven Justification: The novelty is not based on a mere assumption. The analysis in Section 3 ("Number of Missing History Patterns") provides a strong empirical foundation for the entire design. Showing that for 64-bit histories, over 98% of cases see four or fewer patterns is a powerful result that justifies the feasibility of their tagged approach.

---

Weaknesses

1. Incremental Nature of the Architecture: While the core idea of the secondary tag is novel in this application, the overall architecture is an extension of the existing, highly-complex TAGE predictor. The novelty lies in a clever modification, not a fundamentally new prediction algorithm. The work is therefore an evolutionary step, albeit a very important one for making a known technique practical.

2. Potential for Conceptual Overlap in Other Domains: The core concept is "tagging a prediction with the intermediate state required to validate it." While this appears new for branch predictor ahead history, similar concepts may exist in other areas of speculative execution. For instance, certain forms of data prefetching or value prediction might speculatively generate a result and tag it with the dependency information required for its use. The authors should be very careful in positioning this as the absolute first use of such a disambiguation technique in microarchitecture.

---

Questions to Address In Rebuttal

1. On the Novelty of the Implementation: The novelty lies in adding a secondary tag to the main TAGE tables. Have the authors considered alternative implementations of their core insight? For instance, instead of modifying the large TAGE tables, could one use a small, separate "Missing History Pattern Cache" indexed by the ahead PC/history that stores only the few (e.g., 1-4) observed secondary tags and their corresponding predictions? This might avoid polluting the main TAGE structure and could be a different, potentially novel, design point. Please justify the decision to integrate this mechanism directly into TAGE.

2. On the Robustness of the Hash Function: The secondary tag is generated by a hash of intermediate branch targets (Figure 6, page 6). This is a key component of the novel mechanism. How does this design handle pathological cases where different sequences of taken/not-taken branches resolve to the same set of intermediate targets, potentially causing aliasing in the secondary tag hash? Is the target-based hash strictly superior to a simpler hash of the directional bits of the missing history?

3. Final Check on Prior Art: The idea of disambiguating multiple future paths stemming from a single speculative starting point is fundamental. Can the authors state with confidence that no prior work in related fields (e.g., trace caches, block-based predictors, or even speculative multithreading) has used a tagging mechanism to associate a speculative prediction with a specific, yet-to-be-resolved intermediate execution path? A more thorough search for conceptual analogues would strengthen the novelty claim.

Reviewer: The Guardian

Summary

This paper identifies branch predictor cold-start effects as a significant performance bottleneck for latency-sensitive microservices. The authors propose a new hybrid prediction architecture, Similarity-based Branch Prediction (SBP), which leverages the observed high control-flow similarity (CFS) between different requests to the same microservice. The core idea is to use a pre-recorded "reference trace" of a past execution to predict the control flow of a new execution. An instantiation, CHESS, is presented, which optimizes this approach by filtering the reference trace to only include "hard-to-predict" branches, relying on a conventional predictor and static hints for the rest. The authors claim substantial MPKI reductions (up to 94%) and performance within 95% of a fully warm system, at a modest hardware cost of 18.1KB.

While the problem is well-defined and the proposed direction is intriguing, the work rests on a fundamentally fragile assumption of a single, static reference trace. Furthermore, the complex trace reduction heuristics introduce critical blind spots in execution tracking that make the proposed reconvergence mechanism appear unreliable under realistic conditions. The evaluation, while showing impressive numbers, fails to adequately stress-test these core assumptions, calling the robustness and generality of the solution into question.

Strengths

1. Problem Motivation: The paper does an excellent job motivating the problem. The analysis in Section 2, particularly the Top-Down analysis in Figure 1, clearly establishes that branch predictor cold effects ("Frontend Bound Branch Resteers") are a major contributor to performance degradation in microservices.
2. CFS Analysis: The study in Section 3, which quantifies the control-flow similarity across requests, provides a solid empirical foundation for the work. The methodology for identifying reconvergence points and measuring coverage/accuracy is sound and the findings (high similarity) justify the exploration of a similarity-based predictor.

Weaknesses

1. Fundamental Fragility of the Single Reference Trace: The entire SBP/CHESS architecture is predicated on the existence of a single reference trace that is representative of future executions. This is an extremely brittle design choice. The authors state they select the trace that "maximizes coverage" across a training set (Section 6), but this offers no guarantee of robustness against workload drift or even minor variations in request inputs that trigger different control paths. The paper provides no sensitivity analysis whatsoever on the choice of this reference trace. What is the performance if a less-than-optimal trace is chosen? What happens if the workload behavior shifts slightly after the offline profiling period? The current design appears to be a form of extreme overfitting to a training workload, which is untenable for production datacenter environments.

2. Unreliable Reconvergence Mechanism Due to Trace Reduction: The CHESS optimization, which removes "easy-to-predict" (EP) branches to shorten the trace, is a critical flaw. By removing these branches, the predictor loses visibility into the true execution path. The system's state (convergent or divergent) and its understanding of where it is in the program flow now depend on the underlying conventional predictor correctly handling these EP branches.

   Consider the scenario described in Section 5.2: an HP branch is followed by a removed EP branch. The system assumes that after the HP branch, control will eventually arrive at the next HP or rEP branch in the trace. But what if the conventional predictor mispredicts the removed EP branch? The actual execution has now diverged, but the CHESS mechanism is completely unaware of this fact. It will continue to believe it is in a convergent state until it encounters the next HP branch, at which point its prediction will likely be wrong, and its calculated reconvergence point (based on the original, incorrect path) will be invalid. This creates a cascading failure scenario that is not analyzed or even acknowledged. The small-scale example in Figure 7 is insufficient to demonstrate that this logic is sound for complex, real-world control flow.

3. Benchmark-Specific Hardware Costing: The claimed storage cost of 18.1KB is not a general result but an artifact of the specific benchmarks evaluated. The authors derive the sizes for the trace buffer and associated tables by analyzing the characteristics of the eight microservices where CHESS showed a benefit (Section 7, "Storage Requirements"). This is a post-hoc justification, not a principled derivation of hardware requirements. The paper fails to provide any analysis on the worst-case trace length or how storage costs would scale for applications with more complex control flow, rendering the 18.1KB figure potentially misleading.

4. Insufficient Evaluation of System Dynamics: The trace loading mechanism is glossed over. The authors suggest a bulk load via a "privileged hardware control interface" at the start of a request (Section 6). They claim a minimal overhead of 0.4%-1.1% (Section 7), but this appears to be a simple calculation based on trace size and memory bandwidth, ignoring real-world system complexities. It does not account for OS scheduling latency, potential contention on the memory bus from other active cores, or the cost of the privileged operation itself. This is a critical system-level interaction that has been abstracted away to the point of being unrealistic.

Questions to Address In Rebuttal

1. Please provide a sensitivity analysis regarding the choice of the reference trace. What is the performance degradation if the reference trace is chosen to be the one with the median or lowest coverage from the training set, instead of the maximum? How does performance degrade as the test workload increasingly diverges from the training workload used for trace selection?

2. The trace reduction in CHESS creates blind spots. Please provide a detailed analysis of the case where a conventional predictor mispredicts a removed "easy-to-predict" (EP) branch. How does the CHESS mechanism detect and recover from this, given that its own state is now inconsistent with the true execution path? Quantify how often this scenario occurs in your experiments and its impact on the subsequent accuracy of the similarity predictor.

3. The 18.1KB storage cost is derived from the evaluated benchmarks. What is the maximum observed HP+rEP trace length across all requests in your dataset, not just the chosen reference trace? Can you provide a more robust model for how trace length and storage cost scale with program complexity (e.g., number of basic blocks, cyclomatic complexity) rather than a value tuned to a small set of benchmarks?

4. The trace loading overhead of 1.1% seems optimistic. Can you provide a more detailed breakdown of this calculation? Does it include the software overhead of trapping to the OS, the OS scheduling the load, and potential queuing delays in the memory controller under a heavily loaded system, or is it purely a theoretical data transfer time?

Reviewer Persona: The Synthesizer (Contextual Analyst)

Summary

This paper addresses the significant performance penalty caused by branch predictor cold-starts in short-lived microservice executions. The authors identify that the core reason this penalty is so high is that the control flow across different requests for the same microservice is highly similar. The central contribution is a new hybrid branch predictor architecture, Similarity-based Branch Prediction (SBP), which leverages this observation. SBP uses an offline-generated "reference trace" of a typical request's control flow to make highly accurate predictions at runtime. The paper proposes a specific, pragmatic implementation called CHESS, which judiciously applies this similarity-based prediction only to branches identified as "hard-to-predict" by conventional mechanisms, thereby keeping the reference trace manageably small. The evaluation demonstrates that CHESS can reduce branch mispredictions (MPKI) by up to 94% compared to a cold predictor, achieving performance that is within 95% of an ideal warm-predictor baseline with a modest hardware storage cost.

Strengths

1. Excellent Problem Contextualization: The paper does a superb job of situating its contribution within a critical trend in modern computing: the shift towards latency-sensitive, decomposed microservice architectures. The initial analysis in Section 2.2 (page 2), using Top-Down analysis to pinpoint "Branch Resteers" as a major source of cold-start overhead, provides a compelling and quantitative motivation for the work. This is not a solution in search of a problem; it is a direct and well-reasoned attack on a known, significant bottleneck.

2. Elegant Synthesis of Existing Concepts: The core strength of this work lies in its synthesis of several established lines of research. It connects the observation of control-flow similarity (CFS), previously explored in works on web servers [11, 37], with the well-known "cold start" problem in branch prediction. The proposed solution is a logical and powerful next step from prior work like Ignite [54], which warmed up the BTB and a simple bimodal predictor. CHESS extends this "record and replay" philosophy to the far more complex state of a modern TAGE-style conditional predictor. Furthermore, the use of offline profiling via Intel PT places it in conversation with Profile-Guided Optimization (PGO) techniques like Whisper [38] and Thermometer [58], but offers a distinct, dynamic architectural alternative to static binary modification.

3. Pragmatic and Thoughtful Design: The design of CHESS is not a brute-force solution. The insight to filter the reference trace to only include "hard-to-predict" (HP) and a few "retained easy-to-predict" (rEP) branches is the key to its practicality. This reduces the trace length by orders of magnitude (as shown in Figure 12, page 12) while preserving most of the performance benefit. The mechanism for handling divergence and reconvergence is also well-defined, making the hybrid approach robust. This demonstrates a deep understanding of the trade-offs between prediction accuracy and implementation cost.

4. Strong and Convincing Results: The empirical evaluation is thorough and the results are impressive. Achieving performance that closes such a large gap to a warm baseline is a significant achievement. The comparison against multiple baselines, including a warm predictor, a static-hint-based predictor, and an "unbounded" predictor, effectively isolates the benefits of the proposed technique and demonstrates that the problem is truly one of cold state, not predictor capacity.

Weaknesses

My critiques are less about flaws and more about opportunities to further contextualize the work and explore its boundaries.

1. Understated Hardware Complexity: While the storage cost of the reference trace is well-quantified (18.1KB), the logic complexity added to the processor front-end is discussed at a high level. The front-end is one of the most timing-critical parts of a modern core. The machinery required to manage the Trace Buffer, the read pointer, the CSD tracking, and the state transitions between convergent/divergent modes (Figure 8, page 9) could have non-trivial implications for pipeline latency and clock frequency. A more detailed discussion of these implementation trade-offs would strengthen the paper.

2. Sensitivity to Workload Dynamics: The model relies on a single reference trace being representative of the vast majority of requests. The authors reasonably argue that datacenter workloads evolve slowly, allowing for periodic regeneration of traces. However, this assumption could be brittle. A sudden shift in the workload mix or the introduction of a new, popular feature could lead to a significant divergence from the reference trace, potentially degrading performance until a new trace is deployed. An analysis of the system's sensitivity to such "concept drift" would be a valuable addition. How gracefully does performance degrade as the live traffic deviates from the reference trace?

3. Implicit Trade-offs with Static PGO: The paper compares CHESS's accuracy to Whisper [38] but could more explicitly articulate the architectural trade-off space. CHESS requires new, specialized hardware but offers runtime flexibility (a new trace can be loaded without recompilation). Static PGO techniques modify the binary to embed prediction logic, working on existing hardware but requiring a slower re-deployment cycle to adapt to workload changes. A discussion of which scenarios favor one approach over the other would help position the contribution more clearly for system architects.

Questions to Address In Rebuttal

1. Beyond the 18.1KB trace storage, could the authors comment on the anticipated logic overhead and potential timing impact of integrating the CHESS mechanisms into a high-frequency processor front-end? Is the two-cycle override delay mentioned in Section 7 a conservative estimate?

2. How robust is the single reference trace model? Have the authors considered a scenario where the live request distribution is bimodal, with two common but distinct control-flow paths? Would the system require multiple reference traces, and how would it select the correct one at runtime?

3. Could the authors elaborate on the architectural trade-offs between their dynamic, trace-based approach and static, PGO-based approaches like Whisper? Specifically, in what deployment environments (e.g., public cloud FaaS vs. private enterprise microservices) would the hardware cost and runtime flexibility of CHESS be most advantageous compared to a software-only PGO solution?

Paper: Leveraging control-flow similarity to reduce branch predictor cold effects in microservices Reviewer: The Innovator (Novelty Specialist)

---

Summary

The paper proposes Similarity-based Branch Prediction (SBP), a hybrid branch prediction architecture designed to mitigate cold-start effects in microservices. The core mechanism involves recording a "reference execution trace" from a past request and using this trace at runtime to predict the control flow (branch directions and targets) for subsequent, similar requests. The system operates in a "convergent" mode, following the trace, and falls back to a conventional predictor upon a "divergence" (a mismatch between the trace's prediction and the actual execution path). The authors present a specific implementation, CHESS, which optimizes the reference trace by filtering out branches deemed easy-to-predict and applies this replay mechanism primarily to hard-to-predict branches. The stated goal is to achieve performance close to that of a warm predictor by effectively "replaying" the control flow of a canonical execution.

Strengths

The engineering and evaluation of the CHESS system are thorough. The optimizations proposed to reduce the reference trace size (Section 5.1), such as removing direct branches and easy-to-predict conditional branches while retaining key nodes for reconvergence (rEP), are practical and well-reasoned. The performance results are compelling, demonstrating a significant reduction in branch MPKI over a cold baseline.

Weaknesses

The central weakness of this paper, from a novelty standpoint, is that its core conceptual contribution is not new. The fundamental idea of recording a trace of a prior execution to warm up or guide front-end microarchitectural structures has been explored in prior work.

The authors' novel claim is the SBP architecture. However, this architecture is functionally and conceptually an extension of previously published ideas. The most proximate prior art includes:

1. Ignite [54]: This work, cited by the authors, explicitly "records and replays branch-target buffer (BTB) insertions to warm up a cold BTB together with a bimodal predictor." The mechanism is identical at a high level: record a trace of front-end events and replay it to pre-condition the hardware for a subsequent invocation. The primary delta in the present work is applying this concept to a more complex conditional predictor (TAGE-SC-L) rather than just the BTB and a simple bimodal predictor. While this is a valid engineering extension that requires solving new problems (e.g., managing more complex state), it does not represent a new paradigm. The core idea of "replay for branch prediction warmup" is already established by Ignite.

2. Lukewarm [53]: This work, also cited, "records and replays instruction accesses to warm up a cold instruction cache." Again, this establishes the precedent of using a recorded trace from a past serverless invocation to mitigate cold-start effects in a front-end structure (the I-cache). SBP applies the same pattern—record and replay—to a different, albeit related, structure.

3. Trace Caches [51]: The idea of following a pre-recorded path of execution is the foundational concept of a trace cache. While a trace cache stores decoded uops and focuses on fetch bandwidth, the underlying principle of identifying and replaying a common dynamic instruction sequence is the same. SBP can be viewed as a "prediction trace cache" that replays branch outcomes rather than instructions.

The offline analysis component of SBP/CHESS, where a representative trace is selected and optimized, is a well-established methodology from the domain of Profile-Guided Optimization (PGO). Works like Whisper [38] and Thermometer [58] already use extensive offline profiling to create highly specialized prediction hints or logic for difficult branches. SBP's contribution here is the specific format of the output (an explicit trace for replay) rather than the process itself.

Therefore, the paper's contribution is not the invention of a new prediction method, but rather the synthesis and extension of the known "record-and-replay" technique to a modern, complex branch predictor. The "delta" over prior art is one of scope and engineering sophistication, not one of fundamental concept.

Questions to Address In Rebuttal

1. The authors must clearly demarcate the conceptual novelty of SBP from Ignite [54]. Beyond applying the replay concept to a TAGE predictor instead of a BTB/bimodal predictor, what is the fundamental new idea that separates these two works? The paper currently positions itself as a new approach, but it appears to be a direct and logical extension of Ignite's core mechanism.

2. The paper's explicit handling of divergence and reconvergence (Section 4.3) using post-dominator analysis is robust. Is this mechanism itself claimed as novel, or is it the application of known compiler theory [12, 14] to the record-and-replay context? Prior replay-based schemes must have implicitly or explicitly handled divergence; please clarify how SBP's approach is fundamentally different and not just a more formal implementation.

3. Given the significant conceptual overlap with prior art, the contribution seems to lie in the engineering of the CHESS system, particularly the trace reduction heuristics (Section 5.2). Could the authors re-frame their contribution not as a novel prediction paradigm (SBP), but as a set of novel and practical optimizations (CHESS) that make the existing record-and-replay paradigm viable for complex, state-of-the-art predictors?

Paper Title: Cramming a Data Center into One Cabinet, a Co-Exploration of Computing and Hardware Architecture of Waferscale Chip Reviewer Persona: The Guardian (Adversarial Skeptic)

---

Summary

The authors present Titan, an automated co-design framework for wafer-scale systems, with the goal of optimizing performance and integration density within a single cabinet under cost and physical constraints. The framework proposes two primary architectural optimizations: the integration of configurable on-wafer memory dies and the enforcement of vertical area constraints across the hardware stack to improve density. Titan explores a hierarchical design space, from intra-chip core configuration to inter-chip cabinet layout, using a series of analytical models for performance, cost, and physical reliability to prune the space and identify optimal configurations. The paper claims that designs produced by Titan significantly outperform a state-of-the-art Dojo-like baseline.

While the ambition to create a holistic co-design framework for wafer-scale systems is commendable, this work rests on a foundation of overly simplified and critically unvalidated models. The staggering performance gains reported are not substantiated by rigorous evaluation, and the framework’s ability to produce physically realizable and truly optimal designs is highly questionable.

Strengths

1. Problem Definition: The paper correctly identifies a critical and timely research problem. The co-design of computing and hardware architectures for wafer-scale systems, particularly in the context of LLMs, is of significant interest to the community.
2. Holistic Scope: The authors should be credited for the comprehensive scope of their proposed framework. The hierarchical model, which attempts to connect low-level parameters like MAC unit counts to high-level decisions like cabinet chip layout (Section 3, Figure 6), represents a holistic view of the design problem.
3. Ablation Study Structure: The use of distinct baselines (D-Arch, C-Arch, S-Arch) to isolate the benefits of the proposed architectural optimizations is a methodologically sound approach to structuring the evaluation (Section 5.2.3). This allows for a clear, albeit theoretical, attribution of the claimed performance gains.

Weaknesses

My primary concerns with this paper are the severe oversimplifications in its core models, which undermine the credibility of the entire framework and its results.

1. Fatally Simplified Performance Evaluation: The performance evaluation relies on the Astra-sim2.0 simulator with the explicit admission of "ignoring congestion and routing control" (Section 5.2.2). For a wafer-scale system where thousands of cores communicate across a massive mesh network, congestion is not a second-order effect; it is a primary performance bottleneck. Claiming a 10.66x performance improvement for Llama2-72B (Section 5.3) based on a congestion-free model is fundamentally unsound. The reported performance is likely an unobtainable theoretical peak rather than a realistic projection of system throughput.

2. Unvalidated and Abstract Physical Models: The framework’s ability to prune the design space relies on a set of "theoretical prediction models" for interposer reliability, including warpage, thermal, SI, and PI (Section 4.2.3). These are presented as simple analytical equations with un-specified coefficients (Kcool, Kbp, KSI, KPI). There is no evidence that these models have been validated against industry-standard finite element analysis (e.g., Ansys) or electromagnetic simulation tools (e.g., Clarity, SIwave). Using such unverified heuristics to discard vast portions of the design space is perilous; the framework could be systematically discarding valid, high-performance designs or, conversely, retaining physically unrealizable ones. The mention of an in-house prototype (Figure 12) is insufficient validation for a model meant to span a parameter space as large as the one defined in Table 2.

3. Superficial Cost and Yield Modeling: The cost model (Section 4.4) hinges on a yield parameter, Ydie. Yield modeling for wafer-scale integration is a notoriously complex field that depends heavily on defect distribution, redundancy mechanisms, and harvesting strategies. The paper offers no details on its yield model beyond this single variable. It is unclear how this model accurately captures the cost trade-offs of manufacturing a massive interposer with numerous heterogeneous KGDs. Without a credible and detailed cost model, the central claim of optimizing performance "under the same cost constraint" is unsubstantiated. The comparison against the "modeled Dojo tray" is only as meaningful as the accuracy of the model, which is not demonstrated.

4. Architectural Ambiguity: The concept of "on-wafer memory" is central to the computing architecture, yet its implementation is described at a very high level of abstraction (Section 2.2.1). How are these memory dies architecturally integrated? Do they function as a distributed last-level cache, partitioned memory space, or something else? What are the latency and bandwidth characteristics of the die-to-die links connecting them to compute KGDs? The paper does not provide enough detail to assess the feasibility or performance implications of this core proposal. The gains attributed to this feature are therefore speculative.

Questions to Address In Rebuttal

1. Please provide a robust justification for using a congestion-free network model for performance evaluation. Can you present sensitivity studies or data from existing literature to demonstrate that congestion is a negligible factor for the LLM workloads and system scales you are analyzing? Otherwise, how can the performance claims in Figure 13 be considered credible?

2. The reliability models in Section 4.2.3 are critical for design space pruning. Please provide evidence of their validity. Specifically, show a comparison of the predictions from your analytical models (for warpage, IR drop, etc.) against results from established simulation tools for at least three diverse design points (e.g., small chip/low power, large chip/high power, high IO density).

3. Elaborate on the yield model (Ydie) used in your cost calculations. How does it account for defect tolerance and redundancy, which are essential for achieving viable yields in wafer-scale systems? How was this model calibrated, and what is its margin of error when comparing the cost of your proposed T-Arch against the D-Arch baseline?

4. Please provide a more detailed architectural description of the "on-wafer memory" subsystem. What is the coherence protocol, the memory access latency from a compute core, and how does the NoC prioritize traffic between compute-compute and compute-memory communication?

Reviewer: The Synthesizer (Contextual Analyst)

Review Form

Summary

This paper presents Titan, an automated framework for the co-design of wafer-scale computing systems, aiming to "cram a data center into one cabinet." The central thesis is that current wafer-scale systems, while powerful, suffer from inefficiencies due to a lack of coordinated design between the logical computing architecture (cores, memory hierarchy) and the physical hardware architecture (packaging, power, cooling).

To address this, the authors propose a hierarchical, parameterized model of a full cabinet system, from individual compute cores up to the arrangement of wafer-scale chips. The Titan framework uses this model to perform a comprehensive design space exploration (DSE). The core contributions of this framework are:

1. A holistic co-design methodology that simultaneously optimizes compute and physical hardware parameters.
2. The introduction of a "vertical area constraint" as a key heuristic to prune the enormous design space by enforcing area alignment between stacked functional layers.
3. The integration of early-stage physical reliability models (warpage, SI/PI) and cost models to eliminate unfeasible designs and enable multi-objective optimization.

Through simulation, the authors demonstrate that Titan-generated designs significantly outperform a baseline modeled on Tesla's Dojo architecture, and they use the framework to derive valuable, non-obvious insights about the relationship between single-chip area and overall cabinet cost-efficiency.

Strengths

This is an ambitious and important piece of work that sits at the confluence of computer architecture, EDA, and advanced packaging. Its primary strengths are:

1. Holistic, Cross-Layer Scope: The paper's most significant strength is its scope. It successfully bridges the chasm between microarchitectural decisions (e.g., number of MAC units per core, on-wafer memory dies) and system-level physical design (e.g., interposer layers, C4 bump pitch, cabinet layout). This connects two worlds that are often optimized in isolation. This approach is reminiscent of the broader push towards System-Technology Co-Optimization (STCO), and this paper provides an excellent, concrete example of STCO applied to the design of next-generation AI hardware.

2. Grounded in Realistic Technology: The framework is not built in a vacuum. The authors ground their architectural models in real-world, state-of-the-art systems and technologies, referencing Tesla's Dojo (Figure 2), TSMC's InFO_SoW packaging, and a specific process node (12nm). The inclusion of their own in-house prototyped wafer-scale chip (Figure 12) is a powerful addition that lends significant credibility to their cost, yield, and area models, elevating this from a purely theoretical exercise.

3. A Practical Heuristic for a Vast Problem: The design space for a wafer-scale cabinet is combinatorially explosive. The "vertical area constraint" introduced in Section 2.2.2 and modeled in Section 4.2.2 is a simple yet elegant heuristic to manage this complexity. By forcing the area of supporting layers (cooling, substrate) to be proportional to the compute layer, Titan can effectively prune designs that would be spatially inefficient, drastically improving the efficiency of the DSE process.

4. Generates Actionable Architectural Insights: The true value of a DSE framework lies not just in finding a single optimal point, but in revealing the underlying design trade-offs. The analysis in Section 5.4 ("Design Considerations for Single-chip Area") and the accompanying Figure 14 are excellent examples of this. The finding that the largest possible chip is not always the most cost-effective solution for a cabinet—due to discrete chip array configurations and cost constraints—is a crucial, non-obvious insight for system architects.

Weaknesses

While the work is strong, its positioning and some underlying assumptions could be strengthened. These are not fatal flaws but opportunities for improvement.

1. Understated Connection to Broader Fields: The authors could better contextualize their work within the established literature of DSE and STCO. While they have built an impressive system, the paper would have greater impact if it explicitly framed Titan as an advanced STCO framework for wafer-scale systems, drawing clearer parallels to and distinctions from prior work in the EDA and packaging communities.

2. Simplifications in Performance Modeling: The performance evaluation relies on Astra-sim2.0 with "simplified assumptions (ignoring congestion and routing control)" as noted in Section 5.2.2. In massively parallel systems like these, network congestion can be a first-order performance bottleneck. Ignoring it may lead the optimizer to favor designs with high theoretical bandwidth that would not be realizable in practice. This simplification weakens the claims about simulated performance improvements.

3. Abstraction of the Cost Model: The cost model in Section 4.4 is comprehensive for an academic work, including wafer cost, yield, bonding, and cooling. However, the true cost of such systems is also heavily influenced by factors like Non-Recurring Engineering (NRE) costs, the complexity of testing a wafer-scale part, and supply chain logistics, which are not captured. While perfectly modeling this is beyond scope, acknowledging these other major cost drivers would provide a more complete picture.

Questions to Address In Rebuttal

1. The "vertical area constraint" margin, darea, is a critical parameter for the exploration process. The paper states a default value of 0.72 was chosen based on experiments (Section 4.5). Could the authors comment on the sensitivity of the final design quality to this parameter? How might the optimal darea change for different technology nodes (e.g., 3nm vs 12nm) or for different optimization targets (e.g., pure performance vs. performance/watt)?

2. Regarding the performance simulations, can the authors elaborate on the potential impact of network congestion? For example, would including congestion effects likely favor architectures with more on-chip memory (to reduce off-chip traffic) or different on-chip network topologies, even if it meant a lower peak theoretical FLOPS?

3. The Titan framework appears to be highly adept at exploring the design space for Dojo-like tiled architectures. How generalizable is the framework? What key modifications would be necessary to model and optimize a fundamentally different wafer-scale architecture, such as a more monolithic Cerebras-style design with a unified memory fabric, or a heterogeneous system integrating different types of compute dies on the same wafer?

Paper Title: Cramming a Data Center into One Cabinet, a Co-Exploration of Computing and Hardware Architecture of Waferscale Chip Reviewer Persona: The Innovator (Novelty Specialist)

---

Summary

The authors present "Titan," an automated framework for the co-design and exploration of wafer-scale, single-cabinet data center architectures. The framework is built upon a hierarchical, parameterized model that spans from the intra-chip level (compute cores, on-wafer memory) to the inter-chip cabinet level (chip arrays, host interfaces). The central claim of novelty lies in the framework's methodology, which employs two key mechanisms to navigate the vast design space: 1) a "vertical area constraint" to enforce co-optimization across the physically stacked layers of a wafer-scale system (cooling, compute, interposer, substrate), and 2) the integration of early-stage, predictive models for physical reliability (warpage, SI, PI) to prune unfeasible designs before full evaluation. The goal is to automate the discovery of cost-performance optimal wafer-scale systems.

---

Strengths

The primary strength of this work lies in its novel synthesis and systematization of design principles for the emerging domain of wafer-scale systems. While individual components of the framework draw from existing concepts, their integration to address the unique, multi-physics challenges of a monolithic wafer-scale cabinet is new.

1. The "Vertical Area Constraint" Heuristic: The most salient novel concept is the "vertical area constraint" proposed in Section 2.2.2 and implemented in Section 4.2.2. While co-design frameworks exist, this is a simple, physically-grounded, and—to my knowledge—newly articulated heuristic specifically for vertically stacked, wafer-scale packages. It elegantly captures the cross-layer dependency on area, directly targeting the integration density problem and serving as a powerful pruning mechanism in the design space exploration (DSE).

2. Methodological Shift in Reliability Analysis: The paper proposes moving reliability analysis from a late-stage validation step to an early-stage architectural filter. The use of predictive models for interposer warpage, SI, and PI during DSE (Section 4.2.3) is a significant methodological advance over prior art, where such considerations are typically too computationally expensive for architectural exploration. This integration is crucial for making the exploration of such a large and complex design space tractable.

3. Formalization of the Wafer-Scale Co-Design Problem: Existing works on wafer-scale systems like Cerebras' WSE and Tesla's Dojo are point-solution case studies. This paper is the first I have seen to propose a generalized, automated framework to explore the entire design space of such systems. The novelty here is the shift from demonstrating a single instance to creating a methodology for discovering a whole class of optimal instances.

---

Weaknesses

While the synthesis is novel, the work builds heavily on pre-existing concepts, and the novelty of some individual components must be carefully delineated.

1. Incremental Novelty of Component Models: The paper is fundamentally a work of engineering synthesis. The core ideas of parameterized performance/cost modeling, design space exploration, and even reliability modeling are not new in themselves. For example, cost modeling for multi-die systems has been explored in works like "Chiplet actuary" (Feng and Ma, DAC '22), which the authors cite [10]. The reliability models for warpage, SI, and PI are explicitly based on established theoretical formulas from prior works, which are also cited ([49], [52], [18]). The novelty is in their adaptation and integration, not their fundamental formulation.

2. Conceptual Overlap with 3D-IC/Chiplet Co-Design: The concept of adding configurable memory dies alongside compute dies (Section 2.2.1) is a direct extension of existing 2.5D/3D packaging trends (e.g., TSMC's CoWoS-S with HBM). The novelty is not the idea of on-package memory itself, but rather the framework's ability to quantitatively evaluate the trade-offs of including it in a wafer-scale context. This distinction is subtle and could be made clearer. The overall co-design approach bears conceptual resemblance to DSE frameworks for heterogeneous chiplet-based systems, with the key differentiator being the specific constraints and scale of a monolithic wafer.

---

Questions to Address In Rebuttal

1. Pinpointing the Core Novel Mechanism: The concept of system-level co-design is well-established. Please articulate precisely which specific mechanism or model within Titan represents the most significant conceptual leap from prior co-design frameworks for complex, multi-chiplet systems. Is it primarily the "vertical area constraint," or is there another fundamental innovation?

2. Quantifying the Novelty of Reliability Models: Regarding the predictive reliability models detailed in Section 4.2.3, please clarify the delta between your work and the direct application of existing theoretical models from the cited literature (e.g., [49], [52]). What specific adaptations or novel assumptions were required to make these models sufficiently fast and accurate for early-stage architectural DSE, as opposed to late-stage signoff?

3. Robustness of the Vertical Area Constraint: The "vertical area constraint" is an elegant heuristic. However, could you discuss its potential limitations? Are there plausible scenarios where this hard constraint might prematurely prune a non-obvious, globally optimal design? For instance, a design with a significantly larger substrate to accommodate superior power delivery might be discarded despite enabling a disproportionately higher compute performance.

4. Generalizability of the Framework: While the framework is presented for wafer-scale systems, its core principles seem applicable to other complex, 3D-stacked heterogeneous packages. Could the Titan methodology, particularly the vertical constraint and early reliability checks, be considered a novel, general-purpose approach for co-design in the advanced packaging era, beyond just wafer-scale integration?

Paper Title: Dynamic Load Balancer in Intel Xeon Scalable Processor: Performance Analyses, Enhancements, and Guidelines Reviewer: The Guardian

---

Summary

This paper presents an analysis of the Dynamic Load Balancer (DLB), an on-chip accelerator in recent Intel Xeon processors. The authors first conduct a microbenchmark comparison showing DLB's superior throughput over software-based load balancers, but identify that feeding the DLB still consumes significant host CPU cycles. To address this, they propose AccDirect, a system that uses PCIe Peer-to-Peer (P2P) communication to allow a SmartNIC to directly enqueue work descriptors into the DLB, bypassing the host CPU. The evaluation shows that AccDirect maintains performance comparable to host-driven DLB while reducing system power, and outperforms a static hardware load balancer (RSS) in an end-to-end key-value store application by 14-50% in throughput. Finally, the paper provides a set of performance guidelines for configuring DLB.

Strengths

1. The paper provides the first in-depth, public characterization of a novel and relevant piece of commercial hardware (Intel DLB). This exploration of a new feature is valuable to the community.
2. The problem identification is clear and well-motivated. The analysis in Section 3.2 (specifically Figure 4) correctly identifies the host-side enqueue operation as the next bottleneck after offloading the core load-balancing logic, which provides a strong foundation for the proposed solution.
3. The core architectural idea of AccDirect—chaining a peripheral (NIC) to an on-chip accelerator (DLB) via PCIe P2P—is a compelling concept for building more disaggregated and efficient systems.

Weaknesses

My primary concerns with this paper relate to the rigor of the experimental evaluation and the clarity of the claimed contributions. While the ideas are interesting, the evidence provided is not sufficient to fully substantiate the claims.

1. Unconvincing Experimental Baseline: The main end-to-end application performance claim (Section 4.5, Figure 10) relies on a comparison between the proposed dynamic load balancer (AccDirect-DLB) and a static hash-based load balancer (RSS). The chosen workloads (Masstree with mixed GET/SCAN operations) are explicitly designed to have variable service times, a scenario that is known to be the worst-case for static balancers like RSS and the best-case for dynamic balancers. This comparison feels engineered to highlight the benefits of DLB rather than rigorously comparing it against a credible alternative. A state-of-the-art dynamic software load balancer (e.g., inspired by Shenango or Caladan, which are cited but not compared against) would have been a far more appropriate and challenging baseline. As it stands, the 14-50% improvement over RSS is not surprising and its significance is questionable.

2. Omission of Critical Performance Metrics: The central premise of AccDirect is to save host CPU cycles by offloading the enqueue task to a SmartNIC. However, the paper completely omits any quantification of the resource consumption on the SmartNIC. The "SNIC agent" (Section 4.3) runs on the SmartNIC's Arm cores. How many cores are required? What is their CPU utilization under the loads tested in Figures 9 and 10? Without this data, the work has not been "eliminated" but merely "moved." This is a critical omission that undermines the claim of improved efficiency. It is entirely possible that the Arm cores on the SmartNIC become the new bottleneck or that the power saved on the host x86 cores is offset by the power consumed by the Arm cores.

3. Insufficient Substantiation of Power Savings: The abstract and Section 4.5 claim a system-wide power reduction of up to 10%. This claim is based on the data in Figure 9, which shows an absolute power saving of ~30W at the highest request rate. However, the paper never states the baseline total system power from which this 10% figure is calculated. Reporting a relative improvement without providing the denominator is not rigorous. The baseline power measurement for the entire server under the corresponding load must be explicitly stated for this claim to be verifiable.

4. Dilution of Research Contribution: A substantial portion of the paper (Section 6, Figures 11, 12, 13) is dedicated to a detailed parameter-tuning study of DLB. While this information is useful for engineers looking to use this specific Intel product, it reads more like an application note or a user guide than a novel research contribution suitable for a premier architecture conference. This extensive characterization dilutes the paper's focus and makes it feel less like a cohesive research paper and more like a combination of a system proposal (AccDirect) and product documentation.

Questions to Address In Rebuttal

The authors must address the following points to convince me of the paper's validity and contribution.

1. Please justify the choice of RSS as the primary hardware baseline in your end-to-end evaluation (Figure 10). Given the known limitations of static balancing for skewed workloads, how can you claim a significant advantage without comparing against a state-of-the-art dynamic software load balancer?
2. Please provide the resource utilization data for the SmartNIC agent used in the AccDirect experiments. Specifically, state the number of Arm cores utilized on the BlueField-3 DPU and their average CPU utilization percentage for the results shown in Figure 9 and Figure 10.
3. Regarding the 10% system-wide power saving claim, please state the absolute total system power consumption (in Watts) of the server for the baseline dlb-lib configuration at the 14.5 MRPS data point in Figure 9.
4. The evaluation of AccDirect appears to be conducted under conditions highly favorable to dynamic load balancing (i.e., significant service time variance). How do the performance benefits of AccDirect over the baseline (RSS) change as the workload becomes more uniform (i.e., as the variance in service times approaches zero)? A sensitivity analysis is needed to understand the boundaries of your solution's benefits.

Reviewer Persona: The Synthesizer (Contextual Analyst)

Summary

This paper presents a comprehensive study of the new Intel Dynamic Load Balancer (DLB), an on-chip accelerator designed to offload packet scheduling and queue management from host CPU cores. The work is structured into three main contributions. First, it provides a thorough performance characterization of DLB, demonstrating its significant throughput and scalability advantages (up to 100 MPPS) over traditional software-based load balancers, which saturate around 40 MPPS. Second, and most importantly, it identifies a key limitation of the conventional DLB usage model: it still consumes significant host CPU cycles to simply prepare and enqueue work descriptors. To solve this, the authors propose "AccDirect," a novel system architecture that leverages PCIe Peer-to-Peer (P2P) communication to create a direct control path between a network interface card (NIC) and the on-chip DLB. This "accelerator chaining" approach effectively bypasses the host CPU, reducing system-wide power consumption by up to 10% and improving end-to-end application throughput by 14-50% compared to baselines. Third, the paper offers a valuable set of practical guidelines for configuring and optimizing DLB, drawn from an extensive microbenchmark analysis of its advanced features.

Strengths

1. Timeliness and High Relevance: The paper provides the first in-depth, public analysis of a new and important piece of commodity hardware. As datacenters grapple with the "datacenter tax" of managing high-speed networks, understanding and optimizing on-chip accelerators like DLB is of paramount importance to both the systems research community and industry practitioners.

2. Novel and Impactful Systems Contribution: The core idea of AccDirect is a significant contribution. While its constituent technologies (PCIe P2P, SmartNICs, on-chip accelerators) are not new in isolation, the authors' work in integrating them into a cohesive, host-bypassing architecture is a novel and powerful systems concept. This work serves as an excellent blueprint for a future where data flows are orchestrated between on-chip and off-chip accelerators with minimal host CPU intervention. The successful demonstration of chaining an I/O device directly to an integrated accelerator is a key step towards more efficient, accelerator-centric server architectures.

3. Thorough and Methodical Evaluation: The evaluation is comprehensive and compelling. It starts with foundational microbenchmarks to motivate the problem (Section 3, Figure 4, page 5), proceeds to a direct evaluation of the proposed solution's power and performance benefits (Section 4.5, Figure 9, page 8), and culminates in a real-world, end-to-end application benchmark (Masstree KVS, Figure 10, page 9) that demonstrates tangible benefits over both hardware and software baselines. This multi-layered approach provides strong evidence for the authors' claims.

4. Exceptional Practical Value: The detailed characterization study and the resulting guidelines presented in Section 6 are a major strength. The authors demystify a complex hardware component with a vast configuration space, providing clear implications and trade-offs for parameters like port types, wait modes, and priority levels. This section, on its own, is a valuable resource that will enable other researchers and engineers to make effective use of this new hardware.

Weaknesses

My criticisms are less about flaws and more about opportunities to further elevate the work's positioning and impact.

1. Reliance on a Programmable SmartNIC: The current implementation of AccDirect depends on a sophisticated SmartNIC (NVIDIA BlueField-3) to act as the agent that prepares and enqueues work to the DLB. While this is an excellent choice for a research prototype, it limits the immediate applicability of the approach, as such devices are not yet ubiquitously deployed. The paper acknowledges this in the Discussion (Section 5, page 9) but could benefit from a more detailed exploration of what would be required to enable this functionality with less-programmable, commodity NICs.

2. Understated Framing of the Core Concept: The authors frame the work primarily as an analysis and enhancement of DLB. However, the more profound contribution is the demonstration of a general architectural pattern: host-transparent, P2P-based accelerator chaining. This pattern has implications far beyond just DLB and networking. The paper could be strengthened by explicitly framing AccDirect as a case study of this broader architectural principle, connecting it more strongly to the vision of accelerator-centric or disaggregated systems explored in works like Lynx [53].

3. Limited Scope of Application Evaluation: The end-to-end evaluation focuses on an RDMA-based Key-Value Store. While this is a critical datacenter workload, the AccDirect pattern could be highly beneficial for other domains, such as Network Function Virtualization (NFV) service chains, storage disaggregation (NVMe-oF), or data-intensive computing pipelines. A brief discussion on how the AccDirect principles might apply to these other areas would broaden the perceived impact of the work.

Questions to Address In Rebuttal

1. Could the authors elaborate on the path to implementing a system like AccDirect without a fully programmable SmartNIC? What specific, minimal hardware capabilities would a more conventional NIC need (e.g., a flexible DMA engine, limited packet parsing logic) to act as an enqueuing agent for DLB?

2. The concept of P2P accelerator chaining is very powerful. Beyond the NIC->DLB chain demonstrated here, have the authors considered how this pattern could be extended? For example, could a worker core, after being scheduled by DLB, use a similar mechanism to directly chain a task to another on-chip accelerator like Intel's QuickAssist Technology (QAT) or Data Streaming Accelerator (DSA) without returning to a central scheduler?

3. The use of RDMA atomics to manage the DLB credit pool from the SmartNIC is a clever solution to a tricky problem (Section 4.3, page 7). Could you comment on any performance implications of executing atomic operations over the PCIe bus versus on the host CPU? Specifically, how does the PCIe Root Complex handle the contention between PCIe atomics from the NIC and locked instructions from host cores targeting the same memory location?

Review Form: The Innovator (Novelty Specialist)

Summary

This paper presents a comprehensive performance analysis of the Intel Dynamic Load Balancer (DLB), a recently introduced on-chip accelerator. The authors first characterize its performance against software-based alternatives, identifying a significant limitation: the high host CPU cost required to enqueue work descriptors to the DLB at high packet rates. The primary claimed contribution is a system architecture named AccDirect, which establishes a direct control path between a NIC (specifically, a SmartNIC) and the on-chip DLB. This is achieved using standard PCIe Peer-to-Peer (P2P) communication, allowing the SmartNIC to directly enqueue work descriptors into the DLB's hardware queues, thereby completely offloading this "datacenter tax" from the host CPU. The authors demonstrate that this approach saves up to 10% of system-wide power and improves the throughput of an end-to-end application by 14-50% compared to baselines.

Strengths

1. Clever Application of Existing Primitives: The core idea of using PCIe P2P to enable direct device-to-device communication is not new, but its application here is specific and clever. The authors have engineered a solution where an off-chip, peripheral device (a SmartNIC) directly issues control commands (work descriptor enqueues) to an on-chip, root-complex-integrated accelerator (the DLB). This moves beyond the common P2P use case of bulk data transfer (e.g., GPUDirect) and into the domain of direct, fine-grained accelerator control, which is a valuable engineering contribution.

2. Identifies and Solves a Concrete Problem: The paper does an excellent job of identifying a real-world performance bottleneck. The finding in Section 3.2 (Figure 4, page 5) that it requires five host CPU cores just to feed the DLB enough work to reach its 100 MPPS potential is a stark and compelling motivation. AccDirect provides a direct and effective solution to this specific problem.

3. Strong Empirical Results: The demonstrated benefits are significant. A 10% reduction in total system power (Figure 9, page 8) by offloading the enqueue task is a substantial gain in a datacenter context. The end-to-end application improvement further validates that the architectural change translates to real-world performance benefits.

Weaknesses

The primary weakness of this work lies in the degree of its conceptual novelty when viewed against the landscape of prior art in accelerator and SmartNIC offloading.

1. Conceptual Overlap with Prior Work on Accelerator-Centric Architectures: The high-level concept of a SmartNIC acting as the central orchestrator for data and control flow, bypassing the host CPU, has been previously proposed.

   • Lynx [53] proposed an "accelerator-centric" architecture where a SmartNIC offloads both data and control planes, using PCIe P2P to distribute messages directly to other accelerators' memory. While AccDirect's mechanism of writing to a control register (DLB's producer port) is more direct than writing to a memory queue, the fundamental concept of SmartNIC-led P2P dispatch is the same. The authors cite Lynx in their related work (Section 7, page 12) but do not sufficiently distinguish their core idea from it. AccDirect appears to be a highly effective, but specific, instantiation of the Lynx philosophy.

2. Prior Art in Direct P2P Device Control: The idea of one PCIe device directly controlling another via P2P MMIO writes is also not fundamentally new.

   • FlexDriver [11] demonstrated an architecture where an accelerator could host a "NIC driver" to directly control a NIC over PCIe P2P. AccDirect implements the inverse: a NIC (or its embedded CPU) controlling another accelerator (DLB). While the targets are different (on-chip vs. off-chip), the core mechanism of P2P-based device control is conceptually identical. The novelty delta here seems to be in the engineering specifics of targeting the DLB, not in the architectural pattern itself.

3. The "Framework" is a Specific Point-Solution: The authors present AccDirect as a general framework for "accelerator chaining." However, the implementation is tightly coupled to the specifics of the Intel DLB (its BAR structure, queueing mechanism) and a BlueField-3 SmartNIC (using its onboard Arm cores and RDMA capabilities). The choice to use one-sided RDMA verbs as the control primitive (Section 4.2, page 6) is presented as a key enabler, but this appears to be more of a software engineering choice for convenience and generality rather than a fundamental new mechanism. The same control could be achieved with more primitive, direct P2P MMIO writes from the SmartNIC, which is a known capability. This makes the contribution feel less like a new, generalizable framework and more like an exemplary, vertically-integrated system design.

Questions to Address In Rebuttal

1. Please clarify the precise conceptual novelty of AccDirect over Lynx [53] and FlexDriver [11]. Beyond the difference in the specific accelerators being chained, what is the fundamental architectural principle or mechanism in AccDirect that was not already proposed or demonstrated in these prior works? Is the primary contribution the engineering feat of successfully targeting an on-chip, root-complex-integrated device, and if so, what were the non-obvious technical hurdles that make this a distinct scientific contribution?

2. The paper frames AccDirect as a general "framework." To substantiate this claim, could the authors elaborate on how this framework would apply to chaining a different set of accelerators? For example, how would the principles of AccDirect be used to have a SmartNIC directly dispatch compute kernels to an on-chip GPU or schedule tasks on a domain-specific accelerator that does not use a producer-consumer queue model like the DLB?

3. The use of RDMA verbs for P2P control is a major part of the implementation. Is this choice fundamental to the novelty of the idea? Or is it an abstraction layer built on top of the already-existing PCIe P2P MMIO write capability? In other words, does the novelty lie in the use of P2P to control the DLB, or the use of RDMA verbs as the specific API to enact that control?

Reviewer: The Guardian

Summary

The authors present a study of Last-Level Cache (LLC) contention on modern Intel server CPUs, specifically focusing on interactions with high-bandwidth I/O devices. The paper purports to uncover two novel sources of contention: (C1) a "directory contention" where I/O cache lines from DCA-designated ways migrate to hidden "inclusive ways" upon CPU access, conflicting with non-I/O workloads; and (C2) contention within DCA ways between high-throughput storage I/O and latency-sensitive network I/O. Based on these observations, the authors propose A4, a runtime framework that uses Cache Allocation Technology (CAT), performance counters, and a supposedly "hidden" per-device DCA control knob to orchestrate LLC allocation. The framework aims to improve the performance of high-priority workloads (HPWs) without significantly harming low-priority ones (LPWs).

Strengths

1. The paper attempts to tackle a relevant and increasingly challenging problem: managing shared resources (specifically, the LLC) in the face of high-bandwidth I/O, which is critical for modern datacenter performance.
2. The authors provide a step-by-step evaluation (A4-a through A4-d) in Figure 13, which is useful for attributing performance gains to specific components of their proposed solution. This incremental analysis is a methodologically sound way to demonstrate the contribution of each technique.

Weaknesses

My assessment finds significant issues with the paper's foundational claims, methodological rigor, and the robustness of the proposed solution. These must be addressed before the work can be considered for publication.

1. The Central "Directory Contention" Claim (C1) is Built on Speculation, Not Proof: The entire premise of the "directory contention" rests on an observation of increased LLC miss rates for X-Mem when it is allocated to way[9:10] (Figure 3b). The authors then link this phenomenon to a microarchitectural mechanism (migration of I/O cache lines to inclusive ways) described in a prior reverse-engineering study [65]. This is a classic case of correlation presented as causation. The evidence provided is circumstantial at best. The authors fail to provide any direct evidence of this cache line migration. The validation experiment in Figure 4 (disabling DCA removes the contention) only shows that the contention is DCA-related; it does not prove the specific migration mechanism hypothesized by the authors. Alternative explanations, such as complex interactions with the coherence protocol, interconnect contention, or prefetcher behavior, have not been ruled out. Building a significant portion of the paper on an unproven microarchitectural hypothesis is a critical flaw.

2. Unrepresentative Workloads and Lack of Generalizability: The characterization of storage I/O as benefiting "little from DCA" (Section 1, page 1) is a sweeping generalization based on a narrow, and frankly, convenient workload. The experiment in Section 3.2 uses FIO with O_DIRECT and random reads with large block sizes—a workload profile that is explicitly designed to bypass caches and maximize throughput, thus exhibiting poor temporal locality. This is a best-case scenario for the authors' argument but a poor representation of all storage I/O. What about metadata-heavy operations, database index traversals, or other workloads with higher temporal locality that would benefit from being cached via DCA? The paper conveniently ignores these scenarios, which undermines the credibility of claim C2 and the subsequent design choices in A4.

3. The Proposed Solution (A4) is Fragile and Over-Tuned: The A4 framework's robustness is highly questionable.

   • "Hidden Knob": The solution relies on a "hidden feature" (Section 4.2, page 6) to selectively disable DCA for specific PCIe ports. The paper cites register perfctrlsts_0 [26]. Is this a documented, architecturally guaranteed interface for this purpose, or an undocumented MSR that may change or disappear in future CPU steppings or generations? Relying on such features makes the solution brittle and unsuitable for production environments. This is a major practical limitation that the authors do not sufficiently acknowledge.
   • Magic Thresholds: The framework's logic is governed by a multitude of hard-coded thresholds (Table 1, page 9). For example, ANT_CACHE_MISS_THR is set to 90%, and DMALK_DCA_MS_THR to 40%. The sensitivity analysis in Figure 15 is superficial and only explores a small portion of a large, complex parameter space for a single workload mix. It demonstrates that the system is sensitive to these values but provides little confidence that the chosen values are optimal or will generalize to other workloads, systems, or SLOs. The framework appears to be over-fitted to the specific workloads evaluated.

4. Inadequate Comparison with the State-of-the-Art: The evaluation compares A4 against two weak baselines: "Default" (no CAT) and "Isolate" (naive static CAT partitioning). These are well-known strawmen. The paper fails to compare its results against more sophisticated, relevant prior work. For instance, the authors cite Yuan et al. [67] ("Don't forget the I/O when allocating your LLC"), which also proposes a dynamic CAT-based partitioning scheme to mitigate I/O-driven contention. A direct comparison to this or other utility-based dynamic partitioning schemes is essential to properly contextualize the contributions of A4. Without it, the claimed 51% improvement for HPWs is largely meaningless, as it is measured against baselines that are known to perform poorly under contention.

Questions to Address In Rebuttal

The authors must provide clear and convincing answers to the following questions:

1. Can you provide direct microarchitectural evidence (e.g., using specialized performance counters, if available) that I/O cache lines from DCA ways are indeed migrating specifically to the "inclusive ways" (way[9:10]) upon CPU access? If not, how can you definitively rule out other potential causes for the observed contention in Figure 3b?
2. Please justify the generalizability of your storage I/O characterization. Provide data for storage workloads with high temporal locality (e.g., database workloads) and demonstrate that they also do not benefit from DCA, or explain how A4 would handle cases where they do.
3. Clarify the exact nature of the per-device DCA control mechanism. Is this a feature officially documented and supported by the CPU vendor for this purpose? What is the guarantee of its stability across different processor models and microcode updates?
4. Why was A4 not compared against the system proposed in your cited work [67], which appears to address a very similar problem? Please provide such a comparison or a compelling reason for its omission.
5. The A4 framework relies on at least five key thresholds. How would a system administrator realistically set these parameters in a production environment with a dynamic mix of unseen workloads? Please provide a more robust justification for the chosen values beyond the limited sensitivity study presented.

Review Form: The Synthesizer (Contextual Analyst)

---

Summary

This paper identifies and addresses two novel, microarchitecturally-rooted sources of contention in the Last-Level Cache (LLC) of modern datacenter servers. The work focuses on systems with non-inclusive LLCs and high-bandwidth I/O devices, a common configuration in contemporary hardware. The core contributions are twofold: first, the diagnosis of (C1) "directory contention," where I/O data touched by a CPU core unexpectedly migrates to special "inclusive ways" of the LLC directory, contending with non-I/O application data; and (C2) contention within Direct Cache Access (DCA) ways, where high-throughput storage I/O, which benefits little from DCA, pollutes the cache for latency-sensitive network I/O.

To address these issues, the authors propose A4, a runtime LLC management framework. A4 orchestrates LLC resources using existing hardware features like Intel's Cache Allocation Technology (CAT) and performance counters. It intelligently partitions the LLC based on workload priority, safeguards critical I/O buffers in DCA and inclusive ways, and notably, uses a device-specific control knob to selectively disable DCA for antagonistic storage workloads. The evaluation demonstrates that A4 significantly improves the performance of high-priority workloads (by 51%) without harming low-priority ones, effectively untangling these subtle but impactful performance bottlenecks.

---

Strengths

1. Novel and Insightful Problem Diagnosis: The primary strength of this paper lies in its deep and insightful diagnosis of previously unrecognized performance issues. The identification of "directory contention" (Section 3.1, page 4) is a particularly significant contribution. It connects the dots between the reverse-engineered understanding of non-inclusive cache directories (e.g., [65]) and the practical performance implications of I/O data flow. This moves the community's understanding beyond known issues like "DMA bloat" [2] into a more nuanced appreciation of modern cache-I/O interactions.

2. Contextual Relevance and Timeliness: The work is perfectly timed. Datacenters are increasingly deploying servers with non-inclusive LLCs (recent Intel Xeon and AMD Zen CPUs) and ever-faster I/O devices (100+ Gbps NICs and NVMe SSDs). The paper’s finding that storage I/O bandwidth is now on par with network I/O, yet has vastly different cache needs (Section 3.2, page 5), highlights a critical inflection point where old assumptions about DCA being universally beneficial are no longer valid. This paper provides a clear articulation of this emerging problem.

3. Pragmatic and Deployable Solution: A4 is not a theoretical proposal requiring new hardware. It is a software framework built entirely on existing, albeit sometimes obscure, hardware capabilities. By leveraging Intel CAT, performance counters, and a little-known register to control DCA on a per-device basis, the solution is grounded in reality. This pragmatism makes the work immediately relevant to practitioners and operators of large-scale systems looking to improve performance predictability and server utilization.

4. Excellent Explanatory Evaluation: The experimental methodology is a model of clarity. In particular, the incremental evaluation in Figure 13 (page 11), which shows the performance impact of applying each of A4’s strategies (A4-a through A4-d) one by one, is extremely effective. It allows the reader to directly map each proposed solution component to its real-world performance benefit, building a strong and convincing case for the final, complete system.

---

Weaknesses

While the core ideas are strong, the work could be better contextualized and its boundaries more clearly defined.

1. Limited Architectural Scope: The investigation is thoroughly conducted on an Intel Skylake server architecture. However, the non-inclusive LLC design principle has also been adopted by competitors like AMD (Zen architecture) and is present in newer Intel CPUs. The specific mechanism of "inclusive ways" tied to the directory might be an Intel-specific implementation. The paper would be significantly strengthened by a discussion on the generalizability of its findings. For instance, do other non-inclusive directory implementations present similar or different aliasing hazards? Acknowledging this limitation and speculating on the implications for other architectures would broaden the work's impact.

2. Complexity of the Heuristic-Based Framework: The A4 framework relies on a set of five thresholds (T1-T5) and two timing parameters to make its runtime decisions (Section 5.7, page 9). While the authors provide a sensitivity analysis (Figure 15, page 12), this points to a system that may be complex to tune and potentially fragile in production environments with highly dynamic workload mixes. The paper could benefit from a discussion on the vision for deploying such a system—whether these parameters are "set-and-forget" or would require sophisticated, automated online tuning.

3. Understated Relationship to Broader OS/Runtime Scheduling: A4 is essentially a resource scheduler for the LLC. It operates based on a static priority ("HPW" vs "LPW"). In many modern datacenters, workload priority is dynamic and managed by cluster schedulators (like Borg or Kubernetes). The paper misses an opportunity to connect its work to this larger ecosystem. How might A4 integrate with a cluster scheduler that might change a workload’s priority or QoS class on the fly?

---

Questions to Address In Rebuttal

1. Generalizability of "Directory Contention": Your discovery of directory contention is fascinating and hinges on the specific coupling of data ways and directory ways in the Skylake microarchitecture. Could you elaborate on whether you expect this specific contention to exist in newer Intel CPUs (e.g., Ice Lake, Sapphire Rapids) or in AMD's Zen architectures? Does the core principle—that I/O data touched by a core must have its coherence state tracked in a specific, limited directory structure that may alias with application data—hold more generally, even if the implementation details differ?

2. Practicality of Tuning A4: The A4 framework's effectiveness relies on several hand-tuned thresholds. In a real-world datacenter, how do you envision these parameters being set and maintained? Would this require expert human intervention for each new server generation or major software stack, or do you believe a simple, robust set of default values exists?

3. On the "Hidden Knob" for DCA Control: You mention using the perfctrlsts_0 register to selectively disable DCA for storage devices (Section 4.2, page 6). Could you clarify the nature of this control? Is this a documented, officially supported feature for this purpose, or is it an undocumented capability discovered through reverse engineering? The answer would help in assessing the long-term viability and robustness of this part of your solution.

Review Form: The Innovator (Novelty Specialist)

Summary

The authors present a work that identifies two allegedly novel Last-Level Cache (LLC) contention sources in modern Intel server CPUs and proposes a runtime management framework, A4, to mitigate them. The first claimed novel insight (C1) is that DMA-written I/O data, upon being accessed by a CPU core, migrates from Direct Cache Access (DCA) ways to specific, hidden "inclusive ways" tied to the LLC's directory structure, causing a new form of contention with non-I/O workloads. The second insight (C2) is that modern high-bandwidth storage-I/O devices pollute the DCA ways, harming co-located network-I/O, while deriving little benefit themselves. The proposed solution, A4, orchestrates existing hardware features, most notably Intel's Cache Allocation Technology (CAT) and a little-known knob for per-device DCA disabling, to implement mitigation policies derived from these insights.

My evaluation is focused solely on whether these contributions represent a genuine advancement over the state of the art.

Strengths

The primary strength of this paper lies in the identification of what appears to be a genuinely new, microarchitecturally-specific contention mechanism.

1. Novel Insight into Directory Contention (C1): The core novel contribution is the discovery and characterization of "directory contention" as described in Section 3.1 (page 4). While prior art has extensively documented I/O-induced LLC contention (e.g., latent contention [67], DMA bloat [2]), the specific mechanism identified here is new. It builds upon the reverse-engineering work of Yan et al. [65] that exposed the non-inclusive LLC's directory structure, but it goes a step further by identifying the dynamic behavior of I/O cache lines migrating into these specific "inclusive ways". This is a non-obvious performance pathology and represents a true contribution to the community's understanding of modern cache hierarchies.

2. Novel Application of a Hardware Feature (F2): The use of a runtime-accessible knob to selectively disable DCA for specific I/O devices (Section 4.2, page 7) is a significant and novel engineering contribution. The prevailing understanding is that DCA is typically toggled system-wide via BIOS [22]. Exposing and utilizing a mechanism (reportedly via the perfctrlsts_0 register) for fine-grained, per-port control at runtime is a new capability that enables a class of solutions previously considered infeasible. This is a valuable discovery.

3. Novel Policies Derived from Insights: The proposed mitigation strategies are not generic; they are tightly coupled to the novel findings. The "n-Overlap" allocation strategy (Section 4.1, page 6), which intentionally allocates I/O workloads to overlap with inclusive ways to maximize caching efficiency, is a clever and non-obvious policy that would not have been conceived without the insight from C1.

Weaknesses

While the work contains kernels of true novelty, several aspects are incremental advancements or applications of well-known concepts to a new problem context.

1. Incremental Nature of Storage-I/O Contention (C2): The contention between high-bandwidth storage and network I/O in DCA ways (Section 3.2, page 5) is a timely but arguably foreseeable problem. The concept of DCA pollution is not new [18]. The novelty here is in identifying a new, high-bandwidth aggressor (fast NVMe SSDs). Given the known performance characteristics of storage I/O (large blocks, poor temporal locality for streaming reads), its negative interaction with latency-sensitive network packets within a shared resource is an expected outcome. The contribution is one of characterization and quantification, which is valuable, but it lacks the fundamental surprise of the C1 finding.

2. Established Framework Design Pattern: The overall architecture of A4—a runtime daemon that monitors hardware performance counters and dynamically adjusts LLC partitions using CAT—is a well-established design pattern in the literature for performance management (e.g., [48], [66], [67]). The novelty of A4 is not in its structure, but in the specific rules and heuristics it implements, which are derived from the aforementioned insights. The paper should be careful not to overstate the novelty of the framework itself.

3. Conceptual Overlap in Mitigation Techniques: The concept of "pseudo LLC bypassing" (Section 5.5, page 8), where antagonistic workloads are allocated a minimal number of "trash ways," is functionally similar to prior work on utility-based partitioning [48] and dead block management [33, 37], which also seek to identify and isolate cache-unfriendly access streams to prevent them from polluting the cache for other, more deserving workloads. The authors' application of this concept to DMA-bloated I/O streams is sound, but the underlying principle is not new.

Questions to Address In Rebuttal

1. On Directory Contention (C1): The work of Yuan et al. [67] previously identified "latent contention" where non-I/O workloads contend with network-I/O in DCA ways. Please clarify precisely how your "directory contention" differs. Is it a completely separate mechanism, or an evolution of the same root cause now better explained by the non-inclusive directory structure?

2. On the Per-Device DCA Knob (F2): The ability to disable DCA per-port is a cornerstone of your solution to C2. Is this a formally documented and stable feature in Intel's architecture, or is it an undocumented Model-Specific Register (MSR) that could change or be removed in future silicon? The generality and future-proofing of this key mechanism depend on the answer.

3. On Pseudo LLC Bypassing: Please explicitly compare your "pseudo LLC bypassing" for antagonistic I/O workloads to prior academic and industry work on identifying and managing cache-unfriendly or "streaming" data, such as utility-based cache partitioning (UCP) or dead block prediction. What is the fundamental delta that makes your approach novel beyond its application to I/O-generated data?

Title: HardHarvest: Hardware-Supported Core Harvesting for Microservices Reviewer: The Guardian

---

Summary

The authors propose HardHarvest, a hardware architecture designed to support core harvesting in microservice environments. The paper identifies two primary overheads in existing software-based approaches: the latency of hypervisor-driven core reassignment and the performance penalty from flushing/invalidating private caches and TLBs. To address this, HardHarvest introduces three main hardware features: 1) a hardware request scheduler with per-VM queues to accelerate core reassignment, 2) way-partitioning of private caches/TLBs into "Harvest" and "Non-Harvest" regions to preserve Primary VM state, and 3) a novel replacement algorithm that attempts to steer shared application data into the preserved "Non-Harvest" region. The evaluation, conducted via full-system simulation, claims that HardHarvest significantly reduces Primary VM tail latency by 6.0x and increases Harvest VM throughput by 1.8x compared to a state-of-the-art software baseline.

Strengths

1. Well-Defined Problem: The paper correctly identifies a significant and timely problem. The overheads associated with software-based core harvesting, particularly in the context of latency-sensitive microservices, are substantial and well-articulated in Section 3. The motivational analysis is convincing in establishing the potential for a hardware-based solution.
2. Comprehensive Design: The proposed solution is multifaceted, addressing both the control plane (core reassignment) and data plane (cache state) aspects of the problem. This shows a thorough understanding of the challenges involved.
3. Detailed Breakdown of Gains: The cumulative breakdown of performance improvements in Figure 12 is a methodologically sound way to attribute gains to specific architectural features, which is commendable.

Weaknesses

My primary role is to ensure the rigor and validity of new claims against the established state-of-the-art. In this capacity, I have identified several critical weaknesses that challenge the paper's core conclusions.

1. Potentially Unfair Baseline Comparison: The central claim of a 6.0x tail latency reduction rests on the comparison against "state-of-the-art software core harvesting" (Harvest-Term), which is modeled after SmartHarvest [88]. However, a crucial feature of SmartHarvest for mitigating tail latency is its use of an "emergency buffer" of idle cores that can be reclaimed instantly without software overhead. The authors mention this feature in Section 4 (Page 4) but it is entirely unclear whether their simulated software baseline (Harvest-Term) actually implements this buffer. Without it, the baseline is significantly weakened, and the comparison is arguably one against a strawman. The dramatic latency reduction could simply be an artifact of crippling the baseline.

2. Conflation of General-Purpose and Harvesting-Specific Gains: The results in Figure 11 show that HardHarvest not only improves upon software harvesting but also achieves significantly lower tail latency than the NoHarvest baseline. The authors attribute this to "improved cache/TLB replacement and request queuing" (Section 6.1, Page 11). This is a critical issue. It implies that a substantial portion of the claimed benefit is derived from a general-purpose hardware scheduler and a new cache policy, not from the harvesting mechanism itself. The paper is framed as a solution for harvesting, but the evidence suggests it is a paper about a new scheduler that also happens to do harvesting. This conflates two separate contributions and overstates the benefits attributable to the novel harvesting architecture.

3. The Fragility of the Cache Replacement Heuristic: The proposed replacement algorithm (Algorithm 1, Section 4.2.3) is entirely dependent on a heuristic to classify pages as "shared" or "private." The paper proposes a simple temporal heuristic: memory allocated before server.serve() is shared, and memory allocated after is private. This is a fragile assumption that is unlikely to hold for many real-world microservices that might perform lazy initialization, use dynamic data structures, or rely on just-in-time compilation. The paper presents no sensitivity analysis on the accuracy of this classifier. If pages are frequently misclassified, this "smart" replacement policy could easily degrade into a worst-case scenario, polluting the protected Non-Harvest region with transient data or evicting critical shared data. The entire benefit of cache partitioning rests on this unvalidated heuristic.

4. Understated Hardware Complexity and Cost: The paper proposes a non-trivial centralized hardware controller (Figure 9) with dedicated network links, multiple Queue Managers, and VM State Registers. The cost analysis in Section 6.8, based on McPAT, focuses almost exclusively on the storage overhead of the request queue and extra cache bits, concluding a mere 0.19% area overhead. This seems to grossly underestimate the true cost. The analysis likely neglects the area and power of the complex control logic for the scheduler, the cross-VM interrupt mechanism, the dynamic queue management, and the priority multiplexers for the new replacement policy. Furthermore, the centralized design presents a potential scalability bottleneck for future processors with hundreds of cores, a concern that is not addressed.

Questions to Address In Rebuttal

The authors must address the following points directly and with specific evidence from their experiments to convince me of the validity of their work.

1. On the Baseline's Fidelity: Please confirm whether your simulated software baseline (Harvest-Term) implements the emergency core buffer from SmartHarvest [88]. If it does not, please justify why this is still a fair "state-of-the-art" comparison and how your conclusions would change if it were included.
2. On Deconflating Contributions: Can you provide an evaluation that isolates the performance benefit derived purely from the harvesting-specific mechanisms (i.e., cache partitioning and the associated replacement policy) from the benefits of the general-purpose hardware request scheduler and queues? For example, by comparing NoHarvest against a NoHarvest+HardHarvestScheduler configuration.
3. On the Replacement Heuristic's Robustness: What is the performance impact on a Primary VM if the shared/private page classification heuristic has a high error rate (e.g., 25% or 50% misclassification)? Please provide a sensitivity analysis or a more robust defense of why this simple heuristic is sufficient for a general-purpose hardware mechanism.
4. On Hardware Cost Realism: Can you provide a more detailed breakdown of the hardware cost analysis that includes the control logic for the scheduler, queue managers, and replacement policy, not just the storage elements? Please also comment on the scalability of the centralized controller design to processors with significantly more than 36 cores.

Paper: HardHarvest: Hardware-Supported Core Harvesting for Microservices Reviewer: The Synthesizer

---

Summary

This paper presents HardHarvest, a novel hardware architecture designed to enable efficient core harvesting in microservice environments. The authors identify a critical bottleneck in existing software-based harvesting techniques (e.g., SmartHarvest): the high overhead of reassigning cores between Virtual Machines (VMs), which involves hypervisor calls and extensive cache/TLB flushes. These overheads, while tolerable for monolithic applications, are prohibitive for latency-sensitive microservices that operate on sub-millisecond timescales.

HardHarvest's core contribution is a holistic, hardware-first solution that tackles these overheads directly. It proposes a two-pronged approach: 1) a hardware-based request scheduling system with per-VM queues that allows cores to be reassigned between Primary (latency-critical) and Harvest (batch) VMs without hypervisor intervention, and 2) a hardware-managed partitioning of private caches and TLBs into "Harvest" and "Non-Harvest" regions. This partitioning scheme preserves the Primary VM's "hot" state in the Non-Harvest region during harvesting, dramatically reducing the cold-start penalty upon core reclamation. The evaluation, performed via full-system simulation, demonstrates significant benefits, most notably reducing the tail latency of Primary VMs by 6.0x compared to software harvesting, to the point where it is even better than a non-harvesting baseline, while simultaneously increasing Harvest VM throughput by 1.8x.

Strengths

This work's primary strength lies in its excellent synthesis of a critical industry problem with a well-motivated and elegant architectural solution.

1. Timely and Significant Problem: The paper addresses a problem of immense practical and economic importance. The tension between provisioning for peak load (leading to low average utilization) and meeting strict tail-latency SLOs is a central challenge in modern cloud infrastructure. The authors correctly identify that existing software solutions for resource harvesting, developed in the context of longer-running applications, are a poor fit for the microsecond-scale world of microservices. This work is therefore not just an academic exercise but a direct response to a real-world architectural gap.

2. A Conceptual Leap in Core Harvesting: The move from software to hardware for this task is a significant conceptual advance. While prior work has proposed hardware support for microservice scheduling (e.g., µManycore [76], RPCValet [15]), HardHarvest is, to my knowledge, the first to propose a comprehensive hardware framework for resource elasticity at this granularity. It connects the dots between datacenter economics and microarchitecture in a compelling way.

3. Holistic and Co-Designed Solution: The true novelty is not in any single component but in the co-design of the two main features. The hardware scheduler solves the core reassignment latency, while the cache partitioning solves the resulting microarchitectural cold-start problem. This integrated approach shows a deep understanding of the full performance stack, from the hypervisor down to the cache replacement policy. The proposed replacement algorithm (Algorithm 1, page 9), which attempts to steer shared vs. private pages, is a particularly thoughtful refinement that demonstrates this depth.

4. Strong Quantitative Motivation: The "Motivation" section (Section 3, pages 3-4) is exemplary. By systematically measuring and presenting the overheads of hypervisor calls (Figure 4) and cache flushes (Figure 5), the authors build an undeniable case for a hardware-level intervention. This data-driven motivation makes the subsequent design feel necessary rather than contrived.

Weaknesses

The weaknesses of the paper are primarily related to the broader systems context and potential complexities that are not fully explored.

1. System Complexity and Interaction: The paper presents HardHarvest as a self-contained module. In a real processor, this logic would need to interact with a host of other complex features: hardware security enforcers, performance monitoring units, NUMA optimizations, and emerging protocols like CXL. For instance, how would the hardware scheduler's decisions interact with the OS/hypervisor's broader power management or process scheduling policies? The paper could be strengthened by acknowledging and briefly discussing these integration challenges.

2. Security Implications Beyond Flushing: The security model is predicated on flushing the "Harvest" region of the caches to prevent direct data leakage. While this is a necessary first step, the design—which intentionally preserves a Primary VM's state on a core while a Harvest VM executes—creates a novel scenario for side-channel analysis. Could rapid, hardware-managed switching create new timing channels related to resource contention in the core's backend or the control logic of the cache partitioning itself? The brief mention of adding a delay to prevent timing channels (page 8) feels insufficient for a system operating at this level of intimacy between tenants.

3. Robustness of Workload Assumptions: The efficacy of the advanced cache replacement policy hinges on the ability to reliably distinguish between "shared" (e.g., application code, read-only data) and "private" (request-specific) memory pages. The proposed heuristic (based on allocation time relative to a server.serve() call) is clever and likely works for the C++/gRPC-style services evaluated. However, its applicability to microservices built in other ecosystems (e.g., Java with a JIT, Go with its own runtime scheduler) is not obvious. The performance of the system could be sensitive to the accuracy of this classification, making this a potential point of fragility.

Questions to Address In Rebuttal

1. On System Integration: Could the authors comment on the potential interactions between the HardHarvest controller and other modern SoC features, such as Intel's Thread Director or system-wide power management policies? Is there a risk of the HardHarvest hardware scheduler and the OS/hypervisor schedulers working at cross-purposes?

2. On Side-Channel Security: The paper's security model relies on flushing the Harvest region. Could the authors elaborate on potential timing channels or other side-channels that might arise from a Primary VM's state being preserved in the Non-Harvest region while a Harvest VM is executing on the same core? For example, could the Harvest VM infer information by observing the performance effects of the replacement policy (Algorithm 1) acting on the Primary VM's hidden state?

3. On the Generality of the Replacement Policy: The efficacy of the specialized replacement policy hinges on accurately identifying shared pages. How sensitive are the performance benefits to the accuracy of this heuristic? Have the authors considered how this heuristic would apply to microservices built with different frameworks (e.g., those using garbage collection or JIT compilation) where the distinction between shared and private allocations may be less clear?

Reviewer: The Innovator (Novelty Specialist)

---

Summary

This paper proposes HardHarvest, a hardware architecture designed to support core harvesting specifically for microservice environments. The authors' central claim is that this is the first hardware-based solution to this problem, addressing the prohibitive overheads of existing software-based approaches. The core of the proposed architecture consists of two main components: 1) a hardware-based request scheduler that manages core re-assignment between "Primary" (latency-critical) and "Harvest" (batch) VMs without hypervisor intervention, and 2) a hardware mechanism for partitioning private caches/TLBs, coupled with a novel replacement algorithm (Algorithm 1, page 9) that aims to preserve the Primary VM's "warm" state by prioritizing shared pages in a protected partition. The authors claim their solution significantly improves core utilization and Harvest VM throughput while reducing Primary VM P99 tail latency by 6.0x compared to state-of-the-art software harvesting.

---

Strengths

The primary strength of this paper lies in its novel synthesis of architectural concepts to solve a well-motivated problem. While individual components of the solution have roots in prior work, their combination and specific application to inter-VM core harvesting for microservices appears to be genuinely new.

1. Novel Problem-Architecture Mapping: The core idea of creating a dedicated hardware architecture for core harvesting is, to my knowledge, novel. Prior art has focused on software techniques for harvesting (e.g., SmartHarvest [88]) or hardware acceleration for RPC scheduling (e.g., µManycore [76], ALTOCUMULUS [96]), but not the explicit, hardware-managed, dynamic lending and reclamation of cores between different VM types.

2. Novel Cache/TLB Replacement Policy: The most distinct and novel technical contribution is the proposed cache/TLB replacement algorithm (Section 4.2.3). The idea of partitioning the cache is not new (e.g., Intel CAT), but the policy itself is. It introduces awareness of both the region type (Harvest vs. Non-Harvest) and the page type (shared vs. private) into the victim selection process. This fine-grained, policy-driven management to selectively preserve a Primary VM's working set across preemption is a clever microarchitectural technique that I have not seen proposed in prior work.

3. Significant Delta Over Prior Art: The proposed hardware scheduler extends prior work on RPC schedulers by adding logic specific to harvesting: it is aware of VM types (Primary/Harvest), manages core "loans," and handles the preemption and reclamation protocol (Section 4.1.5). This is a non-trivial delta that makes it fundamentally different from a simple request scheduler.

---

Weaknesses

My critique focuses on precisely delineating the novel contributions from the integration of existing ideas, and questioning the robustness of the assumptions upon which the novelty rests.

1. Conflation of Novelty with Integration: The paper presents HardHarvest as a monolithic novel architecture. However, it is more accurately an integration of several concepts, some novel and some drawn directly from prior work. For example, the fast context switching is attributed to µManycore [76], and the need for an efficient flush/invalidate mechanism is acknowledged as a known problem with existing solutions [30, 51]. The impressive 6.0x tail latency reduction is therefore a result of the entire system, and it is difficult to isolate the benefit derived purely from the novel components (the VM-aware scheduler and the replacement policy) versus the benefit from simply implementing previously-proposed hardware accelerations for context switching and cache management. The breakdown in Figure 12 (page 11) is cumulative, which makes it hard to assess the standalone value of the truly new ideas.

2. Novelty Reliant on a Heuristic: The effectiveness of the novel replacement policy (Algorithm 1) is entirely dependent on a software heuristic for classifying pages as "shared" or "private" (Section 4.2.2, page 8). The paper suggests a simple rule: data allocated before server.serve() is shared. While plausible for the frameworks studied, this is not a hardware novelty and may be fragile. The paper does not sufficiently explore the generalizability of this heuristic. If the heuristic fails, the core benefit of the novel replacement algorithm is nullified, even if the hardware is implemented perfectly.

3. Complexity vs. Benefit of the Novel Components: The proposed hardware is significant, requiring a centralized controller, dedicated network, and modifications to every core's cache/TLB replacement logic. While the overall performance gain is large, it is unclear how much of that gain is attributable to the new, complex parts of the design. A simpler design that implements hardware context switching and static cache partitioning (using existing techniques) might achieve a substantial portion of these gains. The paper lacks an ablation study to justify the added complexity of its novel scheduler and replacement policy over simpler hardware extensions.

---

Questions to Address In Rebuttal

1. The central claim is that HardHarvest is the "first" architecture for core harvesting in hardware. While it appears novel in the context of mainstream CPU architecture, can the authors elaborate on whether similar concepts of hardware-managed resource borrowing/lending have been explored in adjacent fields, such as reconfigurable computing (FPGAs) or specialized manycore processors, even if not explicitly termed "core harvesting"?

2. To better isolate the paper's novel contributions, can the authors provide a more direct "apples-to-apples" comparison? For instance, what would the performance be with a baseline that includes previously-proposed hardware context switching and efficient flushing, but uses a simpler hardware scheduler (not VM-aware) and standard cache partitioning (e.g., static ways via CAT) without the novel replacement policy? This would help quantify the specific benefit of the new ideas presented in this work.

3. The novel replacement policy's success is predicated on the shared/private page classification heuristic. How would the performance of HardHarvest degrade if this heuristic were, for instance, only 50% accurate? A sensitivity analysis on the accuracy of this software-level assumption would strengthen the claims about the hardware's utility.

Paper Title: MOPAC: Efficiently Mitigating Rowhammer with Probabilistic Activation Counting Reviewer: The Guardian (Adversarial Skeptic)

---

Summary

This paper identifies the significant performance overhead (cited as 10% on average) of the JEDEC PRAC (Per-Row Activation Counter) mechanism, which is caused by the extended DRAM timings required for every activation's counter update. To mitigate this overhead, the authors propose MoPAC (Mitigating Rowhammer with Probabilistic Activation Counts), a scheme that performs these counter updates probabilistically rather than deterministically. By updating counters for only a small fraction of activations (with probability p), the performance penalty is incurred less frequently. To maintain security guarantees, MoPAC revises the Alert Threshold (ATH) downwards to a new value, ATH*, which is calculated to ensure a target Mean Time To Failure (MTTF) against attacks. The paper presents two implementations: MoPAC-C, a memory controller-side solution requiring two precharge command variants, and MoPAC-D, a fully in-DRAM solution using a small queue (SRQ) to buffer pending updates. The authors claim that for a Rowhammer threshold (TRH) of 500, MoPAC-C and MoPAC-D reduce the 10% PRAC slowdown to 1.7% and 0.7%, respectively.

Strengths

1. Well-Motivated Problem: The paper correctly identifies a critical and practical barrier to the adoption of PRAC. The 10% average performance overhead demonstrated in Figure 2 is a significant cost for a security feature, and addressing this is a worthwhile endeavor.
2. Novel Core Concept: The central idea of trading deterministic, high-cost checks for probabilistic, low-cost checks (compensated by a more sensitive threshold) is a clever approach to amortizing the performance penalty of PRAC.
3. Comprehensive Proposal: The authors present two distinct and well-reasoned implementations (MoPAC-C and MoPAC-D) that explore different points in the design space (MC vs. DRAM-centric), demonstrating a thorough consideration of potential deployment paths.

Weaknesses

My primary responsibility is to ensure the rigor and correctness of the work. Despite its clever premise, the paper's security claims rest on a foundation with several critical, unaddressed flaws.

1. Fundamentally Unjustified Independence Assumption in Security Model: The entire security analysis hinges on the derivation of the acceptable row failure probability, ε. In Section 5.3 (page 7), for a double-sided Rowhammer attack, the paper claims the overall escape probability is Pe2 = Pe1 * Pe1, where Pe1 is the probability of a single aggressor escaping mitigation. This implies that the probabilistic events of failing to mitigate the first aggressor and failing to mitigate the second aggressor are statistically independent. This is a monumental and entirely unsubstantiated assumption. System-level phenomena such as voltage noise, temperature fluctuations, or weaknesses in a hardware pseudo-random number generator could easily introduce correlation. A single period of low entropy from the RNG could cause a burst of missed updates for both aggressors simultaneously, completely invalidating the Pe1 = sqrt(F) derivation in Equation 5. Without a rigorous justification for this independence assumption, the entire security proof is unsound.

2. Inadequate Analysis of Performance Attacks: The analysis of Denial-of-Service attacks in Section 7 is superficial and relies on average-case behavior, which is insufficient for security analysis. The multi-bank attack model (Section 7.2) calculates slowdown based on an average α value of 0.55, derived from a Monte Carlo simulation. An adversary will not target the average case; they will seek to engineer the worst case. The paper provides no analysis of the distribution of α or its minimum possible value. An attacker could potentially find patterns that synchronize the probabilistic updates across banks to achieve a much lower α, leading to a far more effective DoS attack than the paper suggests. A security analysis must be based on worst-case, not average-case, scenarios.

3. Unaddressed Timing Side-Channel Vulnerability: The threat model in Section 2.1 is incomplete. MoPAC-C introduces two precharge commands, PRE and PREcu, with different latencies (as implied by Figure 6, where PRE uses tRP (norm) and PREcu uses tRP (PRAC)). This creates a classical timing side-channel. An attacker can precisely measure the precharge latency to determine whether a counter update occurred or not for a given activation. This leaks the outcome of the random number generator. A sophisticated attacker could use this information to adapt their activation patterns, waiting for a series of non-updates (confirmed via the side-channel) before launching a burst of activations, thereby defeating the probabilistic premise. The paper completely fails to acknowledge or analyze this vulnerability.

4. Discrepancy in Performance Claims and Understated Costs: The abstract and introduction prominently feature the impressive 0.7% slowdown for MoPAC-D at TRH=500. However, Appendix-A reveals that when protecting against Row-Press, the slowdown for MoPAC-D at TRH=500 balloons to 6.8%. Row-Press is a known, practical attack, and omitting its impact from the headline results is misleading. Furthermore, the paper neglects any discussion of the area, power, and, most importantly, verification complexity of the logic required for MoPAC-D's in-DRAM SRQ, its counters, and the associated scheduling/draining logic. Dismissing the cost by comparing SRAM bytes to TRR is insufficient.

Questions to Address In Rebuttal

1. Please provide a rigorous justification for the assumption in Section 5.3 that the probabilistic mitigation failures for the two aggressor rows in a double-sided attack are statistically independent events. What happens to your security guarantees if a weak RNG or systemic noise introduces even a small correlation?
2. Your performance attack analysis in Section 7.2 relies on an average-case parameter α=0.55. Please provide a worst-case analysis. What is the minimum achievable α an attacker could force, and what is the corresponding DoS impact?
3. Please address the timing side-channel inherent in MoPAC-C's design (differing latencies for PRE and PREcu). How do you prevent an attacker from measuring this latency to learn the outcome of the probabilistic choice and adapt their attack strategy accordingly?
4. Please clarify the discrepancy between the 0.7% slowdown for MoPAC-D claimed in the abstract and the 6.8% slowdown reported in Appendix-A when defending against Row-Press. Given that Row-Press is a realistic threat, which of these numbers is the more accurate representation of the mechanism's real-world performance overhead?

Reviewer Persona: The Synthesizer (Contextual Analyst)

---

Summary

This paper addresses a critical and timely problem in hardware security: the significant performance overhead of the JEDEC-proposed Per-Row Activation Counting (PRAC) mechanism for mitigating Rowhammer. While PRAC offers a principled and robust defense against even future, low-threshold attacks, its ~10% average performance penalty—incurred by extending DRAM timings on every activation—presents a major barrier to its widespread adoption.

The authors' core contribution is a simple yet profoundly effective idea: decouple the high cost of counter updates from every memory activation by performing them probabilistically. Termed MoPAC (Mitigating Rowhammer with Probabilistic Activation Counts), their approach only incurs the latency penalty for a small fraction of activations. To maintain security, the mechanism compensates for the sparse sampling by increasing the counter's increment value and carefully lowering the mitigation threshold (referred to as ATH*). The paper presents a rigorous security analysis to derive these safe parameters based on a target Mean Time to Failure (MTTF). Two practical implementation variants are proposed: MoPAC-C (a memory controller-side solution) and MoPAC-D (a fully in-DRAM solution), demonstrating a clear understanding of the hardware ecosystem. The results are compelling, showing that MoPAC reduces the 10% overhead of PRAC to less than 2% for a near-future Rowhammer threshold of 500.

Strengths

1. Addresses a Crucial, Real-World Bottleneck: The primary strength of this paper is its direct and effective targeting of what is arguably the single largest impediment to the adoption of principled in-DRAM Rowhammer defenses. The academic and industrial consensus is moving towards exhaustive tracking mechanisms like PRAC. This work doesn't fight that trend; instead, it makes the trend practical. By transforming PRAC from a theoretically sound but performant-prohibitive solution into a viable one, the work has the potential for significant real-world impact.

2. Elegant and Parsimonious Core Idea: The central concept of applying probabilistic sampling is elegant. It doesn't require a radical redesign of the DRAM array or a complex new tracking structure. Instead, it applies a well-understood technique from computer science (probabilistic counting/sampling) to a new domain with excellent results. This parsimony is a hallmark of strong systems research. The further refinement presented in Section 8 (Non-Uniform Probability) is a thoughtful extension that shows how the core idea can be optimized even more by observing real-world access patterns.

3. Connects Security Theory with System Performance: The authors do not simply propose a heuristic. The security analysis in Section 5.3 is the lynchpin of the paper's credibility. By grounding the choice of probability p and the revised threshold ATH* in a formal model based on acceptable failure rates (Bank-MTTF), they provide a framework for reasoning about the security-performance trade-off. This principled approach is essential for any security mechanism to be taken seriously.

4. Well-Positioned within the Literature: The paper does a good job of situating itself in the broader context of Rowhammer mitigation. It correctly identifies the failure of early ad-hoc solutions like TRR and acknowledges the high storage cost of other principled trackers (ProTRR, Mithril), thereby motivating the industry's shift to PRAC. Crucially, in Section 9.2, it provides a compelling comparison against other low-cost trackers like MINT and PrIDE, arguing that MoPAC's probabilistic counter updates are a more efficient use of limited mitigation time "borrowed" from the refresh budget. This contextualizes MoPAC not just as a PRAC-optimizer, but as a competitive design in the broader space of low-overhead solutions.

Weaknesses

My criticisms are less about fundamental flaws and more about the boundaries and practical implications of the proposed idea.

1. Potential for a New Performance Attack Vector: While the authors analyze performance attacks in Section 7, the analysis is based on a simplified model. Introducing randomness creates a new surface for an adversary to potentially exploit. An attacker could craft access patterns to try and force worst-case scenarios for the probabilistic mechanism, leading to more frequent ABO stalls than with a deterministic system. The paper concludes the impact is less severe than other contention attacks, which is a fair point, but the nature of this new probabilistic vulnerability could be explored more deeply.

2. Understates the Challenge of Standardization: The paper proposes modifications that, while conceptually simple, would require changes to the JEDEC standard. MoPAC-C requires two types of precharge commands, and MoPAC-D requires new state and logic within the DRAM chip itself. The process of amending memory standards is notoriously complex and slow, involving consensus from multiple competing vendors. The paper could benefit from a more explicit discussion of the path to standardization and the potential pushback or implementation challenges from SoC and DRAM vendors.

3. The Fate of MoPAC is Tied to PRAC: The proposal's relevance is entirely contingent on the industry's continued commitment to the PRAC framework. If a different, unforeseen mitigation strategy emerges that supplants PRAC (e.g., a breakthrough in DRAM cell technology that is inherently Rowhammer-resistant, or a radically different low-cost tracker), then this work, while clever, would be solving a problem that no longer exists. This is not a flaw in the work itself, but a contextual risk regarding its long-term impact.

Questions to Address In Rebuttal

1. Regarding the performance attack analysis (Section 7), the throughput loss model 7/(N+7) is a useful first-order approximation. Could the authors elaborate on how this model might change in the face of more complex, multi-bank attack patterns that are specifically designed to exploit the probabilistic nature of MoPAC? How confident are the authors that this simplified model captures the true worst-case performance degradation?

2. Could you elaborate on the practical path to adoption for MoPAC-C and MoPAC-D? What specific changes would be needed in the JEDEC DDR5/DDR6 specifications? For MoPAC-D, what is the estimated silicon area/complexity cost for the per-bank SRQ and associated logic, and how might this affect DRAM vendor cost models?

3. The core philosophy of this paper is to trade deterministic, high-cost security operations for probabilistic, low-cost ones while maintaining a quantifiable security guarantee. Do you see this philosophy being applicable to other areas of hardware security beyond Rowhammer? For example, could similar probabilistic techniques be used to reduce the overhead of memory encryption/integrity counters or other security monitoring features?

Review Form: Innovator (Novelty Specialist)

---

Summary

The paper identifies the significant performance overhead (~10%) of the emerging JEDEC PRAC (Per-Row Activation Counter) standard for Rowhammer mitigation as a major barrier to its adoption. The authors propose MoPAC (Mitigating Rowhammer with Probabilistic Activation Counting), a mechanism to drastically reduce this overhead. The core idea is to perform the costly PRAC counter updates not on every activation, but probabilistically with a small probability p. By doing so, the performance penalty is incurred only for a small fraction of activations. To maintain security guarantees, the proposal includes a rigorous statistical analysis to derive a new, lower Alert Threshold (ATH*) that accounts for potential undercounting due to sampling. The authors present two implementations: a memory-controller-side version (MoPAC-C) and a fully in-DRAM version (MoPAC-D).

---

Strengths

The primary strength of this work lies in its core conceptual contribution: the application of probabilistic counting to the PRAC framework. While PRAC itself is an industry-led evolution of per-row counting concepts (Intel Patent [9], Panopticon [2]), its high, deterministic overhead has been a known, unsolved problem. This paper presents a clean, elegant, and seemingly novel solution to that specific problem.

1. Novel Problem-Solution Pairing: The application of probabilistic methods to reduce the overhead of an exhaustive, per-row tracking mechanism appears to be new. Prior art has explored sampling for limited-entry trackers (e.g., MINT [32] uses deterministic sampling to populate a small tracker) or randomization of the physical space (e.g., RRS [35] randomizes row mappings). However, MoPAC's approach of managing the update frequency of all counters in a full PRAC system is a distinct and novel contribution. It directly targets the latency of the read-modify-write operation inherent to PRAC, which previous academic work on trackers did not need to address.

2. Novel Security Framework for a Probabilistic Defense: A probabilistic mechanism is insecure without a model to bound the failure rate. The security analysis presented in Section 5.3 (page 7) is a crucial and novel component of this work. Deriving the acceptable failure probability (ε) from a target Bank-MTTF and then using the binomial distribution to calculate the required number of critical updates (C) and the revised ATH* provides the theoretical foundation that makes the probabilistic approach viable and trustworthy. This moves the idea from a simple heuristic to a principled defense.

3. Exploration of the Design Space: The proposal of two distinct implementations (MoPAC-C and MoPAC-D) demonstrates a thoughtful exploration of how this novel concept could be integrated into the existing system stack. This is not merely an algorithmic proposal but a tangible architectural one, considering the practical trade-offs between modifying the memory controller and adding complexity within the DRAM chip.

---

Weaknesses

My critique is centered on the precise boundaries of the novelty, as the underlying components are not new in isolation.

1. Incrementalism of NUP: The core intellectual contribution is the introduction of uniform probability to PRAC updates. The extension to Non-Uniform Probability (NUP) in Section 8 (page 12), while clever and effective at further reducing overhead, feels like a logical optimization rather than a separate novel concept. The fundamental leap is made in proposing MoPAC; NUP is an incremental refinement on top of it.

2. Derivative Implementation Primitives: The two proposed implementations, MoPAC-C and MoPAC-D, are logical consequences of the central probabilistic concept. MoPAC-C's requirement for two precharge commands (Section 5.1, page 6) is a simple extension of existing JEDEC capabilities (e.g., PRE vs. PRES). MoPAC-D's in-DRAM queue (SRQ) is a standard architectural pattern. While these are the first proposed implementations for this specific purpose, the building blocks themselves are not novel. The novelty resides entirely in the reason for their use, not the structures themselves.

---

Questions to Address In Rebuttal

1. Distinction from MINT [32]: The paper cites MINT, which uses deterministic sampling (1/p) to select which row activations populate a small, centralized tracker. MoPAC uses probabilistic sampling (p) to decide when to update an existing, dedicated per-row counter. Could the authors explicitly articulate why their probabilistic approach is a conceptually distinct and more powerful primitive than the sampling mechanism used in MINT, particularly in the context of the PRAC framework which already provides exhaustive tracking infrastructure?

2. Prior Art in Probabilistic Hardware Counting: Probabilistic counting is a foundational computer science concept (e.g., Morris's algorithm). While its application to PRAC appears novel, has the literature search included a thorough review of patents and technical reports related to probabilistic or approximate event counting for performance monitoring units (PMUs) or other hardware counters in memory systems? I am interested in whether a functionally identical mechanism has been proposed for a different purpose (e.g., managing power or wear-leveling).

3. Robustness of the Probabilistic Assumption: The security analysis in Section 5.3 (page 7) hinges on the assumption that the selection of an activation for a counter update is a truly independent random event. In a physical implementation, this would rely on a pseudo-random number generator (PRNG). Could a sophisticated attacker, through carefully crafted memory access timing, potentially discover the state of the PRNG or create access patterns that correlate with its output? Such an attack could break the independence assumption and allow for more undercounting than the binomial model predicts. What is the assumed complexity or quality of the PRNG required to uphold the security claims?

Paper Title: PuDHammer: Experimental Analysis of Read Disturbance Effects of Processing-using-DRAM in Real DRAM Chips Reviewer: The Guardian

---

Summary

The authors present an experimental characterization of read disturbance vulnerabilities arising from multiple-row activation patterns, which are foundational to many Processing-using-DRAM (PuD) operations. They term this new vulnerability class "PuDHammer" and characterize two variants: Consecutive Multiple-Row Activation (CoMRA) and Simultaneous Multiple-Row Activation (SiMRA). Using a testbed of 316 commercial DDR4 chips, they claim that these PuDHammer patterns significantly reduce the hammer count required to induce a bitflip (HC_first) compared to conventional RowHammer. Furthermore, they claim PuDHammer bypasses an in-DRAM TRR mitigation mechanism. Finally, they adapt the industry-standard PRAC mitigation and evaluate its performance overhead, concluding it to be substantial.

Strengths

1. Extensive Experimental Scale: The characterization is performed on a large and diverse set of 316 real DDR4 chips from four major manufacturers. This scale lends statistical weight to the general findings regarding CoMRA.
2. Comprehensive Parameter Sweep: The authors have diligently characterized the effects of various operational parameters, including temperature, data patterns, and timing violations. This provides a valuable dataset for the community.
3. Important Core Observation: The fundamental finding that multiple-row activation patterns, particularly CoMRA, exacerbate read disturbance is well-demonstrated across all tested vendors and represents a crucial consideration for the future security and reliability of PuD-enabled systems.

Weaknesses

My primary concerns with this manuscript center on the generalizability of its most dramatic claims, a lack of deep mechanistic explanation, and the realism of the mitigation evaluation.

1. Gross Overstatement and Lack of Generalizability for SiMRA: The paper's most striking claim—a 158.58x reduction in HC_first (Abstract, pg. 1)—is derived from SiMRA. However, the authors concede that SiMRA-induced bitflips were only observed in SK Hynix chips (Section 5.3, pg. 8). To present a vendor-specific behavior as a headline result for a phenomenon named "PuDHammer" is misleading. The abstract and introduction frame the results as general, but the reality is that the most potent attack vector demonstrated is highly limited in its applicability. The footnote explanation hypothesizing that other chips "ignore" commands is conjecture and insufficient justification. This is a critical flaw that undermines the paper's headline contributions.

2. Unsubstantiated Claim of TRR Bypass: The authors claim that PuDHammer "bypasses an in-DRAM RowHammer mitigation mechanism called Target Row Refresh" (Abstract, pg. 1). However, the corresponding experiment in Section 7 (pg. 11) was conducted on a single tested SK Hynix DRAM module. A sample size of N=1 is scientifically indefensible for making such a strong and general claim. This result may be an artifact of that specific module's TRR implementation and cannot be generalized to TRR mechanisms as a whole. The claim is severely overstated.

3. Superficial Physical-Level Analysis: The paper is strong on what happens but critically weak on why. For instance, Observation 14 (pg. 9) notes that SiMRA and RowHammer have opposite bitflip directions. The authors "hypothesize that double-sided SiMRA has a different underlying silicon-level mechanism" but provide no further evidence, simulation, or rigorous analysis. This is a significant finding that demands a more thorough investigation than mere speculation. Without a plausible physical model, the observations, while interesting, lack the explanatory power expected of a top-tier publication.

4. Questionable Realism in Mitigation Evaluation: The evaluation of the adapted PRAC mechanism (Section 8.2, pg. 12-13) relies on a synthetic workload injected into a simulation. This workload consists of "back-to-back one SiMRA with 32-row activation and one CoMRA operation every N ns." It is not clear that this pattern is representative of any real-world PuD application. The reported performance overheads are therefore entirely coupled to an artificial workload whose relevance is not justified. The conclusion that PRAC incurs high overheads is only valid for this specific, potentially unrealistic, workload.

5. Conflation of Distinct Phenomena: The paper bundles CoMRA and SiMRA under the single moniker "PuDHammer." However, the data suggests they may be mechanistically distinct: CoMRA works across vendors while SiMRA appears specific to one; they exhibit different sensitivities to parameters; and SiMRA demonstrates an opposite bitflip directionality. This lumping may obscure important underlying physics and misleads the reader into viewing them as two sides of the same coin, when they may be fundamentally different vulnerabilities.

Questions to Address In Rebuttal

The authors must address the following points directly and with evidence:

1. SiMRA Generalizability: Please justify presenting the SiMRA results, particularly the 158.58x HC_first reduction, as a primary finding of the paper when it was only observed in a single manufacturer's chips. The paper should be reframed to clearly demarcate general phenomena (like CoMRA) from vendor-specific ones (like SiMRA), starting from the abstract.
2. TRR Bypass Evidence: How can the authors justify their strong and general claim about bypassing TRR based on an experiment on a single DRAM module? Please provide data on more modules from different vendors or significantly walk back this claim to reflect the extremely limited scope of the experiment.
3. Physical Mechanism: The paper offers a hypothesis for the opposite bitflip directionality of SiMRA (Observation 14, pg. 9). Can the authors provide any supporting evidence from device-level modeling, literature on charge transport, or other analysis to elevate this from conjecture to a scientifically plausible explanation?
4. Workload Realism: Please provide a clear justification for the choice of the synthetic PuD workload used in the PRAC evaluation (Section 8.2, pg. 13). How does this access pattern map to the operational behavior of known PuD applications (e.g., in-memory databases, graph processing)? Without this, the performance results are of questionable value.

Reviewer: The Synthesizer (Contextual Analyst)

---

Summary

This paper presents the first systematic, experimental characterization of a new and severe form of DRAM read disturbance, which the authors aptly name "PuDHammer." The core contribution is the discovery that multiple-row activation—the fundamental mechanism enabling many Processing-using-DRAM (PuD) operations like in-DRAM copy and bitwise logic—dramatically exacerbates DRAM's vulnerability to read disturbance, far more so than conventional RowHammer attacks.

Through an extensive study on 316 commercial DDR4 chips, the authors demonstrate that PuDHammer can induce bitflips with up to a 158.58x lower activation count (HC_first) compared to the most effective RowHammer patterns. They show this vulnerability is sensitive to various operational parameters, can bypass existing in-DRAM mitigations like TRR, and that adapting the current industry standard mitigation (PRAC) results in prohibitively high performance overheads. This work effectively establishes a critical new bridge between the fields of Processing-in-Memory (PIM) and hardware security/reliability, revealing a potentially fundamental challenge to the safe and secure deployment of PuD systems.

---

Strengths

1. Fundamental and Timely Contribution: This paper sits at the crucial intersection of two highly active research areas. The PIM community has been largely focused on demonstrating functionality and performance, often with the implicit assumption of a reliable substrate. The hardware security community has studied RowHammer extensively but has not considered the unique access patterns of PIM. This work is the first to connect these domains, identifying a problem that is not merely incremental but represents a step-function increase in vulnerability severity. The discovery is foundational for any future work on secure PIM.

2. Impressive Experimental Rigor: The scale and thoroughness of the experimental study are a major strength. Characterizing the effect across 316 chips from four major manufacturers (as detailed in Table 1, page 4) provides strong evidence that PuDHammer is a universal phenomenon in modern DRAM, not an artifact of a specific device. The systematic exploration of a wide parameter space (temperature, data patterns, timing, spatial location) gives the results significant depth and credibility.

3. Clear Real-World Implications: By demonstrating that PuDHammer bypasses an existing mitigation (TRR, Section 7) and that adapting the industry standard PRAC incurs crippling overheads (48.26% average, Section 8.2), the paper immediately grounds its findings in practical reality. This is not a theoretical vulnerability; it is a clear and present danger to the security and isolation properties of future systems that might deploy PuD. This analysis effectively issues a challenge to the community: existing solutions are not sufficient.

4. Opening New Research Avenues: This is precisely the kind of paper that seeds an entire subfield. It raises more questions than it answers, which is a hallmark of significant research. Future work can now explore the underlying device physics, design PIM-aware mitigation strategies, develop new attack methodologies based on PuDHammer, and create secure compilers or runtimes for PIM systems. The paper provides a solid empirical foundation upon which this future work can be built.

---

Weaknesses

While the paper is strong, its focus as a "first characterization" study leaves some areas less developed, which is understandable but worth noting.

1. Limited Exploration of Root Cause: The paper does an excellent job characterizing what happens but is naturally speculative about why PuDHammer is so much more effective than RowHammer. The authors hypothesize about enhanced "trap-assisted electron migration" (Observation 2, page 6) but, as a systems paper, do not provide device-level analysis. While they rightly call for future work here, the paper would be even more impactful with a more detailed theoretical model or preliminary simulation to support their hypothesis.

2. Mitigation Proposals are Preliminary: The analysis of PRAC is excellent for showing the scale of the problem. However, the proposed new countermeasures in Section 8.1 are largely conceptual sketches. This is not a significant flaw, as a full mitigation design is likely beyond the scope of a single characterization paper. Still, the transition from problem identification to solution feels more like a pointer to future work than a complete contribution in its own right.

---

Questions to Address In Rebuttal

1. On the Physics of PuDHammer: The 158.58x reduction in HC_first is a staggering figure. Beyond the general hypothesis of enhanced electron migration, could you elaborate on the potential physical mechanisms? For example, with simultaneous multiple-row activation (SiMRA), is it possible that the massive, concurrent current draw on the power delivery network is causing voltage droops that are the primary driver of vulnerability, as opposed to purely cell-to-cell coupling effects? How does this differ from the mechanism in consecutive activation (CoMRA)?

2. On the Future of PIM Architecture: Your findings paint a challenging picture for the PIM paradigm. Do you view PuDHammer as a fundamental roadblock that might render certain PuD techniques (especially those based on SiMRA) impractical for secure systems, or do you see it as a new set of engineering constraints that can be designed around? In other words, is this an architectural "deal-breaker" or a "call to arms" for more robust PIM designs?

3. On PIM-Aware Mitigations: Given that a generic, low-level mitigation like PRAC imposes such high overheads, what is your vision for a more intelligent, PIM-aware mitigation? For instance, could the memory controller be aware of PuD "sessions" and apply targeted, aggressive refreshing only to neighboring regions of active PuD subarrays, rather than treating every activation as a potential hammer? Could such a semantic approach significantly reduce the observed performance penalty?

Reviewer: The Innovator (Novelty Specialist)

Summary

The authors present an experimental characterization study on the read disturbance effects that arise from multiple-row activation patterns, a core operational primitive of many Processing-using-DRAM (PuD) techniques. The paper identifies and names this phenomenon "PuDHammer," demonstrating that PuD access patterns—specifically consecutive (COMRA) and simultaneous (SiMRA) multiple-row activations—can induce bitflips far more effectively than traditional single-aggressor RowHammer attacks. The core of the work is an extensive empirical analysis across 316 real DDR4 chips, exploring the impact of various parameters on this vulnerability. The authors conclude by evaluating the effectiveness of adapting an existing industry mitigation (PRAC) for this new threat vector.

Strengths

The primary strength of this paper lies in its novel claim: the identification and first-ever systematic characterization of a new class of read disturbance vulnerability. While the constituent concepts are not new—Processing-using-DRAM and RowHammer are both well-established fields—their interaction has been, until now, an unexplored area. My analysis confirms the authors' claim that prior work has not investigated the security and reliability implications of the multiple-row activation patterns inherent to PuD.

The novelty can be broken down as follows:

1. Identification of a New Phenomenon: The core contribution is the empirical proof that the access patterns required for PuD operations create a potent read disturbance vector. Prior PuD literature (e.g., Ambit [29], RowClone [40], ComputeDRAM [73]) focused on demonstrating the functionality and performance benefits of multiple-row activation, overlooking the potential for malicious exploitation or reliability degradation. This paper bridges that critical gap. The "delta" over prior art is therefore the entire investigation itself; it moves from "PuD can be done" to "here are the unintended, severe consequences of doing PuD."

2. Novel Experimental Insights: The paper provides a wealth of new data. For example, Observation 14 (Page 9), which finds that SiMRA and RowHammer have opposite dominant bitflip directions, is a genuinely new and surprising finding. This suggests a fundamentally different underlying physical mechanism, a significant contribution to the community's understanding of DRAM reliability that did not exist previously. The discovery that PuDHammer patterns can reduce the hammer count to the first flip (HC_first) by up to 158.58x (Abstract, Page 1) is not a marginal improvement; it represents a phase change in the severity of read disturbance threats.

3. Systematic Categorization: The distinction between COMRA and SiMRA as separate classes of PuDHammer, each with unique characteristics, is a novel and useful categorization that will inform future work in this area.

Weaknesses

While the central discovery is novel, some of the surrounding elements rely heavily on established methods and concepts, which dilutes the overall novelty of the work.

1. Methodology: The experimental infrastructure (DRAM Bender [75] built on SoftMC [197]) and characterization methodology are standard practice, directly following the path laid by numerous prior RowHammer analysis papers (e.g., [125, 144, 145]). The novelty is in the access patterns tested, not in the methodology used to test them.

2. Countermeasures: The novelty in the countermeasures section (Section 8, Page 12) is limited.

   • The three proposals in Section 8.1 (separating arrays, refreshing post-operation, weighted counting) are logical, high-level extensions of existing security principles like isolation and proactive defense. They lack a novel underlying mechanism and are presented conceptually without implementation.
   • The adaptation and evaluation of Per-Row Activation Counting (PRAC) is an application of an existing, industry-standardized technique to a new problem. While the performance analysis provides new and valuable data points, the core mechanism is not new. The "weighted counting" optimization is a straightforward tuning of this existing counter-based approach rather than a fundamentally new idea. The novelty here is in the application and evaluation, not the invention.

Questions to Address In Rebuttal

1. The core novelty of this work is the experimental characterization of a previously unexplored phenomenon. However, the proposed countermeasures in Section 8.1 appear to be high-level, conceptual extensions of existing principles (e.g., isolation, proactive refresh). Can the authors elaborate on what is fundamentally novel about these proposed countermeasures beyond applying known concepts to this new problem context?

2. The evaluation of an adapted Per-Row Activation Counting (PRAC) mechanism in Section 8.2 is a valuable contribution. However, the mechanism itself is an existing industry standard. Is the novelty here purely in the evaluation, or is there a non-obvious, novel aspect to the adaptation of PRAC to handle multiple simultaneous activations that the paper does not fully elaborate on? The "weighted counting" optimization, for instance, seems like a logical tuning of an existing counter-based approach.

3. Prior works such as ComputeDRAM [73] and others [78, 79] have successfully demonstrated multiple-row activation on COTS DRAM. While their stated goal was not to study read disturbance, did they observe any anomalous behavior or reliability issues that might have hinted at the PuDHammer effect, even if not characterized as such? A more direct positioning against observations from this functionally-closest prior art would strengthen the claim of being the "first."

Paper Title: DREAM: Enabling Low-Overhead Rowhammer Mitigation via Directed Refresh Management Reviewer: The Guardian

---

Summary

The paper addresses the performance and storage overheads of Memory Controller (MC) side Rowhammer mitigations when using the JEDEC Directed Refresh Management (DRFM) standard, as opposed to the hypothetical Nearby Row Refresh (NRR) command assumed by prior work. The authors propose DREAM, a DRFM-aware mitigation framework with two variants. The first, DREAM-R, aims to reduce the performance overhead of randomized trackers (e.g., PARA, MINT) by delaying DRFM commands to increase Rowhammer-mitigation Level Parallelism (RLP). The second, DREAM-C, aims to reduce the storage overhead of counter-based trackers by grouping rows into "gangs" that share a single counter, leveraging the concurrent mitigation capability of DRFMab.

Strengths

1. Practical Problem Formulation: The paper correctly identifies a critical and practical gap in the literature: the significant performance penalty of using the real-world DRFM standard compared to the idealized NRR. Grounding the research in the constraints of an existing JEDEC standard is a notable strength.
2. Clear Core Concepts: The two central ideas—delaying DRFM to improve RLP in DREAM-R and group-tracking to reduce storage in DREAM-C—are intuitive and directly leverage the specified capabilities of the DRFM interface. The graphical summary in Figure 1 provides a clear overview of the proposed mechanisms.
3. Consideration of Secondary Effects: The authors demonstrate foresight by considering the implications of DRFM rate limits (Section 6), a real-world constraint that is often overlooked. While the proposed solution has its own issues, acknowledging the problem is commendable.

Weaknesses

My analysis reveals several significant weaknesses that challenge the security claims and practical viability of the proposed designs.

1. The Security of DREAM-R is Fundamentally Undermined by the DRFM Delay: The core premise of DREAM-R is to delay mitigation. This introduces a vulnerability window between when a row is identified as an aggressor (sampled into DAR) and when it is actually mitigated (DRFM is issued). An attacker can continue to issue activations to this row during this delay.

   • The proposed patch, Active Target-Row Monitoring (ATM), as described in Section 4.4, is an admission of this fundamental flaw. It adds complexity (per-bank registers, counters, and comparators) to solve a self-inflicted problem. The paper does not adequately analyze the overheads and complexity of ATM itself, merely stating it requires "only 3 bytes per bank." This additional hardware is not negligible and complicates the design.
   • The security analysis for PARA in Appendix A relies on a probabilistic model (Gamma distribution). This is insufficient. An adversary is not a random process; they will execute a deterministic, worst-case attack pattern to maximize activations within this delay window. The analysis fails to address such a targeted attack.

2. DREAM-C's Randomized Grouping is Susceptible to Pathological Conflicts: The paper correctly identifies that a naive set-associative grouping leads to hot counters (Figure 15, top). The proposed solution is randomized grouping. However, this is not a panacea.

   • The security of this scheme relies on the unpredictability and quality of the "random masks" initialized at boot time. The paper does not discuss the source of this randomness or its resilience against side-channel analysis or reverse engineering. An attacker with knowledge of the XOR-masking scheme could potentially identify row addresses that collide into the same DCT entry, constructing a pathological workload to create a "hot gang" and trigger frequent, high-overhead DRFMab commands, leading to a Denial-of-Service attack. The paper's claim in Section 6.3 that the likelihood of finding such rows is "less than 1 in a trillion" is an assertion without sufficient proof, especially against a knowledgeable adversary.
   • The results in Figure 15 (bottom) show that for workloads like parest and lbm, the slowdown remains over 10% and 24% respectively at TRH=500 and TRH=250. This directly contradicts the claim in the abstract of "incurring negligible slowdown." These are not edge cases; they represent significant performance degradation under certain access patterns that the randomization failed to adequately smooth out.

3. The Proposed Handling of DRFM Rate-Limits Introduces a Security Hole: To comply with DRFM rate limits, the authors propose a Recent-Mitigated-Address-Queue (RMAQ) that explicitly skips mitigations for rows that have been recently mitigated (Section 6.1, Figure 18).

   • This is a clear and exploitable security vulnerability. An attacker can hammer a row just enough to get it selected for mitigation and placed in the RMAQ. For the subsequent window (up to two tREFI periods), the attacker can hammer this same row with impunity, knowing that the defense will deliberately ignore it.
   • The analysis in Section 6.2 claims this only has a "minor impact" on the tolerated TRH. This downplays the severity. It changes the security guarantee from "a row will be mitigated" to "a row might be mitigated, unless we just did so." This is a qualitative weakening of the defense that is not acceptable for a security mechanism.

4. Inconsistent Claims and Comparisons:

   • As noted, the claim of "negligible slowdown" for DREAM-C is overstated, with the paper's own data showing slowdowns well above 10% for specific workloads.
   • The comparison with ABACUS in Section 5.8 characterizes it as being equivalent to DREAM-C's set-associative design. This is a partial comparison. ABACUS's Sibling Activation Vector (SAV) is a specific mechanism to handle the common case of streaming accesses across banks efficiently. DREAM-C's randomized grouping is a different approach to a similar problem. The comparison does not engage with the technical novelty of the SAV mechanism, potentially presenting an uncharitable view of the competing work.

Questions to Address In Rebuttal

1. Regarding DREAM-R: Please provide a security analysis against a deterministic, worst-case adversary who is aware of the DRFM delay and the ATM mechanism. The current probabilistic model is insufficient. How can you guarantee security when the fundamental design choice is to delay mitigation?
2. Regarding DREAM-C: The security of randomized grouping hinges on the secrecy and strength of the per-bank random masks. What is the assumed threat model regarding the attacker's knowledge of this grouping function? Please provide a more rigorous argument for the resilience of this scheme against an attacker attempting to find collisions and construct a pathological access pattern.
3. Regarding Performance Claims: How do you reconcile the claim of "negligible slowdown" for DREAM-C with your own results in Figure 15, which show slowdowns of 13% for parest (TRH=500) and 24.7% for lbm (TRH=250)? Please define "negligible" in this context.
4. Regarding Rate-Limit Handling: The RMAQ mechanism involves deliberately skipping required mitigations. Please justify how this can be considered a secure design choice. Can you provide an analysis of an attack that specifically exploits the RMAQ by forcing an entry and then hammering the now-unprotected row?

Paper Title: DREAM: Enabling Low-Overhead Rowhammer Mitigation via Directed Refresh Management Reviewer Persona: The Synthesizer (Contextual Analyst)

---

Summary

This paper addresses a critical and timely problem in the domain of hardware-based Rowhammer mitigation: the practical implementation of Memory Controller (MC) side defenses using the real-world JEDEC DDR5 standard. The authors correctly identify that much of the prior academic work in this area has assumed a hypothetical, single-bank Nearby-Row-Refresh (NRR) command, which was not adopted. Instead, JEDEC introduced Directed Refresh Management (DRFM), a multi-bank command that incurs prohibitively high performance overheads when used in a naive, NRR-like fashion.

The core contribution of this work is to reframe DRFM's multi-bank stall from a liability into an asset. The authors introduce the concept of "Rowhammer-mitigation Level Parallelism" (RLP) and propose a new design philosophy, DREAM, to exploit it. DREAM comes in two variants: 1. DREAM-R: For randomized trackers (like PARA and MINT), it decouples row sampling from mitigation by delaying the DRFM command. This delay allows multiple banks to identify and sample aggressor rows, thus increasing the RLP and amortizing the cost of a single DRFM command across several mitigations. 2. DREAM-C: For counter-based trackers, it leverages DRFM's ability to mitigate rows in all banks simultaneously by "ganging" multiple rows (one from each bank) to a single shared counter. This dramatically reduces the storage overhead of counter-based schemes by up to 32x or more.

The paper demonstrates that these approaches can reduce the performance overhead of randomized trackers to levels comparable to or even better than the theoretical NRR command, and can make counter-based tracking practical at very low thresholds with minimal storage.

Strengths

1. Exceptional Timeliness and Practical Relevance: This paper's greatest strength is its grounding in reality. It bridges a significant gap between the theoretical assumptions of past academic proposals and the concrete engineering challenges posed by the available JEDEC DDR5 standard. By being one of the first works to seriously analyze and architect solutions around DRFM, it provides immense value to SoC designers and memory architects who must build secure systems today. This work effectively "course-corrects" the field of MC-side Rowhammer mitigation.

2. Elegant and Powerful Core Insight: The central idea—to exploit, rather than suffer from, DRFM's multi-bank nature—is both simple and profound. The concept of RLP is a clear and useful abstraction. This reframing turns what the paper shows to be a disastrous performance problem (e.g., 49% slowdown for PARA in Figure 5, p. 5) into a manageable one, demonstrating a high degree of intellectual contribution.

3. Comprehensive and Well-Structured Solution Space: The authors do not propose a single, narrow solution. Instead, they thoughtfully tailor their core insight to the two dominant paradigms of MC-side tracking: randomized and counter-based. DREAM-R's temporal batching (delaying) and DREAM-C's spatial batching (ganging) are elegant counterparts. This shows a deep understanding of the broader design space and makes the work widely applicable.

4. Strong Connection to the Broader Landscape: The paper does an excellent job of positioning its contributions. It not only re-evaluates foundational trackers like PARA and MINT but also provides a compelling comparison to recent in-DRAM industry proposals like PRAC (Figure 19, p. 12). By showing that DREAM can offer better performance than PRAC at much lower complexity, it makes a strong case for the continued relevance and potential superiority of MC-side defenses.

Weaknesses

While the core ideas are strong, there are areas where the real-world complexities slightly cloud the elegance of the initial concept. These are not fatal flaws but rather avenues for further refinement.

1. Introduced Complexity in DREAM-R's Security Model: The act of delaying DRFM in DREAM-R necessitates a re-evaluation of the underlying tracker's security guarantees. The paper commendably addresses this by proposing revised tracker parameters and an Active Target-Row Monitoring (ATM) mechanism (Section 4.4, p. 7). However, this adds state (a copy of the sampled row, a counter) and complexity that slightly detracts from the simplicity of the core idea. The solution is sound, but it underscores that the "delay" is not "free" from a security logic perspective.

2. Handling of DRFM Rate Limits: The discussion around JEDEC-mandated rate limits on DRFM (Section 6, p. 11) feels somewhat like a necessary but cumbersome addition. The proposed Recent-Mitigated-Address-Queue (RMAQ) is a pragmatic solution, but it introduces yet another hardware structure and associated logic to manage. While the authors correctly identify that future mitigation techniques could obviate this, it adds a layer of complexity to the current proposal.

3. Potential Second-Order Effects: The paper focuses primarily on the direct performance impact (slowdown) and storage costs. However, the proposed mechanisms, particularly delaying DRFM commands in DREAM-R, could have interesting second-order effects on the memory scheduler, especially in systems with strict Quality-of-Service (QoS) requirements. An impending, high-priority DRFM command that stalls 8 banks could complicate scheduling decisions. This broader system interaction is an area left for future exploration.

Questions to Address In Rebuttal

1. Regarding the Active Target-Row Monitoring (ATM) in DREAM-R: Could the authors elaborate on the security implications if a designer chose to omit ATM for cost reasons and instead relied solely on adjusting the tracker parameters (e.g., using p=1/85 for PARA)? How much does this weaken the security guarantee, and under what threat model would this become a significant vulnerability?

2. The Denial-of-Service (DoS) analysis for DREAM-C (Section 5.5, p. 10) argues that the worst-case throughput reduction is comparable to other memory contention issues like row-buffer conflicts. However, a DRFM-based DoS stalls an entire sub-channel. Could an attacker craft a workload that targets multiple distinct counter "gangs" in rapid succession to create a more sustained and severe system-wide availability issue than traditional bank-level or rank-level contention?

3. The paper argues compellingly against the high intrinsic overhead of PRAC. However, PRAC offloads the tracking logic entirely to the DRAM. DREAM keeps this logic on the MC. Could the authors comment on the trade-offs in terms of verification complexity, power, and area for the MC-side logic proposed in DREAM-R (with ATM) and DREAM-C (with randomized grouping and vertical sharing)? How does this compare to the cost of simply tolerating PRAC's performance hit?

Paper Title: DREAM: Enabling Low-Overhead Rowhammer Mitigation via Directed Refresh Management Reviewer ID: Persona 3 (The Innovator)

---

Summary

This paper presents DREAM, a memory controller (MC) side Rowhammer mitigation framework designed to reduce the high overheads associated with the JEDEC Directed Refresh Management (DRFM) command. The authors identify that naively substituting the hypothetical NRR command with the standardized DRFM command leads to prohibitive performance degradation due to multi-bank stalls. The work's core thesis is that the multi-bank nature of DRFM, while being a source of overhead, can be exploited for parallelism.

The authors propose two distinct contributions based on this insight: 1. DREAM-R: A technique for randomized trackers that decouples row sampling from mitigation. By delaying the issuance of a DRFM command, it allows multiple banks to sample aggressor rows, thereby increasing the number of rows mitigated per command (increasing "Rowhammer-mitigation Level Parallelism" or RLP) and reducing the frequency of multi-bank stalls. 2. DREAM-C: A technique for counter-based trackers that reduces storage overhead. It leverages the DRFMab command's ability to mitigate 32 rows concurrently by grouping rows from 32 different banks to share a single counter. A novel randomized grouping function is proposed to prevent "hot counters" that would otherwise arise from spatially correlated memory accesses.

The paper claims novelty in being the first to analyze the performance of DRFM for MC-side mitigations and in proposing a management layer (DREAM) that makes existing trackers practical with this new standard.

---

Strengths (Novelty-focused Analysis)

The primary strength of this work lies in its novel approach to managing a new, real-world hardware standard (DRFM), rather than proposing yet another hypothetical tracker or mitigation command. The novelty is subtle but significant.

1. Novel Exploitation of a Standard's Properties: The central novel idea is the recognition that DRFM's multi-bank mitigation capability is a double-edged sword. Prior art has largely focused on trackers (PARA, MINT, Graphene) and assumed a simple, single-bank mitigation primitive like NRR. This paper's contribution is to architect a solution specifically for the command that actually exists in the JEDEC DDR5 standard. The concept of exploiting Rowhammer-mitigation Level Parallelism (RLP), as termed by the authors, appears to be entirely new in this context.

2. DREAM-R's Decoupled Management: The idea of delaying a mitigation command to batch operations is the key novelty in DREAM-R. In the context of prior NRR-based work, this would be nonsensical, as there is no parallelism to gain. The novelty is therefore fundamentally tied to the properties of DRFM. I have not seen prior work that proposes decoupling the identification of an aggressor from its mitigation in this manner to improve performance.

3. DREAM-C's Gang-Tracking: The concept of sharing one counter among a "gang" of rows is novel in its application to Rowhammer. While counter-sharing is a general computer architecture technique, its viability here is uniquely enabled by the DRFMab command's ability to mitigate all 32 rows in a single operation. Without this specific hardware feature, the idea would be impractical. The proposal to track a group of rows because they can be mitigated as a group is a genuinely new design point.

4. Novelty in Problem Identification: A significant part of the contribution is identifying and quantifying the performance problem of using DRFM naively (Section 2.7, page 5). While this may seem like evaluation, it is the first work to do so, and this identification is the direct motivation for the novel solutions proposed.

---

Weaknesses (Novelty-focused Analysis)

While the core framework is novel, some of the constituent mechanisms are applications of known principles, which slightly tempers the overall inventive step.

1. Randomized Grouping is an Application of a Known Concept: The insight that set-associative grouping leads to hot counters due to address mapping is excellent (Section 5.2, page 9). However, the solution—using a randomized mapping (XOR with a per-bank mask)—is a classic technique to mitigate aliasing and hash collisions in caches, predictors, and other hardware tables. The application to ganged Rowhammer counters is new, but the underlying principle of randomization to break spatial correlation is not a fundamental discovery. The delta here is in the application, not the invention of the technique itself.

2. Active Target-Row Monitoring (ATM) is a Reactive, Not Proactive, Invention: The ATM component proposed for DREAM-R (Section 4.4, page 7) is an elegant piece of engineering. However, its existence is solely to patch a security vulnerability created by the primary novel idea (delaying DRFM). It is a necessary fix, not a standalone novel concept. Its complexity is a direct cost of the core idea, and while well-executed, it does not represent a separate inventive leap.

---

Questions to Address In Rebuttal

1. The novelty of DREAM-C's randomized grouping rests on its specific application. Could the authors contrast this technique more directly with conceptually similar anti-aliasing schemes used in cache indexing or network packet classification? Clarifying the "delta" from these well-established uses of randomization would help solidify the novelty of this specific part of the contribution.

2. The core insight of the paper is to build a management layer atop a multi-entity hardware command (DRFM). Is this a "one-trick pony" for Rowhammer, or do the authors see this as a new class of optimization? For example, are there other emerging JEDEC commands (e.g., for memory security, error correction) where a similar "management" approach of decoupling and batching could yield analogous benefits?

3. The security of DREAM-R relies on the ATM mechanism to cap the number of unmitigated activations during the DRFM delay. Was this the only design point considered? For instance, could the delay be dynamically sized based on memory traffic intensity, potentially reducing the need for active monitoring in low-intensity phases, or is the fixed-threshold ATM the most robust solution? Exploring this design space would add depth to the novelty claim.

Paper Title: Ecco: Improving Memory Bandwidth and Capacity for LLMs via Entropy-Aware Cache Compression Reviewer ID: The Guardian

---

Summary

The authors propose Ecco, a hardware-accelerated, entropy-aware cache compression framework for Large Language Models (LLMs) integrated at the L2 cache level. The technique combines group-wise non-uniform quantization using shared k-means patterns with Huffman coding to achieve high compression ratios (4x for weights/KV cache, 2x for activations). The core architectural contribution is a novel parallel Huffman decoder designed to overcome the traditional latency and throughput limitations of sequential variable-length decoding. The paper claims significant speedups (up to 2.9x over AWQ) and memory capacity improvements (nearly 4x) while maintaining state-of-the-art model accuracy. However, the work rests on a foundation of an exceedingly complex compression scheme whose hardware feasibility is justified by questionable scaling assumptions and lacks critical ablation studies to validate key design choices.

Strengths

1. Strong Accuracy Preservation: The most compelling aspect of this work is its demonstrated ability to maintain model quality under aggressive, lossy compression. The perplexity results in Table 1 (page 10) and the zero-shot accuracy in Table 2 (page 10) are impressive, showing that the proposed complex quantization and entropy coding scheme is effective at preserving information compared to other state-of-the-art 4-bit methods.

2. Sound Architectural Placement: The high-level architectural concept of integrating a custom compressor/decompressor with the L2 cache (Figure 1, page 2) is a well-established and logical approach to addressing memory bandwidth bottlenecks. This is a more architecturally pure solution than embedding dequantization logic deep within computational kernels.

3. Inclusion of Sensitivity Analysis: The sensitivity analysis regarding decompressor throughput and latency (Figure 14, page 13) is a welcome addition. It demonstrates a clear understanding of the architectural constraints and correctly identifies that the decompressor's performance must be tightly coupled with the cache hierarchy to avoid becoming a new bottleneck.

Weaknesses

My primary concerns with this paper revolve around the practical feasibility of the proposed hardware and the rigor of its evaluation. The claims are strong, but the evidence contains several critical gaps and leaps of faith.

1. Highly Questionable Hardware Cost Analysis: The hardware implementation is enormously complex. The compression algorithm (Figure 4, page 6) involves multi-level normalization, k-means clustering, selection from 64 shared patterns and 4 Huffman codebooks, and variable-length encoding. The decompression pipeline (Figure 8, page 9) requires 64 parallel Huffman decoders with a multi-stage tree-based result aggregator. Yet, the area and power analysis in Section 5.2 (page 11) is superficial at best. The authors synthesize the design in a commercial 28nm process and then simply "scale the area and power metrics to 7nm." This is a fundamentally unsound methodology. Technology scaling is not linear, and factors like wire delay, leakage power, and design rule complexity do not scale predictably in this manner. Claiming this complex machinery occupies less than 1% of the chip area and consumes only 7.36 W based on such a crude estimation is not credible. This weakness fundamentally undermines the paper's claim of practicality.

2. Critical Unsupported Claim in KV Cache Compression: In Section 3 (page 7), the authors state that for online KV cache compression, they replace the computationally expensive Mean Squared Error (MSE) calculation for pattern selection with a simplified min/max value comparison. They then assert that this simplification "incurs only a minimal drop in perplexity." There is absolutely no data presented in the paper to support this claim. An ablation study comparing the perplexity/accuracy of the online min/max method versus the offline MSE-based method is essential for validation. Without this evidence, this critical design choice for the dynamic KV cache—a major component of the overall system—is entirely unsubstantiated.

3. Misleading Performance Baseline Comparison: The performance evaluation in Section 5.3 (page 11) frames the speedup against methods like AWQ and SmoothQuant. While these are relevant SOTA quantization frameworks, they are primarily software-algorithmic techniques that do not presuppose custom hardware acceleration. The authors are comparing their specialized hardware solution against software methods running on general-purpose hardware. This is not an apples-to-apples comparison. A fair comparison would require benchmarking against other hardware-accelerated compression schemes from the computer architecture literature. By failing to do so, the reported speedups appear inflated, as they conflate the benefits of their specific algorithm with the inherent benefits of any hardware acceleration.

4. Convenient Omission of Key Baseline: The authors explicitly exclude Quarot from the performance evaluation (Section 5.3, page 11) because it is slower than the FP16 baseline. This is a critical omission. Table 1 (page 10) shows that Quarot's accuracy is highly competitive with Ecco. The entire narrative of the paper is that complex schemes like Quarot pay a prohibitive runtime cost, which Ecco's hardware solves. The most powerful way to demonstrate this would be to include Quarot in the performance charts (Figure 11, page 12) to visually show its high accuracy but poor latency, thereby perfectly motivating the need for Ecco. By removing it, the authors avoid a direct comparison with their closest competitor on the accuracy-complexity frontier, weakening their argument.

5. Unjustified Critical Path Latency Figure: The entire performance argument hinges on the decompressor being fast enough. The paper claims a 28-cycle latency for the high-ratio decompressor (Section 5.2, page 11). This number is presented as fact without any supporting detail on its derivation. Was this the result of a synthesis run targeting the A100's clock frequency? Does this figure account for the full pipeline including input buffering and output mapping? Given the complexity of the parallel decoder and result aggregator, a 28-cycle latency is aggressive and requires rigorous justification, which is currently absent.

Questions to Address In Rebuttal

1. Please provide a detailed justification for the 28nm to 7nm scaling methodology used in your hardware cost analysis. Acknowledge the non-linear effects of process scaling and explain why you believe your estimation is nonetheless reliable. Better yet, provide an analysis based on a more appropriate technology library or a detailed breakdown of the logic elements that justifies the area claim.

2. You must provide the data from an ablation study that compares the model accuracy (e.g., perplexity on WikiText-2) of your KV cache compression using the proposed online min/max pattern selection versus the offline MSE-based selection. This is necessary to substantiate the claim that the performance drop is "minimal."

3. Please justify the fairness of comparing your custom hardware solution against software-only quantization methods. Discuss relevant prior work on hardware-accelerated cache compression (even if not LLM-specific) and position your work in relation to those architectural baselines.

4. Please provide a detailed breakdown of the 28-cycle decompressor latency. How was this value determined? What were the synthesis constraints (e.g., target clock frequency)? What specific stages are included within this latency figure?

5. Explain the reasoning for excluding Quarot from the performance speedup comparisons in Section 5.3, given that it serves as a key accuracy baseline in Table 1 and represents the exact class of high-complexity, high-accuracy algorithms your hardware aims to accelerate.

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper proposes Ecco, a novel, domain-specific, lossy compression scheme designed to be integrated into the GPU memory hierarchy to alleviate the memory bandwidth and capacity bottlenecks in Large Language Model (LLM) inference. The core contribution is the synthesis of several compression techniques—group-wise non-uniform quantization via shared k-means patterns and entropy coding via Huffman codes—into a hardware-realizable system. Crucially, the authors address the primary obstacle to using variable-length codes like Huffman in a high-performance memory path by designing a novel parallel Huffman decoder with a multi-stage pipeline. This architectural innovation aims to achieve decompression throughput comparable to a GPU L2 cache, making the entire scheme practical. The proposed system is evaluated through extensive simulation, demonstrating significant speedups (up to 2.9x over AWQ) and memory capacity improvements (nearly 4x) while maintaining state-of-the-art model accuracy.

---

Strengths

This work's primary strength lies in its excellent synthesis of ideas from computer architecture, machine learning, and information theory to create a compelling, systems-level solution to a critical problem.

1. Elegant Problem-Solution Fit: The authors correctly identify that the statistical properties of LLM weights and activations exhibit low entropy, which is a perfect target for entropy coding. While software-based quantization methods have exploited this, they often introduce computational overhead (as noted in Section 2.3, page 4). By moving a sophisticated, entropy-aware compression engine into the hardware cache controller, Ecco makes the process transparent to the software stack and avoids polluting the compute kernels with decompression logic. This is a powerful architectural approach that directly maps the nature of the data to the hardware that handles it.

2. Addressing the Core Implementation Challenge: The proposal would be merely a theoretical exercise without a feasible high-throughput decoder for the variable-length Huffman codes. The design of the parallel, pipelined decoder (Section 4.2, page 8 and Figure 8) is the lynchpin of this paper. Recognizing that sequential decoding is a bottleneck and architecting a parallelized solution demonstrates a deep understanding of the practical constraints of memory subsystem design. This transforms a good idea into a plausible engineering proposal.

3. Holistic System Design: The work is not just an algorithm but a well-considered system. The authors have thought through the entire data path, from the fixed-size compressed blocks that align with memory transactions to the trade-offs in the design space exploration (Figure 5, page 7) for k-means patterns and Huffman codebooks. The inclusion of area and power analysis (Section 5.2, page 11), while based on synthesis and scaling, adds a crucial layer of credibility to the proposed hardware's feasibility.

4. Strong Contextualization: The paper does a good job of situating its contribution within the broader landscape. It clearly distinguishes its architectural approach from purely algorithmic methods like AWQ and Quarot and from general-purpose lossless hardware compression found in current GPUs. It correctly frames the problem as a new "memory wall" specific to the LLM era.

---

Weaknesses

The paper's weaknesses are not fundamental flaws but rather areas where the analysis could be deepened to make the proposal even more robust.

1. Sensitivity to Calibration Data: The effectiveness of the entire scheme, particularly the pre-defined k-means patterns and Huffman codebooks, hinges on the calibration dataset being representative of real-world inference workloads. While The Pile is a diverse dataset, LLMs are increasingly used in specialized domains (e.g., code generation, medical analysis) where data distributions might shift significantly. The paper does not explore the sensitivity of Ecco's performance (both in terms of model accuracy and compression efficiency) to such distribution shifts. A more robust system might require some form of lightweight online adaptation, which is not discussed.

2. Lack of a Simpler Hardware Baseline: The performance comparison is primarily against software frameworks (AWQ, SmoothQuant) and an uncompressed FP16 baseline. While insightful, this doesn't fully isolate the benefits of Ecco's complexity. A valuable addition would be a comparison against a simpler lossy hardware compression scheme—for instance, a hardware implementation of simple group-wise uniform quantization without the Huffman coding. This would help quantify the specific gains achieved by the more complex entropy coding stage and better justify its area and power cost.

3. Limited Discussion of Synergies: The paper positions Ecco as an alternative to existing software quantization methods. However, it seems plausible that these approaches could be synergistic. For example, could a model already quantized with a method like AWQ be further compressed by Ecco for additional benefits? Furthermore, Ecco operates at the level of data representation, while techniques like PagedAttention operate at the level of memory management. The potential interplay between these orthogonal hardware and software optimizations is a rich area for discussion that is currently absent.

---

Questions to Address In Rebuttal

1. On Robustness: Could the authors comment on the robustness of the pre-calculated codebooks and k-means patterns to out-of-distribution data encountered during inference? Have you performed any experiments to measure the degradation in accuracy or compression ratio when the inference data statistics differ significantly from the calibration set (The Pile)?

2. On Justifying Complexity: The proposed compressor/decompressor is significantly more complex than the lossless compressors in today's GPUs. To better justify this, could you provide insight into what performance would be achieved by a simpler hardware implementation of just the group-wise non-uniform quantization part of your pipeline, without the subsequent Huffman coding? This would help isolate the contribution of the entropy coding stage.

3. On System-Level Integration: How do you envision Ecco interacting with modern LLM memory management systems like PagedAttention? Since PagedAttention already optimizes memory usage by managing the KV cache in non-contiguous pages, could Ecco be applied on a per-page basis to compound the benefits, effectively storing 4x as many token states within the same physical memory footprint?

Review Form: The Innovator (Novelty Specialist)

---

Summary

The authors propose "Ecco," a hardware-accelerated cache compression system designed to alleviate memory bandwidth and capacity bottlenecks in Large Language Model (LLM) inference. The core of Ecco is a compression scheme that combines several known techniques: group-wise, non-uniform quantization using shared k-means codebooks, followed by Huffman coding for entropy encoding. The central claim to novelty lies not in the compression algorithm's individual components, but in their synthesis into a hardware system and, most critically, in the design of a novel parallel Huffman decoder architecture. This decoder is designed to overcome the inherent sequential limitations of Huffman codes, enabling throughput high enough for on-the-fly decompression between an L2 cache and streaming multiprocessors. The paper presents this architectural solution, evaluates its area and power, and demonstrates significant speedups and memory savings for LLM inference.

---

Strengths

The primary strength of this work lies in its architectural solution to a well-known problem. While the components of the compression scheme are familiar, the key innovation is making a theoretically powerful but practically slow compression technique (Huffman coding) viable at cache-level speeds.

1. Novel Parallel Huffman Decoder Architecture: The most significant novel contribution is the design of the parallel Huffman decoder detailed in Section 4.2 (page 8) and Figure 8 (page 9). The problem of parallelizing variable-length codes is decades old, but the proposed architecture—which uses 64 parallel decoders operating on overlapping 15-bit data chunks and a six-stage tree-based result aggregator—is a specific and clever engineering solution. This design directly addresses the critical path latency and throughput requirements of a GPU memory subsystem, which is a non-trivial architectural challenge. This is the lynchpin that makes the entire proposal feasible.

2. Novel Synthesis for a New Domain: While the constituent compression techniques are not new, their combination and application as an on-the-fly hardware cache compression scheme for LLMs is novel. Most prior work using k-means and Huffman coding (e.g., Deep Compression) has focused on creating statically compressed models for offline storage. Ecco’s contribution is to architect a system that performs this complex, lossy, entropy-aware compression/decompression dynamically as part of the memory hierarchy, which is a fundamentally different operational paradigm.

---

Weaknesses

My main concerns revolve around the paper's framing, which could be interpreted as claiming novelty for the compression primitives themselves, and a need for a more explicit comparison against prior art in parallel decoder design.

1. Constituent Algorithmic Components are Not Novel: The paper's core compression methodology is built upon a foundation of well-established prior art.

   • Huffman Coding for NN Compression: The idea of using Huffman coding to compress quantized neural network weights is not new. It was a cornerstone of the highly influential "Deep Compression" paper (Han et al., 2016, cited by the authors as [25]), which demonstrated its effectiveness years ago.
   • Non-Uniform Quantization via K-Means: Using k-means clustering to generate codebooks for non-uniform quantization is also a standard technique. It was used in the aforementioned Deep Compression work and more recent LLM-specific work like SqueezeLLM (cited as [37]).
   • Group-wise Quantization: This is the standard approach in modern quantization methods like AWQ (cited as [42]) to balance compression ratio and accuracy.

   The paper's novelty is therefore not in what it does algorithmically, but how and where it does it (i.e., in a hardware cache controller). The current framing could be sharpened to de-emphasize the algorithmic components as novel in themselves and focus more squarely on the architectural innovation.

2. Insufficient Differentiation from Prior Parallel Decoder Architectures: While the proposed decoder architecture appears novel in its specific implementation, the paper would be stronger if it explicitly situated its design within the broader literature of parallel Huffman/VLC decoding. Prior work exists on speculative decoding, lookup-table-based methods, and other chunking strategies. A brief discussion of why the proposed overlapping-window and tree-aggregation approach was chosen over these alternatives would better substantiate the novelty and design choices of that core component.

---

Questions to Address In Rebuttal

To strengthen the paper and clarify its precise contribution, I would ask the authors to address the following:

1. Please clarify the primary novel contribution of this work. Can you explicitly differentiate your work from "Deep Compression" and "EIE: Efficient Inference Engine" (Han et al., 2016, cited as [24])? Specifically, while those works also use Huffman coding and hardware acceleration, could you articulate how Ecco's focus on a general-purpose cache controller (as opposed to a full inference engine) and its specific parallel decoding architecture represent a significant delta?

2. The complexity of the proposed compressor/decompressor is substantial, involving bitonic sorters, multiple k-means pattern matchers, and the parallel Huffman decoder. Have you considered simpler lossy compression schemes that could be implemented in hardware? What is the justification for choosing this highly complex scheme over, for example, a simpler block-based transform coding or a more aggressive delta-based compression scheme that might offer a better trade-off between hardware cost and compression ratio?

3. Could you provide more context on the design of your parallel Huffman decoder (Section 4.2)? What are the primary trade-offs of your design (e.g., overlapping windows, fixed number of stages) compared to other approaches for parallelizing variable-length decoding found in prior art?

Paper Title: MeshSlice: Efficient 2D Tensor Parallelism for Distributed DNN Training Reviewer: The Guardian

---

Summary

The authors propose MeshSlice, a novel 2D GeMM algorithm for distributed DNN training. The central idea is to partition (or "slice") collective AllGather (AG) and ReduceScatter (RdS) operations into multiple smaller, partial collectives. This partitioning, combined with software pipelining, is claimed to enable the overlap of communication and computation in both row and column dimensions of a 2D accelerator mesh. The paper also introduces an autotuner that uses analytical cost models to select optimal dataflows, mesh shapes, and communication granularities.

However, the paper's central claims of superiority rest almost entirely on simulated results derived from a model calibrated on a very small-scale system. The included real-hardware experiments not only fail to support the primary claims but actually show MeshSlice underperforming against established baselines due to current software limitations. The validity of the large-scale simulations and the practical feasibility of the proposed approach are therefore highly questionable.

Strengths

1. Problem Motivation: The paper correctly identifies a critical bottleneck in scaling tensor parallelism: the inability of standard "Collective 2D GeMM" algorithms to overlap communication with computation. The motivation to solve this is well-founded.
2. Conceptual Approach: The core idea of slicing a collective operation into multiple partial collectives to facilitate pipelining is conceptually sound and presents an alternative to decomposing them into point-to-point SendRecv operations (as in Wang et al. [34]) or fine-grained broadcasts (as in SUMMA [30]).
3. Autotuner Framework: The recognition that 2D TP performance is sensitive to a complex interplay of hyperparameters (dataflow, mesh shape, sharding, granularity) is astute. Proposing an autotuner to navigate this space is a valuable direction, even if the implementation here has weaknesses.

Weaknesses

1. Over-reliance on Simulation and Lack of Empirical Validation: The primary weakness of this work is that its performance claims are not substantiated by real-world evidence.

   • The impressive speedup figures (e.g., 12.0% and 23.4% faster than the state-of-the-art in Section 1, Abstract) are derived exclusively from the SST simulator (Section 5.1).
   • The simulator itself is calibrated on a small 4-chip TPUv4 ring (Section 4.1, page 9). It is a significant and unsubstantiated leap to assume that a model calibrated on a 4-chip 1D topology can accurately predict the complex network dynamics, contention, and synchronization overheads of a 256-chip 2D torus. Non-linear scaling effects are not adequately addressed.
   • Critically, the real hardware experiments on a 4x4 TPUv4 cluster (Section 5.3, Table 3) show MeshSlice performing worse than the Collective baseline (45.5% vs 47.4% FLOP utilization for GPT-3). The authors attribute this to a lack of software support for overlapping AG/RdS, but this does not excuse the lack of empirical proof. The "MeshSlice Overlap (Estim.)" column in Table 3 is purely speculative and cannot be considered a result.

2. Flawed Validation of the Autotuner's Cost Model: The autotuner's utility depends entirely on the accuracy of its analytical cost models. The validation provided is circular and insufficient.

   • The cost model for communication is a simple linear function (Section 3.2.2, page 8) that ignores network contention, a crucial factor in large, busy torus networks.
   • The validation in Figures 13 and 14 compares the analytical model's predictions against the simulator's results, not against real hardware. This merely demonstrates that the simple analytical model can approximate the behavior of the complex simulator. It provides zero evidence that either the model or the simulator accurately reflects reality at scale.

3. Unclear Practicality and Ignored Implementation Hurdles: The authors admit that their method is not currently feasible on the target hardware (TPUv4) due to the lack of support for asynchronous, overlapped AG/RdS operations (Section 5.3).

   • This is not a minor implementation detail; it is a fundamental barrier to adoption. The paper presents a technique that relies on a software/hardware capability that does not exist in the evaluated public environment.
   • There is no discussion of the software engineering, compiler modifications, or runtime support that would be required to enable this functionality. Without this, the work remains a purely theoretical exercise.

4. Insufficient Justification for Slicing Granularity (S): The paper claims that the slice count S provides a knob to trade off prologue/epilogue overhead against synchronization overhead (Section 3.1, page 5). However, the analysis is superficial. The optimal S is found via an exhaustive search over a small set of integers. A more rigorous analysis would characterize this trade-off more formally and explain why this new partitioning scheme is fundamentally better than existing ones (e.g., Wang et al.'s SendRecv decomposition), which also allows for granularity control via message size.

Questions to Address In Rebuttal

1. How can the authors justify the validity of their 256-chip simulation results when the simulator was calibrated on a 4-chip ring? Please provide specific evidence or analysis showing that network contention and synchronization overheads are modeled accurately at this scale.
2. Given that MeshSlice underperforms on the real 4x4 TPUv4 cluster (Table 3), why should the community accept the speculative "Overlap (Estim.)" figures as evidence of the algorithm's effectiveness? This appears to be a claim based on a hypothetical machine, not the one that was tested.
3. The autotuner's cost model is validated against the simulator, not hardware. Can you provide any data from real hardware (even at the 4x4 scale) that validates the accuracy of your analytical communication cost model?
4. The entire premise of MeshSlice hinges on the availability of asynchronous AG/RdS collectives. Could you detail the specific compiler and runtime system changes required to enable this on a platform like JAX/XLA for TPUs? What is the estimated engineering effort and feasibility of such changes?
5. In the real hardware experiments (Section 5.3.1), Wang et al.'s method also shows a smaller-than-expected speedup, which you attribute to JAX compiler optimizations creating dependencies. Could this same compiler behavior also be creating unforeseen dependencies for MeshSlice, invalidating the core assumption that your partial GeMMs can be effectively overlapped with your partial collectives in a real execution environment?

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper introduces MeshSlice, a novel algorithm for 2D tensor parallelism (TP) in distributed DNN training, designed to overcome the limitations of existing 2D GeMM methods. The core problem is that prior algorithms either incur high communication traffic (Cannon), suffer from synchronization overhead (SUMMA), or cannot effectively overlap communication with computation (Collective-based approaches). The state-of-the-art (e.g., Wang et al.) can only overlap communication in one dimension of the 2D accelerator mesh.

The essential contribution of MeshSlice is a new technique for partitioning collective communication operations (AllGather and ReduceScatter) into multiple partial collectives. This "slicing" enables software pipelining that can overlap communication with computation along both the row and column dimensions of the accelerator mesh simultaneously. This bidirectional overlap is the key mechanism for hiding communication latency more effectively. Complementing the algorithm, the authors present the MeshSlice LLM autotuner, which uses analytical cost models to automate the complex task of finding optimal configurations for dataflow, mesh shape, and communication granularity. The work is evaluated via simulation on TPUv4-like clusters and demonstrates significant performance improvements over existing methods for training large language models like GPT-3 and Megatron-NLG.

Strengths

1. Elegant and Foundational Core Idea: The central concept of partitioning a collective operation into smaller, partial collectives is a powerful and elegant solution to the communication overlap problem. It directly addresses the "all or nothing" nature of the standard Collective 2D GeMM algorithm and provides a principled way to overcome the one-dimensional limitation of prior overlap techniques. This feels less like a heuristic and more like a fundamental primitive that was missing from the distributed computing toolbox for this domain.

2. Excellent Contextualization and Problem Framing: The paper does a superb job of placing itself within the historical and current landscape of distributed training. The introduction and background sections (Sections 1 & 2, pages 1-4) clearly articulate the evolution from 1D TP to 2D TP and provide a concise yet thorough analysis of the trade-offs between Cannon, SUMMA, and Collective-based algorithms. The timeline comparison in Figure 4 (page 5) is particularly effective at visualizing the specific inefficiency that MeshSlice resolves. This demonstrates a deep understanding of the research area.

3. Addresses the Full System Problem: The authors recognize that a novel algorithm alone is insufficient without a means to configure it. The inclusion of the MeshSlice LLM autotuner is a major strength. Optimizing 2D TP involves a complex interplay between sharding, dataflow, mesh topology, and granularity (the slice count S). By providing an automated, model-driven approach to this co-optimization problem (Section 3.2, page 7), the authors elevate the work from a theoretical algorithm to a practical, usable system. This greatly increases its potential for real-world impact.

4. Significant Potential Impact on Hardware Co-Design: This work has clear implications for the future of ML accelerator design. By providing a software technique that can effectively utilize 2D torus interconnects, it strengthens the case for building such topologies over more complex and expensive fully-connected networks, especially at scale. The ability of MeshSlice to adapt to non-square mesh shapes is crucial, as it allows system designers to match the mesh topology to the aspect ratio of the underlying matrix operations, rather than the other way around.

Weaknesses

While this is a strong paper, there are areas where its context and scope could be further explored.

1. Dependency on Physical Mesh Topology: The performance benefits of MeshSlice hinge on the existence of a physical 2D torus interconnect where row and column communications are independent and contention-free. This is characteristic of Google's TPU clusters but not of most GPU clusters, which typically use fat-tree or Dragonfly topologies. The authors briefly acknowledge this in the Discussion (Section 6, page 13), but the challenge is significant. On a logical mesh mapped to a fat-tree, row and column collectives would contend for the same network links, potentially negating much of the benefit from bidirectional overlap. The paper would be stronger if it included a more detailed analysis or preliminary data on the sensitivity of MeshSlice to network contention.

2. Simplicity of the Autotuner's Dataflow Heuristic: The autotuner's method for choosing a dataflow (Section 3.2.1, page 7) is based on a heuristic: keep the largest matrix stationary and default to a non-transposed version to avoid reshuffling data between layers. While this is a sensible and likely effective rule of thumb for standard Transformer architectures, it feels less rigorous than the rest of the paper. It is an open question whether more exotic model architectures or memory access patterns might favor a dataflow that this heuristic would miss.

Questions to Address In Rebuttal

1. Applicability to Contention-Prone Networks: Could the authors elaborate on the viability of MeshSlice for GPU-based supercomputers? When implementing MeshSlice on a logical mesh over a fat-tree network, how would the analytical cost models in the autotuner need to be adapted to account for network contention between the row and column communications? Would the optimal strategy perhaps revert to overlapping in only one dimension, similar to Wang et al., to avoid this contention?

2. Robustness of the Dataflow Heuristic: Regarding the autotuner's dataflow selection, have the authors considered architectures where minimizing the size of the communicated tensors is more important than keeping the largest tensor stationary? For example, in a memory-capacity-constrained scenario, it might be preferable to communicate a smaller tensor even if it requires transposing a larger, stationary one. Can you comment on the robustness of your heuristic outside of the standard LLM training regime?

3. Broader Generalizability of "Partial Collectives": The core idea of slicing collectives into partial collectives to enable pipelining is very compelling. Do you see this as a general-purpose technique? For instance, could this approach be used to optimize other distributed algorithms that are bottlenecked by large, monolithic collective operations, even outside the context of GeMM or machine learning?

Reviewer Persona: The Innovator (Novelty Specialist)

---

Summary

The paper proposes MeshSlice, a new algorithm for 2D General Matrix Multiplication (GeMM) aimed at optimizing 2D Tensor Parallelism (TP) in distributed DNN training. The central novel claim is a technique to partition collective communication operations (AllGather/ReduceScatter) into smaller partial collectives. This partitioning is applied to both the row and column dimensions of the accelerator mesh, enabling communication-computation overlap in both directions simultaneously. This is presented as a solution to the limitations of prior art: Cannon's algorithm is constrained to square meshes, SUMMA has high synchronization overhead, Collective 2D GeMM is blocking, and Wang et al. [34] only achieve overlap in a single dimension. The paper also introduces an autotuner that co-optimizes the dataflow, mesh shape, and a new "slice count" hyperparameter S introduced by the MeshSlice algorithm.

Strengths

1. Clearly Articulated and Novel Core Algorithm: The paper's primary strength lies in its core algorithmic contribution. The proposed method of slicing a collective operation into a series of smaller, partial collectives to enable pipelining is a distinct and novel approach in the context of 2D distributed GeMMs for DNN training. While prior work has explored partitioning, it has focused on different mechanisms with significant drawbacks.

2. Excellent Positioning Relative to Prior Art: The authors demonstrate a comprehensive understanding of the landscape of 2D GeMM algorithms. They correctly identify that Wang et al. [34] represents the closest state-of-the-art and precisely articulate the gap that MeshSlice aims to fill: achieving bidirectional overlap without incurring the high traffic and restrictive constraints of Cannon's algorithm [4]. The claim made in Section 2.3.4 (page 4) that partitioning collectives in both directions would otherwise require Cannon's algorithm effectively frames the problem that MeshSlice's novelty is designed to solve.

3. Introduction of a New, Tunable Parameter: The algorithm introduces the slice count S as a new hyperparameter that controls the granularity of the communication-computation overlap. This is a novel lever for performance tuning that does not exist in the monolithic Collective or Wang et al. algorithms. The autotuner's ability to optimize this parameter is a valuable supporting contribution.

4. Novelty in the Autotuner's Scope: While the use of analytical cost models is a standard technique, the autotuner's novelty lies in its application to the unique search space created by MeshSlice. Specifically, the ability to choose between different 2D dataflows (Section 3.2.1, page 7) and co-optimize this choice with mesh shape and the new slice count S is a more comprehensive approach than seen in related works like PrimePar [33], which is based on the more restrictive Cannon's algorithm.

Weaknesses

1. Incremental Nature of the Core Concept: While the application is novel, the fundamental concept of partitioning a large collective operation into smaller chunks to facilitate pipelining is a well-established pattern in high-performance computing (HPC). The paper would be strengthened by acknowledging that the core novelty is not the invention of "partial collectives" but rather their specific, structured application to solve the bidirectional overlap problem in 2D GeMMs for DNNs.

2. Novelty is Tightly Coupled to Torus Topology: The central claim of efficient, bidirectional overlap relies heavily on the non-contending nature of row and column communications in a physical 2D torus network (as found in TPU clusters). This assumption is critical to the algorithm's performance model. The novelty and associated benefits are less certain on more common interconnect topologies like fat-trees or Dragonfly networks, where logical row and column communications would contend for shared network resources. The discussion in Section 6 (page 13) acknowledges this but does not fully address how the fundamental value proposition of the algorithm might change in these prevalent hardware environments.

3. The "Slicing" Mechanism Itself: The mechanism for slicing the matrix shards (Section 3.1.2, page 6) is a necessary implementation detail, but it is not presented as a fundamental algorithmic novelty in its own right. It is an enabling technique for the core idea of partial collectives. The paper is clear about this, but it is worth noting that the novelty rests on the communication scheme, not the data layout transformation.

Questions to Address In Rebuttal

1. The core mechanism relies on partitioning AllGather/ReduceScatter into a loop of partial collectives on sub-shards. Could the authors clarify if this "partial AllGather/ReduceScatter" is a standard primitive available in the underlying communication libraries (e.g., as part of JAX/XLA) or a custom communication schedule implemented specifically for this work? This would help delineate the software engineering contribution from the conceptual novelty.

2. The key advantage over Wang et al. [34] is achieving overlap in a second dimension. This benefit is maximized on a physical torus where the two dimensions are independent. How does the novelty of MeshSlice hold up on a logical mesh mapped to a physical fat-tree network? Specifically, if row and column communications contend for the same physical links, does the benefit of overlapping the second dimension diminish significantly, potentially making it a marginal improvement over the simpler single-dimension overlap from prior art in that common scenario?

3. The autotuner optimizes dataflow, mesh shape, and slice count S. Table 2 (page 11) shows a significant 21.2% speedup for GPT-3 from dataflow optimization alone. Can the authors provide a sense of the relative importance of the novel contributions within the autotuner? Is the dataflow selection the dominant factor, or does the co-optimization of the new parameter S with the mesh shape provide a similarly crucial contribution to performance?

Reviewer Persona: The Guardian (Adversarial Skeptic)

---

Summary

The authors present OptiPIM, a framework that uses Integer Linear Programming (ILP) to generate what they claim are "optimal" mappings for data-intensive operators on Processing-in-Memory (PIM) accelerators. The central thesis is that existing mapping frameworks designed for ASIC accelerators are inadequate for PIM architectures due to PIM's stringent data layout requirements and the resulting complex cost trade-offs. The paper's primary contributions are a new "PIM-friendly" mapping representation based on a nested loop structure with an expanded set of indexing functions, and a more accurate cost model for estimating data movement that accounts for these layouts. The framework is evaluated on two PIM architectures (HBM-PIM and SIMDRAM) and shows significant speedups over heuristic and adapted ASIC-based mapping methods.

While the problem addressed is both timely and significant, I harbor serious reservations regarding the central claims of optimality, the methodological soundness of the baseline comparisons, and the generalizability of the reported results. The work rests on a foundation that appears strong at first glance but reveals critical weaknesses upon rigorous inspection.

Strengths

1. Problem Motivation: The paper does an excellent job in Section 3 (page 3) of motivating why PIM mapping is a distinct and challenging problem. The example in Figure 2 clearly illustrates how different computation schedules impose vastly different data layout, replication, and movement costs, correctly identifying the core deficiencies of layout-agnostic mapping approaches.

2. Improved Cost Estimation for Unique Indices: The authors correctly identify a key source of inaccuracy in prior work—the use of loop bound products to estimate data size. The proposed estimation function for unique indices in a linear combination (Equation 11, page 8) is more mathematically principled and demonstrably more accurate (Figure 7) than the naive upper-bound approach. This is a solid, albeit localized, contribution.

3. Validation against Exhaustive Search: The claim of optimality is, for the specific workloads tested, substantiated by a comparison against a full exhaustive search (Section 7.5.3, page 12). This provides confidence that, within the confines of their defined search space and cost model, the ILP formulation is indeed finding the best solution.

Weaknesses

1. The Claim of "Optimality" is Circular and Unverified: The paper's most prominent claim is its ability to generate optimal mappings. However, this optimality is only with respect to its own analytical cost model. This model's "ground truth" for validation is a custom-modified version of the Ramulator 2 simulator (Section 7.1, page 10 and Artifact Appendix A.1, page 15). The authors provide no evidence that this simulator has been validated against real PIM hardware or a trusted hardware model. The high correlation shown in Figure 9 only proves that the analytical model is a faithful abstraction of their simulator; it does not prove that either accurately reflects reality. Without validation against a genuine hardware ground truth, the "optimality" is merely a theoretical construct within a self-contained system.

2. Fundamentally Flawed Baseline Comparison: The comparison against the ASIC-based mapping method (COSA [23]) is a classic strawman argument. The authors state they "adopt COSA’s ILP optimization... without considering the fine-grained data layout-related costs" (Section 6.4.3, page 10). This is an inherently unfair comparison. The entire premise of the paper is that these costs are critical. By disabling the very capability that would make the baseline competitive, the authors are not comparing OptiPIM to a state-of-the-art ASIC mapper adapted for PIM; they are comparing it to a deliberately crippled baseline. A scientifically valid comparison would have involved attempting to integrate the new PIM-specific cost functions into the COSA framework to see if it could then find a comparable mapping. The current results only prove that ignoring PIM costs leads to poor PIM performance, which is a trivial conclusion.

3. Scalability of the ILP Formulation is Not Proven: The authors report a maximum optimization time of 4 minutes (Abstract, page 1), suggesting the ILP approach is highly efficient. This claim is based on a limited set of operators from common ML models. ILP problem complexity can scale exponentially with the number of variables and constraints. The paper provides no analysis or empirical data on how the solver runtime scales with key problem parameters, such as the number of loop nests, the magnitude of loop bounds, or the number of tensors. A single, more complex operator from a different domain could potentially cause the solver time to explode, rendering the approach impractical. The claim of efficiency is therefore not robustly supported.

4. Novelty and Expressiveness of the Mapping Representation is Overstated: The proposed mapping representation is described as a key innovation. However, the hierarchical nested loop structure for PU, column, and computation scheduling (Section 4.2, page 5) is conceptually equivalent to standard loop tiling and vectorization, common in many compiler frameworks. The main novelty lies in the exploration of 6 coefficient combinations for the indexing function (Section 4.3, page 6). While this is an improvement over a fixed stride-1 indexing, the paper fails to discuss why this fixed-structure approach is superior to more general and powerful representations like the polyhedral model, which can represent a much broader class of transformations and data layouts. It is not clear that the proposed representation can capture all potentially useful mappings.

Questions to Address In Rebuttal

1. Regarding the ASIC Baseline: Please justify the decision to evaluate the COSA framework without incorporating the PIM-specific cost terms you identified. Could the authors instead augment the ASIC baseline's cost model with these terms and re-evaluate? If this is not possible, please provide a detailed explanation of the fundamental limitations in the baseline's representation that prevent such an integration.

2. Regarding the Ground Truth for Optimality: How was the modified Ramulator 2 simulator itself validated? Can you provide any data correlating your simulator's performance predictions with either real commercial PIM hardware (e.g., Samsung's HBM-PIM or AXDIMM) or a hardware vendor's internal, validated simulation models?

3. Regarding ILP Scalability: Beyond the reported runtimes, can the authors provide data on how the ILP solver time scales as operator complexity (e.g., number of loops, size of loop bounds, complexity of indexing functions) increases? Please characterize the point at which the "within 4 minutes" claim no longer holds.

4. Regarding the Mapping Representation: Please clarify why the proposed nested loop structure with 6 indexing combinations is fundamentally more expressive or efficient for PIM mapping than existing, more general compiler representations like polyhedral models, which are also amenable to ILP-based scheduling. What specific, valid mappings can your representation express that a polyhedral model cannot, or vice versa?

Review Form: The Synthesizer

---

Summary

This paper presents OptiPIM, a framework for optimizing the mapping of data-intensive workloads onto digital Processing-in-Memory (PIM) architectures. The core contribution is the formulation of this complex mapping problem as an Integer Linear Programming (ILP) problem. The authors rightly argue that existing mapping techniques, designed for conventional ASIC accelerators, are insufficient for PIM due to its strict data layout requirements, which introduce significant costs related to data movement and replication that prior models neglect.

To enable this ILP formulation, the paper introduces two key technical innovations: 1) a novel, layout-aware nested loop representation that explicitly models the hierarchical structure of PIM systems (PU allocation, column allocation, and computation scheduling), and 2) more accurate cost models, particularly for estimating the number of unique memory accesses, which prove to be far more precise than the upper-bound estimates used in previous work. The framework is implemented in MLIR and evaluated on two representative digital PIM architectures (HBM-PIM and SIMDRAM). The results demonstrate that OptiPIM can find provably optimal mappings in minutes, delivering significant performance improvements (at least 1.9x, and much higher in many cases) over established heuristic and search-based methods.

---

Strengths

This is an excellent and highly significant piece of work. Its primary strength lies in bringing formal optimization to a problem space that has, until now, been dominated by heuristics and ad-hoc methods.

1. Clear and Compelling Problem Definition: The authors do a masterful job in Section 3 ("Motivation", page 3) of articulating why the PIM mapping problem is fundamentally different from the well-studied ASIC accelerator mapping problem. The example in Figure 2 is particularly illustrative, clearly showing how different mapping choices lead to vastly different data layouts, memory footprints, and data movement costs. This precise problem definition elevates the paper beyond a simple application of a known technique (ILP) to a new domain.

2. Foundational Contribution to PIM Compilation: By successfully formulating the PIM mapping problem for an ILP solver, OptiPIM provides an "oracle" for the community. It establishes a theoretical performance ceiling for a given operator on a given PIM architecture. This is invaluable; it allows researchers to quantify the performance gap of any new, faster heuristic they might develop. It moves the field from "this mapping seems fast" to "this mapping is X% of optimal."

3. Technically Sound and Novel Representation: The proposed layout-aware representation (Section 4, page 5) is well-suited to the problem. The explicit separation of PU allocation, column allocation, and scheduling maps cleanly to the physical realities of PIM hardware. Furthermore, the exploration of different data indexing functions (Section 4.3, page 6) is a subtle but powerful enhancement that expands the design space beyond what previous rigid mapping representations allowed. The accurate cost function for unique inputs (Section 5.3, page 8 and Figure 7) is a concrete improvement over prior art and is critical for the ILP solver to find a truly optimal solution.

4. Rigorous and Insightful Evaluation: The experimental methodology is comprehensive. The validation against a cycle-accurate simulator (Figure 9, page 10) builds confidence in the analytical model. The choice of baselines—including a state-of-the-art PIM heuristic, a search-based method, and an adapted ASIC mapping framework (COSA)—is strong and allows for a nuanced comparison. The performance breakdown plots (Figure 10, page 10) are particularly insightful, clearly showing that OptiPIM wins by minimizing the layout-dependent costs (like input loading) that other methods ignore or model poorly.

---

Weaknesses

The weaknesses of this paper are largely related to the inherent limitations of its chosen approach (ILP) and the scope of the problem addressed, rather than flaws in the execution of the research itself.

1. Scalability of the ILP Formulation: The classic Achilles' heel of ILP is its computational complexity. While the authors demonstrate impressive results with optimization times under 4 minutes per operator, this is for single operators. The complexity of the ILP problem can grow exponentially with the number of variables and constraints. It is unclear how this approach would scale to much larger, fused operators or if it were applied to an entire computational graph simultaneously, rather than on an operator-by-operator basis. The multi-operator optimization discussed in Section 5.6 is a good start but is not explored in depth.

2. Static Workload Assumption: The framework operates at compile-time and assumes that the dimensions of all tensors are known statically. This is a reasonable assumption for many vision models but is increasingly challenged by modern workloads, such as Transformers with variable sequence lengths or dynamic graph structures. The discussion in Section 7.7 acknowledges this, but it remains a significant practical barrier to applying OptiPIM directly in more dynamic execution environments.

---

Questions to Address In Rebuttal

The paper is excellent and I am strongly in favor of acceptance. The following questions are intended to help the authors strengthen the final version and contextualize the work's limitations.

1. On the Nature of Optimality: The paper focuses on optimizing individual operators. Could the authors comment on how the local optimality for each operator translates to global optimality for a full network? Is it possible that the optimal mapping for layer N results in a data layout that is highly inefficient for consumption by layer N+1, thereby requiring a costly data re-layout that negates the gains? Does the multi-operator formulation in Section 5.6 account for these inter-operator layout transformation costs?

2. On the Practicality for Compilers: Given the potential scalability concerns of ILP, do the authors envision OptiPIM being used directly in a production compiler, or do they see its primary role as an offline tool to generate a library of optimal mappings for common operator shapes? Alternatively, could the insights from the ILP solutions be used to train a machine learning model that could predict near-optimal mappings much more quickly?

3. On Architectural Generalization: The framework models two digital PIM architectures. How adaptable is the proposed representation and cost modeling to other classes of PIM, for instance, analog compute-in-memory (CIM) systems? What new constraints or cost functions (e.g., related to ADC/DAC precision, noise, or non-ideal device behavior) would need to be incorporated, and would the problem still be tractable within an ILP framework?

Reviewer: The Innovator (Novelty Specialist)

Summary

The authors present OptiPIM, a framework for generating optimal software-hardware mappings for Processing-in-Memory (PIM) architectures using Integer Linear Programming (ILP). The central claim to novelty is not the use of ILP for mapping—a technique well-established in the ASIC accelerator domain—but rather the development of a novel, PIM-specific mapping representation and associated cost models that make the ILP formulation both feasible and accurate for the unique constraints of PIM. Specifically, the authors' claimed novel contributions are: (1) a layout-aware, multi-level nested loop representation that explicitly models PU allocation, column allocation, and computation scheduling; (2) a "Comprehensive Data Indexing" method that explores a bounded set of linear transformations for data indexing, moving beyond the rigid stride-1 indexing of prior work; and (3) a more accurate cost function for estimating data sizes, which is critical for modeling PIM's layout-dependent costs.

Strengths

The primary strength of this work, from a novelty perspective, lies in its precise and well-reasoned departure from prior art. The authors correctly identify that existing ILP-based mapping frameworks for ASICs (e.g., COSA [23]) are fundamentally unsuitable for PIM due to their assumptions about data layout and movement. The novel contributions directly address these identified gaps:

1. Comprehensive Data Indexing (Section 4.3, Page 6): This is the paper's most significant and genuinely novel contribution. Prior mapping frameworks typically assume a fixed, tiled indexing scheme (e.g., x = x_outer * tile_size + x_inner). The authors' formalization that multiple bijective linear transformations are possible and that for a 3-level tiling there are exactly 6 such combinations (Figure 6b) is a novel insight. Incorporating the choice between these functions as a variable within the ILP formulation is a powerful and original technique that significantly expands the explorable design space to find more optimal data layouts.

2. Accurate Cost Modeling for Unique Indices (Section 5.3.3, Page 8): The second key novel element is the proposed cost estimation function for the number of unique inputs generated by a linear combination of loop variables (Equation 11). The paper provides a clear empirical demonstration in Figure 7 that the prior art's method (product of loop bounds) is highly inaccurate for this task, whereas their proposed function is very precise. This is not merely an incremental improvement; it is a necessary correction to make the ILP objective function meaningful for PIM, where data movement costs stemming from layout are dominant.

3. Structured PIM Mapping Representation (Section 4.2, Page 5): While hierarchical loop representations are not new in themselves, the specific three-level structure proposed (PU Allocation, Column Allocation, Computation Scheduling) provides a clean and effective abstraction for the PIM mapping problem. It correctly separates the different granularities of parallelism and data locality inherent to PIM architectures, making it amenable to a concise ILP formulation. This structured approach is a novel formalization tailored specifically for the PIM domain.

Weaknesses

The paper's claims of novelty should be carefully contextualized. While the specific formulation is new, the foundational components are built upon extensive prior work.

1. Overstated Novelty of the ILP Approach: The use of ILP to solve the accelerator mapping problem is not novel. This has been the core technique in works like COSA [23] and LEMON [52]. The authors' novelty lies entirely in the adaptation and formulation of the problem for PIM, not in the choice of ILP as a solver. The abstract and introduction could be clearer on this distinction, emphasizing that the novelty is the representation and modeling that enables an effective ILP solution for PIM.

2. Insufficient Differentiation from PIM-Specific Mapping Frameworks: The paper primarily contrasts its work against ASIC mapping frameworks. However, other frameworks exist for PIM mapping, such as ARES [7], which also proposes a general representation. While the authors correctly state that ARES uses search-based algorithms, the paper would be stronger if it included a more direct technical comparison of its representation to that of ARES in the main body. A brief discussion on why the ARES representation is unsuitable for an exact ILP formulation, and how OptiPIM's novel representation overcomes this, would more firmly establish its unique contribution within the PIM-specific literature.

Questions to Address In Rebuttal

1. Scalability of Comprehensive Indexing: Section 4.3 and Figure 6 show that for 3 loop levels, there are 3! = 6 valid coefficient combinations to explore. This seems tractable. However, how does this novel aspect of the formulation scale as the number of tiled loop levels (l) increases? The number of combinations would grow as l!. At what point does this enumeration of indexing choices become a bottleneck that makes the ILP problem intractable to formulate or solve?

2. Novelty vs. ARES Representation: The related work section mentions ARES [7], which proposes a "general representation" for mapping operators onto diverse PIM architectures. Could the authors please elaborate on the fundamental limitations of the ARES representation that make it ill-suited for an ILP-based optimization approach? What specific, novel features of the OptiPIM representation enable the concise and efficient ILP formulation presented in this work, which would not be possible if starting from the ARES abstractions?

3. Justification for Linearity of Transformation: The novel indexing in Section 4.3 explores linear transformations of the form d*x2 + e*x1 + g*x0. Is there a theoretical or architectural justification for limiting the search to only linear functions? Have the authors considered whether non-linear indexing functions could provide further benefits, and if so, why were they excluded from the novel formulation?

Review Form: The Guardian

---

Summary

This paper, "HeterRAG," proposes a heterogeneous Processing-in-Memory (PIM) architecture to accelerate Retrieval-augmented Generation (RAG) workloads. The authors identify that the two primary stages of RAG—retrieval and generation—have distinct system requirements. Retrieval is characterized by random memory access over large datasets, demanding high capacity, while generation is memory-bandwidth intensive. To address this, they propose a system combining low-cost, high-capacity DIMM-based PIM ("AccelDIMM") for the retrieval stage and high-bandwidth HBM-based PIM ("AccelHBM") for the generation stage. The system is further enhanced by three software-hardware co-optimizations: locality-aware retrieval, locality-aware generation, and a fine-grained parallel pipeline. The evaluation, conducted through a simulation framework, claims significant throughput and latency improvements over CPU-GPU and other PIM-based baselines.

---

Strengths

1. Well-Motivated Problem: The paper correctly identifies a critical and timely problem. The characterization of RAG workloads in Section 3, using execution breakdowns and roofline models (Figures 2, 3, 4), provides a solid foundation for the proposed solution. The analysis clearly establishes that both stages are memory-bound but with different characteristics, justifying a heterogeneous approach.

2. Logical High-Level Design: The core architectural concept—mapping the capacity-demanding retrieval stage to DIMM-PIM and the bandwidth-demanding generation stage to HBM-PIM—is sound. The authors rightly point out the futility of a naive approach where data is shuttled from DIMMs to HBM over a slow interconnect (Section 3.2), thereby motivating the need for compute capabilities on the DIMM side.

3. Inclusion of Relevant Baselines: The study includes "NaiveHBM" and "OnlyDIMM" baselines, which are crucial for validating the central hypothesis. The poor performance of NaiveHBM effectively demonstrates the interconnect bottleneck, while the comparison against OnlyDIMM helps isolate the benefits of using HBM for the generation stage.

---

Weaknesses

My primary concerns with this manuscript center on the validity of key performance claims, the ambiguity of critical mechanisms, and the overall robustness of the evaluation methodology.

1. Unsubstantiated Performance Claims and Questionable Scaling: The claim of "near-superlinear throughput improvement" for the retrieval stage (Section 5.4, page 11) is a significant red flag. Superlinear speedup is exceptionally rare and requires a strong theoretical justification, such as caching effects that scale non-linearly with the number of nodes. The paper attributes this to "data parallelism," which at best explains linear scaling. Without a rigorous explanation, this claim undermines the credibility of the entire evaluation. The headline performance numbers ("up to 26.5x") are also potentially misleading, as is common with "up to" metrics, and may not reflect average-case behavior.

2. Ambiguity in Core Optimization Mechanisms: The "fine-grained parallel pipeline" is presented as a key contribution (Section 4.4, page 9), but its implementation details are critically underdeveloped. The paper states the host "aggregates retrieval results at fixed intervals" and sends "high-confidence results" ahead. This is vague. How is the interval determined? What is the sensitivity of the system to this hyperparameter? What is the precise, non-heuristic logic for identifying a result as "high-confidence"? The efficacy of this entire optimization hinges on these details, which are absent from the paper.

3. Weakness of Evaluation Methodology: The entire evaluation rests on a simulation framework combining Ramulator and ZSim (Section 5.1, page 10). While simulation is a standard practice, this work fails to account for several real-world complexities:

   • Interconnect Modeling: The high-level interconnect is specified as CXL, but the performance impact of the CXL switch network, protocol overhead, and coherence traffic is not discussed or apparently modeled in detail. These factors can introduce non-trivial latency and limit scalability.
   • Baseline Hardware: The CPU-GPU baseline uses an NVIDIA V100 GPU. While a strong GPU, it is now two generations old. A comparison against a more contemporary architecture (e.g., H100) with significantly higher memory bandwidth and advanced features would provide a much more realistic assessment of HeterRAG's claimed benefits. The chosen baseline may artificially inflate the reported speedups.
   • Comparisons to Prior Work: The comparisons in Section 5.5 are made against results reported in other papers. This is not a scientifically rigorous method, as underlying experimental assumptions (e.g., system configuration, simulator parameters, benchmarks) are invariably different. These comparisons are suggestive at best and cannot be considered conclusive proof of superiority.

4. Incremental Novelty of Components: While the system-level integration is novel, the individual components appear to be implementations of existing ideas. The locality-aware generation is explicitly "inspired by a recent study [87]," the PIM architectures build upon concepts from AttAcc [64] and Newton [25], and vertex caching for retrieval is a standard technique. The paper needs to more clearly articulate the novel architectural contributions beyond the high-level system concept.

---

Questions to Address In Rebuttal

The authors must provide clear and concise answers to the following questions to justify the paper's claims:

1. Regarding "Near-Superlinear" Scaling: Please provide a detailed, evidence-backed explanation for the observed near-superlinear scaling of the AccelDIMM devices. What is the underlying architectural or algorithmic phenomenon that causes the system to scale better than linearly? Standard data parallelism does not suffice as an explanation.

2. Regarding the Fine-Grained Pipeline: Please elaborate on the scheduling algorithm for the fine-grained pipeline. Specifically:

   • How is the aggregation interval determined, and how sensitive is overall performance to this value?
   • What is the exact criterion used by the host to classify a partial retrieval result as "high-confidence" and thus suitable for early forwarding to the generation stage?

3. Regarding Evaluation Baselines:

   • Can you justify the choice of the V100 GPU as the primary baseline, given the existence of newer architectures with substantially higher memory bandwidth and compute power?
   • In the "OnlyDIMM" baseline, how does the design of the bank-level generation unit (BPM) compare in terms of computational throughput and efficiency to the dedicated AccelHBM device? Please clarify if this is a fair, apples-to-apples comparison of generation capability.

4. Regarding Interconnect Overheads: The CXL interconnect is not lossless. What latency and bandwidth assumptions were made for the interconnect in your simulation, and how does system performance degrade as interconnect latency increases or effective bandwidth decreases due to protocol overhead?

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper presents HeterRAG, a heterogeneous Processing-in-Memory (PIM) architecture designed to accelerate Retrieval-Augmented Generation (RAG) workloads. The core contribution is the insightful observation that the two primary stages of RAG—retrieval and generation—have fundamentally different memory requirements. Retrieval is characterized by large data capacity needs and irregular, random memory access patterns, while generation is dominated by high-bandwidth, more regular access.

Instead of proposing a monolithic PIM solution, the authors advocate for a specialized, heterogeneous system. They map the capacity-intensive retrieval stage to low-cost, high-capacity DIMM-based PIM (AccelDIMM) and the bandwidth-intensive generation stage to high-performance HBM-based PIM (AccelHBM). This architectural separation is complemented by several software-hardware co-optimizations, including locality-aware caching and a fine-grained pipeline to overlap retrieval and generation. The paper demonstrates through simulation that this approach significantly outperforms conventional CPU-GPU systems and more naive PIM configurations in terms of throughput, latency, and energy efficiency.

Strengths

1. Excellent Problem-Architecture Mapping (The Core Insight): The paper's primary strength is its clear-eyed identification of RAG as a workload with two distinct phases whose memory characteristics map beautifully onto the two major PIM technologies available today. The retrieval stage, with its massive knowledge bases, is a natural fit for the capacity and cost profile of DIMM-based PIM. The generation stage, bottlenecked by GEMV operations during autoregressive decoding, is a perfect candidate for the high bandwidth of HBM-based PIM. This is not just an application of PIM; it is a thoughtful synthesis of the right tool for the right job, which represents a significant step forward in thinking about system design for complex, multi-stage AI workloads.

2. Strong Grounding in a Critical Workload: The paper addresses a problem of immense practical importance. RAG is rapidly becoming the de facto standard for building knowledgeable and factual AI systems. By focusing on the system-level bottlenecks of this entire workflow, rather than just LLM inference in isolation, the work is highly relevant and has the potential for significant impact. The characterization study in Section 3.1 (page 4), using roofline models and execution breakdowns, provides a compelling, data-driven motivation for the proposed architecture.

3. Holistic and Plausible System Design: The authors present more than just a pair of accelerators; they propose a complete system. The inclusion of a CXL-based interconnect, a clear host-device execution flow (Section 4.1, page 5), and a conceptual software stack (Figure 11, page 9) shows a mature approach to system design. This holistic view makes the proposal more credible and provides a clearer path toward a real-world implementation.

4. Connects Disparate Research Threads: This work serves as an excellent bridge between two very active but often separate areas of architecture research: PIM for LLM inference (e.g., AttAcc, NeuPIMs) and PIM for graph/search algorithms (e.g., RecNMP). By building a system that requires both, HeterRAG effectively synthesizes techniques from both domains, demonstrating how they can be complementary components in a larger system. The authors explicitly acknowledge their debt to prior work (Section 4.2 and 4.3), which is commendable.

Weaknesses

While the core idea is strong, the work could be better contextualized and its limitations more thoroughly explored.

1. The Evolving Nature of RAG: The proposed architecture is tightly coupled to the current dominant RAG paradigm: graph-based ANNS for retrieval followed by autoregressive transformer decoding. However, the RAG space is evolving rapidly. Future techniques might involve different search indexes, simultaneous retrieval and generation, or non-autoregressive models. The paper could benefit from a discussion on the architecture's adaptability to these potential algorithmic shifts. How much of the proposed hardware is special-purpose versus programmable?

2. Understated Role of the Interconnect: The paper uses CXL as the interconnect fabric, which is a sensible choice. However, as the system scales up with many AccelDIMM and AccelHBM units, the all-to-some communication pattern (where retrieved results from many AccelDIMMs are gathered by the host and sent to AccelHBMs) could become a bottleneck. The analysis assumes this overhead is minimal, but a more detailed projection of interconnect traffic under high load would strengthen the scalability claims made in Section 5.4 (page 11).

3. Generalization Claims: In the discussion (Section 4.6, page 9), the authors suggest HeterRAG is well-suited for other workloads like graph processing and recommendation systems. While this is conceptually plausible, the paper is squarely focused on RAG. These claims, while interesting, are speculative without supporting data and might be better framed as promising avenues for future work.

Questions to Address In Rebuttal

1. The fine-grained parallel pipeline optimization (Section 4.4, page 9) is an elegant way to hide latency by overlapping retrieval and generation. Could you provide a more quantitative analysis of its benefit? For instance, what is the typical distribution of retrieval completion times within a batch for your test workloads, and how much of the potential idle time does this pipelining strategy effectively reclaim?

2. The architectural design makes a strong commitment to graph-based ANNS for retrieval. How would the AccelDIMM design need to change to support other popular retrieval methods, such as those based on inverted file indexes (IVF) or product quantization (PQ)? Is the core idea of a DIMM-PIM/HBM-PIM split robust to changes in the underlying retrieval algorithm?

3. Could the authors elaborate on the data path for the retrieved results? The text describes the host aggregating results from AccelDIMMs, mapping vector IDs to documents, and then sending tensors to AccelHBMs. At scale, could this "host-as-router" model become a performance or bandwidth bottleneck? Have you considered a more direct data path between the DIMM and HBM subsystems for certain RAG variants?

Reviewer: The Innovator (Novelty Specialist)

Summary

This paper presents HeterRAG, a heterogeneous Processing-in-Memory (PIM) system designed to accelerate Retrieval-Augmented Generation (RAG) workloads. The core architectural proposal is to combine two distinct types of PIM: a high-capacity, DIMM-based PIM (AccelDIMM) for the retrieval stage (specifically, graph-based ANNS) and a high-bandwidth, HBM-based PIM (AccelHBM) for the generation stage (LLM inference). The authors claim this heterogeneous approach overcomes the capacity/cost limitations of HBM-only systems and the bandwidth limitations of DIMM-only systems. The architecture is supported by three software-hardware co-optimizations: locality-aware retrieval (caching), locality-aware generation (a hardware-accelerated KV cache scheme based on prefix trees), and a fine-grained parallel pipeline to overlap the two stages.

Strengths

From a novelty perspective, the paper's strengths lie not in its high-level architectural concept, but in its specific, system-level co-design choices that are tightly coupled to the RAG workload.

1. Hardware Acceleration of a Recent Software Technique: The most novel contribution is the "locality-aware generation" mechanism (Section 4.4, page 8). The idea of combining prefix trees with selective recomputation for KV cache management is itself very recent, with the authors citing a 2025 paper [87] (CacheBlend). The design of dedicated hardware units (Tree Search Unit, KV Substitution Unit, Token Filtering Unit shown in Figure 9) to accelerate this specific software technique is a genuinely new hardware-software co-design contribution.
2. Nuanced Pipelining: The "fine-grained parallel pipeline" (Section 4.4, page 9) demonstrates a more sophisticated approach than simple coarse-grained overlapping of retrieval and generation. The proposed mechanism of periodically aggregating partial retrieval results and forwarding high-confidence candidates to the generation stage is a clever system-level optimization that exploits the known behavior of best-first graph search algorithms. This is a non-obvious and specific contribution.

Weaknesses

My primary concern with this paper is the overstatement of novelty regarding its core architectural framework. While the integration is complex and the results are compelling, the fundamental ideas are largely derivative of prior art.

1. The Heterogeneous PIM Concept is Not Fundamentally New: The central claim of novelty is a heterogeneous PIM system. However, this is a logical, if not obvious, application of existing principles. The RAG workload is cleanly divisible into two phases with opposing memory requirements: retrieval (high capacity, random access) and generation (high bandwidth, streaming access). Mapping these to DIMM-based PIM and HBM-based PIM, respectively, is a natural system design choice rather than a groundbreaking architectural innovation. The concept of heterogeneous computing and memory systems is well-established.

2. Component PIM Architectures Are Based Heavily on Prior Work:

   • PIM for Retrieval: The use of DIMM-based PIM for accelerating ANNS has been explored. The related work section itself cites MemANNS [15] and DRIM-ANN [14], which use commercial DIMM-PIM for this task. The design of AccelDIMM (Section 4.2, page 6) is an engineering contribution that adopts established techniques like rank-level processing and instruction compression from prior works such as RecNMP [39] and TRiM [65]. The novelty delta here is incremental.
   • PIM for Generation: Similarly, accelerating transformer inference with HBM-based PIM is a very active area of research. The paper explicitly states that AccelHBM (Section 4.3, page 7) "adopt[s] the same mapping scheme as AttAcc [64]" and draws inspiration from "Newton [25]". Therefore, the novelty of AccelHBM itself is minimal; it is an application of known techniques.

3. "Locality-Aware Retrieval" is Standard Practice: The first co-optimization, "locality-aware retrieval" (Section 4.4, page 8), is described as caching frequently accessed vertex vectors and reusing search results as starting points in iterative queries. These are standard caching and heuristic optimization techniques, respectively. While applying them is necessary for a high-performance system, it does not constitute a novel research contribution.

In essence, the paper combines two known PIM acceleration strategies (PIM for ANNS and PIM for Transformers) into a single system. The novelty is in the integration and the two more advanced co-optimizations, not in the headline architectural concept itself.

Questions to Address In Rebuttal

The authors should use the rebuttal to clarify and defend the precise novelty of their contributions.

1. The paper presents the heterogeneous HBM+DIMM architecture as its primary contribution. Given that the components (AccelHBM, AccelDIMM) are heavily based on prior art ([64], [39], [65], etc.), could the authors precisely articulate the novel architectural insight beyond the mapping of RAG stages to suitable memory technologies? What is the fundamental architectural challenge in this integration that this work is the first to solve?
2. Regarding the fine-grained pipeline (Section 4.4, page 9), prior work like PipeRAG [34] also proposes aggressive overlapping of retrieval and generation. Please clarify the key difference and novel step that your interval-based, confidence-aware aggregation mechanism provides over such prior art.
3. Could the authors re-frame their primary contribution? Is it the heterogeneous architecture itself, or is the main contribution the hardware-software co-design for locality-aware generation (accelerating [87]) and the fine-grained pipeline, which are enabled by a heterogeneous architecture? Clarifying this would help position the work more accurately within the literature.

Reviewer: The Guardian

---

Summary

The authors present ATİM, a tensor compiler framework designed to autotune and generate code for Processing-in-DRAM (PIM) systems, specifically targeting the UPMEM architecture. The core contributions are threefold: 1) a unified search space that jointly optimizes host-side data distribution and kernel-side loop transformations; 2) a set of PIM-aware compiler optimizations, most notably for eliminating boundary check overheads; and 3) enhancements to the evolutionary search algorithm to better handle the expanded search space. The paper claims significant performance improvements over hand-tuned libraries (PrIM) and other baselines on both microbenchmarks and layers from the GPT-J model, evaluated on real UPMEM hardware.

---

Strengths

1. End-to-End System: The paper presents a complete, functional system that bridges high-level tensor abstractions down to executable code for a real, commercial PIM architecture. This represents a substantial engineering effort.
2. Real Hardware Evaluation: The primary performance evaluations are conducted on a physical UPMEM server (Section 6), which lends significant credibility to the reported latency numbers, as opposed to relying solely on simulation.
3. Well-Motivated Optimizations: The PIM-aware optimizations detailed in Section 5.3, particularly the analysis of boundary check elimination (Figure 4, Page 4), are technically sound and address a well-understood performance bottleneck on simple in-order cores like the UPMEM DPU.

---

Weaknesses

My primary concerns with this work center on the fairness of the experimental comparisons and the potential conflation of contributing factors, which may lead to an overstatement of the proposed system's core contributions.

1. Baseline Unfairness and the Source of Performance Gains: The headline performance gains (e.g., 6.18x for MTV, 8.21x for GPT-J layers) appear to stem not from a superior compiler per se, but from comparing a system capable of a powerful optimization (2D tiling with hierarchical reduction) against baselines that are artificially constrained. The authors themselves identify this as the key differentiator in Section 7.1 (Page 10): "By applying 2D tiling on both spatial and reduction loop dimensions, ATiM generates a sufficient number of smaller tiles..." The PrIM and even the authors' own "PrIM+search" baselines are limited to 1D tiling. This is not a like-for-like comparison of autotuning frameworks; it is a demonstration of the known benefits of a specific tiling strategy. An expert programmer could implement 2D tiling manually. Therefore, the comparison does not isolate the benefit of ATİM's autotuner from the benefit of a more advanced tiling strategy that the baselines were not configured to use. The work is effectively comparing two different classes of algorithms, which inflates the perceived contribution of the compiler framework itself.

2. Conflation of Search Space and Search Algorithm: The paper introduces both an expanded, joint search space (Section 5.2.1) and a modified search algorithm ("Balanced Evolutionary Search," Section 5.2.3). The results in Section 7.4 (Figure 14, Page 12) attempt to justify the new algorithm but do so by comparing ATİM's full solution against standalone components. A crucial piece of analysis is missing: a clear decoupling of the gains. The performance improvement could primarily come from the richer search space, with the algorithm providing only marginal benefit, or vice versa. Without an experiment that applies a baseline search algorithm (e.g., default TVM) to the new joint search space, it is impossible to attribute the performance gains correctly between these two distinct contributions.

3. Inconsistent Evaluation Methodology: For the main performance results in Section 7.1 and 7.2, the authors use real hardware. However, to evaluate the impact of their PIM-aware optimizations in Section 7.3 (Figure 13, Page 12), they switch to the uPIMulator simulator. This switch is not justified, nor is the simulator validated against the real hardware used elsewhere. Simulators can fail to accurately model memory access contention, DMA overheads, and other microarchitectural effects. Presenting critical performance breakdown data from an unvalidated simulator undermines the conclusions drawn about the real-world impact of these specific optimizations.

4. Unsupported Claims of Generality: The paper focuses exclusively on the UPMEM architecture. While the Discussion (Section 8, Page 12) speculates on extending ATİM to other architectures like HBM-PIM, these claims are entirely unsubstantiated. The current implementation, especially the lowering passes for host/kernel communication and DPU binding, is tightly coupled to UPMEM's programming model. The work as presented does not provide the necessary abstractions or evidence to support its portability.

---

Questions to Address In Rebuttal

1. Regarding the main performance claims in Section 7.1: Can you justify the decision to limit the PrIM+search baseline to 1D tiling? To demonstrate the value of your autotuner, would not a fairer comparison be against a PrIM kernel that is manually optimized by an expert using the same 2D tiling and hierarchical reduction strategy that ATİM discovers?

2. To decouple the contributions of your search space and search algorithm, please provide an ablation study. Specifically, what is the performance of a system using the default TVM evolutionary search on ATİM's proposed joint host-kernel search space? This would isolate the benefit derived purely from the expanded space.

3. Please justify the switch to uPIMulator for the analysis in Section 7.3. Can you provide any data correlating the performance characteristics (e.g., memory stall cycles, instruction mix) reported by the simulator with performance counters or observed behavior on the real UPMEM hardware used in Section 7.1?

4. The paper claims to establish a "foundation for advancing DRAM-PIM programmability" (Abstract, Page 1). Given that the current implementation is specific to UPMEM, what concrete, implemented abstractions in ATİM's design would facilitate porting it to a fundamentally different PIM architecture, such as Samsung's HBM2-PIM or SK Hynix's GDDR6-AiM, which use different execution and data mapping models?

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper presents ATİM, a search-based, optimizing tensor compiler for Processing-in-DRAM (PIM) systems, specifically targeting the commercial UPMEM architecture. The work's essential contribution is bridging the gap between high-level tensor abstractions and the complex, low-level realities of PIM programming. It achieves this by extending the Apache TVM compiler framework to automate the co-optimization of both host and PIM-kernel code. Key innovations include (1) defining and exploring a joint search space that unifies host-side data distribution strategies with kernel-side loop transformations, (2) introducing PIM-aware compiler optimizations to mitigate hardware-specific bottlenecks like boundary checks, and (3) refining the evolutionary search algorithm to effectively navigate this expanded and complex optimization space. The authors demonstrate that this automated approach can generate code that significantly outperforms highly-optimized, hand-tuned libraries on a variety of tensor operations and layers from the GPT-J model.

Strengths

1. Addresses a Critical Problem: The primary obstacle to the widespread adoption of novel accelerators like PIM is not a lack of hardware potential, but the immense difficulty of programming them effectively. This paper directly confronts this programmability and performance portability crisis. By providing a fully automated path from a high-level tensor operation to optimized PIM code, ATİM represents a significant step towards making PIM a viable and accessible architectural player, rather than a niche curiosity.

2. Elegant Conceptual Framing: The central insight—that host and kernel code for PIM must be optimized jointly—is both correct and critical. Unlike a GPU where a powerful runtime and hardware scheduler abstracts away many data placement issues, PIM performance is deeply coupled to how the host partitions and distributes data across thousands of simple processing units (DPUs). The paper’s approach of repurposing TVM's schedule primitives to represent this joint search space (Table 2, page 6) is an elegant and powerful way to frame this complex co-design problem within a proven compiler paradigm.

3. Strong Connection to Broader Compiler Trends: This work fits perfectly within the modern compiler philosophy championed by systems like Halide and TVM, which advocate for separating the algorithmic specification ("what") from the performance schedule ("how"). ATİM successfully demonstrates that this philosophy is not only applicable but essential for taming the complexity of PIM. It serves as an excellent case study on how these domain-specific, search-based compilation techniques can be adapted to unlock the potential of new and unconventional hardware.

4. High-Impact Empirical Validation: The performance gains are substantial and compelling. Outperforming hand-tuned, vendor-adjacent libraries (like the PrIM benchmarks) by factors of up to 8.21x on real-world LLM kernels (Section 7.2, page 11) is a powerful statement. It validates the core hypothesis that a systematic, automated search can discover non-obvious optimizations that even human experts might miss, especially given the vast and interdependent parameter space.

Weaknesses

From the perspective of contextualizing the work's long-term impact, the weaknesses are less about flaws and more about opportunities for broader framing:

1. Hardware Specificity: The work is, by necessity, tightly coupled to the UPMEM architecture. While the discussion (Section 8, page 12) mentions extensibility to other PIM designs (e.g., HBM-PIM), the paper would be strengthened by a more explicit discussion of which principles are fundamental to PIM in general versus which are artifacts of UPMEM. For example, the joint host-kernel search is a universal PIM problem. However, the specific PIM-aware optimizations for boundary checks (Section 5.3, pages 7-8) are a direct consequence of UPMEM’s simple, in-order RISC cores. A clearer separation would help position ATİM as a foundational framework for PIM compilation, not just a UPMEM compiler.

2. The Cost of Automation: The paper rightly focuses on the performance of the generated code, but the cost of the autotuning process itself is a crucial practical barrier. The discussion section briefly notes the overhead is higher than for CPUs. This is a key finding. Quantifying this trade-off more formally (e.g., plotting performance gain vs. tuning time) would provide a more complete picture for practitioners and would situate the work in the broader context of "online" vs. "offline" compilation strategies for ML models.

Questions to Address In Rebuttal

1. The joint search space is the paper's most significant conceptual contribution. Can the authors provide a concrete example of a counter-intuitive trade-off discovered by ATİM? For instance, was there a case where the autotuner selected a less-efficient kernel configuration because it enabled a dramatically better host-side data distribution or reduction strategy, a solution a human programmer focused on kernel optimization might overlook?

2. The PIM-aware optimizations targeting boundary checks (Section 5.3) are fascinating, as they address the limitations of simple in-order cores. These architectural constraints are not unique to PIM; they are also common in resource-constrained hardware like embedded CPUs and edge AI accelerators. Could the authors comment on the potential for generalizing these specific tensor-level branch-hoisting and loop-tightening techniques beyond the PIM domain?

3. Looking forward, how would ATİM's fundamental abstractions need to evolve to support PIM architectures with different compute primitives, such as the fixed-function MAC units in Samsung's HBM-PIM or SK Hynix's GDDR6-AiM? Would the existing TVM schedule primitives for tiling and caching be sufficient, perhaps mapped to new semantics via the lowering process, or would this fundamentally different hardware model necessitate new high-level primitives in the search space?

Reviewer: The Innovator (Novelty Specialist)

Summary

The paper presents ATiM, a tensor compiler designed to generate optimized code for the UPMEM Processing-in-DRAM (PIM) architecture. The authors identify that existing PIM software stacks lack high-level abstractions and systematic optimization frameworks. ATiM's core proposal is to create a unified, search-based autotuning framework that co-optimizes host-level decisions (data distribution across PIM cores) and kernel-level loop transformations simultaneously.

To achieve this, the authors make three primary claims of novelty: 1. The formulation and exploration of a joint search space for host and kernel programs, enabled by repurposing the schedule primitives of the TVM tensor compiler. 2. A set of "PIM-aware" compiler optimizations at the tensor IR level to eliminate performance bottlenecks specific to simple in-order PIM cores, particularly redundant boundary checks. 3. An enhancement to the evolutionary search algorithm to counteract a sampling bias inherent to the expanded PIM search space.

The paper demonstrates that this approach can generate code that outperforms hand-tuned libraries for UPMEM. My review will focus exclusively on the novelty of these contributions relative to prior art.

Strengths

The primary conceptual novelty of this work is the elegant formulation of the joint host-kernel optimization problem. While autotuning for heterogeneous systems is a well-established field, the UPMEM architecture presents a unique challenge where the host's data distribution strategy is not merely a data-copying prelude but fundamentally defines the work performed by each kernel. The key insight and novel mechanism is the repurposing of TVM's existing schedule primitives to describe this coupled space (Section 5.2.1, page 5). Using primitives like split, reorder, and bind—traditionally used for loop transformations—to also define inter-DPU data tiling and mapping is a clever and non-obvious abstraction. It avoids the need to invent an entirely new scheduling language and instead extends the semantics of a known one. This is the paper's strongest and most original contribution.

The second area of notable novelty is in the PIM-aware optimizations (Section 5.3, pages 7-8). While the individual techniques (boundary check elimination, loop-bound tightening, invariant code motion) are known compiler concepts, their application at the TensorIR level is novel and well-justified. The authors convincingly argue that the high-level semantic guarantees of the tensor IR (e.g., knowledge of consumer operations from compute_at) enable transformations that are unsafe or intractable for a low-level compiler. The specific combination of invariant branch hoisting with what is effectively partial dead code elimination (Section 5.3.3) is a particularly strong example of exploiting high-level semantics for an aggressive optimization that a general-purpose compiler would likely avoid.

Weaknesses

While the core ideas are strong, the novelty of some components is incremental when deconstructed.

1. Joint Optimization as a Concept: The overarching idea of co-optimizing host and device code is not fundamentally new. Frameworks for heterogeneous computing have long grappled with decisions about data movement, tiling for memory hierarchies, and kernel scheduling. The novelty here is not the goal of joint optimization, but the specific formulation for the tightly-coupled PIM domain. The paper's contribution should be framed as a novel mechanism for a known problem, rather than the identification of the problem itself.

2. Search Algorithm Enhancements: The "improved search algorithms" described in Section 5.2.3 (page 7) consist of applying balanced sampling and an adaptive epsilon-greedy strategy. These are standard, well-known techniques from the fields of machine learning and search heuristics. The contribution is identifying a specific sampling bias (the "rfactor primitive bias") within their framework and applying an off-the-shelf solution. This is a sound engineering improvement necessary to make their system work well, but it does not represent a novel contribution to the field of search algorithms.

3. Marginal Benefit of Some Optimizations: The PIM-aware optimizations, while novel in their application context, must be weighed against their complexity. The experimental results in Figure 12 (page 12) show that the DMA-aware elimination provides the vast majority of the benefit. The subsequent Loop-bound tightening (LT) and Invariant branch hoisting (BH) provide smaller, though still positive, gains (often in the 5-15% range). The implementation complexity of these passes, especially the logic for hoisting combined with PDCE, may be substantial. For a novel technique to be significant, it should ideally provide a more transformative benefit. The case is made that these small gains matter on resource-constrained cores, but the contribution feels more incremental than foundational.

Questions to Address In Rebuttal

1. On Repurposing Primitives: The core claim rests on repurposing TVM primitives. Prior work in TVM and other tensor compilers already uses scheduling primitives to control data layout, tiling, and memory scope (e.g., mapping to shared vs. global memory in GPUs). Could the authors clarify the fundamental conceptual difference between mapping a tensor tile to a GPU thread block's shared memory (a standard practice) and mapping a tensor tile to a DPU's MRAM (the proposed technique)? Is the novelty simply the target (a PIM DPU) or is there a deeper semantic distinction in how the primitives are being interpreted for the host-level code generation?

2. On the Novelty of High-Level Optimization: The paper argues that optimizations like invariant branch hoisting with PDCE are enabled by the high-level semantics of TensorIR. However, polyhedral compilation frameworks (e.g., Polly [20], Pluto) also operate on high-level loop nest representations with perfect dependence information. Could a state-of-the-art polyhedral framework, when applied to the same problem, not derive an identical or functionally equivalent code transformation? What, specifically, does TensorIR enable that a polyhedral representation does not in this context?

3. On Generalizability of the Search Bias: The solution in Section 5.2.3 addresses the "rfactor primitive bias." Is this bias a fundamental property of the PIM optimization search space, or is it an artifact of the specific evolutionary search algorithm implemented in TVM/Ansor? If it is the latter, then the contribution is more of a patch for a specific framework's limitation rather than a novel, generalizable solution for PIM autotuning. Please clarify.

Review Form: Single Spike Artificial Neural Networks Reviewer Persona: The Guardian (Adversarial Skeptic)

---

Summary

The authors present a computing paradigm, "Single Spike Artificial Neural Networks," that attempts to merge traditional ANNs with the temporal dynamics of SNNs. The core concept is to represent ANN weights as programmable delay elements and activations as the arrival time of a single digital pulse ("spike"). The standard multiply-accumulate operation is mapped to a temporal sequence of delay (for multiplication) and a negative log sum exponential (nLSE) approximation (for accumulation). The authors propose architectural innovations, including a hybrid temporal/digital systolic array, an improved nLSE approximation circuit, and the integration of emerging temporal memories to claim significant improvements in energy efficiency (up to 3.5x over 8-bit digital) and latency (up to 4x over SNNs) on the MLPerf Tiny benchmark suite.

While the underlying mathematical transformation is coherent, the paper's claims rest on a foundation of brittle approximations, optimistic hardware assumptions, and an incomplete analysis of scalability and robustness. The work mistakes a clever but highly-constrained proof-of-concept for a generally applicable and robust solution.

Strengths

1. Consistent Mathematical Framework: The paper's primary strength is the formal mapping of standard ANN arithmetic into "delay space" (Table 1, page 2). The transformation of multiplication into addition (delay) and addition into nLSE is mathematically sound and provides a clear theoretical basis for the work.

2. Identification of Approximation Flaws: The authors correctly identify that prior nLSE approximations are insufficient and introduce bias. The investigation into a complementary "inverse approximation" to improve accuracy (Section 2.2, Figures 3 and 4) demonstrates a rigorous attempt to address a core weakness of the underlying approach.

3. Insightful Analysis of Data Distribution Impact: The analysis in Figure 10 (page 10) is a valuable contribution, as it clearly demonstrates why the proposed method fails on certain benchmarks. Linking the PE tree structure to the distribution of nLSE input differences provides a concrete explanation for the performance degradation on the Anomaly Detection task, revealing the method's sensitivity to network topology and data statistics.

Weaknesses

My primary concerns with this work are the fragility of the core computational primitive, an over-reliance on non-standard hardware to make its most significant claims, and an unconvincing treatment of noise and scalability.

1. The Core Approximation Lacks Generalizability and Robustness: The entire system is built upon the nLSE approximation, which is shown to be fundamentally brittle.

   • Benchmark-Specific Failure: The paper's own results show a catastrophic failure on the Anomaly Detection benchmark, where accuracy drops from an original of ~0.9 to as low as 0.46 with a balanced PE tree (Table 2, page 9). The authors are forced to advocate for a specific, unbalanced 9-tree configuration to recover performance. This is not a sign of a "generally applicable" architecture; it is evidence of a highly-tuned method that is not portable across different network structures or problem domains.
   • Extreme Sensitivity to Noise: The noise analysis in Figure 9 (page 10) is alarming. While the authors frame the voltage variation model as a "worst case scenario," a >15% drop in accuracy with a minor voltage swing indicates a system with razor-thin margins. Real-world systems experience power supply noise, and this level of sensitivity makes the practical deployment of this architecture highly questionable.

2. Headline Claims Hinge on Idealized Temporal Memory: The paper's most compelling claim—a 3.5x energy improvement over digital systolic arrays—is entirely dependent on the use of a memristive temporal memory (TM) system (Section 3.3, page 7). The digital memory (DM) version of their architecture (DS DM) shows performance that is, at best, on par with the digital baseline (Figure 12, page 11).

   • This is a critical flaw in the paper's narrative. The authors are not comparing their novel compute paradigm to the state-of-the-art; they are comparing a system that combines their paradigm with a non-standard, emerging memory technology against a standard-memory baseline. Memristive devices suffer from well-known issues of process variability, limited write endurance, and precision degradation, none of which are adequately modeled or discussed. The claim of "7 bits of precision" (Section 3.3) is optimistic and ignores the significant overhead of the required analog control and readout circuitry.

3. Unconvincing Scalability and Cost Analysis: The paper glosses over fundamental scaling challenges that would arise in any practical implementation.

   • Broadcast and Noise: The proposed broadcast of input signals from a central DTC (Section 3.1, page 5) is a known bottleneck. Figure 13 (page 11) shows the worst-case error growing with array width. The authors dismiss this by stating it is "well under the timing margin," but this linear trend suggests that for larger, more realistic array sizes required for non-trivial networks, this noise will become a dominant error source. The analysis is insufficient to prove scalability.
   • Hidden Cost of Temporal Recurrence: The mechanism for temporal reuse via resynchronization (Section 3, page 5) is described as delaying a signal by a full cycle time T. This is a non-trivial operation. The claim that this delay can simply be "combined with the delay inherent to the nLSE approximation" is convenient but unsubstantiated. What happens if the nLSE computational path is very short? A long, power-hungry delay line would be required, the cost of which is not included in the energy analysis.

Questions to Address in Rebuttal

1. Given the dramatic failure of your method on the Anomaly Detection task with standard balanced tree structures, can you formally define the class of neural networks or data distributions for which your approximation is valid? How would a designer know a priori if your architecture is suitable for their model?

2. Please provide a more honest comparison against the digital baseline by focusing on the "DS DM" results. Can you justify the value of your approach when, using standard digital memories, its energy consumption is comparable to a conventional digital accelerator? Alternatively, can you provide data from fabricated temporal memory arrays that substantiate your claims of 7-bit precision and low energy, including all control and peripheral circuit overhead?

3. The "improved approximation" relies on a "temporal average" circuit (Section 2.2, page 4), which you state is achieved via nLSE(x, y) + ln 2. This is an exact mathematical operation. Please provide the circuit-level implementation of this and analyze its area, energy, and susceptibility to the very noise you show is detrimental in Figure 9. How "approximate" is this circuit in reality?

4. Please provide a quantitative analysis of the area and energy cost of the cycle-time delay T required for temporal recurrence. How often is the inherent computational delay insufficient, thereby requiring an explicit, long delay chain? How does this unaccounted-for cost impact your overall energy claims?

Review Form: Synthesizer Persona

Summary

This paper presents a novel approach for neural network inference, termed "Single Spike Artificial Neural Networks," which compellingly bridges the gap between traditional Artificial Neural Networks (ANNs) and Spiking Neural Networks (SNNs). The core contribution is a mathematical framework and corresponding hardware architecture that executes standard ANN operations in a temporal, or "delay space," domain. By representing numerical values as the arrival time of a single digital pulse (a "spike"), the authors transform the fundamental multiply-accumulate (MAC) operation. Multiplication becomes simple addition of delays, and accumulation is realized through a hardware-efficient approximation of the negative log sum exponential (nLSE) function.

The authors build a complete system concept around this idea, proposing a hybrid temporal/digital systolic array that leverages classical dataflows (e.g., Weight Stationary) while performing computation using temporal primitives. Crucially, their approach can run pre-trained ANNs with minimal modification, thus inheriting the benefits of the mature ANN software and training ecosystem, a major advantage over conventional SNNs. The paper evaluates this approach across the MLPerf Tiny benchmark suite, demonstrating significant potential for energy efficiency gains, particularly when paired with emerging temporal memory technologies, where they project a 3.5x improvement over aggressive 8-bit digital designs.

Strengths

1. Fundamental Novelty and Conceptual Elegance: The paper's primary strength is its core idea. The mapping of ANN arithmetic to a logarithmic time domain (x -> -ln(x)) is both elegant and powerful. It establishes a direct, mathematically sound link between the two dominant paradigms of neural computation. This is not an incremental improvement but a new perspective on how to build neural accelerators, sitting at a fascinating intersection of temporal computing, logarithmic number systems, and traditional computer architecture.

2. Bridging the ANN-SNN Divide: A significant practical strength is the ability to leverage the vast ecosystem of ANN research. SNNs have long been hampered by immature training algorithms and a lack of standardized software. By providing a direct execution path for pre-trained ANNs, this work sidesteps that entire problem, making the energy benefits of spike-based computation immediately accessible. The discussion in Section 1 (page 1) effectively frames this motivation.

3. System-Level Approach: The authors go beyond a mere theoretical concept. They consider the full stack, from the circuit-level implementation of nLSE approximations (Section 2.2, page 4) to a programmable systolic array architecture (Section 3, page 5) and its dataflow (Section 4, page 7). The proposed integration with temporal memories (Section 3.3, page 7) is particularly forward-looking and essential to the claimed energy benefits.

4. Thorough and Realistic Evaluation: The evaluation is comprehensive. The authors analyze the impact of their approximations on accuracy (Section 6.1, page 8), including the effects of hardware noise (Figure 9, page 10) and architectural choices like PE tree size (Table 2, page 9). This detailed analysis lends credibility to their claims and provides valuable insights for future designers. The comparison against both a digital baseline and a state-of-the-art SNN accelerator (SATA) in Section 6.2 (page 10) effectively contextualizes their results.

Weaknesses

While the core idea is strong, the paper could be improved by addressing the following points, which are more about expanding the context and exploring limitations rather than fundamental flaws.

1. Limited Scope of Network Architectures: The evaluation is performed on the MLPerf Tiny suite, which consists of relatively small models. While appropriate for an initial investigation, the scalability of this approach to larger, more complex networks (e.g., Transformers, large CNNs) remains an open question. The dynamic range of activations and weights in such models could pose a significant challenge for the fixed-point temporal quantization scheme. The analysis on broadcast scaling in Section 6.3 (page 11) is a good start, but a broader discussion is needed.

2. Under-explored Connection to Logarithmic Number Systems (LNS): The paper implicitly reinvents many concepts from the well-established field of LNS hardware design. For instance, the difficulty of addition is the central challenge in LNS, just as nLSE is the central challenge here. Explicitly framing the work in the context of LNS could strengthen its theoretical foundation and allow it to draw from decades of research on LNS approximation techniques and error analysis.

3. Complexity of Temporal Synchronization: The paper mentions the need for resynchronization between cycles using "temporal recurrence" (Section 3.1, page 5). While the mechanism is described, the overheads and potential timing closure challenges of managing these precise delays across a large, potentially asynchronous array could be substantial. This feels like a critical implementation challenge that is perhaps understated.

Questions to Address In Rebuttal

1. Dynamic Range and Precision: Your temporal representation relies on a mapping from a numerical value to a physical delay. How does this system handle the wide dynamic range of values seen in larger models? Is the fixed-point quantization scheme (determined by the unit scale and number of bits for the programmable delay) sufficient for models beyond the embedded space, or would a form of temporal "block floating-point" be necessary?

2. The Cost of Conversion: The input data must be converted from the digital domain to the temporal domain (x -> -ln(x)). While this is a one-time cost for weights, it must be done for every input activation. Could you elaborate on the area and energy cost of this initial Digital-to-Time conversion and how it impacts the overall system efficiency, especially for input-bound layers?

3. Beyond ReLU and Max-Pooling: In Section 2.1 (page 3), you demonstrate elegant temporal implementations for ReLU and max-pooling. How would your approach handle other common non-linearities, such as GeLU or Swish, which are prevalent in modern architectures like Transformers? Do these functions have similarly efficient temporal implementations, or would they require costly conversions back to the digital domain?

4. Training in the Loop: You show that regularization during standard training improves robustness (Figure 8, page 9). Have you considered Quantization-Aware Training (QAT) where the specific nLSE approximation and noise models are included in the training loop? It seems this could be a powerful technique to close the remaining accuracy gap and would be a natural next step. Is there any fundamental reason this would not be feasible?

Review Form: The Innovator (Novelty Specialist)

---

Summary

This paper proposes an approach to implement Artificial Neural Networks (ANNs) using temporal computing primitives. The core idea is to transform ANN operations into a logarithmic "delay space," where scalar values are represented by the arrival time of a single digital pulse ("spike"). In this domain, multiplication is implemented as a programmable delay, and accumulation is approximated by a negative log sum exponential (nLSE) function. The authors build a complete system around this concept, proposing a novel noise-tolerant nLSE approximation circuit, a hybrid temporal/digital systolic array architecture to support these operations programmably, and an evaluation of this system when integrated with emerging temporal memories. The work aims to bridge the gap between energy-efficient Spiking Neural Networks (SNNs) and the algorithmically mature world of ANNs.

---

Strengths

The novelty of this work lies not in the base concept, which has been previously explored, but in the specific architectural and circuit-level contributions required to make such a system viable and robust.

1. Novel Approximation Circuit: The most significant and clearly novel contribution is the improved nLSE approximation method detailed in Section 2.2 (page 4). The technique of creating a complementary "inverse approximation" and then performing a "temporal average" of the two to cancel out logarithmic bias is a clever and specific circuit-level innovation. This directly addresses a fundamental accuracy challenge in this domain and appears to be genuinely new.

2. Novel Hybrid Systolic Architecture: The proposed architecture in Section 3 (page 5) is a novel synthesis of concepts. While the systolic dataflow itself is not new, its implementation in a hybrid temporal/digital domain is. The methods for handling spatial reuse (broadcasting) and temporal reuse (resynchronization of temporal signals, discussed at the end of Section 3.1, page 5) are unique to the constraints of this single-spike computing paradigm and represent a new architectural pattern.

3. Novel System-Level Integration: The paper is the first to propose and evaluate the integration of memristive temporal memories (Section 3.3, page 7) into an end-to-end ANN accelerator. While the temporal memory itself is based on prior work [37], its use in this context to eliminate domain conversion overhead (TDCs/DTCs) within a systolic array is a novel systems-level contribution. This provides a clear application-driven context for a technology that has largely been demonstrated in isolation.

---

Weaknesses

The primary weakness of the paper from a novelty perspective is the presentation of the core transformational idea as entirely new, when it is, in fact, an extension of prior art, including the authors' own.

1. Incremental Nature of the Core Idea: The fundamental concept of "delay space arithmetic" for neural network operations is not introduced in this paper. It is a direct extension of the authors' previous work in [23] ("Energy Efficient Convolutions with Temporal Arithmetic"). That paper laid the groundwork for mapping convolutions into the temporal domain using the same delay-for-multiplication and nLSE-for-addition transform. This paper generalizes the concept to full ANNs and builds a programmable architecture, but the foundational mapping is not novel to this work. This should be made much clearer to properly frame the contributions.

2. Low Novelty of Non-Linear Operator Extensions: The extension of the framework to support non-linear operations like ReLU and max-pooling (Section 2.1, page 3) is presented as a key part of the contribution. However, these appear to be direct and mathematically straightforward consequences of the -ln(x) transformation. For instance, max(a, b) in the original domain naturally becomes min(-ln(a), -ln(b)) in the log-time domain. This is more of a necessary implementation detail for completeness rather than a significant conceptual advance.

3. Complexity vs. Benefit Justification: The proposed system introduces significant complexity (hybrid-domain operation, specialized analog-like circuits for approximation, critical timing dependencies) compared to a standard digital systolic array. The results in Figure 12 (page 11) show that the design with digital memory (DS DM) offers performance that is largely on par with, and in some cases less energy-efficient than, a conventional digital implementation. The substantial benefits only manifest with the integration of temporal memories (DS TM), an emerging and not-yet-mature technology. This makes the novelty of the architecture contingent on a future technology to justify its complexity, weakening the claim of a clear and present advancement over the state-of-the-art.

---

Questions to Address In Rebuttal

1. Please clarify the precise delta between the contributions of this paper and your prior work in [23]. The core arithmetic transformation appears identical. Is the novelty limited to the improved nLSE circuit, the generalization beyond convolutions, and the programmable systolic architecture?

2. Could the authors elaborate on the novelty of implementing ReLU and max-pooling in delay space beyond the fact that they are direct mathematical consequences of the logarithmic transform? Were there non-obvious implementation challenges or trade-offs that constitute a novel contribution?

3. Given that the proposed architecture with digital memory (DS DM) shows limited to no energy-delay-product (EDP) improvement over a standard 8-bit digital array, what is the compelling novelty-driven argument for adopting this significantly more complex temporal computing paradigm without relying on the future maturation and availability of high-precision, low-variability temporal memories?

Title: Phi: Leveraging Pattern-based Hierarchical Sparsity for High-Efficiency Spiking Neural Networks Reviewer: The Guardian

---

Summary

The authors propose "Phi," a framework for accelerating Spiking Neural Networks (SNNs) by exploiting pattern-based hierarchical sparsity. The core idea is to decompose the binary spike activation matrix into two levels: Level 1 (vector-wise sparsity), which represents common activation row-vectors as pre-defined patterns whose results can be pre-computed, and Level 2 (element-wise sparsity), a highly sparse correction matrix to handle deviations from these patterns. The paper presents an algorithm-hardware co-design, including a k-means-based algorithm for pattern selection, a pattern-aware fine-tuning (PAFT) technique to increase pattern matching, and a dedicated hardware accelerator to process both sparsity levels efficiently. The authors claim significant speedup (3.45x) and energy efficiency improvements (4.93x) over the state-of-the-art SNN accelerator, Stellar [42].

Strengths

1. Principled Decomposition: The decomposition of the activation matrix into a structured, pattern-based component (Level 1) and an unstructured correction component (Level 2) is a logical approach. It attempts to handle the majority of computation via efficient table lookups (pre-computed PWPs) while managing outliers with a dedicated sparse engine.

2. Comprehensive Co-design: The work is commendably thorough, addressing the problem from the algorithmic level (pattern selection, fine-tuning) to the architectural level (preprocessor, L1/L2 processors). This end-to-end perspective is a clear strength.

3. Detailed Hardware Implementation: The authors provide a detailed description of their proposed hardware architecture, including the pattern matcher, the compressor/packer for Level 2 sparsity, and the reconfigurable adder tree. The design considerations for handling unstructured sparsity in the L2 processor are particularly well-articulated.

Weaknesses

My primary concerns with this work relate to the validity of the evaluation methodology, the justification for key design choices, and the practical implications of the proposed lossy compression scheme.

1. Fundamentally Flawed SOTA Comparison: The central claim of achieving a 3.45x speedup and 4.93x energy improvement hinges on the comparison with Stellar [42]. However, the authors explicitly state in Section 5.1, "For Stellar, we rely on the results reported in the paper." This is an unacceptable methodological flaw for a top-tier architecture conference. Comparing results from one's own simulator against numbers reported in another paper is not a valid, apples-to-apples comparison. Differences in simulator assumptions, process technology characterization (even when targeting the same node), memory models, and benchmark implementation details can lead to significant discrepancies. Without implementing Stellar within the same evaluation framework, the primary performance claims of this paper are unsubstantiated.

2. Weak Motivation and Over-reliance on Visualization: The entire premise is motivated by the t-SNE visualization in Figure 1, which purports to show that SNN activations are more "clustered" than DNN activations. t-SNE is a visualization technique notorious for creating the illusion of clusters where none may exist. It is not a rigorous method for cluster validation. The paper lacks any quantitative analysis (e.g., silhouette scores, variance analysis) to prove that these clusters are statistically significant and that the k-means approach is well-founded. The motivation rests on a subjective visual interpretation rather than rigorous data analysis.

3. Unjustified Accuracy-Performance Trade-off: The Pattern-Aware Fine-Tuning (PAFT) method introduces a non-trivial accuracy degradation. As shown in Figure 11, the accuracy drop for VGG16 on CIFAR-100 is approximately 1.5% (from ~92% for the lossless Phi to ~90.5% for Phi with PAFT). The authors dismiss this as a "minor decrease." This is a subjective judgment. A 1.5% absolute drop can be significant for many applications. This moves the work from a "lossless accelerator" to a "lossy co-design," which should be compared against other lossy techniques like quantization and pruning, not just other SNN accelerators. The paper fails to adequately position and justify this trade-off.

4. Underestimated Hardware and Storage Overheads:

   • The preprocessing logic, particularly the pattern matcher and the packer, appears complex. The pattern matcher must compare each activation row against 128 stored patterns (Section 5.2.2). While implemented as a systolic array, this still represents a substantial area and power cost that is not sufficiently analyzed. The benefit analysis in Section 6.1 feels like a post-hoc justification rather than an integral part of the evaluation.
   • The framework requires storing q=128 patterns of length k=16 for each layer and partition. For deep and wide networks, this calibration data could become substantial. The paper does not analyze the total storage cost of these patterns or the overhead of loading them for each layer.

5. Limited Scalability of the Calibration Process: The paper states that patterns are selected independently for each "model, dataset, layer, and partition" (Section 3.2). This offline calibration step seems computationally intensive and data-dependent. It raises questions about the framework's adaptability. How does Phi perform on a model trained on one dataset but deployed for inference on a slightly different, out-of-distribution dataset? The tight coupling between the calibrated patterns and the training data distribution may represent a significant practical limitation.

Questions to Address In Rebuttal

1. Please provide a compelling justification for comparing your simulated results against the reported results of Stellar [42]. Given the methodological invalidity of this approach, how can the authors stand by their SOTA claims? A fair comparison would require implementing Stellar within your simulation framework.

2. Can you provide a quantitative, statistical analysis of the clustering of SNN activation vectors that goes beyond the subjective t-SNE visualization in Figure 1? This is critical to establishing the foundation of your work.

3. The PAFT fine-tuning results in a lossy scheme. How does the resulting accuracy/performance trade-off compare to established SNN compression techniques like pruning? Why should the community accept a ~1.5% accuracy loss for a 1.26x speedup (as per Section 5.4)?

4. Provide a detailed area and power breakdown of the entire Preprocessor (Matcher, Compressor, Packer, and associated control logic). How does this overhead scale with the number of patterns (q) and the partition size (k)? Show that the preprocessing overhead does not dominate the overall energy savings for layers with low computational intensity.

5. What is the total storage footprint for all calibrated patterns across a model like ResNet-18 or VGG-16? Please clarify how this data is managed and loaded during network execution and account for its overhead.

Review Form: The Synthesizer

Summary

This paper introduces Phi, a novel framework for accelerating Spiking Neural Networks (SNNs) by exploiting a higher-order structure in their activations than has been previously considered. The authors' core observation is that the binary spike activations in SNNs are not randomly distributed but form distinct, recurring patterns or clusters (visualized effectively in Figure 1c, page 2).

Building on this insight, they propose a "pattern-based hierarchical sparsity" that decomposes the activation matrix into two levels: 1. Level 1 (Vector Sparsity): A dense matrix of indices pointing to a small, pre-defined codebook of common activation patterns. The computation for these patterns against the weight matrix is pre-calculated offline, converting most runtime computation into memory lookups. 2. Level 2 (Element Sparsity): A highly sparse correction matrix containing {+1, -1} values to account for the differences (outliers) between the actual activations and the matched patterns.

The paper presents a full algorithm-hardware co-design, including a k-means-based algorithm for discovering patterns and a dedicated hardware architecture to efficiently process both levels of sparsity on the fly. The authors report significant improvements in speed (3.45x) and energy efficiency (4.93x) over the state-of-the-art SNN accelerator, Stellar.

Strengths

1. A Fundamental and Elegant Insight: The primary strength of this work lies in its foundational contribution. The SNN accelerator community has largely focused on optimizing for bit sparsity (skipping zero-activations). This paper makes a conceptual leap by identifying and exploiting vector sparsity (skipping computation for entire recurring patterns). This shifts the optimization target from individual bits to meaningful information chunks, which is a powerful and elegant new perspective. The idea feels fundamental and has the potential to become a standard technique in the field.

2. Bridging Concepts Across Domains: This work serves as an excellent bridge between several important areas of research. The core mechanism is conceptually analogous to Vector Quantization (VQ) or dictionary learning, where a codebook of representative vectors is used to compress information. The Level 2 correction matrix is effectively a clever way to handle the quantization error in hardware. More importantly, this work has strong and immediate relevance to the acceleration of low-bit Deep Neural Networks (DNNs). As the authors rightly point out in Section 6.2 (page 12), techniques like bit-slicing decompose multi-bit DNN matrices into binary ones. The Phi framework could therefore be a highly effective mechanism for accelerating these bit-sliced DNNs, giving the work significance far beyond the SNN niche.

3. Comprehensive Co-Design: The authors present a convincing end-to-end solution. They do not merely propose a software algorithm but have clearly thought through the architectural implications. The design of the Preprocessor (Figure 4, page 7) to handle dynamic pattern matching and the separate, specialized L1 and L2 Processors to handle structured lookups and unstructured corrections, respectively, demonstrates a mature and holistic approach to the problem. This makes the proposed performance gains far more credible than a purely algorithmic study would.

Weaknesses

While the core idea is strong, the paper could be strengthened by further exploring the boundaries and context of the contribution.

1. Limited Exploration of Pattern Genesis: The paper observes the existence of patterns but does not deeply investigate why these patterns emerge. Is this phenomenon inherent to the Leaky-Integrate-and-Fire (LIF) neuron dynamics? Is it a byproduct of certain training methods (e.g., surrogate gradients)? Understanding the origin of this structure would strengthen the theoretical underpinnings and help predict how well the technique might generalize to future SNN models and neuron types.

2. Positioning Could Be Broader: The authors connect their work to bit-slicing in the discussion, but this connection is so powerful that it deserves to be highlighted earlier and more prominently. Framing the work from the outset as a general technique for "pattern-based binary matrix computation" would better capture its potential impact for the wider computer architecture community, which is increasingly focused on extreme quantization in DNNs.

3. Static Nature of the Pattern Codebook: The proposed framework relies on a static, pre-calibrated set of patterns for each layer. While this is a practical starting point, it raises questions about adaptability. For applications with significant domain shift or in continual learning scenarios, a static codebook might become suboptimal. A brief discussion on the potential for dynamic or adaptive pattern updates would add a valuable forward-looking perspective.

Questions to Address In Rebuttal

1. The core observation of clustered activation patterns in Figure 1c is compelling. Can the authors provide some intuition or evidence on how universal this property is? For instance, do these well-defined clusters persist across different SNN architectures (e.g., deep CNNs vs. Transformers like Spikformer) and across different datasets (e.g., temporal event-based data like CIFAR10-DVS vs. static images)?

2. The paper partitions the activation matrix along the K-dimension with a fixed size k=16 (Section 5.2.1, page 10). What is the architectural and algorithmic trade-off here? Would partitioning into 2D blocks or along the activation channel dimension yield a different, perhaps richer, set of patterns? A deeper justification for this 1D partitioning would be helpful.

3. Regarding the connection to quantized DNNs: If one were to apply the Phi framework to a 4-bit weight-and-activation DNN by bit-slicing the activation matrix, how would the proposed approach compare to other state-of-the-art 4-bit accelerators? Specifically, would the overhead of pattern matching and handling two sparsity levels still be advantageous compared to specialized hardware for direct 4-bit multiply-accumulate operations?

Paper Title: Phi: Leveraging Pattern-based Hierarchical Sparsity for High-Efficiency Spiking Neural Networks Review Form: The Innovator (Novelty Specialist)

---

Summary

The authors introduce "Phi," a framework for accelerating Spiking Neural Networks (SNNs) by exploiting a novel form of sparsity they term "pattern-based hierarchical sparsity." The core idea is to decompose the binary spike activation matrix into two components. Level 1 represents rows of the activation matrix (vectors) that closely match a pre-defined dictionary of patterns, enabling the use of pre-computed partial results. Level 2 is a highly sparse correction matrix, containing {+1, -1} values, that accounts for the differences (or "residuals") between the actual activations and the matched patterns. The framework includes an algorithmic component for pattern discovery using a k-means-based approach and a hardware co-design featuring a dedicated architecture to process both levels of sparsity efficiently at runtime. The authors claim significant speedup and energy efficiency improvements over state-of-the-art SNN accelerators.

Strengths

The primary strength of this paper lies in its specific and well-executed application of established compression principles to the unique domain of SNN activations.

1. Novel Application Domain: While the constituent ideas are not entirely new to computer science (as detailed below), their application to the binary, event-driven activation matrices of SNNs is novel. The observation that SNN activations exhibit strong clustering behavior (Figure 1c, page 2) is a key insight, and building a full hardware/software stack around it is a significant contribution.

2. Elegant Residual Representation: The use of a {+1, -1} basis for the Level 2 correction matrix is an elegant and efficient mechanism for representing the residual in a binary space. It naturally handles both types of mismatches (1 in the activation but 0 in the pattern, and vice versa) and is well-suited for hardware implementation.

3. Comprehensive Co-Design: The work presents a complete co-design, from the algorithmic pattern selection method to a detailed hardware architecture. This demonstrates a thorough understanding of the problem and provides a convincing case for the framework's feasibility.

Weaknesses

The paper's primary weakness, from a novelty standpoint, is its failure to adequately contextualize its core mechanism within the broader history of data compression and quantization. The authors present the concept of pattern-based, hierarchical decomposition as a fundamentally new idea, which it is not.

1. Conceptual Overlap with Vector Quantization (VQ): The core idea of Level 1 is functionally identical to Vector Quantization, a concept that dates back decades. In VQ, a "codebook" of representative vectors is created (analogous to Phi's "pre-defined patterns"), and input vectors are replaced by the index of the closest codebook entry. The k-means algorithm, which the authors use for "pattern selection" (Section 3.2, page 5), is the standard algorithm for generating VQ codebooks. The paper does not mention VQ, which is a significant omission of prior art.

2. Conceptual Overlap with Residual/Hierarchical Compression: The two-level Phi sparsity is a form of residual or multi-stage compression. The Level 1 pattern provides a coarse approximation of the activation vector, and the Level 2 matrix provides a fine-grained residual correction. This concept is the foundation of techniques like Residual Vector Quantization (RVQ) and other hierarchical decomposition methods used widely in signal processing and data compression. The paper presents this hierarchy as a novel invention rather than a novel application of a well-known principle.

3. Insufficient Discussion of Prior Art in the DNN Space: While the paper does compare itself to SNN accelerators, its discussion in Section 6.2 ("Relationship with Sparsity and Quantization in DNNs," page 12) misses the most relevant conceptual predecessors in the conventional DNN space. It compares Phi to zero-skipping and bit-slicing but fails to discuss works that use VQ or other dictionary-based methods on weights or activations in traditional DNNs. Acknowledging and differentiating from these works is critical for properly situating the paper's contribution.

The novelty here is not the invention of a pattern+residual scheme, but its specific adaptation and hardware implementation for the unique constraints and opportunities of binary SNN activations. The paper would be substantially stronger if it framed its contribution as such, rather than implying the invention of the core concept itself.

Questions to Address In Rebuttal

1. Could the authors please clarify the novelty of their pattern-based approach in relation to classical Vector Quantization (VQ) and Residual Vector Quantization (RVQ)? How does Phi's Level 1 (pattern matching) and Level 2 (correction matrix) conceptually differ from a one-stage RVQ where the codebook is derived from k-means clustering?

2. The offline calibration step creates a static dictionary of patterns based on a training subset. This seems vulnerable to distribution shift between the calibration set and unseen inference data. Have the authors analyzed the robustness of their selected patterns? How much does performance degrade if the activation patterns at inference time differ significantly from those seen during calibration?

3. The use of pre-computed Pattern-Weight Products (PWPs) trades computation for memory traffic. The paper notes this requires a PWP prefetcher to manage the "heavy memory traffic induced by PWPs" (Section 5, page 9). Could the authors provide a more detailed analysis of this trade-off? Specifically, for very large models or layers, could the storage and bandwidth requirements for PWPs become a new bottleneck that negates the computational savings?

Paper Title: Bishop: Sparsified Bundling Spiking Transformers on Heterogeneous Cores with Error-constrained Pruning Reviewer: The Guardian (Adversarial Skeptic)

---

Summary

The authors present Bishop, a comprehensive HW/SW co-design framework for accelerating spiking transformers. The proposal includes a new data container called Token-Time Bundle (TTB) to manage spatiotemporal workloads, two algorithmic optimizations—Bundle Sparsity-Aware (BSA) training and Error-Constrained TTB Pruning (ECP)—and a heterogeneous hardware architecture composed of dense, sparse, and dedicated attention cores. While the ambition to create a full-stack solution for this emerging model class is noted, the work is undermined by several critical flaws, most notably an inappropriate choice of baselines that likely inflates performance claims, unsubstantiated assertions regarding the "error-constrained" nature of their pruning method, and a lack of evidence for the system's robustness beyond a narrow set of highly-tuned configurations.

Strengths

1. Ambitious Scope: The paper commendably attempts to address the acceleration of spiking transformers from the algorithm level down to the microarchitecture, which is the correct approach for a co-design framework.
2. Structured Workload Management: The central concept of the Token-Time Bundle (TTB) provides a structured primitive for managing sparse spatiotemporal workloads. This approach logically facilitates data reuse and enables coarse-grained computation skipping, which is a sound principle for accelerator design.
3. Problem Motivation: The workload analysis presented in Section 2.2 and Figure 3 correctly identifies that MLP and projection layers, not just the attention mechanism, constitute a significant computational bottleneck in spiking transformers, providing a solid motivation for the overall architectural design.

Weaknesses

This paper suffers from significant methodological and analytical weaknesses that call its central claims into question.

1. Fundamentally Flawed Baseline Comparisons: The claimed speedup and energy efficiency improvements (5.91x and 6.11x) are built on a foundation of inappropriate and poorly-defined baselines.

   • PTB [26] is a Spiking CNN Accelerator: The primary hardware baseline, PTB, was designed for the regular, structured dataflow of convolutional layers. Spiking transformers are dominated by matrix-matrix multiplications (in MLP/projection) and the highly irregular, data-dependent communication patterns of self-attention. Comparing a specialized transformer accelerator to a CNN accelerator is an apples-to-oranges comparison. The data movement, memory access patterns, and compute granularities are fundamentally different. Any reported speedup is therefore suspect, as the baseline is not architecturally suited for the target workload. A valid comparison would require adapting a state-of-the-art ANN transformer accelerator to the accumulate-only nature of SNNs or using a more flexible, state-of-the-art SNN accelerator capable of handling FC-like layers efficiently.
   • Weak and Opaque GPU Baseline: The "Edge GPU" baseline is a NVIDIA Jetson Nano, a low-power device from 2019. This is not a competitive baseline for demonstrating state-of-the-art performance. Furthermore, the authors provide no details on the software implementation. Was it a naive PyTorch implementation, or were optimized kernels (e.g., using CUTLASS, cuSPARSE) or TensorRT employed? Without these details, the GPU comparison is unverifiable and likely represents a lower bound on achievable performance.

2. Unsubstantiated "Error-Constrained" Pruning Claim: The abstract and introduction prominently feature "Error-Constrained TTB Pruning (ECP)" with a "well-defined error bound." However, the paper completely fails to substantiate this claim.

   • Section 5.1 introduces a pruning threshold θp but provides no mathematical formulation linking this threshold to any analytical error bound on the output of the attention layer or the model's final accuracy. The term "error-constrained" implies a formal guarantee or control mechanism, which is absent.
   • The methodology appears to be simple threshold-based magnitude pruning, where the threshold is empirically swept to find a value that doesn't excessively degrade accuracy (as shown in Figure 14). This is empirically-tuned pruning, not error-constrained pruning. The claim is a misrepresentation of the method.
   • Pruning the values (V) is particularly dangerous as it directly removes information from the feature representation. The authors provide no analysis of how ECP avoids catastrophic information loss, relying only on a qualitative image (Figure 8) as evidence.

3. Lack of Robustness and High Sensitivity to Hyperparameters: The proposed system introduces numerous hyperparameters (TTB size (BSt, BSn), stratification threshold θs, pruning threshold θp), and the paper's own analysis suggests the system is brittle.

   • Figure 15 shows that the Energy-Delay Product (EDP) is highly sensitive to the stratification threshold θs. A deviation of the threshold to 20% or 80% results in a significant performance degradation. This indicates that the heterogeneous architecture requires careful, layer-wise tuning and may not be robust to workload variations.
   • Similarly, Figure 16 demonstrates a very narrow "sweet spot" for the TTB volume. This suggests the system is over-fitted to the specific model architectures and datasets tested and undermines claims of general applicability. A truly robust system would exhibit more graceful performance degradation outside the optimal parameter range.

4. Insufficient Architectural Justification and Overhead Analysis:

   • The paper does not provide an ablation study to justify its key architectural decision: heterogeneity. Would a larger, homogeneous core (either sparse or dense) with the same total area/power budget perform better or worse? The necessity of the complex three-core (dense, sparse, attention) design plus a stratifier is assumed, not proven.
   • The overheads associated with the TTB framework are ignored. Managing bundles requires metadata, indexing logic, and packing/unpacking operations. The area, power, and latency costs of this TTB management logic and the stratifier are not detailed in the breakdown in Figure 17, which is a critical omission for a hardware paper.

Questions to Address In Rebuttal

The authors must provide clear and convincing answers to the following questions:

1. Baselines: Please provide a detailed justification for using a Spiking CNN accelerator (PTB [26]) as the primary baseline for a Spiking Transformer accelerator. To make your claims credible, please provide a comparison against a more architecturally relevant baseline (e.g., an SNN-adapted ANN Transformer accelerator). For the GPU baseline, please specify the exact software stack and optimization level used and justify why the Jetson Nano is a representative platform.
2. Error-Constrained Pruning: Provide the precise mathematical definition of the "well-defined error bound" for ECP that you claim exists. How is this error bound analytically linked to the pruning threshold θp? If no such analytical link exists, please retract the "error-constrained" claim and re-frame it as an empirical technique.
3. Robustness: Your performance results appear highly sensitive to the choice of stratification threshold and TTB volume. How would the optimal parameters determined for one model (e.g., Model 3 on ImageNet-100) perform on another (e.g., Model 5 on Google SC) without re-tuning? Please provide evidence to support the generalizability of your approach.
4. Architectural Overheads: Please provide an ablation study that justifies the necessity of your heterogeneous core design over a simpler, homogeneous architecture of equivalent area. Furthermore, provide a quantitative breakdown of the overheads (area, power, and latency) incurred by the TTB management logic and the workload stratifier.

Paper Title: Bishop: Sparsified Bundling Spiking Transformers on Heterogeneous Cores with Error-constrained Pruning Reviewer: The Synthesizer (Contextual Analyst)

---

Summary

This paper presents Bishop, a comprehensive HW/SW co-design framework for accelerating spiking transformers. The authors correctly identify that this emerging class of models, while promising, cannot be efficiently executed on existing SNN or ANN accelerators due to their unique spatiotemporal workload characteristics.

The core contribution is the introduction of the Token-Time Bundle (TTB), a data abstraction that groups spikes across both tokens and time steps. This single, powerful idea serves as the foundation for the entire system. Built around the TTB, the authors propose a holistic solution: 1. A heterogeneous accelerator architecture featuring a stratifier to route TTBs to specialized dense and sparse processing cores, maximizing efficiency based on activation density. 2. A novel Bundle Sparsity-Aware (BSA) training algorithm that encourages structured, TTB-level sparsity, making the workload more amenable to acceleration. 3. An Error-Constrained TTB Pruning (ECP) technique to reduce computation in the costly self-attention mechanism by selectively trimming query, key, and value bundles. 4. A dedicated, reconfigurable attention core that leverages the binary nature of spikes to perform attention calculations using simple AND/Accumulate operations.

The authors demonstrate that this co-designed approach yields significant improvements in speedup (5.91x) and energy efficiency (6.11x) over prior SNN accelerators.

Strengths

1. A Foundational Abstraction for a New Problem Domain: The paper’s greatest strength is its identification of a new, important problem—the acceleration of spiking transformers—and its proposal of a clear, foundational solution. The Token-Time Bundle (TTB) concept (Section 3.2, page 5) is an elegant way to manage the complex spatiotemporal sparsity of these models. It transforms an unstructured, fine-grained problem into a structured, coarse-grained one, which is vastly more amenable to hardware optimization. This abstraction could very well become a standard way of reasoning about and processing these workloads in the future.

2. Exemplary HW/SW Co-design: This work is a textbook example of a holistic, full-stack approach. Rather than designing hardware for a fixed algorithm, the authors modify the algorithm itself to suit the hardware. The BSA training pipeline (Section 4.1, page 5) actively creates the structured sparsity that the heterogeneous cores are designed to exploit. Similarly, the ECP technique (Section 5, page 6) is a model-level optimization that directly maps to reduced hardware activity in their custom attention core. This synergy between algorithm and architecture is what leads to the impressive results and is a model for future research in the domain.

3. Novel and Well-Motivated Architectural Decisions: The architecture is not a monolithic design but a thoughtful composition of specialized units. The use of a "stratifier" to dispatch workloads to either a dense or sparse core (Figure 9, page 8) is a direct and intelligent response to the varying activation densities found in spiking workloads. This is a significant step beyond homogeneous SNN accelerator designs. Furthermore, the design of the TTB spiking attention core, which replaces expensive multiplications with bitwise operations, correctly identifies and tackles the primary computational bottleneck in transformers.

Weaknesses

While this is a strong and well-executed paper, its context within the broader landscape of AI acceleration could be strengthened.

1. Missing Contextual Comparison to ANN Transformer Accelerators: The paper's primary baseline is a prior SNN accelerator (PTB) [26]. While this is a necessary and fair comparison within the neuromorphic field, it leaves a critical question unanswered. Spiking transformers are ultimately competing with conventional ANN transformers on performance and efficiency. A crucial piece of context would be to compare Bishop's end-to-end efficiency (e.g., energy-per-inference for a given accuracy) against a state-of-the-art sparse ANN transformer accelerator (running an appropriately quantized and pruned ANN-ViT). Without this, it is difficult to assess whether the entire SNN-based approach, even when highly optimized, provides a true efficiency advantage over the incumbent ANN paradigm.

2. Scalability and Overhead of the TTB Abstraction: The TTB is a powerful concept, but its practical implementation involves overheads (e.g., metadata for active/inactive bundles, routing logic in the stratifier). The paper evaluates models of moderate scale. A discussion on how this management overhead scales to much larger transformer models (e.g., with thousands of tokens or hundreds of layers) would be valuable. Does a point exist where managing the bundles becomes a bottleneck itself, or where the "sweet spot" for bundle size (Figure 16, page 12) changes dramatically?

3. Clarity on the "Error-Constrained" Pruning: The abstract promises a "well-defined error bound" for the ECP technique. However, the description in Section 5 (page 6) presents it as a process of trimming bundles based on user-specified thresholds (θp, θq). It is not immediately clear how these thresholds translate to a formal, predictable error bound on the model's output. Is this an empirically derived relationship, or is there a theoretical grounding? Clarifying this would strengthen the claims of the ECP method.

Questions to Address In Rebuttal

1. Could the authors provide an estimate, even if it's a "back-of-the-envelope" calculation, of how Bishop's energy-delay product (EDP) for a task like CIFAR100 would compare to a leading sparse ANN transformer accelerator executing a similarly performing, quantized MobileViT or DeiT model? This would help position the work in the broader context of efficient AI.

2. Regarding the ECP mechanism, can you elaborate on the process of determining the pruning thresholds? How is the "error bound" established and maintained? Is it a hard constraint, or a target that is achieved via iterative training and tuning?

3. The TTB concept seems broadly applicable to spatiotemporal workloads. Have the authors considered its applicability beyond spiking transformers, perhaps to other event-based models like spiking LSTMs or models for dynamic vision sensor processing? A brief comment on the generality of the core idea would enhance the paper's impact.

Recommendation: Accept. This is a pioneering work that provides the first comprehensive acceleration framework for an important and emerging class of models. The core TTB abstraction is novel and powerful, and the holistic HW/SW co-design is executed exceptionally well. This paper will likely be highly influential in the fields of neuromorphic engineering and specialized computer architecture.

Paper Title: Bishop: Sparsified Bundling Spiking Transformers on Heterogeneous Cores with Error-constrained Pruning Reviewer Persona: The Innovator (Novelty Specialist)

---

Summary

The authors present Bishop, a hardware/software co-design framework for accelerating spiking transformers. The core proposal consists of several interconnected components: (1) a data container called Token-Time Bundle (TTB) to group spatiotemporal workloads for data reuse; (2) a heterogeneous accelerator architecture with a "stratifier" to route TTBs to either a dense or a sparse processing core; (3) a Bundle Sparsity-Aware (BSA) training algorithm to induce structured sparsity at the TTB level; (4) an Error-Constrained TTB Pruning (ECP) technique to prune low-activity queries and keys in the attention mechanism; and (5) a dedicated spiking attention core that uses simplified AND-Accumulate operations. The authors claim this is the first dedicated accelerator framework for spiking transformers and demonstrate significant speedup and energy efficiency gains over prior SNN accelerators.

Strengths

The primary strength of this work lies in the tight integration and synthesis of its components. While individual concepts may have precedents, the authors have constructed a cohesive end-to-end system where the software optimizations (BSA, ECP) are explicitly designed to create data structures (sparse TTBs) that the hardware architecture (heterogeneous cores) is specifically built to exploit. This holistic co-design approach for the niche but growing domain of spiking transformers is commendable.

Weaknesses

My evaluation is focused exclusively on the novelty of the core ideas presented. While the system as a whole is new, a deconstruction of its constituent parts reveals that many of the foundational concepts are evolutionary extensions of prior art rather than revolutionary inventions.

1. The "Token-Time Bundle" (TTB) is conceptually similar to prior work. The idea of batching spikes over time to improve data reuse is not new. Jeong et al. [26] proposed "Parallel Time Batching" (PTB) for spiking CNNs, which this paper cites. The TTB (Section 3.2, page 948) extends this concept by adding a token dimension (BSn) to the time dimension (BSt). While this is a logical and necessary adaptation for transformer architectures, it represents an incremental step—a dimensional extension of a known technique—rather than a fundamentally new data packing paradigm.

2. Heterogeneous dense/sparse architectures are a well-established design pattern. The use of separate processing units for dense and sparse computations, managed by a routing or stratification unit (Section 5.2, page 950), is a known technique for optimizing workloads with varying sparsity. This principle has been explored in general-purpose architectures (e.g., NVIDIA's Ampere) and in prior SNN accelerators that aim to skip inactive neuron computations. The novelty here is not the heterogeneous architecture itself, but its application to workloads structured as TTBs. The contribution is in the integration, not the architectural concept.

3. Bundle Sparsity-Aware (BSA) Training applies a known principle to a new structure. Sparsity-aware training, particularly structured pruning where groups of parameters or activations are zeroed out, is a vast field of research. The BSA algorithm (Section 4.1, page 948) introduces a regularization term to encourage entire TTBs to become empty. This is a clever application of structured pruning, but the underlying mechanism—adding a group sparsity regularizer to the loss function—is a standard technique. The novelty is the choice of the target structure (the TTB), which is a direct consequence of the hardware design, not a fundamental advance in training algorithms.

4. The spiking attention core's "AND-Accumulate" is an implementation detail, not an algorithmic novelty. The paper highlights a reconfigurable core that uses "AND" and "Accumulate” operations (Section 5.5, page 952). However, the simplification of matrix multiplication to accumulation is an inherent property of event-based processing in SNNs. When a spiking query vector is multiplied by a spiking key matrix, the computation naturally reduces to accumulating the key rows corresponding to the spike locations in the query vector. The "AND" operation is simply a hardware realization of identifying these co-located spikes. The contribution is the design of a dedicated hardware unit that performs this known computation efficiently on TTB-formatted data, not the invention of the simplification itself.

5. The claim of being the "first dedicated hardware accelerator" needs careful qualification. The authors state in the abstract and introduction that Bishop is the "first dedicated hardware accelerator architecture... for spiking transformers." However, they also cite Qi et al. [49], which they describe as a "simple spiking transformer architecture." While Bishop is undoubtedly more sophisticated and presents a full co-design framework, the existence of [49] challenges the unqualified claim of being "first." A more accurate claim would be the "first HW/SW co-design framework" or the "first sparsity-aware heterogeneous accelerator" for this domain. The lack of a quantitative comparison against [49] is a notable omission.

Questions to Address In Rebuttal

1. The Token-Time Bundle (TTB) is presented as a key contribution. Could the authors please elaborate on the fundamental novelty of TTB beyond being a two-dimensional extension (token and time) of the one-dimensional Parallel Time Batching (PTB) concept from prior art [26]?

2. The paper claims to be the "first dedicated hardware accelerator... for spiking transformers." Could the authors please provide a more detailed differentiation from the work of Qi et al. [49]? A qualitative discussion on why Bishop's approach is fundamentally different and a justification for the absence of a quantitative benchmark comparison would strengthen the paper's positioning.

3. Regarding the BSA training algorithm (Section 4.1, page 948), the core idea is to apply a group sparsity regularizer. How does this technique fundamentally differ from established methods for structured pruning, other than the fact that the target "group" is the author-defined TTB?

4. The complexity vs. benefit trade-off for the heterogeneous core design is unclear. Introducing a stratifier and two separate core types adds significant design complexity. Given that this architectural pattern is known, what is the "delta" in performance gain that justifies this complexity specifically for spiking transformers, compared to a more unified, flexible sparse architecture?

Paper Title: RAGO: Systematic Performance Optimization for Retrieval-Augmented Generation Serving Reviewer: The Guardian

---

Summary

The authors present RAGO, a framework for optimizing the performance of Retrieval-Augmented Generation (RAG) serving systems. The paper first introduces RAGSchema, a structured abstraction to describe the diverse landscape of RAG pipelines. Using this abstraction, the authors characterize the performance of four representative RAG paradigms, identifying shifting bottlenecks between retrieval and inference components under various configurations. They then propose the RAGO framework, which performs an exhaustive search over a design space of task placement, resource allocation, and batching policies to find Pareto-optimal system configurations. The evaluation, conducted using a custom simulation framework, claims that RAGO can achieve up to a 2x increase in QPS/chip and a 55% reduction in TTFT compared to a baseline extended from LLM-only serving systems.

Strengths

1. Problem Formulation: The paper correctly identifies a timely and critical problem. As RAG systems move into production, the complexity of their serving pipelines—composed of heterogeneous computational stages—presents a significant optimization challenge. The work does a commendable job of structuring this problem.
2. Workload Abstraction (RAGSchema): The proposed RAGSchema (Section 3.2, page 4) is a logical and useful abstraction. It provides a structured vocabulary for defining and comparing complex RAG pipelines, which is a necessary first step for any systematic analysis.
3. Performance Characterization: The analysis in Section 5 offers valuable insights into RAG system behavior. The identification of shifting bottlenecks (e.g., retrieval dominating in hyperscale scenarios for small models, Section 5.1, vs. encoding dominating in long-context scenarios, Section 5.2) is well-articulated and highlights the core challenge that the paper aims to solve. The sensitivity analyses regarding model size, query numbers, and sequence lengths are particularly informative.

Weaknesses

My primary concerns with this paper center on the methodological rigor of the evaluation and the novelty of the proposed optimization technique. The claims of optimality and significant performance gains rest on a foundation that appears non-verifiable and potentially fragile.

1. Reliance on a Non-Validated, In-House Simulator: The paper's entire quantitative analysis is predicated on an "in-house calibrated XPU simulator" (Section 4, page 6). The authors state it is "well-correlated with the production-grade XPU accelerators" but provide absolutely no evidence to substantiate this claim. There are no correlation plots, error analyses, or quantitative comparisons to real hardware measurements. Similarly, the retrieval performance model is "calibrate[d]... using internal production datasets." This lack of transparency and validation makes it impossible to assess the credibility of the results. The findings could be artifacts of the simulator's specific assumptions rather than reflections of real-world system behavior. Without rigorous validation, the results are fundamentally irreproducible and untrustworthy.

2. "Optimization" via Brute-Force Search: The core of the RAGO framework is an "exhaustive search" (Algorithm 1, page 11). While functional, this is the most basic possible approach to exploring a design space. To present brute force as a novel optimization framework is a significant overstatement. The paper lacks any discussion on the scalability of this search. What is the size of the configuration space for the evaluated workloads? How long does RAGO take to find the "optimal" schedule? A framework that requires hours or days of search to configure a system is impractical. The contribution here appears to be the enumeration of a search space, not a sophisticated method for navigating it.

3. Potentially Weak Baseline: The claimed 2x performance improvement is measured against a baseline described in Section 7.1 (page 11). This baseline collocates RAG components with the LLM's prefix stage and uses a "carefully tune[d]" 1:1 prefix:decode resource ratio. While not a naive strawman, it is questionable whether this represents a strong, state-of-the-art deployment. A skilled systems engineer would likely already consider disaggregating a component known to be a bottleneck (like the long-context encoder in Case II). The significant gains reported may be partially inflated by comparing against a configuration with obvious, well-understood inefficiencies. The paper does not sufficiently justify that this baseline represents a legitimate, production-quality manual optimization.

4. Oversimplification of System Dynamics: The performance models, based on roofline principles (Figure 4, page 6), inherently simplify complex system interactions. For instance, the analysis of iterative retrieval stalls in Section 5.3 (page 9) explicitly assumes zero latency for retrieval and prefix stages to isolate the batching effect. This is an unrealistic condition that likely magnifies the observed slowdown. The model does not appear to account for system-level effects such as network contention, OS scheduler jitter, or nuanced cache interactions, all of which can significantly impact end-to-end performance and invalidate the clean separation of stages assumed by the simulator.

Questions to Address In Rebuttal

1. Simulator Validation: Can the authors provide quantitative evidence validating their XPU and retrieval simulators against real hardware? This should include correlation plots and an analysis of prediction error (e.g., MAPE) across a set of representative model and retrieval configurations.
2. Scalability and Practicality of RAGO: What is the runtime of the RAGO exhaustive search for the case studies presented in the paper? How does the search time scale with the number of pipeline stages and the granularity of resource allocation options? At what point does this brute-force approach become intractable?
3. Baseline Justification: Please provide a stronger justification for why the chosen baseline represents a state-of-the-art, manually optimized system. Why is this specific collocation strategy and 1:1 resource split the correct point of comparison, as opposed to other plausible heuristic-based configurations?
4. Generality of Architectural Conclusions: The analysis is based on a specific family of "XPU" accelerators with parameters detailed in Table 2 (page 5). How would the key findings—particularly the bottleneck locations for each paradigm—change if run on an accelerator with a different compute-to-memory-bandwidth ratio or a different interconnect topology (e.g., an NVIDIA H100 GPU)?
5. Impact of Placement Heuristics: In Section 6.1 (page 10), you state that RAGO restricts collocation to "consecutive neighbors to avoid excessively complicating the search space." How do you know that an optimal configuration does not involve collocating non-neighboring stages? What is the potential performance loss introduced by this simplifying heuristic? This constraint seems to contradict the claim of finding a truly optimal schedule.

Review Form: The Synthesizer

Summary

This paper presents RAGO, a systematic framework for understanding, characterizing, and optimizing the serving performance of Retrieval-Augmented Generation (RAG) systems. The authors' core contribution is a three-pronged approach to tackle the immense complexity of modern RAG pipelines. First, they introduce RAGSchema, a structured abstraction to formally describe the diverse landscape of RAG workloads. Second, using this schema, they conduct a comprehensive performance characterization of four representative RAG paradigms, revealing non-obvious performance bottlenecks that shift dramatically depending on the workload. Finally, they build RAGO, a system optimization framework that uses these insights to explore a vast scheduling policy space—spanning task placement, resource allocation, and batching—to find Pareto-optimal configurations. The authors demonstrate that RAGO can achieve up to a 2x increase in queries-per-second (QPS) per chip and a 55% reduction in time-to-first-token (TTFT) latency compared to a strong baseline extended from LLM-only serving systems.

Strengths

The most significant strength of this work is its conceptual contribution of bringing a principled, systematic methodology to the chaotic and rapidly evolving domain of RAG system optimization. This paper elevates the conversation from ad-hoc tuning of individual components to a holistic, co-design problem.

1. The RAGSchema Abstraction: The introduction of RAGSchema (Section 3.2, page 4) is a standout contribution. In a field where new RAG variants emerge constantly, this abstraction provides a much-needed canonical language for describing and comparing workloads. By capturing key performance-relevant attributes (pipeline stages, model sizes, retrieval parameters), it creates a foundation for reproducible research, benchmarking, and systematic optimization that was previously lacking. It effectively tames the complexity of the problem space.

2. Insightful Workload Characterization: The performance characterization in Section 5 is excellent and provides immense value to the community on its own. By analyzing four distinct paradigms (hyperscale, long-context, iterative, and rewriter/reranker), the authors demonstrate that there is no "one size fits all" solution. The findings—such as the database encoder becoming the bottleneck in long-context RAG (Section 5.2, page 8) or the subtle idleness effects of batched iterative retrievals (Section 5.3, page 9)—are non-obvious and critical for practitioners and system designers. This analysis effectively maps the problem terrain that RAGO is designed to navigate.

3. Holistic Optimization Space: RAGO addresses the full, coupled optimization problem. It doesn't just tune batch sizes; it considers the interplay between task placement (collocation vs. disaggregation), resource allocation across heterogeneous components (CPU servers and ML accelerators), and batching policies. This holistic view is crucial, as the paper shows that decisions in one dimension profoundly impact the others. This connects disparate research threads from LLM serving (prefix/decode splitting) and distributed systems into a unified framework for RAG.

4. Contextualization and Future-Looking Implications: This work provides a clear bridge between the worlds of ML model design, information retrieval, and computer systems/architecture. The finding that retrieval becomes a dominant bottleneck as ML accelerators improve (Figure 7a, page 8) offers a concrete directive for future hardware design, making a strong case for co-designing retrieval and inference accelerators (as explored in works like Chameleon [50]). RAGO provides the analytical framework needed to reason about such future systems.

Weaknesses

The weaknesses of the paper are less about flaws in the execution and more about the boundaries of its scope and potential areas for future expansion.

1. Abstraction of Quality: RAGSchema and RAGO primarily operate on system performance metrics (latency, throughput). While the authors acknowledge that RAG parameters (e.g., number of retrieved documents, percentage of database scanned) affect model quality (recall), this critical quality-performance trade-off is outside the core optimization loop. In a real-world deployment, a user might specify a target recall, which would constrain the search space. Integrating this dimension would make the framework even more powerful.

2. Scalability of the Optimization Search: RAGO relies on an exhaustive search to find the Pareto frontier (Section 6.2, page 11). While feasible for the paradigms explored, this approach may face scalability challenges as RAG systems evolve into more complex, dynamic, and conditional agentic workflows. A discussion on how this framework might incorporate heuristic or learning-based search strategies for more complex future workloads would be valuable.

3. Static Pipeline Assumption: The framework appears to assume a relatively static RAG pipeline defined at the outset by RAGSchema. Agentic systems may involve dynamic, data-dependent execution paths (e.g., deciding to call a tool or perform another retrieval based on the content of a generated token). The current framework doesn't seem to explicitly model this dynamism, which represents the next frontier of complexity.

Questions to Address In Rebuttal

1. The decoupling of performance and quality is a pragmatic choice. Could the authors elaborate on how they envision RAGO being used in a production setting where a product owner might provide a quality constraint, such as a minimum retrieval recall? Would this simply prune the search space, or would it require a more fundamental change to the optimization objective?

2. Your baseline system is a thoughtfully tuned extension of an LLM-only system, which is a strong point of comparison. Could you comment on why the hybrid task placement strategies explored by RAGO (Figure 17b, page 12) are so effective compared to a more naive "collocate everything with the prefix" strategy that one might intuitively adopt?

3. Given the exhaustive search methodology, could you speculate on the computational cost of running RAGO itself? How long does it take to generate a Pareto frontier for one of the case studies, and how do you see this scaling as RAG pipelines incorporate more optional stages and components?

Paper Title: RAGO: Systematic Performance Optimization for Retrieval-Augmented Generation Serving Reviewer Persona: The Innovator (Novelty Specialist)

---

Summary

The authors present a framework for optimizing the serving performance of Retrieval-Augmented Generation (RAG) systems. The work makes three primary claims of contribution: (1) RAGSchema, a structured taxonomy for describing diverse RAG workloads; (2) a detailed performance characterization of four RAG paradigms using this schema, which reveals significant performance variability and shifting bottlenecks; and (3) RAGO, a system optimization framework that performs a systematic search over scheduling policies—specifically task placement, resource allocation, and batching—to find Pareto-optimal configurations for a given RAG workload.

Strengths

The primary novel contribution of this work lies in the synthesis of well-known systems optimization principles and their application to the specific, emerging domain of RAG serving. While the individual techniques employed by the RAGO framework are not new, their holistic integration to navigate the complex, heterogeneous (CPU for retrieval, XPU for inference), and multi-stage pipeline of RAG systems is a novel endeavor.

Specifically, the paper's novelty rests on:

1. A Unified Optimization Framework for a Novel System Class: Prior work has optimized LLM serving (prefix/decode splitting) or retrieval systems (ANN algorithms) in isolation. Other RAG-specific works, such as Chameleon [50] or PipeRAG [51], have proposed point solutions for specific bottlenecks (retrieval acceleration, iterative retrieval stalls). This paper is the first I am aware of to propose a generalizable and systematic framework that co-optimizes the entire end-to-end RAG pipeline, considering the interplay between all its optional and mandatory stages (encoder, rewriter, retrieval, reranker, prefix, decode).

2. Codification of the RAG Search Space: The RAGSchema abstraction, while fundamentally a taxonomy, serves as a necessary and novel contribution in the context of this work. It formalizes the configuration space of RAG pipelines, which is a prerequisite for any systematic optimization. By defining this structure, it enables the RAGO framework to operate methodically, a step beyond ad-hoc optimizations.

Weaknesses

While the application of the framework is novel, its constituent components and underlying methodology lack fundamental novelty. The work is more a feat of rigorous systems engineering and integration than one of conceptual invention.

1. Lack of Algorithmic Novelty in the Optimization Framework: The core of RAGO is a systematic, exhaustive search over a discretized space of scheduling policies, guided by an analytical cost model (Algorithm 1, page 11). This methodology is well-established and conceptually identical to design-space exploration frameworks in other domains. For instance, Timeloop [88] and MAESTRO use this exact approach (analytical modeling + systematic search) to find optimal dataflows for DNN accelerators. The authors have effectively built a "Timeloop for RAG serving scheduling," which is a valuable engineering contribution but not a new optimization paradigm.

2. The "Novel" Decisions are Applications of Prior Art: The key scheduling decisions RAGO explores are direct extensions of known techniques:

   • Task Placement (Collocation vs. Disaggregation): The central placement decision explored by RAGO is whether to group (collocate) or separate (disaggregate) pipeline stages. This directly mirrors the "phase splitting" of prefix and decode stages, a concept already thoroughly explored in prior LLM serving literature such as Splitwise [89] and DistServe [132]. RAGO merely applies this known principle to a pipeline with more stages.
   • Batching Policies: The use of techniques like continuous batching for the decode stage is considered standard practice in modern LLM serving systems like Orca [120] and vLLM [62]. RAGO incorporates this as a known best practice rather than introducing a new batching methodology.

3. Characterization as the Primary Insight: Much of the paper's intellectual weight rests on the characterization study (Section 5, pages 7-9), which demonstrates how bottlenecks shift depending on the RAGSchema. While insightful, a characterization study's novelty is contingent on it revealing profoundly counter-intuitive truths. The findings here—that retrieval can be a bottleneck at hyperscale (Case I), or that a small encoder can be a bottleneck on a huge input (Case II)—are logical consequences of Amdahl's Law applied to a new pipeline structure. They are valuable confirmations but not paradigm-shifting discoveries.

In summary, the paper's novelty is confined to its specific application domain. It does not introduce a new type of algorithm, a new systems primitive, or a fundamentally new theoretical insight into system performance. The contribution is the creation of the first such systematic optimizer for RAG, not the invention of a new kind of optimizer.

Questions to Address In Rebuttal

1. The search methodology in RAGO appears to be an exhaustive search over a pre-defined and discretized policy space. How is this approach fundamentally different from prior design-space exploration frameworks like Timeloop [88], beyond the target domain? Could the authors elaborate on any novel search or pruning strategies that were required to make this exploration tractable for the RAG domain?

2. The concept of disaggregating compute stages with different workload characteristics (compute-bound vs. memory-bound) is central to recent LLM serving systems [89, 132]. Can the authors clarify what new principles of task placement RAGO introduces beyond applying this known heuristic to a wider array of pipeline stages (e.g., rewriter, reranker)?

3. RAGSchema is presented as a key contribution. While it is a clear and useful abstraction, taxonomies themselves are not always considered novel research contributions. Could the authors argue why this abstraction is more than a descriptive framework and constitutes a novel scientific contribution in its own right, perhaps by demonstrating how it enables insights or optimizations that would be impossible otherwise?

Paper Title: Transitive Array: An Efficient GEMM Accelerator with Result Reuse Reviewer Persona: The Guardian (Adversarial Skeptic)

---

Summary

The authors introduce a novel sparsity paradigm, "transitive sparsity," which aims to reduce computations in GEMM by reusing partial results. The core idea is that if the binary representation of one row (a "TransRow" after bit-slicing) is a subset of another, the result for the latter can be computed by adding the difference to the former's result. This dependency is modeled using a Hasse graph. To manage these dependencies and enable parallel execution, the paper proposes a hardware mechanism called the "Scoreboard," which determines an execution order. The authors present the "Transitive Array," a multiplication-free accelerator architecture designed around this principle. Evaluations on LLaMA and ResNet models claim significant speedup and energy reduction over state-of-the-art accelerators like Olive and BitVert.

Strengths

1. The fundamental concept of identifying subset relationships in bit-sliced matrices to reuse partial sums is a logically sound approach to reducing redundant additions. It represents a form of fine-grained common subexpression elimination.
2. The formalization of this dependency using a Hasse diagram (Section 2.3) provides a solid theoretical foundation for the proposed sparsity pattern.
3. The architectural design is explicitly "multiplication-free," relying on shifters and adders post-bit-slicing. If the overhead is properly managed, this can lead to an efficient PE design.

Weaknesses

My primary concerns with this manuscript lie in the experimental methodology, the unquantified overhead of its core mechanism, and several unsubstantiated claims that undermine the impressive results.

1. Severely Limited and Potentially Unrepresentative Evaluation Scope: The evaluation on LLaMA models is critically flawed. The authors state in Section 5.1 that "we only extract the first Transformer block with a prefill sequence length of 2048." This is wholly insufficient for the following reasons:

   • The computational characteristics and data distributions (weights and activations) of the first block are not necessarily representative of the entire model. Different layers can have vastly different properties.
   • Crucially, the evaluation focuses exclusively on the prefill phase. LLM inference performance is often dominated by the memory-bound, latency-critical token-by-token decoding phase. By omitting an analysis of the decode phase, the paper ignores a major part of the problem and presents a one-sided performance picture that favors their compute-bound optimizations.

2. Unquantified and Potentially Prohibitive Overhead of the Dynamic Scoreboard: The paper’s claims of supporting dynamic activation patterns in attention layers hinge entirely on the "Dynamic Scoreboard." However, the actual runtime cost of this unit is never properly analyzed. The Scoreboarding algorithm (Algorithms 1 & 2) involves forward and backward passes over all potential nodes (up to 256 for 8-bit TranSparsity) to build a dependency forest.

   • The paper fails to provide any cycle-level analysis of the Scoreboard's latency. How many cycles does it take to process a sub-tile and generate the Scoreboard Information (SI) before the PEs can begin execution? This latency is directly on the critical path for every tile processed dynamically.
   • While an area figure is provided in Table 2, the energy consumption of this complex, on-the-fly graph traversal and optimization is not broken out. It is simply absorbed into the "Core" energy in Figure 11. This lack of transparency makes it impossible to assess if the savings in the PEs are nullified by the overhead of the Scoreboard.

3. Flawed and Unfair Baseline Comparisons: The performance comparisons, particularly for attention layers, are not rigorous.

   • In Section 5.7, the authors compare their 8-bit quantized Transitive Array for attention against a 16-bit BitFusion baseline. This is not an iso-precision comparison. It is expected that a lower-precision implementation would be faster. The comparison is meaningless without a proper 8-bit baseline.
   • The authors dismiss other state-of-the-art accelerators (Olive, BitVert) from the attention analysis by claiming they "do not support Attention layers" (Section 5.7). This is a strong claim that requires more justification. While these works focus on pre-processed weights, it is not proven that they are fundamentally incompatible. This exclusion conveniently removes the most relevant competitors from a key part of the evaluation.

4. Over-reliance on Favorable, Potentially Non-Generalizable Data: The entire hardware design and scheduling strategy is predicated on a key statistic mentioned in Section 4.6: "only approximately 1.67% of TransRows in our design have distances greater than 1."

   • Where does this number come from? Is it an average across all layers and models, or a single data point from the LLaMA-1-7B first layer? The performance of the pipeline, especially potential stalls in the PPE array, is extremely sensitive to this value. The paper provides no evidence that this property holds across diverse models and workloads.
   • The design space exploration in Section 5.2 is performed using a "random 0-1 matrix." The authors themselves show in Figure 13 that real data and random data have different characteristics. Basing fundamental design decisions like tiling size and bit-width on random data, rather than a thorough characterization of real-world model data, is methodologically weak.

5. Unsupported Theoretical Claims: In Section 2.4, the paper asserts that the Hasse graph can be divided into "T independent trees for T-bit transitive sparsity." This is presented as fact but lacks a formal proof or a detailed argument. This property is fundamental to their parallelization strategy, and its validity cannot be taken on faith. Is this an inherent property of the graph, or a consequence of their heuristic pruning (e.g., "assign only one prefix to every node"), which might lead to suboptimal reuse?

Questions to Address In Rebuttal

1. Please provide end-to-end inference performance for LLaMA models, including both the prefill and, critically, the token decode phases. How does Transitive Array perform on metrics like time-to-first-token and token latency?
2. Provide a detailed breakdown of the dynamic Scoreboard's latency in cycles. For a given sub-tile size (e.g., 256xT), how many cycles of overhead does the Scoreboard introduce before the processor can begin its work? How does this overhead scale with tile size and sparsity?
3. Please justify the comparison in Figure 12 by either providing results against an 8-bit attention baseline or explaining why such a baseline is impossible to construct for BitFusion. Furthermore, provide a more rigorous argument for why accelerators like Olive and BitVert cannot be adapted for attention layers.
4. Please provide data showing the distribution of dependency distances (as in Figure 9d) across all layers of the evaluated LLaMA models, as well as for other models like ResNet. How robust is the "1.67% of distances > 1" assumption?
5. Please provide a more formal argument or proof for the claim in Section 2.4 that the dependency graph can be decomposed into T independent trees. Explain whether this is a natural property or a result of the Scoreboard's heuristic choices, and what the performance trade-offs of those choices are.

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper introduces "transitive sparsity," a novel and intriguing paradigm for accelerating General Matrix Multiplication (GEMM) in the context of bit-sliced, quantized deep neural networks. The core idea is to identify and exploit subset relationships between binary row vectors in a bit-sliced weight matrix. Instead of recomputing partial sums from scratch for each row, the proposed "Transitive Array" accelerator reuses the result from a "prefix" row (a subset) and performs a small number of additional accumulations to compute the result for a "suffix" row (a superset).

To manage the complex dependencies this creates, the authors propose a principled "Scoreboard" mechanism that represents the relationships as a Hasse diagram, allowing for the efficient, on-the-fly generation of an optimal execution order. The paper presents a complete hardware architecture that is multiplication-free and designed to handle both static weights and dynamic activations, a key challenge for attention mechanisms in LLMs. The evaluations demonstrate significant speedup and energy reduction over contemporary bit-serial and quantization-aware accelerators like BitVert and Olive.

Strengths

The primary strength of this work lies in its conceptual novelty and the depth of the proposed solution.

1. A Fundamentally New Perspective on Sparsity: The community has extensively explored value-based sparsity (zero weights/activations) and, more recently, bit-level sparsity (zero-skipping in bit-serial computation). This paper moves beyond simply skipping work and introduces a powerful concept of reusing work. "Transitive sparsity" is a genuinely new way to frame computational redundancy in quantized GEMM. It connects the problem to the rich mathematical field of order theory and opens a new avenue for optimization.

2. Principled and Elegant Formalization: The decision to model the reuse dependencies using a Hasse diagram (Section 2.3, page 3) is particularly insightful. This is not a heuristic-driven approach but a formal one that allows the authors to reason systematically about execution order, parallelism, and load balancing. This principled foundation gives the work a sense of robustness and intellectual depth that is often missing in purely empirical accelerator designs.

3. An End-to-End Architectural Solution: The authors do not stop at the conceptual level. They present a comprehensive architectural design, the Transitive Array, that tackles the practical challenges of their idea. The design of the Scoreboard unit, with its forward and backward passes to build a "balanced forest" of dependencies (Figure 5, page 5), is a sophisticated piece of microarchitecture. Critically, the inclusion of a dynamic Scoreboard to handle the online nature of activation tensors in attention layers demonstrates a keen awareness of the primary bottleneck in modern LLMs. This elevates the work from a clever trick for FC layers to a potentially viable solution for entire Transformer models.

4. Significant Potential Impact: If transitive sparsity is as prevalent as the authors' analysis suggests (e.g., theoretical 87.5% sparsity for LLaMA-7B mentioned on page 2), this work could have a substantial impact on the design of future low-precision accelerators. It suggests that merely building efficient pop-count and accumulate units is not enough; co-designing hardware to understand and exploit deeper structural patterns within the bit representations could yield another major leap in efficiency. The multiplication-free nature of the design further positions it as an extremely low-power solution.

Weaknesses

The weaknesses of the paper are primarily related to the potential overhead of its complexity and questions about the generality of its core assumption.

1. Hardware Overhead and Scalability of the Scoreboard: The Scoreboard is the brain of the operation, but its complexity is non-trivial. It involves sorting (Hamming-order), graph traversal (forward/backward passes), and bitmap manipulation. While the paper asserts this is efficient, a more detailed analysis of its area, power, and latency overhead is needed. It is unclear how this overhead scales with larger bit-widths (e.g., T=16) or larger tile sizes, and whether the Scoreboard's latency could become the new bottleneck in the pipeline, especially in the dynamic, on-the-fly scenario.

2. Generality of Transitive Sparsity: The entire premise hinges on the frequent occurrence of these subset/superset bit patterns. The evaluation is strong for LLaMA models, but the phenomenon needs to be characterized more broadly. How prevalent is transitive sparsity in other model families like Vision Transformers, CNNs, or diffusion models? How is it affected by different quantization algorithms (e.g., rounding modes, affine vs. symmetric quantization) or data distributions? While the work is compelling for LLMs, its claim as a general GEMM accelerator would be strengthened by a broader empirical study of this phenomenon.

3. Complexity of Data Movement: The result reuse mechanism implies a complex data movement pattern. A processing element working on a "suffix" row needs to fetch a partial sum computed by a potentially distant "prefix" row. The paper mentions a distributed prefix buffer and a Benes network (Section 4.4, page 7), but the potential for bank conflicts, network contention, and the management of this buffer (e.g., replacement policies for "prefix misses") feel underexplored. This data-forwarding network could introduce significant overhead and complexity that might erode some of the computational gains.

Questions to Address In Rebuttal

1. Could the authors provide a more detailed analysis of the area, power, and latency overhead of the dynamic Scoreboard unit? Specifically, how does its critical path compare to the data path computation, and how does it scale as the tile size (N) and bit-width (T) increase?

2. The performance of the Transitive Array is fundamentally tied to the prevalence of transitive sparsity. While the results on LLaMA models are promising, have the authors analyzed its prevalence across a wider range of model architectures (e.g., ViTs, different CNN backbones) and quantization schemes? How sensitive is the technique to the underlying weight and activation distributions?

3. The prefix buffer is central to enabling result reuse. Could the authors elaborate on the data access patterns to this buffer and the potential for bank conflicts or network contention, especially in the distributed design? How does the system handle a 'prefix miss' if the required partial sum has been evicted or belongs to a different tile that is not currently being processed?

Review Form: The Innovator (Novelty Specialist)

---

Summary

This paper introduces a novel sparsity paradigm for GEMM acceleration, termed "transitive sparsity." The core idea is to exploit redundancies within a bit-sliced representation of a quantized matrix. Instead of merely skipping zero-valued bits, the proposed method identifies when the set of active bit positions for one row (a "TransRow") is a superset of the active bit positions for another. In such cases, the partial sum computed for the subset row can be reused as a starting point for the superset row's computation, requiring only the accumulation of the difference.

To manage the complex dependencies this creates, the authors formalize the relationships between TransRows using a Hasse diagram. They then propose a hardware accelerator, the "Transitive Array," which features a "Scoreboard" unit to dynamically determine the optimal, dependency-aware execution order. The architecture is multiplication-free, relying on XOR operations to identify difference bits and adders for accumulation. The authors claim significant speedup and energy reduction over state-of-the-art bit-slice and quantization-aware accelerators.

Strengths

The primary strength of this paper is the genuine novelty of its core concept. My analysis of the prior art confirms that "transitive sparsity" represents a new and fundamentally different way to conceive of sparsity in the context of bit-sliced computation.

1. Novel Sparsity Paradigm: The central contribution, "transitive sparsity," appears to be a genuinely novel paradigm. Existing bit-slice accelerators (e.g., Bit-Pragmatic, BitVert) primarily focus on zero-skipping, which is a form of unstructured sparsity exploitation. This paper shifts the focus to exploiting structural relationships between non-zero bit patterns. The idea of reusing computation from (0011) to compute (1011) (Figure 1, page 2) by treating the former as a sub-problem of the latter is a clear departure from prior work.

2. Elegant Formalization: The use of a Hasse graph (Section 2.3, page 3) to represent the partial ordering of TransRows is a particularly elegant and novel formalization for this specific problem. It provides a solid theoretical foundation for the dependency analysis and elevates the contribution beyond a mere ad-hoc optimization.

3. Qualitative Shift from Prior Art: This work can be conceptually framed as applying common sub-expression elimination (CSE) to the bit-level operations of a GEMM. While CSE is a well-known concept, its specific application to the bit-patterns of rows within a single GEMM to enable partial sum reuse in a hardware accelerator is, to my knowledge, new. It exploits a type of redundancy that is orthogonal to and deeper than the zero-value sparsity targeted by most contemporary accelerators.

Weaknesses

While the core idea is novel, its realization in the Transitive Array introduces considerable complexity, and the paper does not fully delineate the boundaries of its applicability.

1. High Implementation Complexity for Novelty: The proposed Scoreboard mechanism (Section 3, page 4) is a highly complex and stateful piece of hardware. It must record all present TransRows, build a dependency graph via forward and backward passes, and balance workloads across lanes. This is an order-of-magnitude more complex than the simple zero-detection logic it seeks to improve upon. The novelty comes at the cost of significant control logic overhead, which may have implications for timing, area, and power not fully captured by the high-level comparison in Table 2 (page 9).

2. Scalability of the Core Idea: The Hasse graph representation scales exponentially with the bit-width T. For the 8-bit TransRows explored (2⁸ = 256 nodes), the Scoreboard is manageable. However, the novelty of this approach seems questionable for wider data types (e.g., 16-bit), where a graph with 2¹⁶ nodes would be computationally intractable to analyze at runtime. The paper acknowledges this implicitly by selecting T=8, but a more explicit discussion of this fundamental scaling limitation is warranted.

3. Narrowing the Scope of Novelty: The concept of result reuse via delta computation is not new in a broader sense (e.g., temporal reuse in video processing, differential data transmission). The paper’s novelty lies strictly in its intra-GEMM application based on static bit patterns. A brief discussion situating the work relative to these broader concepts of computational reuse would help to more precisely define the "delta" of the contribution.

Questions to Address In Rebuttal

1. Overhead of the Scoreboard: Could the authors provide a more detailed analysis of the latency and energy overhead of the dynamic Scoreboard unit itself? The Scoreboarding process involves sorting and multiple graph traversal passes (Figure 5, page 5). How many cycles does this process take for a given tile size (e.g., 256 rows), and how does this pre-computation latency impact overall pipeline efficiency, especially for smaller GEMM operations?

2. Scalability to Wider Bit-Widths: The paper’s design space exploration (Section 5.2, page 8) concludes that 8-bit is optimal. Does this imply that the novelty of transitive sparsity is fundamentally limited to low-precision (<= 8-bit) operands? How would the authors propose to apply this concept to 16-bit or 32-bit integer GEMM, where the Hasse graph becomes intractably large?

3. Differentiation from Broader Concepts: The concept of reusing partial products bears a resemblance to common sub-expression elimination (CSE) in compilers or certain multiplier architectures that aim to reduce the number of partial products. Can the authors further differentiate their contribution from these broader concepts, clarifying why its application to bit-sliced DNN weights is a non-obvious and significant inventive step?

4. Sensitivity to Data Distribution: How sensitive is the performance of "transitive sparsity" to the underlying data distribution? The comparison in Section 5.9 (page 11) between real and random data is a good start. However, do the benefits hold for non-DNN workloads or for quantization schemes (e.g., non-uniform) that might produce less structured or less-repeating bit patterns? This would help clarify if the novelty is in a general-purpose GEMM optimization or one highly co-designed for the statistical properties of quantized neural networks.

Paper Title: Light-weight Cache Replacement for Instruction Heavy Workloads Reviewer: The Guardian (Adversarial Skeptic)

---

Summary

The authors propose PACIPV, a static, low-overhead LLC replacement policy based on an extension of Re-Reference Interval Prediction (RRIP). The core contribution is the use of separate Insertion and Promotion Vectors (IPVs) for demand and prefetch accesses, which are determined via an exhaustive search over a heuristically constrained design space. The authors claim that for instruction-heavy workloads, PACIPV achieves performance competitive with or slightly better than a complex, state-of-the-art policy (Mockingjay) and superior to the baseline SRRIP, while incurring significantly less hardware overhead. A central, and counter-intuitive, finding is that this performance gain is achieved despite an increase in the Last-Level Cache (LLC) Misses Per Kilo-Instruction (MPKI), which the authors attribute to a reduction in the average L1I demand miss latency.

Strengths

1. Low Hardware Overhead: The paper convincingly argues that PACIPV's hardware cost is minimal, representing a negligible addition to a standard SRRIP implementation. This is a clear and quantifiable advantage over complex, table-based predictors like Mockingjay or SHiP, making the proposal pragmatically appealing.

2. Systematic Design Space Exploration: The methodology of constraining the vast IPV design space with heuristics and then performing an exhaustive search is more rigorous than relying on stochastic methods like genetic algorithms. This approach lends confidence that the selected IPVs are indeed optimal within the defined constraints.

3. Logical Separation of Access Types: The fundamental idea of treating demand and prefetch accesses with distinct policies is sound. Prefetches are speculative and may have different reuse characteristics than demand requests; a policy that acknowledges this distinction is well-motivated.

Weaknesses

My primary concerns with this work center on the central performance claim, the generality of the findings, and the rigor of the evaluation methodology.

1. The Central Performance Claim is Unsubstantiated: The paper's most critical argument—that higher LLC MPKI leads to better performance—hinges on a tenuous causal link to reduced L1I average demand miss latency (Section 5.1, page 8). This explanation is insufficient and lacks rigorous proof.

   • The authors hypothesize that the retained blocks are more critical, or that the additional misses are to non-critical blocks that do not gate execution due to memory-level parallelism (MLP). This is pure speculation. The paper provides no data on MLP, no analysis of the criticality of the misses incurred by PACIPV versus other policies, and no distribution of miss latencies. "Average" latency can be a misleading metric; a policy could reduce latency for many non-critical misses while increasing it for a few highly critical ones, resulting in a net performance loss not captured by the average.
   • Without a quantitative analysis connecting the specific eviction decisions made by PACIPV to a tangible reduction in critical-path stalls, the core performance claim is an observation that conflates correlation with causation.

2. Questionable Generality and Overfitting: The chosen IPVs appear to be highly tuned to the specific characteristics of the CVP training workloads, undermining the claim of a broadly applicable static policy.

   • The results on the GAPS workloads are telling (Section 5.2.3, page 10). When using the IPVs trained on CVP (PACIPV-I), the policy significantly underperforms Mockingjay. It only becomes competitive when retrained specifically on GAPS (PACIPV-D). This strongly suggests that the policy does not generalize well across workload domains (instruction-heavy vs. data-heavy) and is essentially overfitting to the training set's microarchitectural behavior.
   • This contradicts the premise of finding a robust, static policy. Instead, it suggests that the methodology of searching for IPVs is effective, but the resulting static policy is brittle. The authors briefly mention set-dueling (page 6) but do not pursue it, which seems like a missed opportunity to address this clear limitation.

3. Opaque Workloads Weaken the Evaluation: The evaluation relies heavily on the CVP benchmark suite, which the authors admit are opaque commercial traces whose "actual identities... are not revealed publicly" (Section 4.2, page 7).

   • This presents a major methodological flaw. We cannot independently verify their classification as "instruction heavy." The paper provides no characterization of these workloads (e.g., instruction footprint, L1I MPKI, branch misprediction rates, IPC).
   • Without this crucial context, it is impossible to understand why PACIPV performs well or to assess whether the results are applicable to any broader class of applications. The findings are tied to a black-box workload set, limiting their credibility and impact.

4. Inconsistent and Overstated Claims Against SOTA: The paper's claims regarding its performance against Mockingjay are inconsistent.

   • The abstract claims PACIPV "improves performance over a state-of-the-art LLC replacement policy (Mockingjay)".
   • However, Figure 1 (page 1) shows a 0.1% lead, which is well within the noise margin of microarchitectural simulation. Figure 4 (page 8) is described as PACIPV "roughly matches the performance of Mockingjay".
   • This is an overstatement of the contribution. A 0.1% difference does not constitute an "improvement"; it constitutes a statistical tie. The authors should present their results with more precision and less embellishment. The true benefit is achieving this parity with far lower cost, not by "outperforming" the SOTA.

Questions to Address In Rebuttal

The authors must address the following points to strengthen the paper for publication:

1. Prove the IPC vs. MPKI Link: Can you provide a detailed analysis to substantiate your claim that higher LLC MPKI leads to higher IPC? This would require, at a minimum:

   • An analysis of memory-level parallelism (MLP) for the different policies.
   • A distribution of miss latencies, not just the average.
   • Data on whether the misses introduced by PACIPV are on the critical path of execution.

2. Justify Workload Selection: Please provide a detailed microarchitectural characterization of the CVP workloads used for training and testing. Justify their label as "instruction heavy" with metrics such as L1I miss rates, instruction cache footprint, and front-end vs. back-end stall breakdowns. Why should the community trust results derived from these opaque traces?

3. Reconcile the Generality Claim: How do you reconcile the claim of a robust, static policy with the GAPS results (Figure 10), which clearly show a significant performance degradation when the policy is not retrained for the target workload domain? If the IPVs are so workload-sensitive, doesn't this argue against a static implementation and in favor of a dynamic approach like set-dueling, which you mention but do not evaluate?

4. Clarify the SOTA Comparison: Please revise your claims of "improving performance" over Mockingjay. Acknowledge that the performance is, at best, statistically equivalent and reframe the contribution around achieving this parity with substantially lower hardware complexity.

Paper Title: Light-weight Cache Replacement for Instruction Heavy Workloads Reviewer: The Synthesizer (Contextual Analyst)

---

Summary

This paper introduces PACIPV, a lightweight Last-Level Cache (LLC) replacement policy specifically tailored for the increasingly prevalent instruction-heavy workloads found in modern systems. The core contribution is not the invention of a new mechanism from scratch, but rather an elegant and effective synthesis of two existing ideas: Re-Reference Interval Prediction (RRIP) and Insertion/Promotion Vectors (IPVs). By applying the IPV concept to the coarse-grained 2-bit state of RRIP (instead of the fine-grained state of LRU), the authors make the policy design space small enough to be searched exhaustively.

The key innovations are twofold: first, the separation of IPVs for demand and prefetch accesses, making the policy inherently prefetch-aware in a simple manner; and second, the demonstration that this low-complexity approach can achieve performance competitive with, or even slightly better than, a highly complex state-of-the-art policy like Mockingjay for the target workload class. The authors provide a compelling analysis, showing that while PACIPV may increase LLC Misses Per Kilo-Instruction (MPKI), it improves overall performance by reducing the average demand miss latency at the L1 instruction cache, indicating it is more effective at retaining performance-critical blocks.

Strengths

1. Excellent Problem Motivation and Timeliness: The work is grounded in a well-established trend documented in recent systems research (e.g., [2, 3, 11, 25]): the growing importance of the instruction-delivery front-end as a performance bottleneck in warehouse-scale computers. By focusing specifically on instruction-heavy workloads, the paper addresses a problem of significant and immediate relevance to both industry and academia.

2. Elegance and Simplicity of the Proposed Solution: The paper's greatest strength lies in its "less is more" philosophy. Instead of adding more tables, complex predictors, or machine learning structures, it re-purposes and combines existing lightweight mechanisms. The decision to base the policy on RRIP's 4-state recency representation is a crucial insight, as it transforms the intractable design space of LRU-based IPVs into a manageable one that can be optimized with a principled, exhaustive search. This stands in stark contrast to the high hardware overhead and verification complexity of policies like Mockingjay, as clearly highlighted in Table 2 (page 10). This simplicity makes the proposal highly practical and adoptable.

3. Insightful Performance Analysis: The authors present a sophisticated analysis that goes beyond top-line speedup numbers. The observation in Figures 5 and 6 (page 8) that PACIPV improves IPC despite a higher LLC MPKI is counter-intuitive and interesting. Their hypothesis—that the policy is better at reducing L1I demand miss latency by prioritizing the right blocks in the LLC—is well-reasoned and provides a deeper understanding of the mechanism's behavior. Furthermore, the discussion in Section 6.2.2 (page 12) explaining why PC-based predictors (used by SHiP and Mockingjay) are fundamentally ill-suited for instruction streams is a critical insight that beautifully contextualizes why a simpler, non-PC-based approach can win here.

Weaknesses

1. Static Policy in a Dynamic World: The primary limitation is the static nature of the IPVs, which are determined offline during a design-time training phase. While the authors demonstrate that the resulting vectors generalize well across a suite of similar workloads, a single static policy is unlikely to be optimal for the vast diversity of applications and phases running in a real-world datacenter. The "PACIPV-best" results in Figure 8 (page 10) clearly show the performance left on the table by not adapting per-workload.

2. Specialization vs. General-Purpose Performance: The work is explicitly and successfully targeted at instruction-heavy workloads. However, the results on the data-heavy GAPs suite (Figure 10, page 11) show that the instruction-trained vectors significantly underperform a policy trained on GAPs itself, and lag Mockingjay by a non-trivial margin. While retraining closes this gap almost entirely, it underscores that the policy, as presented, is a specialist. A general-purpose processor needs a policy that is robust across all workload types. The current proposal would require an adaptation mechanism to be universally effective.

3. Implicit Dependence on Prefetcher Characteristics: The policy's "prefetch-aware" nature is achieved by having a separate IPV for prefetch-initiated accesses. This is a clean design, but the optimal prefetch IPV is surely dependent on the behavior (e.g., accuracy, timeliness, aggressiveness) of the instruction prefetcher (EIP). The paper does not explore this sensitivity. For instance, would a less accurate prefetcher benefit from a more conservative prefetch IPV that inserts blocks at a higher (less-likely-to-be-kept) RRPV?

Questions to Address In Rebuttal

1. Regarding the static nature of the policy: Given the significant gap between the static PACIPV and the per-workload PACIPV-best (Figure 8, page 10), have the authors considered a lightweight dynamic adaptation mechanism? For example, instead of full set-dueling, could a simple runtime mechanism (perhaps hinted by the OS or a simple hardware counter) switch between a few pre-selected IPV pairs (e.g., one for instruction-heavy phases, one for data-heavy phases)? This could potentially bridge the gap to PACIPV-best without incurring the complexity of policies like Mockingjay.

2. The results on the GAPs suite (Figure 10, page 11) are very interesting. They demonstrate the value of training for the right workload class. The authors mention set-dueling as a potential solution. Could you elaborate on how you envision this working? Would dueling between just two candidate pairs—the best "instruction-heavy" pair and the best "data-heavy" pair found in your study—be sufficient to create a robust, general-purpose policy that performs well everywhere?

3. How sensitive are the optimal demand and prefetch IPVs to the specific instruction and data prefetchers used (EIP and Berti)? If a different, perhaps more aggressive, instruction prefetcher were used, how do you hypothesize the optimal prefetch IPV would change? A brief discussion on this co-design aspect would strengthen the paper's conclusions about prefetch-awareness.

Paper Title: Light-weight Cache Replacement for Instruction Heavy Workloads Reviewer Persona: The Innovator (Novelty Specialist)

---

Summary

This paper proposes PACIPV, a static LLC replacement policy that extends the concept of Insertion/Promotion Vectors (IPVs) to a Re-Reference Interval Prediction (RRIP) baseline. The central novel claim, as I interpret it, is the use of two distinct, static IPVs—one for handling demand accesses and another for prefetch accesses. The authors leverage the coarse-grained nature of RRIP's 2-bit recency states to create a design space small enough for exhaustive search, contrasting with prior IPV work that required genetic algorithms for larger LRU state spaces. The paper's thesis is that this extremely simple, low-cost mechanism can achieve performance competitive with, or even slightly better than, complex state-of-the-art policies like Mockingjay, particularly for instruction-heavy workloads.

---

Strengths

From a novelty perspective, the paper has several strengths:

1. The Dual-Vector Mechanism: The core idea of maintaining two completely separate Insertion and Promotion Vectors for demand and prefetch streams appears to be novel. While prior work has certainly made replacement policies "prefetch-aware" (e.g., KPC, SHiP++, Mockingjay), they typically do so by using prefetch confidence to alter a single policy's decision or by modifying a specific promotion rule. The PACIPV approach of defining two distinct, complete policy vectors is a clean, simple, and previously unexplored mechanism for specializing replacement behavior.

2. Application of IPVs to RRIP: The original IPV concept, as presented by Jiménez [22] and Teran et al. [55], was applied to tree-based pseudo-LRU. Applying this vector-based policy definition to RRIP is a logical but non-obvious extension that has not been explored in prior literature. This adaptation is what enables the methodological novelty discussed next.

3. Methodological Novelty in Policy Search: The authors make a valuable contribution by demonstrating that the design space for RRIP-based IPVs is small enough to be searched exhaustively. This is a significant departure from the original IPV work [22] which relied on genetic algorithms to navigate the astronomical search space of 16-way LRU. By showing that an optimal static policy can be found deterministically for a given training set, the authors provide a more robust design methodology than prior heuristic approaches.

4. Novelty in Simplicity: The most compelling aspect of this work is the novelty of its "less is more" result. It is a novel finding that a mechanism with virtually no hardware overhead beyond baseline SRRIP (a few dozen static bits for the vectors) can match the performance of Mockingjay, a policy requiring kilobytes of state, sampled caches, and complex prediction logic. The contribution here is not just the mechanism itself, but the insight that extreme complexity may be unnecessary for this problem domain.

---

Weaknesses

My concerns are focused exclusively on the scope and positioning of the novel contribution:

1. Incremental Nature of the Contribution: The paper's novelty rests on the specific combination of three existing concepts: RRIP, IPVs, and differentiating policy for prefetches. While the specific combination is new, the conceptual leap is modest. The paper would be stronger if it more rigorously differentiated its dual-vector approach from the mechanisms in SHiP++ [58] (mentioned in Section 2.3) and KPC [33]. These policies also treat prefetches differently; the authors should more clearly articulate why their dual-vector mechanism represents a fundamental advancement over these other forms of differentiation, rather than just an alternative implementation.

2. Overstatement of Performance Novelty: The performance improvement over the most complex state-of-the-art, Mockingjay, is marginal at best (3.3% vs 3.2% speedup over LRU, as shown in Figure 1, page 1). The true novelty lies in achieving this performance parity with drastically lower complexity. However, the paper's narrative sometimes frames this as a performance win. The contribution's framing should be precise: the novelty is a new point on the Pareto curve of complexity vs. performance, not a new absolute performance record.

3. Static Nature Limits General Novelty: The proposed policy is fundamentally static; the vectors are hard-coded at design time based on a training set. While this is the source of its simplicity, it raises questions about the generality of the contribution. The methodology for finding the vectors is general, but the specific vectors presented are not. The work does not sufficiently explore how brittle these "optimal" vectors are to changes in the underlying microarchitecture—for instance, a different front-end prefetcher or a different core design. The novelty claim would be stronger if it included an analysis of this sensitivity.

---

Questions to Address In Rebuttal

To strengthen the paper's claims of novelty, I would like the authors to address the following:

1. The core novel mechanism is the use of distinct IPVs for demand and prefetch accesses. Can you please elaborate on the fundamental differences between this dual-vector approach and the way prior prefetch-aware policies, such as SHiP++ and KPC, differentiate their handling of prefetched blocks? Is the dual-vector approach strictly more expressive or does it simply offer a different trade-off?

2. Your methodology for finding optimal static IPVs is a key part of your contribution. However, these optimal vectors are trained with a specific configuration (e.g., Berti and EIP prefetchers). How sensitive are the derived vectors to these components? If a different, equally effective instruction prefetcher were used instead of EIP, how much performance would be lost by retaining the current vectors, and would a full retraining be necessary to restore performance parity?

3. Given that the performance gain over a complex policy like Mockingjay is ~0.1%, would you agree that the primary novel contribution is the demonstration that a vastly simpler policy can achieve performance parity, rather than a contribution to absolute performance? Clarifying this in the paper's narrative would help to properly position its novelty.

Reviewer Persona: The Guardian (Adversarial Skeptic)

---

Summary

This paper proposes "LazyGPU," a GPU architecture designed to improve performance by reducing memory traffic. The core ideas are threefold: 1) a LazyCore that defers memory load requests until the data is actually needed, aiming to reduce memory system congestion; 2) a Zero Cache that stores masks to identify and eliminate memory transactions for data that is entirely zero (LazyCore+①); and 3) an instruction-level optimization that suspends loads for operands of ⊗ instructions (e.g., multiply) when the other operand is zero (LazyGPU). The authors evaluate their proposal using MGPUSim on a set of benchmarks, including ResNet-18 and LLaMA 7B, claiming significant speedups, particularly for sparse workloads.

While the proposed mechanisms are individually interesting, the work suffers from a reliance on an outdated architectural baseline, a critical logical inconsistency in its core mechanism, and an insufficient evaluation that fails to substantiate its primary claims against a rigorously defined state-of-the-art. The hardware overhead analysis is also overly simplistic and likely underestimates the true cost.

Strengths

1. The high-level concept of combining lazy execution with sparsity optimizations to tackle the GPU memory wall is a valid research direction.
2. The ⊗ instruction optimization (referred to as optimization ② in Section 4.3) is a novel, fine-grained technique for eliminating dead memory accesses by linking instruction semantics directly to memory system behavior.
3. The evaluation includes modern and highly relevant machine learning workloads (ResNet-18, LLaMA 7B), which is commendable.

Weaknesses

1. Outdated Architectural Baseline: The entire evaluation is built upon MGPUSim simulating an AMD GCN3 (R9 Nano) architecture, which is nearly a decade old. Modern high-performance GPUs (e.g., NVIDIA Hopper, AMD CDNA) feature vastly different memory subsystems, including technologies like HBM3, large last-level caches (e.g., Infinity Cache), and specialized hardware for asynchronous data movement (e.g., Tensor Memory Accelerator). The memory congestion problem, which is the central motivation for LazyCore, manifests very differently on these systems. The paper's core premise—that simply deferring loads is a net win—is not proven to be relevant for current hardware, rendering the performance results highly questionable in terms of generalizability.

2. Contradictory Claims Regarding Sub-Block Sparsity: The paper's logic for handling sparse data is critically flawed. In Section 3 (page 5, "Challenge 1"), the authors correctly state a major problem: "...it is not feasible to eliminate such memory transactions where the required portion of the data is zero as memory systems lack this information..." This acknowledges that the memory system operates at a fixed transaction granularity (e.g., 32B) and cannot natively handle requests for partial, all-zero data within that block. However, Section 4.2 and Figure 14 then claim that LazyCore+① is the solution, eliminating far more requests than can be accounted for by full 32B-block sparsity (which Figure 4 shows is very low, e.g., 2.7% for ResNet-18 inference). The paper never explains how the LazyCore overcomes this fundamental "not feasible" barrier. How does the core communicate the exact byte-level requirements of a strided load to the Zero Cache and memory system to enable this optimization? Without a precise hardware mechanism, the claimed benefits from optimization ① are unsubstantiated.

3. Insufficient and Poorly Defined Competitive Baseline: The authors position their work as an alternative to "eager execution." However, their comparison is superficial. At the end of the first paragraph of Section 5.2 (page 10), they provide two speedup numbers (1.26x and 1.02x) for an "eager execution with zero caches" baseline. There is absolutely no detail provided on this baseline. Was it simply the baseline MGPUSim with a zero cache added? Did it include a modern, aggressive hardware prefetcher, which is the hallmark of eager execution systems? A rigorous study would implement and evaluate against a strong, well-defined eager execution baseline. As it stands, the paper compares its complex lazy design against a strawman.

4. Oversimplified Hardware Overhead Analysis: The analysis in Section 5.5 is incomplete to the point of being misleading. It calculates storage costs for "Busy Bits" and "Address Upper Bits" and arrives at a negligible 0.009% area overhead. This analysis completely ignores the significant costs of the associated control logic:

   • The Lazy Unit itself, which must track dependencies for all pending loads.
   • The additional tag arrays, comparators, and control logic for the Zero Caches (stating they are "repurposed" from normal caches is not a zero-cost operation; it reduces the effective size of the data/instruction cache and requires new logic).
   • The modifications to the instruction decoder and issue stage to identify ⊗ instructions and suspend/reactivate their associated loads. The actual area, power, and latency overhead of these complex logic structures is unstated and is almost certainly much higher than reported.

5. Unexamined Performance Penalty in Low-Contention Scenarios: The lazy execution model inherently adds latency to every memory access that is not eliminated. The authors claim that Thread-Level Parallelism (TLP) hides this, but their own data in Figure 3a shows that LazyCore only provides a benefit when the number of wavefronts is very high (>2048). For workloads with fewer active wavefronts, performance is either the same or worse than the baseline. This demonstrates a critical trade-off that the paper fails to adequately address: the proposed architecture may actively harm performance on any workload that does not fully saturate the memory system.

Questions to Address In Rebuttal

1. Please provide a compelling justification for using the GCN3 architecture. How do you expect the proposed mechanisms to interact with modern memory features like NVIDIA's TMA or AMD's Infinity Cache? What evidence can you provide that the observed memory congestion patterns and the resulting benefits of your design are not artifacts of this outdated baseline?

2. Please provide a precise, register-transfer-level (RTL-like) description of the mechanism that reconciles the "not feasible" challenge from Section 3 with the capabilities claimed for LazyCore+①. How does the core inform the memory hierarchy of the specific bytes a wavefront requires from a 32-byte memory transaction, such that a request can be elided if only that specific subset is zero?

3. Please provide a detailed specification of your "eager execution with zero caches" baseline used for comparison in Section 5.2. Detail the prefetching policy, MSHR configuration, and any other relevant parameters. A more thorough, head-to-head comparison is required.

4. Please provide a more comprehensive hardware overhead analysis that includes area and power estimates for the control logic of the Lazy Unit, the Zero Cache, and the modified instruction front-end, not just the storage bits.

5. Please analyze and discuss the performance of LazyGPU in scenarios with low memory contention or for latency-sensitive kernels. At what point does the inherent latency penalty of the lazy approach begin to dominate the benefits of congestion reduction?

Paper Title: The Sparsity-Aware LazyGPU Architecture Reviewer: The Synthesizer (Contextual Analyst)

---

Summary

This paper presents LazyGPU, a novel GPU architecture designed to mitigate memory bandwidth contention by fundamentally reconsidering when memory requests should be issued. Instead of the conventional eager approach (e.g., prefetching), which issues requests as early as possible, LazyGPU deliberately delays them. This "lazy execution" model creates a critical look-ahead window, which the authors leverage to enable two powerful, sparsity-aware optimizations. First, by integrating a Zero Cache, the architecture can check if the data required by a wavefront is entirely zero and, if so, completely eliminate the memory request. Second, by analyzing upcoming instructions, it can identify and eliminate "dead" memory requests whose fetched values would have no impact on the program's outcome (e.g., a value that will be multiplied by zero). The authors evaluate their proposal on a range of workloads, demonstrating significant speedups, particularly on sparse neural network models like ResNet-18 and LLaMA 7B.

Strengths

The core strength of this paper lies in its elegant synthesis of three distinct architectural concepts—lazy execution, zero-value caching, and dead instruction elimination—into a cohesive and impactful solution for a critical problem.

1. Novel Synergistic Mechanism: The central insight is not just applying lazy execution to GPUs, but recognizing that laziness is an enabling mechanism. The delay inherent in the lazy model provides the necessary time and information to make intelligent decisions about memory traffic. While prior work has explored Zero Caches (as cited in Section 2, page 3), those proposals often still issue memory requests concurrently with the zero-mask check. LazyGPU’s approach of checking before issuing the request to the memory system is a significant conceptual advance that directly attacks bandwidth consumption, not just latency.

2. Addressing a Timely and Critical Problem: The paper is exceptionally well-positioned at the confluence of two major trends in high-performance computing: the "memory wall" and the increasing prevalence of sparsity. As models like LLMs grow, unstructured sparsity from techniques like pruning is becoming a key tool for managing computational cost. Current hardware, like NVIDIA's sparse tensor cores, often requires structured sparsity (e.g., 2:4 patterns). LazyGPU's mechanism is inherently suited to handle unstructured, fine-grained sparsity, making it highly relevant to the future of efficient machine learning acceleration.

3. Compelling Performance Results: The empirical results strongly support the architectural claims. The 2.18x speedup on LLaMA 7B inference at 60% sparsity (mentioned in the Abstract, page 1) is particularly compelling and immediately grounds the work in a high-impact domain. The methodical breakdown of performance gains from the baseline to LazyCore, LazyCore+①, and the full LazyGPU (Figure 9, page 10) provides a clear and convincing narrative of where the benefits originate.

Weaknesses

While the core idea is strong, the paper could benefit from a broader discussion of its place within the larger architectural landscape and the potential second-order effects of its design.

1. Tension with Existing Architectural Philosophies: The proposal fundamentally pushes back against the decades-long trend of "eager" and speculative execution. A key missing piece of the discussion is how LazyGPU would interact with other standard components of a modern GPU memory system, particularly hardware prefetchers. A lazy execution core and an aggressive, eager prefetcher are philosophically opposed. Does LazyGPU obviate the need for prefetching, or would the two mechanisms need a complex protocol to coexist without working at cross-purposes? A deeper exploration of this tension would better situate the work.

2. Implicit Assumptions about Workload Parallelism: The paper argues that GPUs' massive thread-level parallelism (TLP) is well-suited to hide the additional latency introduced by the lazy model. This is a plausible and intuitive argument. However, it remains an implicit assumption. The analysis would be stronger if it explored the limits of this assumption. For instance, how does performance scale as TLP decreases? Kernels with high register pressure or significant thread divergence might not have enough active wavefronts to hide the latency, potentially turning the lazy approach into a net negative.

3. Scope of Instruction-Based Elimination: The optimization to eliminate loads based on subsequent instructions (Section 4.3, page 8) is a powerful idea. The paper focuses primarily on multiply and multiply-add instructions, which are certainly dominant in the evaluated ML workloads. However, this concept could be generalized. For example, a load whose value is destined only for a logical AND with a register known to be zero could also be eliminated. A broader discussion on the potential classes of instructions amenable to this optimization would strengthen the generality of the contribution.

Questions to Address In Rebuttal

1. Could the authors elaborate on the level of thread-level parallelism (TLP) or wavefront occupancy required to effectively hide the latency introduced by the lazy execution model? At what point (e.g., in low-occupancy kernels) does the overhead of laziness start to outweigh its memory-saving benefits?

2. The paper contrasts lazy execution with eager approaches like prefetching. How do the authors envision LazyGPU interacting with a conventional hardware prefetcher? Would the prefetcher need to be disabled, or could the two mechanisms be made to work synergistically (e.g., by having the lazy unit inform the prefetcher)?

3. The instruction-aware optimization for eliminating dead memory requests is very compelling. Have the authors considered the potential for applying this optimization to a wider range of instructions beyond multiply-based ones (e.g., logical operations, shifts)? What is the estimated potential of such generalizations?

Review Form: The Innovator

---

Summary

The authors propose LazyGPU, a GPU microarchitecture designed to mitigate memory contention by fundamentally changing when memory requests are issued. The core idea is to employ "lazy execution" for memory instructions, deferring the issuance of a load request from the decode/issue stage until the point where a subsequent instruction actually requires the data. The paper presents this core idea in three stages:

1. LazyCore: A baseline implementation of lazy execution on a GPU, which reorders memory requests by prioritizing those that are blocking computation.
2. LazyCore+①: The integration of lazy execution with a Zero Cache. The delay inherent in lazy execution provides a natural window to query the Zero Cache first, allowing memory requests to be eliminated non-speculatively if the required data is all-zero.
3. LazyGPU (LazyCore+①②): An additional optimization that leverages the lazy execution window to inspect the consuming instruction. If the consumer is an instruction like multiplication and its other source operand is zero, the memory request for the value to be multiplied is eliminated as "dead."

The authors evaluate this architecture on a range of benchmarks, with a focus on sparse neural networks like ResNet-18 and LLaMA 7B, demonstrating significant speedups by reducing memory system pressure.

---

Strengths

The primary strength of this work lies in the synergistic combination of pre-existing concepts. The authors correctly identify that the principal weakness of eager execution—issuing memory requests that may later prove unnecessary—can be addressed by the principal strength of lazy execution—delay. The most novel insight is using the delay window created by lazy execution to enable more effective, non-speculative application of other known optimization techniques (namely, Zero Caches and dead value elimination). This creates a powerful feedback loop: lazy execution enables better filtering of memory requests, which in turn reduces the memory contention that lazy execution was designed to mitigate in the first place.

---

Weaknesses

My analysis focuses exclusively on the novelty of the proposed ideas, measured against the body of prior art. While the combination of techniques is interesting, the novelty of the constituent parts is limited.

1. The Concept of Lazy Execution is Not New: The authors themselves cite "LaZy superscalar" [8] (Aşılıoğlu et al., ISCA 2015), which introduced this concept for CPU architectures. The claim of novelty in the present work, therefore, rests on the argument that its application to and implementation for GPUs is a novel contribution. The paper claims this is an "underexplored design scheme" for GPUs (Section 2, page 2), but does not sufficiently articulate the unique architectural challenges of the SIMT model that required a fundamentally new solution beyond what was proposed for CPUs.

2. Zero Caches are Not New: The concept of a cache that stores metadata about zero-value blocks is well-established. The authors cite the foundational works by Dusser et al. [26] (ICS 2009) and Islam and Stenstrom [36] (PACT 2009). In those works, the zero-check often runs in parallel to a main memory request, which is then cancelled. The "delta" here is that LazyGPU's delay makes this check-then-issue flow non-speculative. This is a clever integration, but it is an incremental refinement of how to use a Zero Cache, not a new concept in itself.

3. Instruction-Aware Elimination of Memory Requests is Conceptually Similar to Prior Work on Sparsity: The core idea of optimization ②—eliminating a load because it will be used in a multiply-by-zero operation—is a form of dynamic dead value identification. This is conceptually related to prior work on sparsity-aware processing. For example, "SAVE: Sparsity-aware vector engine" [29] (Gong et al., MICRO 2020) proposed a mechanism for CPUs to skip computation and memory accesses for operations involving zero-valued data by tracking data validity. While the mechanism in LazyGPU (tied to the lazy execution pipeline) is different from SAVE's, the high-level goal of exploiting zero-valued operands to eliminate work is identical. The paper needs to more clearly differentiate its contribution from this and other sparsity-aware execution paradigms.

The novelty of this paper is therefore not in any single primitive, but entirely in the specific integration of three known ideas. The significance of the contribution hinges on whether this integration is non-obvious and solves unique challenges specific to the GPU domain.

---

Questions to Address In Rebuttal

1. Regarding Novelty over SAVE [29]: The optimization to eliminate memory requests for operands of instructions like multiply-add when another operand is zero (optimization ②) appears functionally similar to the goals of SAVE. Please clarify the fundamental novelty of your approach. Is the primary contribution the non-speculative nature of the optimization, which is enabled by the lazy execution pipeline, thus avoiding the potential complexities of speculation-and-recovery mechanisms?

2. Regarding Novelty of Lazy Execution on GPUs: The foundational concept of lazy execution was proposed for CPUs in [8]. Beyond stating that this is "underexplored" for GPUs, please elaborate on the specific, novel microarchitectural contributions required to adapt this concept to a massively parallel SIMT architecture. For instance, what challenges arose in managing pending requests for an entire wavefront versus a single thread, and how does your design for storing request information in the physical register file (Figure 6, page 6) represent a novel solution to these challenges?

3. Regarding the Complexity/Benefit Trade-off: The claimed hardware overhead of 0.009% of the total die size (Section 5.5, page 13) seems exceptionally low, given that it requires adding state (busy bits) to a large physical register file and logic to store partial address information. Could you provide a more detailed breakdown of this cost, perhaps relative to the area of the SM or the register file itself, rather than the entire die? A small percentage of a large die can still be a significant absolute area, and a more contextualized figure is needed to properly evaluate the novelty of the implementation's efficiency.

Reviewer Persona: The Guardian (Adversarial Skeptic)

---

Summary

This paper presents an evaluation of Ruche Networks, an on-chip network topology that augments a standard 2-D mesh with uniform, long-range "express" links. The authors posit that this approach retains the physical design advantages of a mesh while overcoming its performance scaling limitations. Through RTL-level simulations with both synthetic and benchmark-driven traffic, the authors conclude that Ruche Networks are superior to conventional 2-D mesh and 2-D torus topologies in terms of performance, power, area, and cycle time. The core argument rests on the idea that adding physical express links is a more cost-effective method for improving performance than implementing virtual channels (VCs) as required by a torus.

Strengths

1. The evaluation methodology is grounded in RTL-level implementations of the router microarchitectures, which provides a more credible basis for area, timing, and power analysis than high-level simulation models.
2. The paper commendably uses a combination of synthetic traffic patterns (Section 4.1) and a comprehensive suite of execution-driven parallel benchmarks (Section 4.6), allowing for both micro-architectural stress testing and system-level performance characterization.
3. The analysis in Section 4.5 and Table 4, which explicitly considers the relationship between bisection bandwidth, memory bandwidth, and network aspect ratio, is a valuable and clear-headed piece of analysis that helps frame the design space.
4. The specific performance diagnosis for the half-torus on the Jacobi benchmark (Section 4.6), where nearest-neighbor communication becomes a worst-case scenario, is an insightful observation that lends credibility to the simulation framework.

Weaknesses

Despite its strengths, the paper's central claims of cost-effectiveness and superiority are built on a foundation of questionable assumptions and critical omissions in the analysis. My primary concerns are as follows:

1. Critically Flawed Area and Energy Accounting: The paper's primary argument is that Ruche is "cost-effective." However, the cost analysis is fundamentally flawed.

   • The area comparison in Figure 7 and Table 2 is explicitly limited to the router logic. It completely ignores the physical area cost of the long-range Ruche channels themselves, including the significant area consumed by repeaters needed to drive these global wires. A network defined by its long-range links cannot have its area cost evaluated while ignoring the area of those links. This omission invalidates the "area efficiency" and "area-normalized speedup" claims (Table 6).
   • Similarly, the initial energy analysis in Table 3 explicitly states: "This result does not include the energy dissipated by long-range links outside the tile area." This is a fatal omission. The very mechanism purported to provide benefit is excluded from the cost analysis. While a wire energy model is introduced later in Section 4.9, it appears to be an oversimplified first-order model whose results—that wire energy is a "very small percentage of the total energy" (Figure 13)—are deeply counter-intuitive for long, repeated global wires and require much stronger validation.

2. Unconvincing Baseline for 2-D Torus: The paper's negative characterization of the 2-D torus relies on a potentially uncharitable baseline implementation.

   • The argument in Figure 3c that a VC router must "discard one of the mesh crossbars" is a specific design choice, not an inherent property of VC-based routers. An alternative implementation could have maintained the crossbar bandwidth. This choice seems designed to cripple the torus baseline from the outset.
   • The reported saturation throughput for the 16x16 torus under uniform random traffic is only 19% (Figure 6). This figure is suspiciously low for a well-designed torus network and suggests that the baseline may be under-provisioned (e.g., insufficient VCs, suboptimal allocator design) and not representative of a state-of-the-art implementation. The claims of Ruche's superiority are weakened if the comparison is made against a strawman.

3. Bifurcated and Inadequately Justified Evaluation: The paper abruptly switches from evaluating "Full Ruche" with synthetic traffic (Section 4.1) to evaluating "Half Ruche" for the more realistic benchmark-driven analysis (Sections 4.5-4.9). The justification provided—that all-to-edge traffic only requires horizontal links—is insufficient.

   • This split raises immediate suspicion. Why was the supposedly superior Full Ruche topology not carried through to the benchmark evaluation? One might infer that the full cost (in area, power, or routing complexity) of a Full Ruche network was too high to show a benefit in a more realistic setting, which would significantly weaken the paper's overall claims. The authors must demonstrate the performance of Full Ruche on the benchmark suite to present a complete and honest evaluation.

4. Oversimplified Physical Design Argument: The paper claims Ruche is "physically scalable" based on the regularity of its tile-based layout (Figure 2). This is a superficial argument that ignores the profound physical design challenges of implementing such a topology at scale. The paper fails to discuss or quantify the impact of increased routing congestion from adding numerous global wires, the difficulty of timing closure across these multi-tile links, or the potential for crosstalk and signal integrity issues. The simple repeater model in Section 4.9 is inadequate for addressing these first-order VLSI concerns.

Questions to Address In Rebuttal

1. Please provide a revised area analysis (akin to Figure 7) that includes the area of the repeaters required for all Ruche links and an estimate for the routing area overhead based on the number of additional wiring tracks consumed. How does this affect the "area-normalized speedup" metric in Table 6?

2. Please justify your torus router implementation. Specifically, why was the choice made to halve its crossbar bandwidth relative to a multi-mesh (as depicted in Figure 3), and can you provide evidence that its performance (e.g., 19% saturation in Figure 6) is representative of a competitive, modern torus design?

3. To provide a consistent evaluation, please present the benchmark speedup, latency, and energy results (Figures 10-13) for the Full Ruche topology. If its performance is not superior to Half Ruche, please explain the architectural reasons for this outcome.

4. The wire energy calculation in Section 4.9 appears to be a primary source of the paper's strong energy efficiency claims. Can you provide a more detailed breakdown of this model, including the assumptions made for repeater sizing, leakage power, and wire parameters (e.g., how was 0.2 pF/mm validated for your target 12nm process)? Please provide a sensitivity analysis showing how total energy changes if wire capacitance/energy-per-bit is 2x or 5x higher than your estimate.

Paper Title: Evaluating Ruche Networks: Physically Scalable, Cost-Effective, Bandwidth-Flexible NoCs Reviewer Persona: The Synthesizer (Contextual Analyst)

---

Summary

This paper presents a comprehensive and compelling evaluation of Ruche Networks, a Network-on-Chip (NoC) topology that augments the standard 2-D mesh with regular, physical, long-range "skip" links. The authors situate their work in the well-known context of the scalability limitations of 2-D mesh, particularly its bisection bandwidth bottleneck, which is increasingly problematic for modern data-intensive manycore architectures. The core contribution is not the invention of Ruche Networks, but rather the first rigorous, RTL-level, execution-driven evaluation that fills a critical gap left by prior analytical work.

Through detailed simulations using both synthetic traffic and a suite of parallel benchmarks, the authors systematically compare Ruche against 2-D mesh and 2-D folded torus—its most practical competitor. Their findings demonstrate that Ruche offers a superior design point, achieving higher throughput and lower latency at a reduced area and power cost compared to a virtual-channel-based torus. The work provides strong evidence that Ruche Networks represent a highly practical and effective solution for scaling on-chip interconnects, preserving the physical design advantages of mesh while overcoming its primary performance bottlenecks.

Strengths

1. High-Quality and Comprehensive Evaluation: The paper's primary strength is the depth and realism of its evaluation methodology. By moving beyond analytical models and high-level simulations to RTL-level implementations (for area, power, and timing analysis in Section 4.2, Page 5) and full-system, execution-driven simulation (Section 4.6, Page 8), the authors provide a level of evidence that is both convincing and highly valuable to the community. This rigorous approach gives significant weight to their claims of superiority over torus and mesh.

2. Excellent Contextualization and Problem Framing: The authors do an outstanding job of placing Ruche Networks within the broader landscape of NoC research. The introduction (Section 1, Page 1) astutely points out why historical solutions like concentration and simple channel widening are based on "outdated assumptions" that no longer apply to modern stream-based, data-intensive workloads. The comparison against folded torus is particularly insightful, as it represents the most direct and physically-plausible alternative for adding long-range links to a mesh-like structure. Table 1 (Page 3) provides a clear and useful taxonomy of topologies based on physical scalability criteria.

3. Focus on Practicality and Physical Design: A key theme of the paper is its grounding in the realities of modern chip design. The authors consistently emphasize that Ruche retains the regular, tileable structure that makes 2-D mesh so popular (Figure 2, Page 3). This focus on physical realizability is a critical differentiator from more esoteric topologies that may look good on paper but are impractical to route on a 2D die. The analysis of "depopulated" crossbars (Figure 5, Page 5) is an excellent example of a practical, cost-saving optimization.

4. Clear Demonstration of a Superior Design Space: The paper successfully makes the case that Ruche offers a better set of trade-offs than its competitors. It demonstrates that the architectural complexity of virtual channels required for a deadlock-free torus negates many of its theoretical bandwidth advantages (Figure 6, Page 6). Ruche, by contrast, achieves deadlock freedom through simple dimension-ordered routing while using its hardware resources more efficiently to provide higher crossbar bandwidth. The results presented in the energy analysis (Figure 13, Page 11) are particularly striking, showing that half-torus can actually consume more total energy than 2-D mesh due to router overhead, a pitfall that Ruche avoids.

Weaknesses

While this is a strong paper, there are areas where its context and claims could be further broadened and strengthened.

1. Limited Exploration of Routing Algorithms: The evaluation is exclusively based on Dimension-Ordered Routing (DOR). While DOR provides a simple and effective deadlock-free mechanism, it is unable to route around congestion. The addition of numerous long-range links in the Ruche topology seems to create a path diversity that is ripe for exploitation by adaptive routing algorithms. A discussion of how Ruche might perform with even a simple adaptive scheme would provide a more complete picture of its potential. Without this, it's unclear if the full capability of the added physical links is being realized under heavy, non-uniform traffic.

2. The Premise of Underutilized Wiring: The justification for adding Ruche links rests on the premise, cited from [27], that 2-D meshes typically underutilize available wiring tracks between tiles. While this is a plausible and well-established observation, the paper would be stronger if it provided some quantitative data from its own physical design flow to support this. For example, showing wiring congestion maps or utilization statistics for a baseline mesh versus a Ruche network (e.g., RF=3) would turn this premise from a cited fact into a demonstrated reality within the context of their own experiments. How close to the practical wiring limit does a high Ruche Factor push the design?

3. Narrow Quantitative Comparison to Other Express Topologies: The paper provides a good qualitative comparison to topologies like MECS and Flattened Butterfly in Section 3 (Page 3). However, the quantitative evaluation is limited to mesh and torus. While a full RTL-level comparison is likely out of scope, including even a high-level simulation-based comparison against a topology like MECS could help readers better situate Ruche's performance. Is Ruche's advantage due to its constant-radix routers, its specific link placement, or both? A broader comparison could clarify the specific sources of its efficiency.

Questions to Address In Rebuttal

1. The choice of DOR is practical, but could you comment on the potential of using adaptive routing with Ruche Networks? Given that Ruche routers are simpler and faster than VC-based routers, would adding the minimal logic for adaptivity (e.g., extra VCs for deadlock avoidance) still result in a more efficient design point than the highly complex allocators found in a baseline torus?

2. Your work is predicated on the availability of VLSI wiring resources to implement the long-range Ruche links. Can you provide any concrete data from your place-and-route experiments regarding wire track utilization or routing congestion? This would significantly bolster the argument that Ruche is not just performant but also physically non-disruptive to implement.

3. The comparison to 2-D torus is well-motivated and excellent. Could you elaborate on the decision not to include a quantitative performance comparison against other express-link topologies like MECS? Would the increasing router radix of MECS make it a non-starter from an area/timing perspective even in the 16x16 networks you evaluated?

Paper Title: Evaluating Ruche Networks: Physically Scalable, Cost-Effective, Bandwidth-Flexible NoCs Reviewer Persona: The Innovator (Novelty Specialist)

---

Summary

The paper presents a detailed evaluation of Ruche Networks, a 2D mesh topology augmented with regular, equidistant long-range physical links. The authors implement RTL-level routers for Ruche, 2D mesh, and 2D torus topologies, and perform a comparative analysis based on synthetic traffic and execution-driven simulations of parallel workloads. The study provides a characterization of Ruche Networks in terms of performance (latency, throughput), area, power, and cycle time, arguing that Ruche provides a superior trade-off compared to mesh and torus.

The core architectural concept, "Ruche Networks," was previously proposed by the same authors in [15] and [25]. The primary contribution of this work is therefore not the introduction of a new network topology, but rather its comprehensive, hardware-level characterization and comparison against established alternatives. The novel claims are centered on the experimental insights derived from this evaluation, such as the effectiveness of depopulated crossbars and the scalability benefits of the Ruche Factor.

Strengths

The strength of this paper lies in its rigorous and detailed evaluation methodology, which moves significantly beyond the analytical models presented in the authors' prior work. The provided RTL-level implementations, synthesis results for area and cycle time (Figure 7, Page 7), and power analysis (Table 3, Page 7) provide a concrete and valuable grounding for the architecture's claims. This level of detail is essential for transitioning an architectural concept from a theoretical proposal to a viable engineering solution. The comparison against a virtual-channel-based 2D torus is particularly useful, as it directly contrasts two distinct methods for achieving deadlock-free, long-range connectivity.

Weaknesses

From the perspective of conceptual novelty, the paper's contribution is limited.

1. Recycled Core Idea: The central architectural idea—the Ruche topology—is not new. It was introduced in the authors' previous publications, specifically [15] "Ruche Networks: Wire-Maximal, No-Fuss NoCs" (NOCS 2020) and [25] "Implementing Low-Diameter On-Chip Networks..." (NOCS 2020). This paper is explicitly positioned as an evaluation of a known entity, making it an incremental contribution rather than a foundational one.

2. Well-Established General Concept: The broader concept of augmenting a 2D mesh with physical express links or bypass channels is a well-explored area in the network-on-chip literature. Topologies like Flattened Butterfly [17], MECS [12], and various other hierarchical or express-link-based designs have long sought to reduce the diameter of mesh networks. While Ruche offers a specific, physically-aware implementation with its equidistant links and constant-radix routers, it exists within this established paradigm. The paper does not introduce a fundamentally new way of thinking about network topology.

3. Standard Microarchitectural Optimizations: The proposed "depopulated" router variant (Figure 5, Page 5) is a direct application of a standard design practice. Router crossbars are commonly optimized by removing paths that are illegal under the chosen routing algorithm (in this case, DOR). This is a well-known technique to reduce area and power and does not constitute a novel microarchitectural contribution. The paper's contribution here is merely the quantification of this standard technique in the context of Ruche.

In essence, the paper does an excellent job of evaluating an existing idea but presents little in the way of new conceptual frameworks, algorithms, or architectural primitives.

Questions to Address In Rebuttal

1. The authors explicitly state that this paper aims to fill the evaluation gap left by their prior work [15, 25]. Beyond demonstrating that Ruche performs well, what is the single most significant and surprising conceptual insight derived from this evaluation? That is, what fundamental trade-off or principle did this hardware-level study reveal that was not already predictable from the high-level concept?

2. The core idea of Ruche is the addition of regular, equidistant "skip" links. This is topologically similar to other regular graph structures, such as k-ary n-cubes with additional chords. Could the authors articulate the fundamental topological novelty of Ruche that distinguishes it from this broader class of networks, beyond the specific tile-based physical implementation methodology?

3. One of the key results is that a simple Ruche configuration (e.g., RF=2, depopulated) yields most of the performance gains. While a valuable engineering guideline, this outcome seems predictable: adding any low-cost bisection bandwidth to a bisection-limited mesh should yield significant returns, with diminishing gains thereafter. Can the authors argue why this result is a novel finding rather than an empirical confirmation of first-order network theory?

Paper Title: Garibaldi: A Pairwise Instruction-Data Management for Enhancing Shared Last-Level Cache Performance in Server Workloads Reviewer: The Guardian

---

Summary

This paper identifies the "instruction victim problem" in server workloads, where instruction cachelines are evicted from the shared Last-Level Cache (LLC) due to contention with data cachelines. The authors argue that this is detrimental because an instruction miss can stall the frontend, preventing access to even hot, already-cached data. They propose Garibaldi, a pairwise instruction-data management scheme for the LLC. The mechanism introduces a "pair table" to track the relationship between instruction lines and the data lines they access. The hotness of data accesses (determined by hit/miss status under a baseline replacement policy) is used to calculate a "miss cost" for the corresponding instruction. Instructions with a high miss cost are selectively protected from eviction using a query-based mechanism. Additionally, the scheme prefetches data associated with unprotected instructions upon an instruction miss. The evaluation, performed on a simulated 40-core system, claims significant performance improvements over state-of-the-art LLC management schemes like Mockingjay.

---

Strengths

1. Problem Motivation: The paper provides a compelling motivation. Figure 1 (page 1) effectively illustrates that the instruction fetch (ifetch) component of CPI is a significant bottleneck in server workloads, particularly in multi-core configurations, which is often overlooked in traditional LLC management schemes focused solely on data.
2. Core Concept: The fundamental idea of creating a feedback loop from data access outcomes back to instruction management at the LLC level is logical. Decoupling instruction and data management, as is traditionally done, ignores their inherent dependency, and this work attempts to address that.
3. Evaluation Scope: The authors have conducted an extensive evaluation, including 16 server workloads, comparisons against multiple state-of-the-art policies (DRRIP, Hawkeye, Mockingjay), and a battery of sensitivity studies.

---

Weaknesses

My analysis finds that while the motivation is sound, the proposed mechanism rests on several questionable assumptions and its claimed benefits are not as robustly supported as presented.

1. Gross Oversimplification of the "Miss Cost" Metric: The paper's central concept of "opportunity cost" is implemented as a simple n-bit saturating counter, incremented on a data hit and decremented on a data miss (Section 4.1, page 6). This is a fundamentally flawed proxy for cost. A data miss that goes to main memory incurs a penalty hundreds of cycles longer than an LLC hit. Treating these two events as symmetric opposites (+1/-1) fails to capture the non-linear and highly variable cost of memory accesses. An instruction leading to one catastrophic data miss is far more costly than one leading to several data hits. The mechanism, as described, cannot distinguish this and is therefore unlikely to accurately identify the most critical instructions.

2. Fragility and Questionable Scalability of the Pair Table: The core hardware structure, the pair table, appears to be both fragile and a potential scalability bottleneck.

   • The performance of Garibaldi is critically dependent on a large pair table. The sensitivity study in Figure 14(c) (page 12) shows a steep performance decline as the table size is reduced from the 16K-entry default. This undermines the claim of a modest "0.6% of LLC capacity" overhead (page 10), as this overhead is not optional but essential for the mechanism to function.
   • The tracking mechanism relies on a per-core "helper table" to map a PC to an instruction physical address (IL_PA) (Section 5.1, Figure 8, page 8). The paper is completely silent on the miss rate of this helper table and the performance consequences of a miss. If this lookup fails, the entire chain of correlating a subsequent data access back to its instruction line breaks down. This is a critical omission that calls the viability of the entire tracking scheme into question.

3. Unconvincing Experimental Comparisons and Analysis: The evaluation contains analyses that weaken, rather than strengthen, the paper's claims.

   • The comparison against partitioning-based protection in Figure 14(d) (page 12) appears to be a strawman argument. The authors themselves state in Section 2.2 that way-partitioning is challenging in modern processors with high core counts and low associativity. To then implement a simplistic version and show it performs poorly does not validate Garibaldi; it merely confirms a known limitation of a naive approach. A more rigorous comparison against a state-of-the-art dynamic partitioning scheme is required.
   • The analysis of the paired-data prefetcher is insufficient. Figure 14(a) (page 12) shows that performance peaks when tracking only one or two data lines (k=1, 2) and then degrades. The authors offer no substantive explanation for this degradation. Is it due to prefetch-induced cache pollution? What are the accuracy, coverage, and timeliness metrics of this prefetcher? Without this analysis, the prefetching component feels ad-hoc and its benefit is not rigorously established.

4. Incomplete Problem Diagnosis: The paper attributes the high ifetch CPI from Figure 1 primarily to LLC instruction misses without providing a rigorous breakdown. Frontend stalls are a complex phenomenon resulting from L1-i cache misses, L2 misses, branch mispredictions, and other pipeline hazards. The paper does not quantify what fraction of these stalls are uniquely attributable to LLC misses. Without this data, it is impossible to assess whether an LLC-level intervention like Garibaldi is the most effective solution, or if the observed problem is better addressed at the L1/L2 level or via improved branch prediction.

---

Questions to Address In Rebuttal

The authors must address the following points directly to demonstrate the technical soundness of their work:

1. On "Miss Cost": Please provide a quantitative justification for using a simple, symmetric +/-1 counter as a proxy for "miss cost." Show data comparing this metric's predictions of instruction criticality against a more realistic oracle based on actual stall cycles incurred by dependent data accesses.

2. On Helper Table Failures: What is the measured miss rate of the per-core helper tables in your simulations? Crucially, what is the defined behavior and performance impact when a helper table lookup fails during a data access, preventing the correlation to an instruction line?

3. On Prefetcher Performance: Please provide a detailed analysis explaining the performance degradation seen in Figure 14(a) for k > 2. This analysis must include metrics for prefetcher accuracy and the extent of cache pollution caused by the additional prefetches.

4. On Problem Attribution: Please provide a detailed breakdown of the ifetch stall cycles reported in Figure 1. What percentage of these stalls are caused by L1-i misses that hit in L2, L2 misses that hit in the LLC, and finally, LLC misses that go to memory? This data is essential to validate the premise that the LLC is the primary locus of the problem.

Reviewer: The Synthesizer (Contextual Analyst)

---

Summary

The authors address the increasingly critical problem of frontend stalls in server workloads, specifically targeting instruction misses in the shared Last-Level Cache (LLC). They observe that modern, data-centric LLC management policies overlook the high cost of an instruction miss, which stalls the pipeline even if the data it needs is already resident in the cache.

The central contribution is "Garibaldi," a novel management scheme that redefines the value of an instruction cache line not by its own reuse characteristics, but by the "opportunity cost" associated with the hot data it subsequently accesses. It achieves this through a "pairwise" mechanism that tracks instruction-data relationships, propagates data hotness (as determined by the underlying cache replacement policy) back to the corresponding instruction, and uses this information to selectively protect these high-cost instructions from eviction. The mechanism also includes a conservative prefetcher for paired data when an unprotected instruction misses. Evaluated on top of a state-of-the-art LLC policy (Mockingjay), Garibaldi demonstrates a significant 6.1% average performance improvement for server workloads, highlighting the efficacy of this new perspective.

---

Strengths

1. Novel and Insightful Problem Framing: The paper’s primary strength lies in its excellent conceptualization of the "instruction victim" problem at the LLC level. The authors compellingly argue that the cost of an instruction miss is intrinsically linked to the hotness of the data it gates. The CPI stack analysis in Figure 1 (page 1) and the reuse distance breakdown in Figure 3 (page 4) provide a strong empirical foundation for this claim, effectively demonstrating that a significant and growing bottleneck exists where the authors propose to intervene.

2. An Elegant Conceptual Bridge: This work beautifully synthesizes concepts from two often-disparate domains of computer architecture: frontend pipeline optimization and shared cache management. By bringing awareness of frontend stalls into the LLC's policy decisions, Garibaldi bridges a crucial gap. The analogy drawn between their "instruction victim" and the well-known "inclusion victim" problem (citing work like QBS [33] from which they draw inspiration for their query-based mechanism) is particularly sharp and helps to situate the contribution within a broader theoretical context.

3. Pragmatic and Orthogonal Design: The proposed mechanism is well-considered from an implementation standpoint. It cleverly piggybacks on the hit/miss signals from existing advanced replacement policies (like Mockingjay) to define data "hotness," making the design orthogonal and broadly applicable. The details, such as the Pair Table, the dynamic threshold adjustment via coloring (Section 5.2, page 8), and the decoupled storage for data addresses, demonstrate a thoughtful approach to balancing effectiveness with hardware overhead, which is analyzed and shown to be modest (Table 2, page 10).

4. Strong and Convincing Results: The performance improvements are not trivial. Achieving an additional 6.1% speedup on top of a powerful baseline like Mockingjay is a significant result in the context of LLC management. The per-workload analysis in Section 7.2 (page 11) effectively links performance gains to reductions in ifetch stall cycles, closing the loop and validating the paper's central hypothesis.

---

Weaknesses

While the core idea is strong, its presentation and exploration could be broadened to better understand its position in the design space.

1. Limited Scope of the "Pairing" Concept: The pairing mechanism, which links an instruction PC to the physical addresses of the data it accesses, is effective but may be a simplification of real-world program behavior. The paper does not deeply discuss how the mechanism copes with instructions involved in complex access patterns, such as pointer-chasing or accesses to highly dynamic data structures, where a single static instruction may touch a wide and changing set of data lines. The reliance on a small, fixed number of tracked data lines per instruction (k=1) is justified for latency but its implications for these more complex patterns are underexplored.

2. Under-examined Interplay with Advanced Prefetchers: The baseline system rightly includes advanced instruction (I-SPY) and data (GHB) prefetchers. However, the interplay between these components and Garibaldi is a rich area for discussion that is largely absent. For instance, a perfect instruction prefetcher would obviate the need for Garibaldi's protection mechanism entirely. It would be valuable to understand if Garibaldi is primarily compensating for prefetcher inaccuracies or if there is a more fundamental synergy. Could Garibaldi's pairwise information, for example, be used to create an even more powerful, data-aware instruction prefetcher?

3. Dependence on a Specific Workload Characteristic: The mechanism is predicated on the "many-to-few" access pattern (many cold instructions, few hot data) identified in server workloads. The paper correctly notes that in workloads like kafka, where both instruction and data reuse are low, the trade-off made by Garibaldi is not beneficial (Section 7.2, page 11). This suggests that the system's effectiveness is highly dependent on this characteristic. The work would be strengthened by a more formal characterization of the workload properties that define its ideal operating conditions and a discussion of potential dynamic mechanisms to modulate Garibaldi's aggressiveness in phases where its core assumptions do not hold.

---

Questions to Address In Rebuttal

1. The pairwise tracking currently maps an instruction line to a small, fixed number of data lines (k=1 in the final evaluation). Could the authors elaborate on how the scheme would handle instructions with more complex data access patterns (e.g., pointer chasing through a list, where data addresses are not stable)? Does the mechanism risk "polluting" its pairing information in such cases, and is the aging mechanism sufficient to manage this?

2. The baseline configuration includes an advanced instruction prefetcher (I-SPY). Could the authors discuss the interplay between Garibaldi and instruction prefetching in more detail? For example, does Garibaldi primarily rescue misses that the prefetcher fails to cover, or is there a more complex interaction? Could the pair table's data address information be used to create a more powerful, data-aware instruction prefetcher?

3. The analysis for the kafka workload (Section 7.2, page 11) is insightful, showing that the mechanism is not beneficial when both instruction and data reuse is low. Could the authors propose a dynamic mechanism or heuristic to detect such application phases and perhaps throttle or disable Garibaldi's protection policy to avoid degrading data cache performance in these scenarios? For example, could the P(D_miss | I_miss) metric used for threshold adjustment also serve this purpose?

Review Form: The Innovator

Summary

The authors present "Garibaldi," a hardware mechanism designed to mitigate the "instruction victim problem" in the shared Last-Level Cache (LLC) for server workloads. They observe that cold instruction lines are often evicted to make space for hot data lines, leading to frontend stalls when those instructions are needed again, even if the data they would access is already resident in the LLC.

The core claim of novelty lies in a "pairwise instruction-data management" scheme. This scheme explicitly tracks the relationship between an instruction cache line and the data cache lines it subsequently accesses. The "hotness" of data accesses (as determined by whether they hit or miss in the LLC under a baseline policy) is used to compute a "miss cost" for the corresponding instruction line. This miss cost is stored in a dedicated "pair table" and serves two purposes: 1) to selectively protect high-cost instruction lines from eviction using a query-based mechanism, and 2) to trigger a prefetch of associated (cold) data lines when an unprotected instruction line misses.

Strengths

1. Novel Inversion of Information Flow: The central concept of propagating data cacheline hotness back to the instruction cacheline is a conceptually novel approach. The vast majority of prior art in PC-based cache management and prefetching uses the instruction's identity (the PC) to predict the future behavior of data (e.g., Hawkeye [32], Mockingjay [56]). This paper inverts this dependency, using the observed behavior of data to inform the cache management policy for the instruction. This represents a genuine, albeit subtle, shift in perspective.

2. Problem Formulation: The paper provides a clear and well-motivated articulation of the "instruction victim problem" at the LLC level. The insight that an instruction miss can be more costly than a data miss if the data is already resident is a sharp observation that effectively frames the need for a new solution.

3. Integrated Mechanism: The use of a single new structure, the pair table, to drive both selective protection and prefetching is an elegant design choice from a conceptual standpoint. It treats the instruction-data pair as a single entity to be managed holistically.

Weaknesses

1. Constituent Mechanisms are Not Novel: While the synthesis is new, the underlying building blocks are well-established.

   • Query-Based Protection: The selective protection mechanism is an explicit adaptation of Query-Based Selection (QBS), which the authors correctly cite from Jaleel et al. [33]. The novelty is not in the mechanism itself, but in the new metric used to drive it (the "miss cost").
   • PC-to-Data Association: Tracking associations between a PC and the data addresses it touches is the foundational principle of many advanced prefetchers and cache replacement policies (e.g., [32], [72]). The pair table is, in essence, a hardware structure to cache these associations, which is not a fundamentally new idea.
   • Paired Prefetching: The idea of prefetching a data line upon an access to a related (instruction) line is a form of group or correlated prefetching. The novelty here is slim, resting on the trigger (an instruction miss) and the source of the correlation (the dynamically populated pair table).

2. High Implementation Complexity for the Achieved Gain: The proposed hardware is far from trivial. As detailed in Table 2 (page 10), Garibaldi requires approximately 194KB of SRAM storage for a 40-core configuration. This includes a 120KB main pair table, 40KB of per-core helper tables, and a 32KB D_PPN table. This represents a significant area and power overhead. The LLC controller logic is also made more complex, requiring new lookup paths and dynamic threshold management. The geometric mean speedup of 6.1% over a very strong baseline (Mockingjay) is commendable, but it is debatable whether this performance gain justifies the substantial new hardware complexity. The novelty of the idea is tempered by the brute-force nature of its implementation.

3. The "Delta" Over Prior Art is Primarily in Application, Not Invention: The core contribution is the creation of the "miss cost" metric and its application to existing management techniques. This is an engineering innovation rather than a foundational one. The paper does not propose a new caching theory or a fundamentally new architectural paradigm, but rather a new heuristic to plug into existing ones. For a mechanism of this complexity, a more significant conceptual leap would be expected.

Questions to Address In Rebuttal

1. The novelty rests on using data hit/miss status to manage instruction lines. How is this fundamentally different from a feedback-directed approach where the LLC replacement policy simply learns that PCs leading to data hits are "high value" and should be prioritized? Could a simpler scheme that just increments a priority counter for an instruction line's PC upon a subsequent data hit achieve a similar outcome with less hardware?

2. The paper proposes a large, centralized pair table that tracks physical addresses. This requires a complex helper table structure to map a data access's PC (virtual address) to its instruction line's physical address (IL_PA), as shown in Figure 8 (page 8). This seems to introduce a multi-step, latency-intensive process into the LLC controller. Please clarify the latency and critical path implications of this lookup process required on every data access that updates the pair table.

3. Given the high cost of the proposed structure, have the authors considered a more lightweight approach? For example, could the instruction-data pairing information be stored directly within the LLC tags/metadata for resident instruction lines, avoiding the need for a large, decoupled pair table? This would limit tracking to cached instructions but would dramatically reduce the area overhead. What is the justification for the large, off-path table structure versus a more integrated, albeit less comprehensive, approach?

Paper Title: NetCrafter: Tailoring Network Traffic for Non-Uniform Bandwidth Multi-GPU Systems Reviewer: The Guardian (Adversarial Skeptic)

---

Summary

The authors present NetCrafter, a system comprising three techniques—Stitching, Trimming, and Sequencing—designed to optimize network traffic in multi-GPU systems with non-uniform interconnect bandwidths. The motivation stems from the observation that inter-cluster links are significant bottlenecks. The proposed techniques aim to improve bandwidth utilization by combining partially-filled flits (Stitching), reduce total traffic by only sending requested portions of cache lines (Trimming), and prioritize latency-critical page table walk traffic (Sequencing). The evaluation, conducted using the MGPUSim simulator, claims an average speedup of 16% over a baseline non-uniform configuration.

While the problem is relevant, this paper's central claims rest on a series of questionable assumptions and methodological choices. The analysis of performance trade-offs is superficial, particularly regarding the Trimming mechanism's impact on spatial locality and the latency implications of Flit Pooling. Furthermore, the work's applicability is severely limited by its simplistic memory and coherency model, which ignores the complexities of modern hardware-coherent systems.

Strengths

1. Well-Motivated Problem: The paper correctly identifies that non-uniform interconnect bandwidth in emerging multi-GPU systems (e.g., Frontier) is a critical performance bottleneck.
2. Systematic Observations: Section 3 provides a clear, data-driven analysis that forms the basis for the three proposed techniques. The identification of unused bytes in flits, partial cache line utilization, and the critical nature of PTW traffic are logically presented.
3. Component-level Sensitivity: The authors perform some sensitivity analysis, for instance, on the Flit Pooling delay (Section 5.4, Figures 18 and 19), which demonstrates an attempt to justify design parameter choices.

Weaknesses

1. Fundamentally Flawed Rationale for Trimming: The paper’s core defense of Trimming rests on a weak and unsubstantiated claim. The authors state that since Trimming is only applied to inter-cluster requests, it "does not entirely negate the spatial locality benefits of natural fetching of the cacheline" (Section 4.3, pg. 7). This is a hand-wavy dismissal of a critical performance principle. Spatial locality is valuable regardless of whether the data resides on a local or remote cluster. By truncating cache line transfers to 16 bytes, the authors are explicitly gambling that no other data in that line will be needed, destroying any potential prefetching benefit. The evaluation in Section 5.3 (Figure 16) compares against a strawman "all-trimming" sector cache baseline. A proper evaluation would quantify the performance lost due to spoiled spatial locality on inter-cluster requests and compare against more intelligent hardware prefetching mechanisms that would be negatively impacted by this design.

2. Unjustified Latency Cost of Flit Pooling: The Stitching mechanism requires a companion technique, Flit Pooling, which intentionally stalls flits in a queue for up to 32 cycles hoping a merge candidate appears. The authors mitigate the obvious performance risk by creating "Selective Flit Pooling," which exempts PTW-related flits (Section 4.2, pg. 7). This solution is overly simplistic. It assumes that PTW requests are the only form of latency-critical traffic in the system. What about other synchronization primitives, atomic operations, or critical metadata reads that are not part of a PTW? These would be unduly delayed, and the paper provides no analysis of the latency impact on the broader distribution of non-PTW network packets. The claim that this delay is acceptable is not sufficiently supported.

3. Grossly Simplified Coherency and Memory Model: The paper's evaluation and design assume a software-managed coherence model and explicitly state that "remote L2 data is not cached in the local L2 partition" (Section 2.1, pg. 3). This is a profound simplification that invalidates the work's applicability to many state-of-the-art and research systems that employ hardware coherence. Hardware coherence protocols (e.g., MESI variants) would generate a high volume of small, latency-sensitive control packets (e.g., invalidations, writebacks, acknowledgments). The paper completely ignores this entire class of network traffic. How would these critical control packets interact with Flit Pooling? Would an invalidation message be delayed by 32 cycles? The claim in Section 4.5 (pg. 9) that NetCrafter "can also seamlessly complement any underlying hardware coherence mechanisms" is pure speculation and is not backed by a single piece of evidence or analysis.

4. Baseline Configuration Exaggerates Benefits: The primary evaluation is performed on a system with an 8:1 bandwidth ratio (128 GB/s intra-cluster vs. a meager 16 GB/s inter-cluster). While such asymmetry exists, this extreme configuration provides a near-perfect environment for traffic reduction techniques to show benefit. The performance gains are likely inflated by this choice. The sensitivity study in Figure 22 shows, as expected, that the benefits shrink as the ratio becomes less skewed. The headline 16% average speedup is therefore conditional on a highly bottlenecked network that may not be representative of all designs.

Questions to Address In Rebuttal

1. On Trimming and Spatial Locality: Please provide a quantitative analysis of the L1 cache miss penalty and overall performance impact resulting from the destruction of spatial locality for inter-cluster requests. How does your selective Trimming compare against a baseline that has a standard prefetcher active for all requests, including inter-cluster ones?

2. On Flit Pooling Latency: The "Selective" aspect of Flit Pooling only protects PTW traffic. What is the empirical evidence to support the assumption that no other traffic is latency-sensitive? Please provide data showing the latency distribution of non-PTW packets with and without the 32-cycle pooling delay.

3. On Coherency Model: The claim of compatibility with hardware coherence is unsubstantiated. Please articulate precisely how NetCrafter's mechanisms (Stitching, Pooling, and Sequencing) would classify and handle the control and coherence traffic (e.g., invalidations, upgrades, acknowledgments) generated by a directory-based coherence protocol. Would these critical messages be delayed by Flit Pooling?

4. On Performance Attribution: In Figure 14, the final results bar combines the effects of "Stitching + Trimming + Sequencing." To properly assess the contributions, please provide a performance breakdown that shows the incremental speedup of each technique individually over the baseline (i.e., Baseline vs. Baseline+Stitching, Baseline+Stitching vs. Baseline+Stitching+Trimming, etc.). This is essential for understanding which components of NetCrafter are providing the claimed benefit.

Paper: NetCrafter: Tailoring Network Traffic for Non-Uniform Bandwidth Multi-GPU Systems Review Form: The Synthesizer (Contextual Analyst)

---

Summary

This paper addresses the performance bottlenecks arising from non-uniform interconnect bandwidth in modern hierarchical multi-GPU systems, a design pattern exemplified by HPC systems like Frontier. The authors identify that the lower-bandwidth links connecting clusters of GPUs are a primary source of contention and performance degradation.

The core contribution is NetCrafter, a suite of three complementary, flit-level network traffic management techniques designed to improve the efficiency of these constrained links. The techniques are: 1. Stitching: Combines partially filled flits destined for the same location to reduce padding/overhead and improve link utilization. 2. Trimming: Fetches only the necessary bytes of a cache line (e.g., 16B instead of 64B) over the slow links, reducing total data transfer volume for requests with low spatial locality. 3. Sequencing: Prioritizes latency-critical network traffic, specifically Page Table Walk (PTW) related packets, over bulk data transfers to prevent head-of-line blocking on critical operations.

Through simulation on a Frontier-like multi-GPU model, the authors demonstrate that NetCrafter achieves an average performance improvement of 16% and up to 64% across a diverse set of applications.

---

Strengths

The primary strength of this work lies in its pragmatic and well-motivated approach to a real, timely, and increasingly critical problem.

1. Excellent Problem Contextualization: The paper is grounded in the clear architectural trend of building large-scale GPU complexes using hierarchical interconnects. This isn't a purely academic exercise; it's a direct response to the design challenges faced by current and future exascale systems. The motivation is clear, compelling, and well-supported by examples from industry.

2. Synthesis of Proven Concepts: The work's elegance comes not from a single radical invention, but from the insightful synthesis and application of established networking and architecture principles to the specific domain of multi-GPU systems.

   • Trimming is a clever, dynamic, and link-aware application of the core idea behind sectored caches. The decision to only trim on slow inter-cluster links (as discussed on page 7, Section 4.3) is particularly insightful, as it mitigates bandwidth pressure where it matters most while preserving the prefetching benefits of full cache line transfers on high-bandwidth local links.
   • Stitching is a flit-level analogue to message coalescing or TCP piggybacking, effectively tackling the classic problem of fragmentation and padding overhead. The addition of "Flit Pooling" to increase stitching opportunities demonstrates a thoughtful design process.
   • Sequencing is a direct application of Quality of Service (QoS) principles. The identification of PTW traffic as the most latency-critical component (Observation 3, page 5) is a key insight that allows for a simple yet highly effective prioritization scheme.

3. Strong, Data-Driven Motivation: Each of the three techniques is justified by a clear observation backed by data presented in the Motivation and Analysis section (Section 3, pages 4-5). This foundational analysis (e.g., Figures 6 and 7 showing underutilized flits and cache lines) makes the subsequent design choices feel logical and well-founded, rather than arbitrary.

4. Thorough and Rigorous Evaluation: The experimental methodology is robust. The authors use a respected simulator (MGPUSim), a relevant system configuration, and a diverse set of workloads. The sensitivity studies on Flit Pooling delay (Section 5.4), flit size, and especially the varying bandwidth ratios (Figure 22, page 12) are crucial for establishing the generality and robustness of the proposed solution. The direct comparison of their Trimming approach against a standard sectored cache baseline (Figure 16, page 11) is a particularly strong piece of analysis that validates their nuanced design.

---

Weaknesses

The weaknesses of the paper are minor and relate more to its positioning and potential future scope rather than fundamental flaws in the core idea.

1. Incremental Nature of Individual Components: While the synthesis is novel, the constituent ideas are conceptually related to prior work in broader fields. Trimming relates to sectored caches, Stitching to packet packing, and Sequencing to QoS. The paper would be slightly stronger if it more explicitly framed its contribution as the novel adaptation and co-design of these principles for the unique traffic patterns of non-uniform multi-GPU interconnects.

2. Understated Hardware Complexity: The paper argues for low overhead, citing ~16KB of SRAM. While the storage overhead is indeed small, the logical complexity added to the network switch is non-trivial. The NetCrafter controller (Figure 13, page 8) requires queue parsing, candidate searching for stitching, timers for pooling, and prioritization logic. This could potentially impact the switch's pipeline depth and critical path latency, a factor that is abstracted away by the fixed 30-cycle latency assumption.

3. Limited Scope Regarding Coherence: The work assumes a software-managed coherence model, which is common today. However, the field is moving towards hardware-coherent multi-GPU systems. Such systems introduce new classes of small, latency-critical traffic (e.g., invalidations, acknowledgments, probes). It is a missed opportunity to not discuss how NetCrafter's mechanisms could be extended to manage this coherence traffic, which would be a perfect candidate for both Stitching and Sequencing.

---

Questions to Address In Rebuttal

1. The evaluation assumes the NetCrafter logic fits within the baseline 30-cycle switch latency. Can the authors provide more reasoning on the feasibility of this? Specifically, how does the search for a stitching candidate within the Flit Pooling mechanism avoid extending the critical path of the switch pipeline?

2. The work is situated within a software-coherent memory model. How do the authors envision NetCrafter adapting to a future hardware-coherent multi-GPU system? Would coherence messages be treated as a new, high-priority traffic class for Sequencing, and would they be good candidates for Stitching with other control packets?

3. Could the authors comment on the potential for negative interactions between the mechanisms? For instance, does the Trimming mechanism, by creating smaller response packets, reduce the opportunity for the Stitching mechanism to find "parent flits" with enough empty space to be useful? Or do they primarily act on different types of traffic, minimizing interference?

Review Form: The Innovator

Summary

The paper proposes NetCrafter, a system designed to optimize network traffic in multi-GPU systems characterized by non-uniform interconnect bandwidth. The authors identify that traffic on the slower, inter-cluster links is a primary performance bottleneck. To address this, NetCrafter employs a combination of three techniques: 1) Stitching, which combines partially filled flits to improve link utilization; 2) Trimming, which fetches only the necessary portions of a cache line (sub-blocks) for requests traversing the slow links; and 3) Sequencing, which prioritizes latency-critical page table walk (PTW) traffic over bulk data traffic. The authors claim this combination of techniques is novel and results in significant performance improvements, averaging 16% across their evaluated workloads.

Strengths

The paper addresses a timely and practical problem. As multi-GPU systems scale using hierarchical and non-uniform networks (e.g., Frontier, Aurora), managing the traffic on lower-bandwidth links is of paramount importance. The authors correctly identify key sources of network inefficiency. The evaluation appears thorough, covering a diverse set of applications and performing sensitivity studies.

Weaknesses

My primary concern, and the focus of this review, is the fundamental novelty of the proposed techniques. While the authors present NetCrafter as a combination of "novel approaches," the core concepts underlying each of the three pillars—Stitching, Trimming, and Sequencing—are well-established principles in computer architecture and networking.

1. Stitching is functionally equivalent to packet/message coalescing. The idea of aggregating smaller data units into a larger transmission unit to amortize header overhead and improve link utilization is not new.

   • The paper's own Related Work section (Section 6, Page 12) acknowledges similar concepts like TCP piggybacking [79, 88] and batching in NICs [41, 80].
   • More directly, Finepack [62] proposed dynamic coalescing of small writes/stores in a multi-GPU context. The authors of NetCrafter claim their approach is more general by operating at the flit level for various packet types. However, this appears to be an incremental extension of the same core concept rather than a fundamentally new idea.
   • The proposed "Flit Pooling" mechanism, which delays a flit to find a stitching partner, is conceptually identical to techniques like interrupt coalescing/moderation in network cards [67, 89], where an event is delayed to allow for batch processing.

2. Trimming is a direct application of sectored/sub-blocked caches. The mechanism of fetching only a portion of a cache line is a classic architectural technique, known as sub-blocking or sectoring, designed to reduce memory bandwidth consumption at the cost of more fine-grained metadata.

   • The authors acknowledge the concept of sectored caches [35, 74] in Section 4.3 (Page 7).
   • The only "novel" aspect presented is the policy of applying this technique selectively, i.e., only for requests traversing the slower inter-GPU-cluster links. While this is a sensible engineering decision to balance bandwidth savings against potential harm to spatial locality, a new application policy for an old mechanism does not, in my view, constitute a novel technical contribution for a premier architecture conference.

3. Sequencing is a standard application of Quality of Service (QoS). Prioritizing latency-sensitive control or metadata traffic over bulk data traffic is a foundational concept in network design, often implemented using virtual channels or priority queues.

   • The insight that PTW traffic is latency-critical is also well-documented in prior work on GPU virtual memory, as noted by the authors' own citations [7, 28, 44, 81] in the Introduction (Section 1, Page 1).
   • Combining a known problem (PTW latency) with a standard solution (traffic prioritization) is sound engineering but lacks research novelty.

In summary, the paper's contribution appears to be the system-level integration and specific application of three pre-existing optimization principles to the multi-GPU interconnect problem. While the engineering and evaluation of this integrated system are valuable, the work does not introduce a fundamentally new architectural concept. The performance gains of 16% on average, while respectable, are not so transformative as to justify the claims of novelty for what is essentially a clever recombination of known techniques.

Questions to Address In Rebuttal

The authors should use the rebuttal to convince the program committee of the work's novelty by addressing the following points directly:

1. On Stitching: Beyond applying the idea to more packet types, what is the fundamental conceptual difference between flit-level Stitching and prior art in message/packet coalescing (e.g., Finepack [62])? How is "Flit Pooling" conceptually novel compared to decades of batching/delay mechanisms used in networking hardware to improve efficiency?

2. On Trimming: The underlying mechanism for Trimming is a sectored cache fetch. Do the authors contend that the novelty lies exclusively in the policy of when to apply it? If so, please argue why a new application policy for a decades-old mechanism is a significant enough contribution for ISCA.

3. On the Combination: The primary argument for novelty may rest on the synergistic combination of these three techniques. Please elaborate on any non-obvious, emergent benefits that arise from combining these three specific techniques that would not be achieved by implementing them independently. Are there specific interactions between Stitching, Trimming, and Sequencing that create a "whole is greater than the sum of its parts" effect?

Excellent. Here is a peer review of the paper from the perspective of "The Guardian."

---

Review Form

Reviewer: The Guardian (Adversarial Skeptic)

Summary

The authors present ANSMET, a DIMM-based near-data processing (NDP) system for accelerating Approximate Nearest Neighbor Search (ANNS). The core proposal is a "hybrid partial-dimension/bit early termination" (ET) algorithm, which aims to reduce memory traffic by calculating a lower-bound distance on partially fetched vectors and terminating further accesses if this bound exceeds a threshold. This ET scheme is supported by a sampling-based offline process to optimize the data layout for fetch efficiency. While the paper addresses a relevant problem, the central claims rest on a heuristic methodology whose robustness is not sufficiently established. Furthermore, the headline claim of "no accuracy loss" is conditional and potentially misleading, obscuring significant performance and storage overheads required to maintain it.

Strengths

1. The paper correctly identifies that a large fraction of distance computations in ANNS are for "rejected" vectors, providing a clear motivation for an early termination strategy.
2. The evaluation is conducted across a reasonable set of seven diverse, large-scale datasets, which provides breadth to the experimental results.
3. The proposed hardware architecture is a pragmatic extension of existing DIMM-based NDP concepts, and the breakdown of system components is clearly described.

Weaknesses

1. The "No Accuracy Loss" Claim is Misleading. The central premise of the early termination algorithm is that it guarantees no accuracy loss. However, a deep reading of Section 4.2 ("Offline common prefix elimination") and Table 5 reveals this is not strictly true under all configurations.

   • To achieve no accuracy loss for outlier vectors (those not matching the common prefix), the system must store the original, non-compressed vector separately and perform a "re-check." This incurs both storage overhead and, more critically, additional memory accesses (Table 5a shows up to 1.4% extra accesses for the default 0.1% outlier threshold). This performance penalty is not adequately factored into the main performance results and undermines the efficiency claims.
   • The alternative, which is implied to be more efficient, involves dropping the least significant bits of outlier elements (Figure 4c), which by the authors' own admission leads to significant accuracy loss (Table 5b shows a catastrophic -34.7% accuracy drop). The paper cannot simultaneously claim "no accuracy loss" as a general feature while also presenting and depending on configurations that either lose accuracy or incur non-trivial overhead to preserve it.

2. The Sampling-Based Data Layout Optimization is Heuristic and Lacks Rigor. The entire data layout, which is critical to the performance of the hybrid ET, is determined by an offline sampling of just 100 vectors from billion-scale datasets (Section 4.2). This methodology is highly suspect.

   • The choice of parameters (nc, nF, TC) is entirely dependent on this minuscule sample being perfectly representative of both the global data distribution and, implicitly, the future query distribution. There is no analysis to demonstrate that this holds. What is the performance degradation if the sample is not representative?
   • The use of the 90th percentile distance from the sample as the reference threshold is arbitrary. Figure 11b shows this percentile yields the lowest KL divergence for the DEEP dataset, but there is no theoretical justification or evidence that this specific value is optimal or even suitable for other datasets. The methodology appears to be curve-fitting on a small sample.

3. The Absolute Improvement in Fetch Utilization is Underwhelming. Figure 10 is presented as a success, showing that ANSMET (NDP-ETOpt) improves effectual data fetch utilization to 11.1% from a baseline of 6.0%. While this is a relative improvement, an 11.1% utilization rate is still exceptionally low. This suggests that even with the proposed complex optimizations, nearly 90% of all data fetched from DRAM remains useless. This fundamentally questions the overall efficacy of the approach; it makes an extremely inefficient process slightly less inefficient, rather than solving the core problem.

4. Baseline Comparisons May Not Be Robust. The paper compares against NDP-DimET as its representation of prior art. However, it is not clear if this baseline is an optimized implementation of dimension-level ET or a strawman. State-of-the-art dimension-level ET techniques can be more sophisticated than what is implied here. Without a more detailed specification of this baseline, the claimed 1.52x speedup of the hybrid ET is not verifiable.

Questions to Address In Rebuttal

1. Please clarify the "no accuracy loss" claim. For the mode that guarantees accuracy, provide a rigorous analysis of the performance and storage overhead of the re-check mechanism across all datasets, not just SPACEV. The main results in Figure 6 should be updated to reflect this overhead.

2. Justify the sampling methodology. Provide a sensitivity analysis showing how the final system performance (QPS) changes with (a) the number of vectors in the sample (e.g., 10, 100, 1000, 10000) and (b) the chosen distance threshold percentile (e.g., 70%, 80%, 90%, 95%). This is critical to prove the method is robust and not just a fragile heuristic.

3. Provide a more detailed specification of the NDP-DimET baseline. What specific algorithm from prior work ([25, 69, 86]) does it implement? How was it optimized to ensure a fair comparison against your proposed hybrid technique?

4. The final fetch utilization rate is still very low (~11%). Can the authors comment on the fundamental reasons for this remaining inefficiency? Does this suggest a ceiling on the effectiveness of any early termination strategy that operates at this level of granularity?

Of course. Here is a peer review of the paper from the perspective of "The Synthesizer."

---

Review Form

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper presents ANSMET, a hardware-software co-design that accelerates Approximate Nearest Neighbor Search (ANNS) by tackling its primary bottlenecks: memory bandwidth and wasted computation. The core contribution is the synthesis of two key ideas: (1) a practical DIMM-based Near-Data Processing (NDP) architecture to offload distance calculations, and (2) a novel, lossless "hybrid early termination" algorithm. This algorithm computes a conservative lower bound of the distance to a query vector by incrementally fetching and processing partial data—either a subset of dimensions or, more uniquely, a subset of the most significant bits within each dimension. This enables the system to terminate work on unpromising vectors much earlier, often before the full data has even been fetched from DRAM. The authors support this co-design with a systematic, sampling-based methodology for optimizing the in-memory data layout to maximize the effectiveness of early termination. The work is evaluated in a cycle-accurate simulation environment, demonstrating significant speedups on billion-scale datasets.

Strengths

The true strength of this paper lies in its holistic, problem-driven approach and its excellent positioning within the broader landscape of systems and algorithm research.

1. Elegant Problem-Solution Coupling: The authors correctly identify the fundamental inefficiency in ANNS workloads: a large fraction of memory accesses and computations are for vectors that are ultimately rejected (as shown clearly in Figure 1, page 3). Their proposed hybrid early termination is not just a clever trick; it is an elegant solution directly tailored to this specific problem. By stopping data movement and computation at the earliest possible moment, it addresses the inefficiency at its root.

2. A Novel Synthesis of Algorithmic Primitives: The concept of early termination is not new, nor is the idea of bit-serial processing. However, the authors' contribution is to create a powerful synthesis. They combine dimension-level termination (a known technique) with a novel bit-level termination, creating a hybrid approach. More importantly, they build the necessary system infrastructure around it, particularly the data layout optimization (Section 4.2, page 5), which transforms the raw algorithm into a practical systems solution. The use of prefix entropy to guide a dual-granularity fetch strategy is particularly insightful.

3. Pragmatic Hardware Choices and Contextual Awareness: The choice of a DIMM-based NDP architecture, as opposed to a more exotic 3D-stacked memory approach, is a sign of mature systems thinking. The authors understand that billion-scale vector databases require capacity above all else, a domain where DIMMs excel. They correctly place their work in conversation with the rich body of literature on DIMM-based NDP for recommendation systems (e.g., RecNMP, TensorDIMM), noting the architectural similarities but highlighting the unique algorithmic opportunities that ANNS presents.

4. Connecting Algorithm Theory to Hardware Reality: This work serves as an excellent case study in how abstract algorithms can be re-imagined in light of concrete hardware characteristics. The 64-byte granularity of a DRAM access is no longer just a parameter to be tolerated, but a constraint to be exploited. The entire data layout optimization scheme is built around making every 64-byte transfer as informative as possible for the early termination heuristic. This is a wonderful example of true hardware-software co-design.

Weaknesses

The weaknesses identified are less about fundamental flaws and more about the scope and positioning of the work, which could be expanded to clarify its broader applicability.

1. Focus on a Single Indexing Family: The evaluation and discussion are heavily centered on HNSW, a graph-based index. While HNSW is state-of-the-art and highly representative, ANNS is a diverse field. It would strengthen the paper to discuss in more detail how the ANSMET approach would apply to other major index families, such as inverted file (IVF) based methods. The access patterns are different (more sequential scans within inverted lists), which might change the dynamics of early termination and load balancing across NDP units.

2. Implications for Dynamic Databases: The proposed data layout transformation is an offline preprocessing step. This is perfectly reasonable for static datasets. However, a key feature of modern vector databases is support for incremental insertions and updates. A discussion on the practicality and overhead of applying these transformations in a dynamic environment would be valuable. Does a new vector need to be fully transformed before it's searchable? How does this impact ingestion latency?

3. Under-explored Interaction with Vector Quantization: The paper briefly mentions compatibility with vector quantization (Section 4.3, page 7), but this interaction is critical. Product Quantization (PQ) and its variants are the industry standard for reducing the memory footprint of billion-scale indexes. Quantization fundamentally alters the bit-level representation and error profile of the data. It is not immediately clear how much benefit the bit-level termination provides on top of data that has already been compressed to 8-bit integers. A more detailed analysis or evaluation of this interplay would significantly increase the paper's practical impact.

Questions to Address In Rebuttal

1. Beyond HNSW, could the authors elaborate on the expected performance and potential challenges of applying ANSMET to an IVF-based index? Specifically, how would the more structured memory access patterns of scanning an inverted list interact with the proposed hybrid partitioning and adaptive polling schemes?

2. Regarding the offline data layout optimization: What is the authors' vision for integrating this approach into a dynamic vector database that handles frequent insertions? Would the overhead of the bit-level transformations on newly inserted vectors negate some of the search-time benefits?

3. Could the authors provide more intuition or preliminary data on the effectiveness of hybrid early termination when applied to vectors that have already undergone Product Quantization (PQ)? Does the common prefix elimination technique still yield significant benefits on the quantized codes, or does the benefit shift primarily to the partial dimension aspect of the technique?

4. The current design cleanly separates index traversal (CPU) from distance comparison (NDP). This is a sensible division of labor. Looking forward, do the authors see any potential in offloading parts of the index traversal itself, such as fetching the neighbor lists in HNSW, to the NDP units to further reduce CPU-memory traffic?

Of course. Here is the peer review from the perspective of "The Innovator."

---

Review Form

Reviewer: The Innovator (Novelty Specialist)

Summary

The paper, "ANSMET," presents a hardware-software co-design to accelerate Approximate Nearest Neighbor Search (ANNS). The authors propose integrating a DIMM-based Near-Data Processing (NDP) architecture with a novel early termination (ET) strategy. The core claim of novelty resides in this ET strategy, which is a hybrid of partial-dimension and partial-bit evaluation. By fetching and processing a vector piece-by-piece (in terms of both dimensions and the bits within each dimension’s representation), the system computes a conservative lower bound on the distance. If this bound exceeds the current threshold, subsequent memory accesses and computations for that vector are terminated, saving bandwidth and compute cycles without accuracy loss. This core idea is supported by a sampling-based offline method to optimize data layouts for the ET strategy and a heterogeneous execution model where the host CPU handles index traversal and the NDP units handle distance calculations.

Strengths

1. Genuinely Novel Algorithmic Contribution: The central contribution—the hybrid partial-dimension/bit early termination—is a novel concept in the ANNS acceleration space. Prior work on ET has largely focused on either the vector level [52] or the dimension level [25, 69]. While bit-level ET has been explored in BitNN [32] for low-dimensional point clouds, that work was confined to bit-serial processing. ANSMET proposes a more generalized and flexible framework that combines dimension- and bit-level termination and, crucially, operates on variable-sized chunks of bits rather than serially. The "dual-granularity fetch" mechanism detailed in Section 4.2 (page 6) is a direct and novel outcome of this core idea.

2. Strong Co-Design Principle: The paper successfully demonstrates a tight coupling between the novel algorithm and the system design. The algorithmic need for non-contiguous, prioritized data access (e.g., most significant bits first) directly motivates the non-trivial data re-layout scheme. The proposed sampling-based offline optimization to determine the fetch granularity (nc, nF) and common prefix length is a clever and novel engineering solution that makes the core algorithmic idea practical. This full-stack thinking, from algorithm to data layout to hardware, is a significant strength.

3. Novel Application of Existing Concepts: The use of "offline common prefix elimination" (Section 4.2, page 6) is a creative application of a well-known compression technique to the problem of ANNS. Identifying and omitting low-entropy prefixes in vector elements to reduce data movement is a novel approach in this specific context.

Weaknesses

1. Architectural Foundation is Not Fundamentally New: While the paper claims to be the first DIMM-based NDP system for ANNS, the underlying hardware architecture is an adaptation of prior art. DIMM-based NDP systems with buffer-chip logic for accelerating high-dimensional vector operations are well-established in the context of recommendation systems [42, 47, 82]. The hardware described in Section 5 (page 7), including the QSHRs and parallel compute units, is conceptually very similar to these previous designs. The novelty is therefore not in the base architecture, but in its specific tailoring and co-design for the proposed ET algorithm. The paper should be more explicit about this distinction.

2. Complexity Introduced by Offline Analysis: The entire system's effectiveness hinges on a complex offline preprocessing and data transformation stage. This stage requires statistical analysis of a data sample (e.g., prefix entropy, early termination frequency as shown in Figure 3, page 5) to determine optimal layout parameters. While the authors argue the cost is amortized, this dependency introduces a significant novelty-vs-practicality tradeoff. For highly dynamic vector databases with frequent insertions and updates, the novelty of this static optimization may become a practical bottleneck, as the learned layout could become stale.

3. The "Delta" Over Prior Art May Be Data-Dependent: The performance gains from the most novel components (dual-granularity fetch and prefix elimination) appear to be highly dependent on the statistical properties of the dataset. For instance, if a dataset's vector elements have high-entropy prefixes, the benefit of common prefix elimination would be negligible. Similarly, if the early termination probability is not concentrated in a specific bit-range, the dual-granularity fetch may offer little advantage over a simpler, uniform-chunk approach. The novelty is sound, but its significance might not be universal across all possible high-dimensional vector distributions.

Questions to Address In Rebuttal

1. Regarding the hardware architecture, can the authors precisely delineate what aspects of their NDP unit design (Figure 5, page 7) are fundamentally novel, beyond the parameterization required for the hybrid ET algorithm? How does the required logic differ in principle from what was proposed in NDP systems for recommendation models like RecNMP [42] or TensorDIMM [47]?

2. The novelty of the system relies heavily on offline analysis and a static data layout. How does the proposed ANSMET framework handle dynamic vector databases where new vectors are continuously inserted? Would this require periodic, system-wide re-analysis and data re-shuffling, potentially negating the online performance benefits?

3. The core idea of prioritizing more significant bits is intuitive. However, could the authors comment on the robustness of their sampling-based parameter tuning? What is the performance degradation if the sampling set is not representative of the true query distribution, leading to suboptimal choices for the prefix length or the coarse/fine-grained bit steps (nc, nF)? This would help assess the true "delta" of this novel contribution in real-world, unpredictable scenarios.

Review Form

Reviewer: The Guardian (Adversarial Skeptic)

---

Summary

The paper proposes DReX, an algorithmic-hardware co-design for accelerating dense vector retrieval, primarily for Retrieval-Augmented Generation (RAG) applications. The core idea is a two-stage process: 1) an in-DRAM filtering mechanism called Sign Concordance Filtering (SCF) that uses simple logic (PIM Filtering Units or PFUs) to prune the search space by comparing only the sign bits of query and corpus vectors, and 2) a near-memory accelerator (NMA) that performs an exact nearest neighbor search on the much smaller, filtered set of vectors. The system is architected as a CXL Type-3 device using LPDDR5X memory. The authors claim that DReX is dataset-agnostic, accurate, and significantly outperforms state-of-the-art ANNS methods on both CPU and GPU platforms.

Strengths

1. Clear Co-design Philosophy: The work presents a clear and compelling case for an algorithmic-hardware co-design approach. The tight integration of the SCF algorithm with the in-DRAM PFU logic and the specialized data layout (Section 5.2, page 6) is a well-considered piece of engineering.
2. Detailed Architecture: The proposed hardware architecture is detailed and appears plausible. The choice of LPDDR5X over HBM is well-justified based on capacity and shoreline pin limitations (Section 5.4, page 7). The distribution of NMAs per memory package is a sound design choice for scalability.
3. Inclusion of Ablation Study: The ablation study in Section 7.2 (page 12) is valuable. It effectively isolates the performance contributions of the near-memory exact search component (N/A--->NMAs) versus the full DReX system with in-memory filtering (PFUs--->NMAs), providing a clearer picture of where the speedups originate.

Weaknesses

1. Fundamental Flaw in the "Dataset-Agnostic" Claim: The central claim that DReX is "dataset-agnostic" (Abstract, page 1) is not only unsubstantiated but is directly contradicted by the authors' own analysis. The entire premise of Sign Concordance Filtering relies on the assumption that vector distributions are centered around zero, making similarity correlatable with sign agreement. The authors admit this reliance in Section 3 (page 4), stating "many of these embedding vectors demonstrate distributions... centered on or near zero." This is a strong assumption, not a general property. More damningly, the discussion in Section 8 and Figure 18 (page 14) explicitly demonstrates the catastrophic failure of SCF on a "pathologically constructed dataset" (i.e., a non-negative dataset), where filtering performance becomes worse than random. The proposed solution, Iterative Quantization (ITQ), is an admission that the core SCF algorithm is, in fact, highly dataset-dependent and requires a preprocessing step to enforce the necessary data properties. This preprocessing overhead is not evaluated, and its necessity fundamentally undermines the paper's core premise of generality.

2. Misleading Performance Comparisons: The performance comparisons in Figure 11 (page 10) are presented as an algorithm-to-algorithm showdown, but they are fundamentally a platform-to-platform comparison. The DReX system is a custom-designed accelerator with an enormous internal memory bandwidth of 1.1 TB/s (Table 2, page 9). It is compared against general-purpose CPUs with 282 GB/s and GPUs with 3.35 TB/s of memory bandwidth. While the GPU has higher peak bandwidth, HNSW and other graph-based methods exhibit irregular access patterns that fail to saturate it, whereas DReX's design is tailored for sequential streaming. The massive speedups reported (e.g., 270x over CPU IVF-SQ) are therefore more indicative of the benefits of specialized high-bandwidth hardware for brute-force computation than the superiority of SCF over ANNS. A fair comparison would require acknowledging that the baselines are severely bandwidth-bottlenecked on their respective platforms.

3. Insufficiently Rigorous Competitor Evaluation: The comparison against the ANNA accelerator (Section 7.1.3, page 11) is based on a "first-order model" constructed by the authors (Section 6, page 9). Comparing a detailed simulation of a proposed architecture against a high-level analytical model of a competing architecture is not a rigorous or convincing evaluation. This approach is susceptible to modeling errors and optimistic assumptions that could unfairly favor the authors' proposal.

4. Questionable Power and Area Modeling: The power analysis in Section 7.4 (page 13) relies on applying a power breakdown model from an HBM paper (Lee et al. [40]) to their LPDDR5X-based system. The authors must provide justification for why power characteristics of these two very different memory technologies can be considered analogous. Furthermore, the PFU area overhead is calculated based on a synthesis in a 16nm logic process and then scaled, with an assumed 10x area penalty for a DRAM process (Section 6, page 8). This is a rough estimation, and the actual implementation costs of integrating non-trivial logic into a cutting-edge DRAM periphery could be substantially higher. The reported "modest" overheads rest on these fragile assumptions.

Questions to Address In Rebuttal

1. Please reconcile the claim of your method being "dataset-agnostic" with the evidence presented in Section 8 that SCF fails completely on non-zero-centered data. If the solution is to use ITQ, please provide a full evaluation of the computational and storage overhead of this mandatory preprocessing step and incorporate it into your end-to-end performance results.
2. Can you justify the fairness of comparing your specialized, high-bandwidth hardware platform against general-purpose CPUs and GPUs? Please provide an analysis that decouples the gains from the SCF algorithm itself versus the gains from having a massive, dedicated memory bandwidth for the exact search phase. For instance, what is the performance if the baseline ANNS algorithms were run on a hypothetical platform with equivalent memory bandwidth to DReX?
3. Please defend the decision to compare DReX against a "first-order model" of the ANNA accelerator rather than a more rigorous, published simulation framework or implementation. What steps were taken to validate that your model accurately represents the performance and bottlenecks of the ANNA architecture?
4. Provide a more robust justification for your power and area modeling. Specifically, why is it valid to use an HBM power breakdown for an LPDDR5X system? What evidence supports the 0.1 mm² area for a PFU implemented in a real DRAM process, beyond the high-level 10x penalty assumption?

Of course. Here is a peer review of the paper from the perspective of "The Synthesizer."

---

Review Form

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper addresses a critical and timely bottleneck in modern AI systems: the performance of dense vector retrieval for Retrieval-Augmented Generation (RAG). The authors correctly identify the problematic trade-off between slow but accurate Exact Nearest Neighbor Search (ENNS) and fast but often inaccurate and dataset-specific Approximate Nearest Neighbor Search (ANNS).

The core contribution is DReX, a compelling algorithmic-hardware co-design that aims to deliver the accuracy of ENNS with performance surpassing ANNS. The proposal is built on two key ideas: 1. An elegant and computationally inexpensive algorithm called Sign Concordance Filtering (SCF), which uses the sign bits of vector dimensions to perform a high-throughput, online filtering of a vector database. 2. A hierarchical hardware architecture that implements this co-design, featuring in-DRAM PIM Filtering Units (PFUs) to execute SCF with massive parallelism, and near-memory accelerators (NMAs) to perform an exact similarity search on the small, filtered set of candidate vectors.

The authors present a holistic system design, including specific DRAM data layouts and a CXL-based integration strategy. Their evaluation, culminating in a 6.2-7x reduction in time-to-first-token for a representative RAG application, convincingly demonstrates the system's potential.

Strengths

1. Excellent Problem-Solution Fit: The paper targets a high-impact, real-world problem. The retrieval step is a well-known performance and quality limiter for RAG. The proposed solution is not merely an acceleration of an existing algorithm but a ground-up rethinking of the problem from a co-design perspective, which is precisely the right approach.

2. Elegance of the Core Algorithm: The Sign Concordance Filtering (SCF) method (Section 4, page 4) is the paper's conceptual jewel. It is simple, intuitive, and, most importantly, exceptionally well-suited for hardware implementation (requiring little more than bitwise XORs and popcounts). This avoids the complexity of trying to implement something like graph traversal (HNSW) in hardware and instead creates an algorithm that thrives on the vast, simple parallelism available in DRAM.

3. Holistic and Credible System Design: This is a strong systems paper. The authors have considered the full stack, from the algorithm down to the data layout in memory (Section 5.2, page 6), the PIM logic in the DRAM periphery (Section 5.3, page 6), the near-memory compute unit (Section 5.4, page 7), and the system integration via CXL. The justification for using LPDDR5X over HBM is well-reasoned and adds to the design's credibility. This end-to-end thinking is a significant strength.

4. Connecting Architectural Gains to Application-Level Impact: A major strength of the evaluation is the direct line drawn from the retrieval throughput improvements (Figure 11, page 10) to the reduction in application-level time-to-first-token (TTFT) (Section 7.3, Figure 15, page 12). This is often missing in architecture papers, which can get lost in micro-benchmarks. By showing a tangible benefit to the end-user of an LLM application, the authors make a powerful case for their work's significance.

5. Anticipation of Limitations: The discussion in Section 8 (page 14) proactively addresses the most obvious critique of SCF: its dependency on data distributions being centered around zero. By showing that a standard technique like Iterative Quantization (ITQ) can effectively mitigate this pathological case, the authors substantially strengthen their claims of generality and robustness.

Weaknesses

While this is an excellent paper, its positioning within the broader landscape of in-memory processing could be strengthened.

1. Context within the Broader History of In-Memory Filtering: The architectural pattern of using simple in-memory logic for coarse-grained filtering followed by more powerful near-memory processing for fine-grained evaluation is a classic idea in the database accelerator and processing-in-memory communities. While the application to dense vectors for RAG is novel and the specific SCF algorithm is new, the paper would benefit from acknowledging this lineage. Placing DReX as the latest and most sophisticated application of this long-standing pattern would strengthen its academic context rather than weaken its novelty.

2. Limited Exploration of the Algorithmic Design Space for In-DRAM Filtering: SCF is presented as the primary solution, and it is a very good one. However, the paper could be even more impactful by briefly discussing why SCF is the right choice compared to other potential hardware-friendly, online filters. For instance, were other simple primitives considered, such as filtering based on a few key quantized dimensions, or a simple form of Locality-Sensitive Hashing (LSH)? A short discussion justifying the choice of SCF over these alternatives would add depth.

3. The Economic and Practical Viability of Modified DRAM: The proposed system relies on custom logic within the DRAM die (the PFUs). This is a notoriously high bar for adoption in the industry. While the power and area analysis (Section 7.4, page 13) is good, the paper could benefit from a paragraph discussing the path to adoption. Is this a feature that could be standardized by JEDEC? Could it be a high-margin custom product for a specific hyperscaler? Acknowledging this practical hurdle and suggesting a path forward would make the work more complete.

Questions to Address In Rebuttal

1. The success of this work is predicated on the elegance of the SCF algorithm. Did the authors consider or prototype other simple, hardware-amenable online filtering techniques (e.g., based on quantized centroids, or a subset of vector dimensions)? A brief discussion on why SCF was chosen over potential alternatives would be enlightening.

2. Can the authors further contextualize their contribution in relation to the broader history of in-memory/near-data filtering for search and database applications? While the application to RAG is new, the architectural pattern feels familiar. Acknowledging this and highlighting what makes the DReX design uniquely suited for high-dimensional vectors would strengthen the paper's positioning.

3. The proposed system requires significant modifications to the memory subsystem (custom DRAM and NMAs). Could the authors comment on the cost-benefit trade-off from a Total Cost of Ownership (TCO) perspective? How does the cost of a DReX system compare to a baseline system that achieves similar performance by simply scaling out more commodity servers with GPUs running an optimized ANNS library?

Review Form

Reviewer: The Innovator (Novelty Specialist)

Summary

This paper presents DReX, an algorithmic-hardware co-design for accelerating dense vector retrieval, primarily for Retrieval-Augmented Generation (RAG) applications. The authors identify the core novelty as a two-stage process: 1) A computationally lightweight, online filtering algorithm called Sign Concordance Filtering (SCF), which is implemented using in-DRAM Processing-in-Memory (PIM) logic (PFUs). This stage prunes the vast majority of non-candidate vectors without moving them off-chip. 2) An exact nearest neighbor search (ENNS) performed on the much smaller, filtered set of vectors using a near-memory accelerator (NMA).

The central novel claim is not the algorithm or the hardware in isolation, but their tight co-design. Specifically, the identification of an algorithm (SCF) that is simple enough for efficient PIM implementation yet effective enough to drastically reduce the search space for a subsequent, more complex near-memory processing stage.

Strengths

1. Novelty of the Co-Design: The primary strength of this work is the symbiotic relationship between the proposed algorithm and hardware. While PIM and near-memory accelerators for search are not new concepts, the authors have identified a particularly elegant primitive (Sign Concordance Filtering) that is exceptionally well-suited for in-DRAM implementation (bitwise XOR and accumulation). This avoids the complexity of implementing more sophisticated indexing or hashing schemes in memory. This specific co-design appears to be novel.

2. Online vs. Offline Filtering: The proposed SCF is an online filter, distinguishing it from the dominant paradigm of offline index construction used in ANNS methods like HNSW and IVF. This is a significant conceptual departure. By avoiding a rigid, pre-computed index structure, DReX offers flexibility for dynamic datasets where vectors are frequently added or updated, a point briefly touched upon in Section 8 (page 14). This online nature is a key part of its novelty.

3. Clear Delta from Prior Art: The authors build upon existing work, including what appears to be their own (IKS [61]), which focused on near-memory ENNS acceleration. The novelty of DReX is clearly articulated as the addition of the in-DRAM SCF pre-filtering stage (the PFUs). The paper effectively demonstrates that this new component is responsible for the majority of the performance gain over a pure near-memory ENNS accelerator. This represents a significant and non-obvious extension of prior art.

Weaknesses

1. Algorithmic Proximity to LSH: The core SCF mechanism, while presented as a novel heuristic, bears a strong conceptual resemblance to certain variants of Locality Sensitive Hashing (LSH), particularly SimHash, where the sign of dot products with random vectors forms a hash. SCF uses the signs of the vector's own components, which can be viewed as dot products with axis-aligned basis vectors. While functionally distinct (no random projections), the underlying principle of using sign agreement as a proxy for angular similarity is a well-established concept. The paper would be stronger if it more directly confronted this similarity and provided a clearer analysis of why its direct, axis-aligned approach is superior to a PIM-implemented LSH scheme for this specific problem.

2. Limited Novelty of the NMA Component: The paper acknowledges in Section 7.1 (page 10) that the ENNS-only configuration is "equivalent to IKS [61]". This implies the near-memory accelerator (NMA) architecture for the second stage is not, in itself, a novel contribution of this work. The novelty is therefore confined to the PFU filtering logic and the system-level pipeline connecting the PFU and NMA. This should be stated more explicitly upfront in the architectural description (Section 5) to precisely frame the paper's contribution.

3. Generality of the Core Mechanism: The effectiveness of SCF is predicated on the assumption that vector embeddings are distributed somewhat symmetrically around zero. The authors rightly identify this limitation in Section 8 (page 14) and show a pathological case where performance collapses. While they propose a known technique (ITQ) as a remedy, this highlights that the core novel mechanism is not universally applicable without data pre-processing. The novelty is therefore in a mechanism that works exceptionally well for a specific, albeit common, data distribution.

Questions to Address In Rebuttal

1. Please clarify the novelty of Sign Concordance Filtering (SCF) with respect to established LSH techniques. Could you provide a brief theoretical or empirical comparison against a PIM-amenable LSH scheme (e.g., a single-table SimHash)? What are the specific trade-offs in terms of hardware complexity, filtering efficacy, and recall that make SCF a superior choice for this co-design?

2. The paper discusses a pathological case (Figure 18, page 14) requiring ITQ pre-processing. In a dynamic RAG environment where new documents are constantly added, what is the overhead of applying or updating an ITQ rotation to the corpus? Does this requirement for a global data transformation compromise the "simple update" story that is a key advantage of DReX over ANNS methods?

3. Could the authors be more precise about the novel contributions within the Near-Memory Accelerator (NMA) architecture itself? Beyond leveraging the design from IKS [61], are there any specific modifications or optimizations made to the NMA to better handle the sparse and temporally unpredictable stream of candidate vectors produced by the PFU stage? Or is the novelty purely in the addition of the PIM front-end?

Review Form

Reviewer: The Guardian (Adversarial Skeptic)

---

Summary

The paper presents EOD, a co-designed hardware/software system for low-latency Graph Neural Network (GNN) inference in an inductive setting. The core idea is to mitigate the latency caused by neighborhood explosion and data preparation by precomputing hidden features for all training nodes. To manage the resulting memory overhead, the authors propose a "multi-layer concatenate" compression scheme (ZVC). These algorithmic changes are supported by a custom DIMM-based Near-Memory Processing (NMP) architecture designed for efficient aggregation of the precomputed, compressed features.

While the paper identifies a valid and important problem, the proposed solution rests on a foundation of precomputation that introduces critical methodological flaws. The evaluation framework compares fundamentally different amounts of online work, leading to inflated performance claims, and fails to account for the substantial hidden costs required to maintain the system's accuracy, thereby undermining its claimed real-world viability.

Strengths

1. Clear Problem Identification: The paper correctly identifies that the Preparation and Memcpy steps, rather than pure computation, are the primary bottlenecks for real-time GNN inference services. The latency breakdown in Figure 3(a) effectively motivates the need to address these stages.

2. Novel Compression Heuristic: The observation of an inverse relationship in sparsity between consecutive GNN layers (Figure 5) and the proposed multi-layer concatenation method to exploit this for ZVC compression (Section 4.2) is a clever algorithmic insight. This technique effectively increases the minimum number of zero-values per node, enhancing the efficacy of the chosen compression scheme.

Weaknesses

1. Fundamental Evaluation Flaw: Unaccounted Maintenance Cost: The entire premise of precomputation introduces data staleness. As new nodes and edges are added during online service, the precomputed embeddings for training nodes become outdated. The authors acknowledge this and show the resulting accuracy degradation in Figure 15. Their proposed solution is a "periodic re-precomputation (refresh)." This refresh operation is computationally equivalent to performing GNN inference on the entire training graph, a massively expensive and high-latency process. This cost is completely omitted from the performance evaluation. A system that requires periodic high-latency downtime or background computation to remain accurate cannot be fairly evaluated on its low-latency inference capabilities alone. This omission is a critical flaw that invalidates the claims of providing a practical low-latency solution.

2. Misleading Baseline Comparison: The headline speedup claims (e.g., 17.9x geometric mean end-to-end) are derived from comparing the proposed EOD system against a standard GPU baseline. This is an apples-to-oranges comparison. The GPU baseline performs the full L-hop neighborhood traversal and feature gathering online (Preparation step), while EOD offloads this expensive traversal to an offline precomputation step. EOD's online workload is fundamentally smaller and simpler. A fair comparison would be against a GPU baseline that also leverages precomputed data (the "GPUpre" case in Figure 14). As the paper itself states on page 12, the speedup of EOD over GPUpre is a much more modest 1.14-1.35×. The massive reported speedups are an artifact of an inequitable experimental setup, not a revolutionary performance gain in like-for-like inference.

3. Overstated Aggregation Performance: The paper prominently features aggregation speedups of over 900x (Abstract, Figure 13). While technically representing the performance on one sub-task, this is misleading. For a system paper focused on end-to-end latency, this cherry-picked metric dramatically inflates the perceived contribution. The end-to-end latency, which is the only metric that matters to the end-user, shows far more modest gains.

4. Unquantified Cumulative Accuracy Loss: The proposed system introduces at least three distinct approximations: a. Pruning of target-to-train edges (Section 4.1). b. An "adjusted ReLU threshold" to increase sparsity (Section 4.2). c. The inherent staleness of precomputed embeddings between refreshes (Section 6.4). The paper analyzes these in isolation (or not at all, in the case of pruning's specific % drop) but never presents a clear analysis of their cumulative impact. The final accuracy of the EOD system at the moment just before a refresh cycle, compared to a gold-standard model with no approximations, is never reported. This leaves the true accuracy cost of the system entirely ambiguous.

Questions to Address In Rebuttal

1. What is the wall-clock time required for the "periodic re-precomputation" on the datasets evaluated (e.g., Products, Reddit)? How does this "maintenance latency" compare to the aggregated inference latency served during one refresh period? Please justify how a system requiring such a costly refresh operation can be considered a "low-latency" solution in a continuously operating online service.

2. Please justify the use of the standard GPU as the primary baseline for end-to-end speedup claims, given that it performs a fundamentally larger online workload (L-hop traversal) than EOD. Why should the results not be primarily framed in comparison to the "GPUpre" baseline, which represents a more direct, apples-to-apples comparison of inference hardware for a precomputation-based algorithm?

3. Provide a table detailing the final, end-to-end inference accuracy of the EOD system under the combined effects of all three approximations (pruning, ReLU thresholding, and data staleness at the end of a refresh cycle) compared to a non-approximated baseline model.

4. The multi-layer concatenation compression scheme appears to depend on the properties of the ReLU activation function. How does this technique perform with other common GNN models that use different activation functions (e.g., LeakyReLU, GeLU) or architectures that do not rely on simple activation sparsity (e.g., GAT)? Please comment on the generality of this core contribution.

Of course. Here is a peer review of the paper from the perspective of "The Synthesizer."

---

Review Form:

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper presents EOD, a hardware-software co-designed system aimed at tackling the critical challenge of low-latency inference for Graph Neural Networks (GNNs) in an inductive, real-time setting. The authors correctly identify that the primary bottlenecks are not just the GNN computation itself, but the extensive data preparation and host-to-device data transfer, exacerbated by the "neighborhood explosion" problem.

The core contribution is an elegant decoupling of the inference process. The authors propose to precompute the computationally heavy and data-intensive propagation among the existing "train" nodes offline. This transforms the online inference problem from an L-hop graph traversal to L separate 1-hop aggregations from the precomputed train node features to the new "target" nodes. To manage the storage overhead of these precomputed features, they introduce a novel "concatenated ZVC" compression scheme that exploits sparsity patterns across different GNN layers. This algorithmic approach is supported by a DIMM-based Near-Memory Processing (NMP) architecture designed specifically to perform the parallel 1-hop aggregations on the compressed data, thereby minimizing data movement and leveraging memory-side parallelism. The result is a system that dramatically reduces end-to-end latency for small-batch GNN serving.

Strengths

The true strength of this paper lies in its insightful problem formulation and the holistic, co-designed solution it proposes.

1. Addressing the Right Problem: The GNN acceleration literature is crowded with work on full-batch training or transductive inference. This paper wisely targets the inductive, mini-batch inference scenario, which is far more representative of real-world, latency-sensitive applications like fraud detection and real-time recommendation. The analysis in Section 3 (page 4), which clearly shows that data preparation and memory copy dominate latency, is a crucial observation that correctly frames the entire problem and motivates the need for a system-level solution beyond a simple compute accelerator.

2. Elegant Algorithmic/Hardware Co-Design: This is not a paper that simply throws hardware at a problem. The algorithmic innovation—precomputing train-to-train propagation—is the key enabler. This single decision fundamentally changes the nature of the online workload, making it vastly more regular and amenable to a specialized NMP architecture. The subsequent compression scheme (Section 4.2, page 5) is a clever and necessary component to make the precomputation practical from a memory capacity perspective. The hardware is therefore not a generic NMP solution, but one tailored to execute the specific "concatenate aggregation" task enabled by the algorithm. This synergy is the paper's most compelling feature.

3. Connecting Disparate Research Threads: This work serves as an excellent synthesis of several important trends in computer science. It sits at the intersection of:

   • GNNs: Taking models like GraphSAGE out of the lab and into production environments.
   • Near-Memory Processing: Applying the principles pioneered in domains like recommendation systems (e.g., RecNMP) to the unique data access patterns of GNN inference.
   • Systems for ML: Recognizing that deploying ML is a full-stack problem, where data movement, I/O, and preprocessing are often more critical than the matrix multiplications themselves.

By building these bridges, the paper provides a valuable blueprint for future research in practical ML systems.

Weaknesses

The weaknesses of the work are not in its core idea, which is sound, but in the assumptions that bound its current applicability. As a synthesizer, I see these less as flaws and more as the most fertile grounds for future work.

1. The Static "Training Graph" Assumption: The precomputation strategy is highly effective but hinges on the set of training nodes and their features being largely static. In many real-world systems (e.g., social networks or e-commerce platforms), the graph is constantly evolving with new users, products, and interactions that become part of the "known" graph. The paper's proposed solution of "periodic re-precomputation" (Section 6.4, page 12) is a practical but reactive fix. This approach avoids the crucial question of how to incrementally and efficiently update the precomputed embeddings as the base graph changes, which is a significant challenge for real-world deployment.

2. Single-Node System Abstraction: The evaluation is performed in the context of a single server with an in-memory graph database. This is a reasonable starting point, but the largest and most valuable graphs are almost always distributed across a cluster of machines. The paper acknowledges this limitation in the discussion (Section 7, page 13). In a distributed setting, the data-fetching and aggregation step would involve network latency, which could easily become the new dominant bottleneck, potentially negating some of the benefits of NMP. The paper would be strengthened by a more thorough discussion of how the EOD paradigm might extend to a distributed memory or storage environment.

3. Limited Generality of Observations: The "concatenated ZVC" compression method is motivated by the empirical observation that sparsity in hidden features can be inversely correlated across layers (Figure 5, page 6). This is an interesting finding, but its generality is not established. It may be a specific artifact of the GraphSAGE model with ReLU activations on the tested datasets. It is unclear if this property holds for other popular architectures (e.g., GATs) or different activation functions, which could limit the effectiveness of the proposed compression scheme.

Questions to Address In Rebuttal

1. Could the authors elaborate on the cost and complexity of the "refresh" or re-precomputation step? In a production environment with a constant stream of updates, how would one determine the optimal refresh frequency to balance the trade-off between inference accuracy (which decays as the graph becomes stale) and the computational cost of the refresh?

2. While a full distributed implementation is beyond scope, could the authors speculate on the architectural changes EOD would require to function in a sharded graph environment? For instance, would NMP modules need to communicate with each other (e.g., via a technology like DIMM-Link, mentioned in their citations), and how would the precomputed features be managed and accessed across the cluster?

3. Regarding the concatenated ZVC compression: Have you investigated if the observed inverse sparsity correlation holds for GNN models other than GraphSAGE, or for different non-linearities besides ReLU? How critical is a high compression ratio to the overall performance of EOD, and how gracefully does the system's performance degrade if the compression is less effective?

Review Form

Reviewer: The Innovator (Novelty Specialist)

Summary

This paper, "EOD: Enabling Low Latency GNN Inference via Near-Memory Concatenate Aggregation," proposes a co-designed hardware/software system to accelerate real-time, inductive Graph Neural Network (GNN) inference. The authors identify two primary bottlenecks: the data preparation/transfer overhead and the "neighborhood explosion" problem.

The core of their proposed solution consists of three main components: 1. An algorithmic optimization based on precomputing the hidden features for all training nodes (tr-to-tr propagation), thereby reducing the online inference workload to L separate 1-hop aggregations instead of a single L-hop aggregation. 2. A compression scheme that concatenates the hidden features of a node across all GNN layers before applying Zero Value Compression (ZVC), which aims to improve the compression ratio by averaging out sparsity variations between layers. 3. A DIMM-based Near-Memory Processing (NMP) architecture specifically designed to accelerate the tr-to-tar aggregation step on data stored in this concatenated and compressed format.

While the paper demonstrates substantial performance improvements, the novelty of the core constituent ideas is limited. The primary contribution lies in the specific synthesis and co-optimization of these components into a functional and high-performance system for a very specific workload.

Strengths

1. Novel System-Level Integration: The primary strength of this work is the coherent and tightly-coupled integration of pre-existing concepts (precomputation, ZVC, NMP) into a specialized system. While the individual parts are not new, their combination to solve the low-latency inductive GNN inference problem is a novel system-level engineering contribution.

2. Minor Algorithmic Novelty in Compression: The idea of concatenating hidden features across multiple layers before applying ZVC (Section 4.2, Pages 5-6) is a clever, albeit incremental, optimization. By combining feature vectors that may have anti-correlated sparsity patterns (as shown in Figure 5), the authors create a more favorable data distribution for compression. I have not seen this specific technique applied in prior GNN acceleration work.

3. Specialized Hardware Co-design: The NMP architecture is novel in its specialization. It is not a generic NMP unit; it is purpose-built to handle the proposed concatenated ZVC data format, including a custom instruction (Agg-instruction, Figure 10, Page 8) and an integrated ZVC decompressor (Figure 9d, Page 8). This demonstrates a deep co-design.

Weaknesses

The central weakness of this paper, from a novelty perspective, is that its foundational algorithmic and architectural pillars are built upon well-established prior art.

1. Core Algorithmic Premise is Not Novel: The main algorithmic trick—pruning tar-to-tr edges and precomputing tr-to-tr propagation to simplify online inference (Section 4.1, Page 4-5)—is not a new idea. This exact concept of decoupling training and test node computations to accelerate inductive inference has been explored before. Specifically, the work by Si et al., "Serving graph compression for graph neural networks" (ICLR 2023) [51], which the authors cite, proposes this very same decoupling strategy as its central contribution. The current paper re-implements this known technique as the basis for its hardware acceleration, but the fundamental insight is not original.

2. Architectural Paradigm is Not Novel: The use of DIMM-based NMP for accelerating aggregation-heavy workloads is a known pattern. Prior works such as RecNMP [24] and Tensordimm [28] established the viability of placing logic on a DIMM's buffer chip to process embedding table lookups for recommendation models. More directly, GNNear [64] and GraNDe [61] have already proposed using DIMM-based NMP to accelerate GNNs. EOD follows this established architectural template. Its novelty is confined to the specific logic implemented within the NMP unit, which is tailored to its specific data format, rather than a fundamentally new NMP architecture.

3. Compression Primitive is Standard: The use of Zero Value Compression (ZVC) to exploit activation sparsity is a standard technique in ML accelerators, as seen in works like Rhu et al., "Compressing DMA engine" (HPCA 2018) [47]. The novelty in EOD lies only in the pre-processing step (concatenation), not in the compression mechanism itself.

In essence, the paper takes a known algorithm from [51], implements it on a known architectural template from [24, 61, 64], and uses a standard compression primitive [47] with a minor pre-processing twist. The impressive speedup numbers are a result of aggressively applying this known precomputation algorithm, which fundamentally changes the problem from an L-hop graph traversal to L independent table lookups—a task for which NMP is known to be highly effective.

Questions to Address In Rebuttal

1. The core precomputation strategy described in Section 4.1 appears functionally identical to the decoupling method proposed in your cited work [51]. Please clarify, with technical precision, what the novel algorithmic difference is between your method and the one in [51]. If there is no significant difference, please justify why building hardware for a known algorithm constitutes a sufficient contribution for this venue.

2. The multi-layer concatenation technique for ZVC is presented as a key enabler. Is this technique generally applicable to other GNN acceleration scenarios, or is its utility strictly limited to a context where all layers' hidden features are precomputed and stored? Its novelty is proportional to its generality.

3. Given that the primary performance gain comes from transforming the problem into one that is embarrassingly parallel (L independent 1-hop aggregations), how much of the benefit is from the NMP architecture versus the algorithm itself? Figure 14a shows that even a "GPUpre" baseline (precomputation on GPU) achieves a significant speedup (12.0-16.4x). This suggests the algorithm is the dominant factor. Please argue for the significance of the NMP architecture's novel aspects beyond simply being a good fit for a known memory-bound problem.

Here is the completed review form from the perspective of "The Guardian."

---

Review Form

Reviewer: The Guardian (Adversarial Skeptic)

Summary

The authors present RAP, a reconfigurable in-memory automata processor designed to support three distinct automata models: NFA, NBVA, and LNFA. The central thesis is that real-world regex workloads are diverse (as shown in Figure 1), and a single-mode processor is inherently inefficient. By reusing 8T-SRAM cells for character classes (CCs) and bit vectors (BVs) and implementing specialized logic for different modes, the authors claim significant improvements in energy efficiency and compute density over state-of-the-art (SotA) specialized processors.

However, the paper's claims of superiority are predicated on a series of unquantified overheads, arbitrary compilation heuristics, and a potentially biased evaluation framework. While the motivation is sound, the presented evidence lacks the rigor required to fully substantiate the claimed benefits of reconfigurability.

Strengths

1. Clear Motivation: The paper effectively motivates the need for a reconfigurable architecture. The analysis in Figure 1 (page 1) provides a compelling, data-driven rationale that different workloads benefit from different automata representations, which is the foundational premise of this work.
2. Core Architectural Concept: The fundamental idea of repurposing the same in-memory fabric (8T-SRAM and local switches) to implement distinct computational models (NFA state matching, NBVA bit vector storage/manipulation, and LNFA Shift-And execution) is a technically sound and resourceful approach.

Weaknesses

1. Unquantified Overhead of Reconfigurability: The paper repeatedly claims the reconfigurable design has "little overhead" (Abstract, Section 3) but fails to provide a quantitative analysis to support this. The architecture requires additional components that are not present in a single-mode processor:

   • The NBVA mode requires metadata to identify BV-STEs, an overflow checker, and control logic for the pipelined bit-vector-processing phase (Section 3.1, page 5).
   • The LNFA mode repurposes pipeline buffers and requires a "specialized routing path for the active vector to facilitate bitwise shift" (Section 3.2, page 7). These components consume static and dynamic power and silicon area, irrespective of the mode being used. The paper does not provide a breakdown of these overhead costs, making it impossible to assess the true cost-benefit of the proposed reconfigurability. The improvements shown might be entirely consumed by these overheads in workloads that do not heavily leverage all modes.

2. Arbitrary Compiler Heuristics: The decision-making process in the compiler relies on unsubstantiated heuristics. Specifically, in Section 4.2 (page 9), the compiler will use LNFA mode only if the required state expansion "does not increase the number of states by more than 2x." This "2x" threshold is arbitrary. There is no sensitivity analysis or justification provided for this specific value. A different threshold could dramatically alter the NFA/LNFA workload distribution and, consequently, the final performance results. This undermines the claim that the compiler finds the "best automata model" (Section 2, page 2).

3. Subjective Design Space Exploration (DSE): The selection of "optimal" parameters from the DSE in Section 5.3 (page 10 and Figure 10) appears subjective. The authors state, "We choose the depth that improves energy and area while offering acceptable throughput." The term "acceptable" is not defined quantitatively. For example, in Figure 10(a) for the Snort benchmark, depth=32 offers better area than the chosen depth=16. It appears parameters were selected to present the most favorable narrative rather than by applying a consistent, rigorous optimization function. This potential for post-hoc rationalization casts doubt on the optimality of the final configuration.

4. Opaque System-Level Aggregation: A critical detail regarding performance normalization is buried in Section 5.5 (page 11). The authors state that if an array's throughput in NBVA mode falls below 2 Gch/s, they "assign another RAP array to work on the same regexes to share the workload." This effectively duplicates hardware resources for certain benchmarks. It is unclear if this duplication is reflected in the final area, power, and compute density figures presented in Figure 12. If a benchmark required two arrays, its area and power should be doubled, and its compute density halved. The paper does not specify which benchmarks required this treatment, making the final summary comparisons in Figure 12 potentially misleading.

5. Potentially Biased Benchmark Comparison: The evaluation in Tables 2 and 3 compares RAP against SotA ASICs (CAMA, BVAP, CA) using a new set of benchmarks curated by the authors. While claimed to be more "up-to-date," this prevents a direct, apples-to-apples comparison against the published results for those architectures, which were often evaluated on standard suites like ANMLZoo. Without a characterization showing how these new benchmarks differ from established ones, it is difficult to determine if RAP's superior performance stems from its architecture or from a benchmark set that happens to align perfectly with its strengths.

Questions to Address In Rebuttal

1. Please provide a detailed area and power breakdown of the hardware components required specifically for reconfigurability. This includes the mode-selection logic, control logic for the NBVA pipeline, the specialized LNFA active-vector routing, and any other overheads not present in a baseline single-mode NFA processor like CAMA.
2. Justify the "2x" state-expansion heuristic for LNFA compilation (Section 4.2). Provide a sensitivity analysis showing how the overall system performance changes if this threshold is set to 1.5x, 2.5x, or 3x.
3. Define the precise, quantitative cost function used to select the final parameters from your design space exploration (Figure 10). Specifically, how did you weigh throughput penalties against area and energy savings to define an "acceptable" trade-off?
4. Clarify the system-level mapping policy mentioned in Section 5.5. Which of the evaluated benchmarks required the use of multiple RAP arrays to meet the 2 Gch/s throughput target? How are the duplicated area and power consumption for these benchmarks accounted for in the final aggregated results presented in Figure 12?